ConfigSys

A Configuration Management Method Based On XML/XSLT

Chapter 1: The Madness
Have a lot of servers, odds are pretty good most of them all fall into one of several categories and it's very likely those are running the same components in various combination and providing the same services clustered or not with only a few words and values scattered across each in a dozen or more places, all in separate files with no coordination in between one another other than a careful admin edited them, that actually separate one host from another. You hatch another server whether virtual or real and then log in and edit all the files with a text editor to suit it's purpose every time you need one and if a server dies you start from scratch. Mistakes are made, maybe you skip a step or an entire file or just do it a bit differently and then start the debugging and head scratching. Once inconsistencies are introduced the smooth upgrade path and migration forward just became a herculean task of compounded frustration and manual labor instead of the automated sheer bliss it should be. This document will introduce one method of centralizing all the information that separates them in a central XML file store and converting configuration files and even scraps of script to XSL templates and use them to configure hosts in a reliable, repeatable fashion and aid in automating server build, software installation, deployments, maintenance and moving builds from zone to zone with predictable results.

Chapter 2: The Method
Take all the minute details and errata that separates one host from another and put them all in one central place on your network. For this purpose we'll use XML since it has a variety of editors, becomes almost self documenting with good descriptive tag names and does not require a schema making the design even more flexible. Then use a template, for our purpose we'll use XSLT since it's robust, well documented and provides not only interpolation and text substitution but logic, math and more to make this even more flexible than it has to be. For this example let's pretend we are building portal servers that need a database connection, a directory server connection and a web server configuration all unique to each, we would start by creating a doc with all these unique bits of information for one host and it would look something like this:

<?xml version="1.0"?> <node> <dbms-dbname>lportal3</dbms-dbname> <dbms-host>dbmsbox3</dbms-host> <dbms-jndi>jdbc/LiferayPool</dbms-jndi> <dbms-port>3306</dbms-port> <dbms-pword>fishm0nger</dbms-pword> <dbms-user>dbmsguy</dbms-user> <httpd-group>apache</httpd-group> <httpd-name>appserver3</httpd-name> <httpd-port>80</httpd-port> <httpd-ssl-key-name>lportal</httpd-ssl-key-name> <httpd-user>apache</httpd-user> <ldap-cn>cn=Directory Manager</ldap-cn> <ldap-dn>dc=fullsack,dc=com</ldap-dn> <ldap-host>dir.fullsack.com</ldap-host> <ldap-port>389</ldap-port> <ldap-pword>fl0un9er</ldap-pword> </node>

I use, "node" for the root document tags, use whatever suits the device as long as it matches your stylesheet templates. The names of tags are what is meaningful for you to use and only need to match the tags used in your stylesheets, keep in mind they will be sorted by tag name and use it wisely. Save your XML data files to /usr/local/Templates/Type/Host or where it matches the templates directory value in the configsys/WEB-INF/web.xml file and the device name as in this example, a Host. ConfigSys will work on multiple object types, just add a, "type", named directory in templates/Type and fill it with XML files and it will show up as an additional line on the main menu. Use the pattern <the device's hostname>.xml for the XML files names so it will be easy to apply your configurations in scripts later. Now copy that files contents to one named NEW.xml in the same directory then edit and remove any values truly unique and leave a set of defaults, ConfigSys will use this to create new configurations quickly through a web browser form, edit NEW and give it a new file name. The multiple device named directories can be used to sort and group your devices into more manageable sets. This document will not teach you beyond the basic in and out of XSL but will only show you only most basic form of the transformation, how to protect the text you want passed through and how to interpolate it with the text in between the tags of your XML document. First create a stylesheet in /usr/local/templates or where you have the configsys/WEB-INF/web.xml pointing to. First develop a working configuration file we need, copy it to /usr/local/templates/orig for reference. Our example will be a simple ssl.conf to point to the right license file for this host and looks like:

LoadModule ssl_module modules/mod_ssl.so Listen 443 AddType application/x-x509-ca-cert .crt AddType application/x-pkcs7-crl .crl SSLPassPhraseDialog builtin SSLSessionCache dc:UNIX:/var/cache/mod_ssl/distcache SSLSessionCacheTimeout 300 SSLMutex default SSLRandomSeed startup file:/dev/random 512 SSLRandomSeed connect file:/dev/random 512 SSLCryptoDevice ubsec <VirtualHost _default_:443> ErrorLog logs/ssl_error_log TransferLog logs/ssl_access_log LogLevel warn SSLEngine on SSLProtocol all -SSLv2 SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW SSLCertificateFile /etc/pki/tls/certs/localhost.crt SSLCertificateKeyFile /etc/pki/tls/private/localhost.key <Files ~ "\.(cgi|shtml|phtml|php3?)$"> SSLOptions +StdEnvVars </Files> <Directory "/var/www/cgi-bin"> SSLOptions +StdEnvVars </Directory> SetEnvIf User-Agent ".*MSIE.*" \ nokeepalive ssl-unclean-shutdown \ downgrade-1.0 force-response-1.0 CustomLog logs/ssl_request_log \ "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b" </VirtualHost>

Some careful analysis reveals the only thing we need to change per host is the name and location of the keys in bold above. We store our keys in /etc/httpd/certs and named for the host. But first we must mark up the areas we want to preserve as character data by bracketing inside a tag of, "<![CDATA[", and, "]]>. Next replace the text we want to substitute with the XSL tag to select the value in the XML tag of the same name outside of the CDATA tag. We'll need to describe it as an XML file, and instruct the processor to output it as UTF-8 text with no <?xml version?> declarations. The sole template inside our stylesheet, (there could be many), matches on the document root tag and reads in the values we'll use and select on the field we need with a little boiler plate text added like so:

<?xml version="1.0"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:output method="text" version="1.0" encoding="UTF-8" indent="no" omit-xml-declaration="yes"/> <xsl:template match="node"><![CDATA[# LoadModule ssl_module modules/mod_ssl.so Listen 443 AddType application/x-x509-ca-cert .crt AddType application/x-pkcs7-crl .crl <Files ~ "\.(cgi|shtml|phtml|php3?)$"> SSLOptions +StdEnvVars </Files> <Directory "/var/www/cgi-bin"> SSLOptions +StdEnvVars </Directory> NameVirtualHost *:443 <VirtualHost *:443> SSLEngine on]]> SSLCertificateFile /etc/httpd/certs/<xsl:value-of select="httpd-ssl-key-name"/>.crt SSLCertificateKeyFile /etc/httpd/certs/<xsl:value-of select="httpd-ssl-key-name"/>.key <![CDATA[ErrorLog logs/ssl_error_log TransferLog logs/ssl_access_log LogLevel warn </VirtualHost> ]]> </xsl:template> </xsl:stylesheet>

Chapter 3: The Installation And Usage
Unzip the application configsys-#.#.zip into the webapps directory of your Java application server. If running ControlTier this might be ControlTier-3.6.0/pkgs/jetty-6.1.21/webapps/. Unzip the configsys-templates.zip into /usr/local. Chown -R both webapps/configsys and /usr/local/templates to your installation user. Once unzipped restart your application service and configsys should be found at http://ControlTierServerName:8080/configsys. Edit the file webapps/configsys/WEB-INF/web.xml and change the URL's and directories in context parameters to match your installation. Click edit and you should see your XML files, new and first host listed. Clicking on them loads them into a web form to edit and save, NEW will need to be named before saving. If you use an existing file's name it will be overwritten. You can open a file and then change the name field to make a copy. Click view and you should see your XSL templates on the left and your XML files on the right. Select two and submit to see the transformation of them. This is the raw output of the apply servlet and how you will make use of your configuration data. As your install user on your target host you can use the following syntax of curl or wget to fetch, transform and write to disk with a command pattern of, "apply stylesheet document" as in:

sudo curl -s "http://ControlTierServer:8080/configsys/apply?style=ssl-conf&doc=`hostname`&type=Host&" -o /etc/httpd/conf.d/ssl.conf or sudo wget "http://ControlTierServer:8080/configsys/apply?style=ssl-conf&doc=`hostname`&type=Host&" < /etc/httpd/conf.d/ssl.conf

or you can still do this without java or a application server by making a transformation locally with a perl script like so:

#!/usr/bin/perl # usage: # apply.pl stylesheet xmlfile # example: # ./apply.pl ssl.xsl Type/Host/mrsblapp00399.xml # # import required modules use XML::XSLT; use XML::DOM; # get args my $xslfile = $ARGV[0]; my $xmlfile = $ARGV[1]; # create an instance of XSL::XSLT processor my $xslt = eval { XML::XSLT->new ($xslfile, warnings => 1, debug => 0) }; # some error handling here ... if ($@) { die("Sorry, Could not create an instance of the XSL Processor using $xslfile.\n"); } # transform XML file using the XSL style sheet and get DOM my $xmldom = eval { $xslt->transform($xmlfile) }; if ($@) { die("ERROR: Could not transform XML file $xmlfile.\n"); } # use dom obj to expand pre-defined entities print $xmldom->expandEntityRefs($xmldom->toString); # reclaim resources $xmldom->dispose(); $xslt->dispose();

In one intstance I used CGI programming to fill out a form and generate the XML document then apply the transformation to generate configurations with a local web server running as the install user. Uses are mostly limited by you for instance instead of configuration files it could be generating Makefiles or any thing you need reliable results on. I hope this has helped you in your mission and XML/XSLT will continue to be another useful tool in your System Engineering toolbox.

H8 linus_AT_fullsack.com for this, last update 8/27/12.