Tuesday, April 27, 2010

Novell IDM XSL to Global Find/Replace

I heavily use Novell IDM if you haven't noticed from my other blog articles.  Recently, I needed to use Novell IDM to do a global find/replace.  Typically, this can be done in policy very quickly by just replacing all instances of a character with another character (or series of characters).

I hit an issue where the characters that I was doing the replacement with were special characters in the context of the IDM, so they were not output correctly.  In order to get the replacement done correctly, I needed to do it in the Output Transformation policy so they didn't get fubar'd while passing the text around.

So, what I did was make an XSL Template to do a global find/replace, then call the function.  I was pretty proud of my code, its pretty elegant the way it was implemented.

    <xsl:template name="globalReplace">
        <xsl:param name="outputString"/>
        <xsl:param name="target"/>
        <xsl:param name="replacement"/>
        <xsl:choose>
            <xsl:when test="contains($outputString,$target)">
                <xsl:value-of select="concat(substring-before($outputString,$target),$replacement)"/>
                <xsl:call-template name="globalReplace">
                    <xsl:with-param name="outputString" select="substring-after($outputString,$target)"/>
                    <xsl:with-param name="target" select="$target"/>
                    <xsl:with-param name="replacement" select="$replacement"/>
                </xsl:call-template>
            </xsl:when>
            <xsl:otherwise>
                <xsl:value-of select="$outputString"/>
            </xsl:otherwise>
        </xsl:choose>
    </xsl:template>

So, we pass this function our information.  If the target isn't present, it just throws back the string unmodified.  If the target is present, it takes the substring up to that string, tacks on the replacement, then recursively calls itself until all replacements are done.

Nice little piece of code for XSL to hold on to.

Cisco VPN Client for 64-bit Windows 7

So, I've stumbled upon a VPN client that I can use for a 64-bit windows OS.  Someone posted this and I thought I'd pass it along because it works!  Its called Shrew VPN, frmo a company called Shrew Soft Inc.  I downloaded it for free and am using it on 64-bit Windows 7 Ultimate and it seems to be working like a champ!

http://www.shrew.net/software

Enjoy!

3DES Encrypt data with Novell IDM

If you check the blog prior to this one, I developed a method to call a web service URL from within any driver for Novell IDM.  This is all good, but what happens if we need to pass sensitive data in this URL.  We can't just do a GET operation with a URL that has a password in clear text, that's just asking for trouble.

What we do to protect this data is to throw some 3DES encryption on it before we throw the args on the end.  With the 3DES encryption, we also need a function to URLEncode the contents so the URL is written in a language the browser can send across.  Once again, I used some ECMAScript and code to get this done.

Please note, you will need to generate your own encryption key, then also use this key and similar code on the remote side to decrypt the contents to read it.

Here is the code.  The first portion is two functions.  The first function encrypts the data string passed to it, then passes it to the second function which will URLEncode it.

importPackage(Packages.javax.crypto);
importPackage(Packages.javax.crypto.spec);
importPackage(Packages.java.security.spec);
importPackage(Packages.java.io);
importPackage(Packages.sun.misc);
importClass(java.net.URLEncoder);

function DESEncrypt(theString) {
    try {
            var secretKey   = new SecretKeySpec(new Packages.sun.misc.BASE64Decoder().decodeBuffer(new java.lang.String("thisiswhereyouputyourkey")), "DESede");
            var ecipher = new Cipher.getInstance("DESede");
            ecipher.init(Cipher.ENCRYPT_MODE, secretKey);
            var utf8 = new java.lang.String(theString).getBytes("UTF8");
            var enc = ecipher.doFinal(utf8);
            return EncodeURLString(new Packages.sun.misc.BASE64Encoder().encode(enc));
        } catch (e) {
            return e.toString();
        }
       
        return null;
}

function EncodeURLString(theContents) {
    try {
        return new URLEncoder.encode(theContents, "UTF8");
    } catch (e) {
        return e.toString();
    }
    return null;
}

Once this ECMAScript is saved, pushed up, then called into your IDM, you can call it like this:

            <do-set-local-variable name="lv.EncryptedArgs" scope="policy">
                <arg-string>
                    <token-xpath expression="es:DESEncrypt($myArgs)"/>
                </arg-string>
            </do-set-local-variable>

Now, you have your encrypted arguments stored in a local variable, all nice and URL encoded.  Just tack it on the end of a URL and call it with the code defined in the previous blog article and you have now sent encrypted data across the wire.

Web Services with Novell IDM

So I have a pretty unique request.  A customer asked me to integrate IDM with a system that doesn't have a standard interface that I could use to integrate.  It did, however, have an API that we could use to create our own interface.  The customers programmer decided that the easiest way to implement this would be to setup a web service for me to call. 

While initially, I thought this was a candidate for the SOAP driver, I realized that this driver is overkill for the simplicity of this implementation.  This is a sample of how we would be creating a user in the system from IDM:

http://www.server.com/addUser?username=testuser&fname=test&lname=user&password=Passw0rd&othreattribute=other1&....

SOAP is complete overkill, I just need to construct a simlpe URL and call a linux wget on that URL.  The resulting status message would be returned instead of a typical HTML page when the API code processed the request.

I did some research and decided the easiest way to implement this would be to use ECMAScript to call some custom code that would very easily call my URL and return the result for me.  I would shove the result into a local variable in policy then jam that sucker into an attribute on the user object in eDirectory.  Seems like a very simple implementation, here is how I was able to do it.

Below is the ECMAScript for the function that I wrote.  Its a simple function, urlGet. You pass it the URL and it returns whatever is returned when a GET operation is performed on that URL.

importClass(java.net.URL);
importClass(java.io.InputStreamReader);
importClass(java.lang.StringBuilder);

function urlGet(urlString) {
try {
var url = new java.net.URL(String(urlString));
var stream = url.openStream();
var reader = new java.io.InputStreamReader(stream, "UTF-8");
var sb = new java.lang.StringBuilder();
var c;
while ((c = reader.read()) != -1) {
sb.append(String.fromCharCode(c));
}
return sb.toString();
} catch(e) {
return e.toString();
}
}

Now, we post that ECMAScript and use it in our driver with the following.  It will use the URL that is stored in the Local Variable myURL and return the results back to lv.URLResult.

            <do-set-local-variable name="lv.URLResult" scope="policy">
                <arg-string>
                    <token-xpath expression="es:urlGet($myURL)"/>
                </arg-string>
            </do-set-local-variable>

Program some logic to store the result in some sort of status attribute, then business logic to construct the correct URL and your can very easily IDM enable a web service based system.

Wednesday, March 24, 2010

Locked ESX Virtual Machines

Here is the scenario.  Power outage at 2am.  The outage lasted longer than our battery life.  We have in our roadmap plans to implement scripts to do graceful shutdowns when a low battery signal comes from the UPS, but for the time being, we do not have that.

We initially had some fun bringing everything back up in order.  It gets particularly fun when your AD Domain Controllers are all virtual, DNS is all virtual, and DHCP is virtual.  Get some nice little chicken and egg issues, but we have learned our lesson and are going to create a DNS, DHCP, and DC that are physical, so they can come up before the virtual environment.

The real meat of our problem was some of the virtual machines.  It wasn't directly related to the power outage either.  Our core switches, which are on separate UPS and did not lose power, decided to go Tango Uniform right after we got most of the boxes back up.  Unfortunately, our Netgear core switches are not sending logs to a syslog server and the log files do not persist a reboot (I know, I think its stupid too).  This means we don't have any way of knowing why they went stupid on us.

So, we have learned some lessons and moved on.  On to what I want this article to reflect.  When we lost the switches, the virtual machines lost their connection to their vmdk's.  We use NFS to connect to the datastore, so when we restored the switches, most of the virtual machines just flushed their writes and went on their merry way.  Some virtual machines, however, did not do this.  I inspected the vmware.log file stored with the virtual machine files to see what happened, and I noticed that on all of the virtual machines that were locked up (most of which were Windows XP boxes) had the following log message:

Mar 23 13:34:38.731: vmx| VMX has left the building: 0.

So, we have determined that VMWare just gave up on trying to talk to its VMDK file after some amount of time and the VMX decided to ditch this party.   Ok, so here is the procedure I had to go through to get the darn things back.

First, we need to get the virtual machine into an 'off' state.  This is not easy, nor is it intuitive.  What I had to do was the following.

From the service console of the ESX Server running the VM, find the vmid of the virtual machine in question.  To do so, run the following command and grep for the Virtual Machine name,

cat /proc/vmware/vm/*/names | grep vdi-rivey

The return should  start with vmid=####.  Take this number into the next command, where we are looking for the VM Group.

less -S /proc/vmware/vm/####/cpu/status

You are looking for the vmgroup, which will look something like vm.####.  Next, using the VM group ID number, feed it into the following command to run an effective kill -9 on the VM within the VMKernel.  Note:  be sure to run this command as root (or use sudo).

/usr/lib/vmware/bin/vmkload_app --kill 9 ####

Once this command is complete, the virtual machine still shows as though its in a Powered On state.  To get the VIC to figure this out, I had to restart some daemons on ESX to force VI to figure it all out.  Please note, I disabled VMWare HA and DRS on my cluster because of some of the issues I was having, I am not sure what HA will do with the VM's on this ESX server if you run this command while they HA is enabled.

service vmware-vpxa restart
service mgmt-vmware restart

The virtual machines running on this ESX Server and the ESX Server itself will grey out in your VIC while the services restart.  When everything is back to normal, the VM in question will now be grayed out, with (invalid) appended to the name.

Next, I removed the virtual machine from the inventory, then browsed to it in the datastore, right clicked on the vmx file, and added it back into my inventory.  It still is not ready to boot because it has a couple of .lck directories (lock files).  I browsed in the service console to the virtual machine, went into its directory and ran the following command to blow away all of the locks

rm -rf .lck*

After this was done, I was able to boot the VM back up!  Unfortunately, good ole Windows had some issues on 3 of my 60+ virtual machines.  These virtual machines boot, but promptly lock up.  I am not sure why this happened to a small subset of VM's, but I am attributing this to corruption of the disk.  The OS lost access to the disk and we killed the virtual machine without flushing the writes, so that could have been the problem.  Luckily, we take snapshots every night of the volume that holds the VM's at midnight.  I simply copied the entire VM directory from this backup, blew away the lock files again, added it to inventory, and bam, instant restore from backup.

Everything is back up and running.  We have a few infrastructure changes to make to help our recovery from a down state much quicker, we have a new reason to push for the scripts to bring everything down gracefully, and I have a procedure for unlocking a virtual machine.  We have also (again), verified that our backups are working like a champ!

Friday, March 12, 2010

Novell IDM XPATH

 This one is a pretty fun example.  I have a user coming from a payroll system.  The user has a PayrollCode identifier on them.  Unfortunately, this Payroll identifier code is not completely unique, so I have to query another object in eDirectory to get the uniqueCode.   To do this I will be using XPATH.  I have posted the actions XML of the rule below.  I'll break it apart and explain.

        <actions>
            <do-set-local-variable name="lv.PRCode" scope="policy">
                <arg-string>
                    <token-attr name="PRCode"/>
                </arg-string>
            </do-set-local-variable>
            <do-set-local-variable name="facnode" scope="policy">
                <arg-node-set>
                    <token-xpath expression='query:search($destQueryProcessor,"subordinate","","dn\of\subtree\I\want\to\search","ObjectClassName","PRCode",$lv.PRCode,"ReturnAttr1,uniqueCode,ReturnAttr3")'/>
                </arg-node-set>
            </do-set-local-variable>
            <do-for-each>
                <arg-node-set>
                    <token-local-variable name="facnode"/>
                </arg-node-set>
                <arg-actions>
                    <do-trace-message level="1">
                        <arg-string>
                            <token-xpath expression="$current-node[1]/attr[@attr-name='ReturnAttr1']/value"/>
                        </arg-string>
                    </do-trace-message>
                    <do-set-local-variable name="lv.ReturnAttr1" scope="policy">
                        <arg-string>
                            <token-xpath expression="$current-node[1]/attr[@attr-name='ReturnAttr1']/value"/>
                        </arg-string>
                    </do-set-local-variable>
                    <do-if>
                        <arg-conditions>
                            <and>
                                <if-global-variable mode="nocase" name="GCVAttr1" op="equal">$lv.ReturnAttr1$</if-global-variable>
                            </and>
                        </arg-conditions>
                        <arg-actions>
                            <do-set-dest-attr-value class-name="User" name="uniqueCode">
                                <arg-value>
                                    <token-xpath expression="$current-node[1]/attr[@attr-name='uniqueCode']/value"/>
                                </arg-value>
                            </do-set-dest-attr-value>
                            <do-set-dest-attr-value class-name="User" name="Attr3">
                                <arg-value>
                                    <token-xpath expression="$current-node[1]/attr[@attr-name='ReturnAttr3']/value"/>
                                </arg-value>
                            </do-set-dest-attr-value>
                        </arg-actions>
                        <arg-actions/>
                    </do-if>
                </arg-actions>
            </do-for-each>
            <do-if>
                <arg-conditions>
                    <or>
                        <if-op-attr name="uniqueCode" op="not-available"/>
                        <if-op-attr name="ReturnAttr3" op="not-available"/>
                        <if-op-attr mode="nocase" name="uniqueCode" op="equal"/>
                        <if-op-attr mode="nocase" name="ReturnAttr3" op="equal"/>
                    </or>
                </arg-conditions>
                <arg-actions>
                    <do-trace-message level="1">
                        <arg-string>
                            <token-text xml:space="preserve">No matching facility object found, uniqueCode and Attr3 not set.  Veto'ing transaction.</token-text>
                        </arg-string>
                    </do-trace-message>
                    <do-veto/>
                </arg-actions>
                <arg-actions/>
            </do-if>
        </actions>


Ok, now for the breakdown.  The first section actually executes the meat of  our sample, its the XPATH portion.  first I set a local variable so I don't have to query back on my JDBC driver if the attribute is not readily available.  I can just grab it and store it once in our policy.  Then, I run the XPATH query and set the nodeset to another local variable.

           <do-set-local-variable name="lv.PRCode" scope="policy">
                <arg-string>
                    <token-attr name="PRCode"/>
                </arg-string>
            </do-set-local-variable>
            <do-set-local-variable name="facnode" scope="policy">
                <arg-node-set>
                    <token-xpath expression='query:search($destQueryProcessor,"subordinate","","dn\of\subtree\I\want\to\search","ObjectClassName","PRCode",$lv.PRCode,"ReturnAttr1,uniqueCode,ReturnAttr3")'/>
                </arg-node-set>
            </do-set-local-variable>

The query uses the destQueryProcessor.  We put in the DN of the subtree we want to search (so the query doesn't take forever).  We are looking specifically at objects of class "ObjectClassName".  We are matching the PRCode attribute with the value in the lv.PRCode local variable.  Finally, for each resulting object we find, we want to grab ReturnAttr1, uniqueCode, and ReturnAttr3 attributes from it.

The next thing we are going to do is loop through all of our resulting nodes.

            <do-for-each>
                <arg-node-set>
                    <token-local-variable name="facnode"/>
                </arg-node-set>
                <arg-actions>
                    <do-trace-message level="1">
                        <arg-string>
                            <token-xpath expression="$current-node[1]/attr[@attr-name='ReturnAttr1']/value"/>
                        </arg-string>
                    </do-trace-message>
                    <do-set-local-variable name="lv.ReturnAttr1" scope="policy">
                        <arg-string>
                            <token-xpath expression="$current-node[1]/attr[@attr-name='ReturnAttr1']/value"/>
                        </arg-string>
                    </do-set-local-variable>

We grabbed the local variable facnode that is holding our resulting set.  Foreach will loop through each result.  I put some debug code in there to echo out in the trace file the result of each loop, its not necessary but nice to help step through the code in the trace.  In the result, we grab the ReturnAttr1 value and set it to a local variable lv.ReturnAttr1.  The next thing we are going to do is verify if lv.ReturnAttr1 meets our other criteria of matching a GCV.

                   <do-if>
                        <arg-conditions>
                            <and>
                                <if-global-variable mode="nocase" name="GCVAttr1" op="equal">$lv.ReturnAttr1$</if-global-variable>
                            </and>
                        </arg-conditions>

Pretty straight forward.  If there is a match, we execute the following section of code.

                      <arg-actions>
                            <do-set-dest-attr-value class-name="User" name="uniqueCode">
                                <arg-value>
                                    <token-xpath expression="$current-node[1]/attr[@attr-name='uniqueCode']/value"/>
                                </arg-value>
                            </do-set-dest-attr-value>
                            <do-set-dest-attr-value class-name="User" name="Attr3">
                                <arg-value>
                                    <token-xpath expression="$current-node[1]/attr[@attr-name='ReturnAttr3']/value"/>
                                </arg-value>
                            </do-set-dest-attr-value>
                        </arg-actions>
                        <arg-actions/>
                    </do-if>
                </arg-actions>
            </do-for-each>

If there is a match, I grab the values of the other two attributes (uniqueCode and ReturnAttr3) and stuff them in attributes on the Current User object I am processing.  If not, it will continue looping through the objects.  Once the loop is finished, I want to verify that I found a result and kick back a trace message and veto if I did not find a match.

           <do-if>
                <arg-conditions>
                    <or>
                        <if-op-attr name="uniqueCode" op="not-available"/>
                        <if-op-attr name="ReturnAttr3" op="not-available"/>
                        <if-op-attr mode="nocase" name="uniqueCode" op="equal"/>
                        <if-op-attr mode="nocase" name="ReturnAttr3" op="equal"/>
                    </or>
                </arg-conditions>
                <arg-actions>
                    <do-trace-message level="1">
                        <arg-string>
                            <token-text xml:space="preserve">No matching facility object found, uniqueCode and Attr3 not set.  Veto'ing transaction.</token-text>
                        </arg-string>
                    </do-trace-message>
                    <do-veto/>
                </arg-actions>
                <arg-actions/>
            </do-if>
        </actions>

Thats all there is to it!  The XPATH was easily used to run off and grab stuff out of eDirectory that was not previously available to me.  I can pick it up and use other dirxml logic to process through what I have very easily.

Thursday, March 11, 2010

Novell IDM XSL - Change Attribute to Proper Case

Today I was working on a Novell IDM project and I needed to use some XSL to call an external Java to format some text.  So, when I created my policy, I created an XSLT policy instead of a standard DirXML policy.  My input values looked something like the following:

        <modify-attr attr-name="Given Name">
             <remove-all-values/>
        </modify-attr>
        <modify-attr attr-name="Given Name">
            <value>
                ROBERT
            </value>
        </modify-attr>
        <modify-attr attr-name="Surname">
             <remove-all-values/>
        </modify-attr>
        <modify-attr attr-name="Surname">
            <value>
                IVEY
            </value>
        </modify-attr>
        <modify-attr attr-name="Full Name">
             <remove-all-values/>
        </modify-attr>
        <modify-attr attr-name="Full Name">
            <value>
                ROBERT IVEY
            </value>
        </modify-attr>


 So, here is what the meat of my XSL policy looked like to transform it.  First, we find a match on the tags we want and store the attribute name and its value in some local variables:

    <xsl:template match="add-attr[@attr-name='Given Name']     |add-attr[@attr-name='Surname']">
        <xsl:variable name="attrName" select="./@attr-name"/>
        <xsl:variable name="newVal" select="./value/text()"/>

The next thing we do is ensure that there is a value.  This is done because on modify events there are two tags, the <remove-all-values/> one and the one with the new value to add.  We don't want to send a blank value and output two different tags.

         <xsl:choose>
        <xsl:when test="$newVal">

Now that we have matched the tag and ensured there is a value, lets go ahead and write the new version of the xml element and call the template to replace the value.  We can usee and otherwise statement to copy everything that isn't being replace (IE the <remove-all-values/> tags that we matched but didn't rewrite) and close up all of our xsl tags.

        <add-attr attr-name="{$attrName}">
            <value>
                <xsl:call-template name="convertCase">
                    <xsl:with-param name="UCData" select="$newVal"/>
                </xsl:call-template>
            </value>
        </add-attr>
        </xsl:when>
        <xsl:otherwise>
            <xsl:copy-of select="."/>
        </xsl:otherwise>
        </xsl:choose>
    </xsl:template>

Lets have a look at the xsl template "convertCase" thats being called.  Its very simple and calls some java methods that are included using a jar file that we added to our IDM server.

    <xsl:template name="convertCase">
        <xsl:param name="UCData"/>
        <xsl:variable name="LCData" select="util:lowerString($UCData)"/>
        <xsl:variable name="newData" select="util:capitalizeWords($LCData)"/>
        <xsl:value-of select="$newData"/>
    </xsl:template>

The parameter UCData is passed to the method util:lowerString and stored in LCData.  Then, LCData is passed over to util:capitalizeWords and the new value is stored in newData.  Notice how newData is the selected value that we use in the replacement up in the xsl template for the modify.

This template is added to copy through everything we didn't match:

    <xsl:template match="@*|node()">
        <xsl:copy>
            <xsl:apply-templates select="@*|node()"/>
        </xsl:copy>
    </xsl:template>

And our output should be exactly the same, except our snippet was modified to look like this:


        <modify-attr attr-name="Given Name">
             <remove-all-values/>
        </modify-attr>
        <modify-attr attr-name="Given Name">
            <value>
                Robert
            </value>
        </modify-attr>
        <modify-attr attr-name="Surname">
             <remove-all-values/>
        </modify-attr>
        <modify-attr attr-name="Surname">
            <value>
                Ivey
            </value>
        </modify-attr>
        <modify-attr attr-name="Full Name">
             <remove-all-values/>
        </modify-attr>
        <modify-attr attr-name="Full Name">
            <value>
                Robert Ivey
            </value>
        </modify-attr>

Please note, the java classes are custom code delivered by a consultant.  I do not have the source code, nor can I distribute this code without their permission.  A little time with a string tokenizer should help recreate this functionality, but I am by no means a java programmer.  Here is what all of our code looks like when we slap it together:

    <xsl:template match="add-attr[@attr-name='Given Name']     |add-attr[@attr-name='Surname']">
        <xsl:variable name="attrName" select="./@attr-name"/>
        <xsl:variable name="newVal" select="./value/text()"/>
         <xsl:choose>
        <xsl:when test="$newVal">
        <add-attr attr-name="{$attrName}">
            <value>
                <xsl:call-template name="convertCase">
                    <xsl:with-param name="UCData" select="$newVal"/>
                </xsl:call-template>
            </value>
        </add-attr>
        </xsl:when>
        <xsl:otherwise>
            <xsl:copy-of select="."/>
        </xsl:otherwise>
        </xsl:choose>
    </xsl:template>
    <xsl:template name="convertCase">        <xsl:param name="UCData"/>
        <xsl:variable name="LCData" select="util:lowerString($UCData)"/>
        <xsl:variable name="newData" select="util:capitalizeWords($LCData)"/>
        <xsl:value-of select="$newData"/>
    </xsl:template>
    <xsl:template match="@*|node()">
        <xsl:copy>
            <xsl:apply-templates select="@*|node()"/>
        </xsl:copy>
    </xsl:template>