ColdFusion 9 init script

Cloud migration is happening at GCA!  Very exciting stuff.  The cloud is great, we will no longer have to manage our own hardware, worry about the power situation, backup battery, snapshots, maintenance crap, or anything else associated with hosting our own websites. 

The fun part of the whole thing is rebuilding the coldfusion stuff up in the cloud.  Not a big deal, been there done that.  The only quirk to the whole thing about building a Coldfusion 9 webserver on Ubuntu 10.04 LTS is that it does not have a valid startup script, so I created my own.  Its nothing fancy, but gets the job done.  Enjoy

#! /bin/sh

case "$1" in
  start)
        echo "Starting ColdFusion9" >&2
        /opt/coldfusion9/bin/coldfusion start
        exit 3
        ;;
  restart)
        echo "Restarting ColdFusion9" >&2
        /opt/coldfusion9/bin/coldfusion restart
        exit 3
        ;;
  stop)
        echo "Stopping ColdFusion9" >&2
        /opt/coldfusion9/bin/coldfusion stop
        exit 3
        ;;
  *)
        echo "Usage: $0 start|stop|restart" >&2
        exit 3
        ;;
esac

:

ECMAScript to run a command on windows

I had the need to try to run a batch script on the command line using Novell IDM.  I was able to develop a quick function in ECMAScript in order to accomplish this task.  Simply call the following function from policy and pass it the full string of what you want run on the command line and it'll return any output it may return.  Very simple, yet infinitely useful.

importPackage(Packages.java.io);
importPackage(Packages.java.util);

function runCommand(commandString)
{
        var runtime = new java.lang.Runtime.getRuntime();
        var process = runtime.exec(commandString);
        var is = process.getInputStream();
        var isr = new Packages.java.io.InputStreamReader(is);
        var br = new Packages.java.io.BufferedReader(isr);
        var line = new java.lang.String();
        var fulltext = new java.lang.String();
       
        while ((line = br.readLine()) != null )
        {
            fulltext = fulltext + line;
        }
       
        return fulltext;
}

Performance Tuning eDirectory

I have a particular eDirectory server that is one beefy sucker.  This server is running Windows Server 2008 64bit with a whopping 32GB of memory.  It's running 64bit eDirectory and the DIBFiles directory is about 1.2GB in size.  I was looking at this box, which was only using about 500MB of memory.  While the box is not struggling whatsoever, I was thinking, with 32GB of memory, why not start using some of that to jack up the performance as much as possible.

The first thing I did was max out the Java Heap Size for IDM (see yesterdays blog post) to 2GB (maximum for 32bit java).  If, for whatever reason, the box actually uses all of this memory in the heap, I'll still have at least 28GB of memory left over.  Why not just throw the entire directory into cache so that it performs wicked fast.  I searched around on the Google debugger until I found an article by Novell on performance tuning with eDirectory.  In this article I found some information on how to change the cache settings.  There is a file (C:\Novell\NDS\DIBFiles\_ndsdb.ini on windows) that has memory settings.  I made a quick change, then bounced eDirectory, and now my box will use up to 75% available memory for eDirectory cache.

The file before changes looked like this:

preallocatecache=true
cache=200000000

After my changes:

preallocatecache=true
cache=HARD, %:75, MIN:200000000

It took a while for it to use the cache, but after about 24 hours, dhost.exe is using up just over 3GB of memory instead of the 500MB it had accumulated over the period of a couple months.

For reference, the eDirectory performance tuning article was located here:  http://www.novell.com/documentation/edir88/pdfdoc/edir88tuning/edir88tuning.pdf

Performance Tuning Novell IDM

Working with IDM in a large environment.  I needed to tune some memory parameters on IDM because I was getting Out of Memory errors on my JDBC drivers.  The errors looked like the following:

DirXML Log Event -------------------
     Driver:   \VAULT\ORG\ESC\DirXML\DRIVERSET\HMS S00177
     Channel:  Publisher
     Status:   Error
     Message:  Code(-9010) An exception occurred: java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2786)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
    at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
    at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:272)
    at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:276)
    at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122)
    at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212)
    at java.io.BufferedWriter.flush(BufferedWriter.java:236)
    at java.io.PrintWriter.flush(PrintWriter.java:276)
    at com.novell.nds.dirxml.driver.jdbc.util.JDBCLib.GetStackTrace(Unknown Source)
    at com.novell.nds.dirxml.driver.jdbc.util.JDBCLib.UnhandledException(Unknown Source)
    at com.novell.nds.dirxml.driver.jdbc.JDBCPublicationShim.start(Unknown Source)
    at com.novell.nds.dirxml.driver.jdbc.JDBCPublicationProxy.start(Unknown Source)
    at com.novell.nds.dirxml.engine.Publisher.run(Publisher.java:420)
    at java.lang.Thread.run(Thread.java:619)

I was able to tweak with these settings straight from Designer, but could also use iManager.  For iManager, you simply go to Identity Manager Overview, then select Driverset properties once you are looking at the driverset.  In the properties, it is under the Identity Management tab, Misc section (see screenshot below).

For designer, you simply go to the properties of the driverset object, select java and set your environment for the heap there (see below).

As you can tell, the values are the same (as they should be).  I set it to 640MB, which is fairly high, but I will be running many JDBC drivers in this driverset on this server, so I will use that memory.  The default value is 64MB, and it is recommended to increase this in increments of 64-128MB at a time.  In order for these settings to take effect, eDirectory must be restarted.  Please note, this setting is on a per server basis and must be tweaked as such.

There is the potential to set this value too high.  One symptom that you will observe if this value is set too high is the dhost.exe process will consume a very small amount of memory relative to the previous it was using and your IDM drivers do not start at all.  This is a hint that you have set the value too high and need to set it lower.

If you are using a tree that only has one server and you hose this up, it is possible that eDirectory will not even start.  This becomes problematic because you need eDirectory started in order to change this setting.  To resolve your problem, you can prevent the DirXML portion of the eDirectory stack from starting by renaming the following file and starting eDirectory.  At this point you should be able to change the cache value and move on:

C:\Novell\NDS\dirxmllib.dll

eDirectory Installation Errors

More troubleshooting fun today.  Large customer, lots of users, lots of drivers, needed more horsepower.  How do we get more horsepower?  That's easy, throw another server in the mix.  Step one of adding a new server to IDM, install eDirectory and join it to the tree.  This is where I got hung up and for some reason it actually took a little while to click in my head.

I was starting the installation, then receiving timeout errors.  Come to find out, I had forgot to turn off the stupid Windows Firewall.  It was blocking the replication and all of the other things I need to work to get the tree working.  My install should work now, but I need to go through and cleanup that old install and get cracking on the new one, there are a few things that I have to do for this to work.

First, I had to do the basic Add/Remove programs thing.  Of course, being Windows, this rarely, if ever, actually removes the entire program.  The next step was to go remove the C:\Novell directory in its entirety.  I had to escalate to admin privileges for this.

At this point, you would think it works, but it does not.  The stupid NDS console thing still exists in the control panel and needs to be removed.  This is the tricky one.  To remove this (and complete the uninstall), you must remove the following file:

C:\Windows\System32\NDScpa.cpl

Now, you can actually start the eDirectory installation again, but you have another problem.  The eDirectory object name for the server already exists.  Due to the installer crashing out early, you must go and manually delete two objects from the directory.  The server object and the ndsPredicateStats object.  Both will be named pretty obviously specific to your server (unless you changed them, in which case, shame on you).

After this, you can continue the installer (this time without the windows firewall!) and everything is happy.

Novell IDM Syntax Violation Errors

I was recently working on a driver.  I had finished the driver and everything was working great.  I then started getting the following errors on my driver:

Code(-9010) An exception occurred: novell.jclient.JCException: createEntry -613 ERR_SYNTAX_VIOLATION

After yanking out some of the hair on my scalp, I took two traces, one of a user that was created successfully, and one of a user that kicked back this error.  I looked at the XML document after all the driver logic was finished just before it tried to create the account.

What I found was there were very few differences (I hope not), but what ended up standing out is that in one of the traces, the user had a blank value for their Title attribute, and looked similar to this:

<add-attr attr-name="Title">
   <value type="string"/>
</add-attr>

Why is this significant?  Because if you look at the schema for eDirectory, the attribute Title is sized, with a minimum length of 1 character, meaning a blank attribute is not valid.

The resolution for this was simple.  I found all attributes that had sizing restrictions on them, then simply did a check and stripped them out if they had a blank value.  Here is what the Title attribute sample looked like:

<rule>
   <description>Strip Title if blank</description>
   <comment xml:space="preserve">If title is a blank value, strip it so it doesn't cause a syntax violation.</comment>
   <conditions>
      <and>
          <if-class-name mode="nocase" op="equal">User</if-class-name>
          <if-op-attr mode="nocase" name="Title" op="equal"/>
       </and>
   </conditions>
   <actions>
       <do-strip-op-attr name="Title"/>
   </actions>
</rule>

Once this was done all of the errors disappeared!

Sharing Policies between Drivers in Novell IDM

Working with Novell IDM to integrate an HMS Payroll System hosted on an AS400 system in a DB2 database.  The customer has approximately 120 of these databases that need to be interfaced.  In order to accomplish this, I need to create 120 separate JDBC drivers.

I was able to architect and implement the drivers in such a way that all of the policies are identical.  The only differences between them are the connection parameters and some GCV's.  With all of the policies identical, I wanted to deploy them in such a way that updates to the drivers are easier than duplicating the modification 120 times.

By creating a new Library container within the driverset, I was able to reference the policies within the drivers from the shared Library.  This allows all of the drivers to subscribe to the same policy and XSL objects.  When an update needs to occur for the 120 drivers, the policy in question is updated in the library, then all of the drivers are restarted. 

To expedite the process of restarting all of the drivers, I simply restart DirXML on the servers hosting the drivers.  This will force a restart of these drivers, so long as they are set to automatic startup.

Please note that the Filter cannot be shared as it must exist on the driver object, so changes to filter will still require changing all 120 drivers individually.