Wednesday, June 2, 2010

Performance Tuning eDirectory

I have a particular eDirectory server that is one beefy sucker.  This server is running Windows Server 2008 64bit with a whopping 32GB of memory.  It's running 64bit eDirectory and the DIBFiles directory is about 1.2GB in size.  I was looking at this box, which was only using about 500MB of memory.  While the box is not struggling whatsoever, I was thinking, with 32GB of memory, why not start using some of that to jack up the performance as much as possible.

The first thing I did was max out the Java Heap Size for IDM (see yesterdays blog post) to 2GB (maximum for 32bit java).  If, for whatever reason, the box actually uses all of this memory in the heap, I'll still have at least 28GB of memory left over.  Why not just throw the entire directory into cache so that it performs wicked fast.  I searched around on the Google debugger until I found an article by Novell on performance tuning with eDirectory.  In this article I found some information on how to change the cache settings.  There is a file (C:\Novell\NDS\DIBFiles\_ndsdb.ini on windows) that has memory settings.  I made a quick change, then bounced eDirectory, and now my box will use up to 75% available memory for eDirectory cache.

The file before changes looked like this:

preallocatecache=true
cache=200000000

After my changes:

preallocatecache=true
cache=HARD, %:75, MIN:200000000

It took a while for it to use the cache, but after about 24 hours, dhost.exe is using up just over 3GB of memory instead of the 500MB it had accumulated over the period of a couple months.

For reference, the eDirectory performance tuning article was located here:  http://www.novell.com/documentation/edir88/pdfdoc/edir88tuning/edir88tuning.pdf

Tuesday, June 1, 2010

Performance Tuning Novell IDM

Working with IDM in a large environment.  I needed to tune some memory parameters on IDM because I was getting Out of Memory errors on my JDBC drivers.  The errors looked like the following:

DirXML Log Event -------------------
     Driver:   \VAULT\ORG\ESC\DirXML\DRIVERSET\HMS S00177
     Channel:  Publisher
     Status:   Error
     Message:  Code(-9010) An exception occurred: java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2786)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
    at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
    at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:272)
    at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:276)
    at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122)
    at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212)
    at java.io.BufferedWriter.flush(BufferedWriter.java:236)
    at java.io.PrintWriter.flush(PrintWriter.java:276)
    at com.novell.nds.dirxml.driver.jdbc.util.JDBCLib.GetStackTrace(Unknown Source)
    at com.novell.nds.dirxml.driver.jdbc.util.JDBCLib.UnhandledException(Unknown Source)
    at com.novell.nds.dirxml.driver.jdbc.JDBCPublicationShim.start(Unknown Source)
    at com.novell.nds.dirxml.driver.jdbc.JDBCPublicationProxy.start(Unknown Source)
    at com.novell.nds.dirxml.engine.Publisher.run(Publisher.java:420)
    at java.lang.Thread.run(Thread.java:619)

I was able to tweak with these settings straight from Designer, but could also use iManager.  For iManager, you simply go to Identity Manager Overview, then select Driverset properties once you are looking at the driverset.  In the properties, it is under the Identity Management tab, Misc section (see screenshot below).


For designer, you simply go to the properties of the driverset object, select java and set your environment for the heap there (see below).


As you can tell, the values are the same (as they should be).  I set it to 640MB, which is fairly high, but I will be running many JDBC drivers in this driverset on this server, so I will use that memory.  The default value is 64MB, and it is recommended to increase this in increments of 64-128MB at a time.  In order for these settings to take effect, eDirectory must be restarted.  Please note, this setting is on a per server basis and must be tweaked as such.

There is the potential to set this value too high.  One symptom that you will observe if this value is set too high is the dhost.exe process will consume a very small amount of memory relative to the previous it was using and your IDM drivers do not start at all.  This is a hint that you have set the value too high and need to set it lower.

If you are using a tree that only has one server and you hose this up, it is possible that eDirectory will not even start.  This becomes problematic because you need eDirectory started in order to change this setting.  To resolve your problem, you can prevent the DirXML portion of the eDirectory stack from starting by renaming the following file and starting eDirectory.  At this point you should be able to change the cache value and move on:

C:\Novell\NDS\dirxmllib.dll