Today, I finally found the time to take a closer look into the Domino JNA project. The project has been created by Karsten Lehmann, mindoo.
The goal of the project is to provide low level API methods that can be used in Java to speed-up the retrival of data; especially, if you have to deal with lots of data, this can be a significant performance impact.
My szenarion is the following:
Read entries from a Notes Application into a Java bean and add data from another document in another application to the bean.The number of documents in the source application can be from 1 – n. I do not “own” the design of the source database, so I cannot modify it.
In this article, I will concentrate on reading the source application in the fastest way possible. I will show, how I do it right now and also, how you can use Domino JNA.
Here is a small piece of Java to measure the duration of data retrival.
public class StopWatch {
private long startTime = 0;
private long stopTime = 0;
private boolean running = false;
public void start() {
this.startTime = System.nanoTime();
this.running = true;
}
public void stop() {
this.stopTime = System.nanoTime();
this.running = false;
}
// elaspsed time in milliseconds
public long getElapsedTime() {
long elapsed;
if (running) {
elapsed = (System.nanoTime() - startTime);
} else {
elapsed = (stopTime - startTime);
}
return elapsed;
}
// elaspsed time in seconds
public long getElapsedTimeSecs() {
long elapsed;
if (running) {
elapsed = ((System.nanoTime() - startTime) / 1000);
} else {
elapsed = ((stopTime - startTime) / 1000);
}
return elapsed;
}
}
I am using “fakenames.nsf” in my sample code. You can download the two sample databases fakenames.nsf and fakenames-views.nsf from this URL:
ftp://domino_jna:domino_jna@www2.mindoo.de
Next, place them in the data folder of your IBM Notes Client.
here is the code, that I use in my application. It uses a NotesNavigator to traverse the view. It then opens the underlying document for each entry found using entry.getDocument() and prints values for some items to the console.
I need to do it this way, because I need the contents of some items to identify the document in the other database. Unfortunately not all needed values are in the view. So just reading the column values is not an option.
import lotus.domino.Database;
import lotus.domino.Document;
import lotus.domino.NotesFactory;
import lotus.domino.NotesThread;
import lotus.domino.Session;
import lotus.domino.View;
import lotus.domino.ViewEntry;
import lotus.domino.ViewNavigator;
public class Domino {
public static void main(String[] args) {
try {
StopWatch stopWatch = new StopWatch();
NotesThread.sinitThread();
Session session = NotesFactory.createSession();
stopWatch.start();
Database dbData = session.getDatabase("", "fakenames.nsf");
View view = dbData.getView("People");
ViewNavigator navUsers = null;
ViewEntry vweUser = null;
ViewEntry vweTemp = null;
Document docUser = null;
view.setAutoUpdate(false);
navUsers = view.createViewNav();
navUsers.setEntryOptions(ViewNavigator.VN_ENTRYOPT_NOCOUNTDATA + ViewNavigator.VN_ENTRYOPT_NOCOLUMNVALUES);
vweUser = navUsers.getFirst();
navUsers.setCacheGuidance(Integer.MAX_VALUE, ViewNavigator.VN_CACHEGUIDANCE_READSELECTIVE);
while (vweUser != null) {
docUser = vweUser.getDocument();
System.out.println(
docUser.getItemValueString("lastname") + ", " + docUser.getItemValueString("firstname"));
vweTemp = navUsers.getNext(vweUser);
docUser.recycle();
vweUser.recycle();
vweUser = vweTemp;
}
stopWatch.stop();
System.out.println(stopWatch.getElapsedTimeSecs());
} catch (Exception e) {
e.printStackTrace();
} finally {
NotesThread.stermThread();
}
}
}
Now to Domino JNA. Here is the code. First, I get all IDs from the documents in the view and then I take the result to get the underlying documents and the data.
import java.util.EnumSet;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.concurrent.Callable;
import com.mindoo.domino.jna.NotesCollection;
import com.mindoo.domino.jna.NotesCollection.EntriesAsListCallback;
import com.mindoo.domino.jna.NotesDatabase;
import com.mindoo.domino.jna.NotesIDTable;
import com.mindoo.domino.jna.NotesNote;
import com.mindoo.domino.jna.NotesViewEntryData;
import com.mindoo.domino.jna.constants.Navigate;
import com.mindoo.domino.jna.constants.OpenNote;
import com.mindoo.domino.jna.constants.ReadMask;
import com.mindoo.domino.jna.gc.NotesGC;
import lotus.domino.NotesException;
import lotus.domino.NotesFactory;
import lotus.domino.NotesThread;
import lotus.domino.Session;
public class DominoApi {
public static void main(String[] args) throws NotesException {
try {
NotesGC.runWithAutoGC(new Callable() {
@Override
public Object call() throws Exception {
StopWatch stopWatch = new StopWatch();
NotesThread.sinitThread();
Session session = NotesFactory.createSession();
stopWatch.start();
NotesDatabase dbData = new NotesDatabase(session, "", "fakenames.nsf");
NotesCollection colFromDbData = dbData.openCollectionByName("People");
boolean includeCategoryIds = false;
LinkedHashSet allIds = colFromDbData.getAllIds(includeCategoryIds);
NotesIDTable selectedList = colFromDbData.getSelectedList();
selectedList.clear();
selectedList.addNotes(allIds);
String startPos = "0";
int entriesToSkip = 1;
int entriesToReturn = Integer.MAX_VALUE;
EnumSet returnNavigator = EnumSet.of(Navigate.NEXT_SELECTED);
int bufferSize = Integer.MAX_VALUE;
EnumSet returnData = EnumSet.of(ReadMask.NOTEID, ReadMask.SUMMARY);
List selectedEntries = colFromDbData.getAllEntries(startPos, entriesToSkip,
returnNavigator, bufferSize, returnData, new EntriesAsListCallback(entriesToReturn));
for (NotesViewEntryData currEntry : selectedEntries) {
NotesNote note = dbData.openNoteById(currEntry.getNoteId(), EnumSet.noneOf(OpenNote.class));
System.out.println(
note.getItemValueString("lastname") + ", " + note.getItemValueString("firstname"));
note.recycle();
}
stopWatch.stop();
System.out.println(stopWatch.getElapsedTimeSecs());
return null;
}
});
} catch (Exception e) {
e.printStackTrace();
} finally {
NotesThread.stermThread();
}
}
}
Now, what do you think? Which code is faster? Domino JNA? Well, not really in my szenario.
I have done a couple of tests on a local machine for both code sample.
The average time ( from 100 runs each ) for my code to get 40.000 documents from the “fakenames.nsf” and to print the values from the firstname and lastName item is 9.70 seconds; the average for Domino JNA is 10.65 seconds.
This does not mean that Domino JNA does not have any advantage over the standard IBM Domino Java API; It depends on the szenario. And in my szenario, there is no advantage in using Domino JNA. It would only result in an advanced complexity and platform dependancies.
If you have read this far, here is an extra for you. I played with the options and found that the
navUsers.setCacheGuidance(Integer.MAX_VALUE, ViewNavigator.VN_CACHEGUIDANCE_READSELECTIVE);
is a significant performance boost.
Without setting the cache guidance, the average time to get the data out of the application was 11.40 seconds. I could not see any difference in using VN_CACHEGUIDANCE_READALL instead of VN_CACHEGUIDANCE_READSELECTIVE.
It is bad practice to use System.out.println in a microbenchmark, because the output to the console is extremly slow. Retry your benchmark witout the System.out in the loop and you will get other results.
ok, good point.
Domino JNA: 8.08 sec
Legacy API: 4.05 sec
Ulrich, there is another problem in your legacy code. You should set CacheGuidance before you access the ViewNavigator with the getFirst. For an explanation of Read_selective you can have a look at Karsten Lehmanns blog post: https://www.mindoo.com/web/blog.nsf/dx/17.01.2013085308KLEB9S.htm
Another one: Why are you using bufferSize=Integer.Max_value. This value is much to high. As far as i now the value buffer size is the number of entries which the viewNav should read in one transaction. Normally i recommend 400 for this value. With Integer.Max_Value you set it to 4 billions.
Your code using Domino JNA can be improved.
This line alone returns all note ids in the view in the view order:
LinkedHashSet allIds = colFromDbData.getAllIds(includeCategoryIds);
There is no need to use the selectedList afterwards. You are basically reading the view twice. The first time to get all note ids, the second time to reread those entries and fill the summary buffer with all column names and values:
EnumSet returnData = EnumSet.of(ReadMask.NOTEID, ReadMask.SUMMARY);
Something you explicitly disabled in your legacy API sample code:
navUsers.setEntryOptions(ViewNavigator.VN_ENTRYOPT_NOCOUNTDATA + ViewNavigator.VN_ENTRYOPT_NOCOLUMNVALUES);
The main trick to speed up view scanning is to decide which data really needs to be written into the summary buffer by Domino.
The summary buffer for a single NIFReadEntries call is always max 64 KB. A note id takes 4 bytes. So with one single call you might read about 16.000 rows (maybe less because of some overhead or delimiters).
When you tell Domino to use READ_MASK_SUMMARY, it writes all column names and values for each view entry into the buffer. Depending on the number of view columns and the column value lengths, this can mean that only 70 rows can be read with one call (I have those kind of view in an absence management application of a customer).
seems to me you need to time it again using the updated info, Ulrich.
btw, I tried helping to port this to iSeries. but I lack development resources.