For those who are always ‘cutting edge’, here is a small hint that some of the new Java methods in the DominoResultsProcessor and Database class are not working on Domino 12.0.1 for Linux.
The largest transport aircraft in the world, the Antonov AN-225 Mriya was presumably irretrievably destroyed at the beginning of the unprovoked, international law violating WAR OF AGGRESSION between Russia and Ukraine.
However, this is only a material asset.
Much worse is the suffering of the civilian population caused by Russian aggression and the inhumane actions of the Russian invading forces in this WAR OF AGGRESSION. At the same time, Russian dictator is holding the peoples of the free world hostage and condemning them to inaction by threatening a nuclear first strike in the event that the free world intervenes militarily in the conflict. And those in Russia who have the power to stop this insane despot are siding with him out of sheer fear instead of following their conscience and acting.
This WAR OF AGGRESSION is not the war of the overwhelming part of the Russian population. It is the war of a ruthless, power-obsessed, cowardly autocrat for whom no lie against the Russian people is too brazen to push through his plans. With all his might.
As much as I would like to see the Russian people revolt. . In a climate of repression, elimination of freedom of expression, prohibition of independent reporting, and the threat of rotting in the gulag for 15 years if you use the word WAR, it is understandable that resistance to the regime in Moscow is difficult or impossible to achieve.
Let us not give up hope that the people of the free world, together with the part of the Russian population that opposes the WAR in Ukraine, will find a way to end the WAR and bring the aggressors to justice.
An airplane can be rebuilt. A destroyed world and the mistrust between peoples caused by the war, on the other hand, is difficult to restore.
I wanted to upgrade my Windows 2016 Domino 12.0.1 HF18 server to Domino 12.0.1 HF24 today to fix an issue with DAOS.
You need to uninstall any existing HF prior to install a new one. When I start the HF installer to revert the HF 180 installation I get an error message even when I start the executable as administrator.
Sysinternals ProcExplorer does not show any running HCL / Notes/ Domino task. I also set the existing Domino service to “Manual” and restarted the VM.
I can reproduce this behaviour on another installation. Not sure, if this issue only occur when you run Domino in a VM.
The only workaround at the moment is to uninstall Domino keeping all data and settings, restart the computer and reinstall Domino. Now you can apply the HF.
Domino 12.0.1 includes an interesting enhancement to the compact task options. Together with the options -c and -X, -RESTARTensures that a compact that terminates because the limit set with -X is exceeded continues at exactly the same point during the next run. This ensures that even large files are compressed, which at the moment would never be fully compressed due to exceeding the time limit.
load compact -c -X 2 -restart db.nsf
compresses the file db.nsf with copy-style compact, but stops the compression after 2 minutes. In the file system you can see a db.restart file.
If the compression is stopped after 2 minutes, then the db.restart file remains. During the next run, more data is copied from db.nsf to db.restart. This is repeated until db.nsf is completely compressed. Then db.nsf is renamed to db.orig, and db.restart is renamed to db.nsf. Finally, db.orig is deleted. This completes the compression.
It is important to know that the –RESTART option can also be used together with the -REPLICA option. But there it has a completely different function.
DBMT uses only the -c option for the compact. If DBMT determines that the database would be added to the dbmt_compact_filter.ind file if it exceeds the timelimit, it passes in the –RESTART option.
Here is some additional information that is not (yet) documented.
Compact –RESTART requires active transaction logging. An enhancement request is pending at HCL, which issues a corresponding message on the Domino Server console if transaction logging is disabled in the advanced properties of an application. The change will probably not come until Domino 12.0.2.
Compact –RESTART currently only works in case the compression is aborted after reaching the time limit. In other cases (server crash, termination of the compact task) the .restart file is deleted. Again, I have submitted an enhancement request to extend the functionality accordingly. Also this change will probably not come until Domino 12.0.2.
HCL Ambassadors nominations are open from 01-OCT to 31-OCT-2021.
HCL Ambassador is a distinction that HCL awards select members of the community that are both experts in their field and are passionate about sharing their HCL knowledge with others. HCL Ambassadors are exactly that, ambassadors. Importantly they are not employees, but their commitment to sharing their expertise has a huge impact on the HCL community. Whether they are blogging, writing books, speaking, running workshops, creating tutorials and classes, offering support in forums, or organizing and contributing to local events – they help make HCL’s mission of making technology play nice, possible.
HCL Ambassadors are eager to bring their technical expertise to new audiences both in person and online around the world.
More information about the program can be found here.
You can nominate yourself or someone else.
If you want to nominate ME, I would be happy. Permission is hereby explicitly granted.
HCL recently released a new version of the C API Toolkit for Notes/Domino 12.0 on Flexnet.
Good news, makefiles are back and included in the samples.
There are changes and additions in the following header files: nsfnote.h, nsfdb.h, dbmisc.h,osfile.h, osenv.h, osmisc.h, nif.h, client.h, idtable.h.
Aside from other new methods, NSFQueryDBExt2 and NSFQueryDBAddArgs are some interesting candidates to play with.
NSFQueryDBExt2 runs Domino Query Language (DQL) queries. Its inputs are a database handle, the query text, length of query text, and a set of flags. The input query must be translated into LMBCS prior to execution.
NSFQueryDBAddArgs is used to build a list of values to input into a Domino Query Language (DQL) query, specifying arguments through QUEP_ARGVAL structure.
QUEP_ARGVAL structure has been added to dbmiscods.h to build the argument value.
typedef struct
{
WORD type; /* Standard NSF types: TYPE_TEXT, TYPE_NUMBER, TYPE_TIME. */
DWORD length; /* For all textual values. */
DWORD ordinal; /* The nth argument in the query (so, the list needn't be in order). */
BOOL bBinaryForm; /* Whether NUMBER and DATETIME values have been created in the Value area. */
char ArgName[16]; /* Argument names can only be 16 bytes long. */
char Value[256]; /* The stringized value. */
} QUEP_ARGVAL
Some new flags for query processing have also been added to dbmisc.h.
QUEP_NO_EXEC //Explain only mode, only plan and return the explain output.
QUEP_DEBUG //Produce debugging output (notes.ini setting is independent of this).
QUEP_VIEWREFRESH //Refresh all views when they are opened (default is NO_UPDATE).
QUEP_EXPLAIN //Governs producing Explain output.
QUEP_PARSEONLY //To check for syntax only - stops short of planning.
QUEP_DSGNCATREFRESH //Before running the query, build/refresh the design catalog, will have no effect if you have less than Designer privileges on the database.
QUEP_DSGNCATREBUILD //Before running the query, rebuild the design catalog, will have no effect if you have less than Designer privileges on the database.
More new methods can be found in Chapter 1-2 in the “HCL C API Notes/Domino 12.0User Guide“
Recently there was a discussion in the German Notes forum about an error when loading a JDBC driver. https://atnotes.de/index.php/topic,63166.0.html To be clear, the problem has so far only occurred with the driver for MySQL. The driver used is the same as the one used for HCL Traveler when Traveler uses a MySQL database as backend datastore in HA mode. I could not reproduce the described error in Traveler, but in a DOTS plugin and in a Java agent. My tests were performed with Domino 12. But I assume that the behavior is also reproducible in V11.0.1.x. Here is a description of my tests and instructions on how to solve the problem. Once again. The problem is only with the JDBC MySQL driver and is not with Domino. Rather, it is a problem in the driver’s code, but it is affecting Domino.
Here is the code I used in a Java agent. I also use the same code in my DOTS plugin.
In the simplest variant, the drivers to be used are simply copied to DominoPrgmDir/ndext and can then be accessed via
Class.forName(MYSQL_DRIVER);
The directory ndext is already in the classpath. But I use the drivers from the HCL Traveler installation. The directory is not in the classpath because Traveler brings its own JVM. So we need to extend the classpath before using the driver. This is done by the updateClasspath() method.
When you run the agent, you will get the following error.
AMgr: Start executing agent 'driver' in 'driver.nsf'
Agent Manager: Agent error: Exception in thread "AgentThread: JavaAgent"
Agent Manager: Agent error: java.lang.ExceptionInInitializerError
Agent Manager: Agent error: at java.lang.J9VMInternals.ensureError(J9VMInternals.java:146)
Agent Manager: Agent error: at java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:135)
Agent Manager: Agent error: at sun.misc.Unsafe.ensureClassInitialized(Native Method)
Agent Manager: Agent error: at java.lang.J9VMInternals.initialize(J9VMInternals.java:87)
Agent Manager: Agent error: at java.lang.Class.forName(Class.java:347)
Agent Manager: Agent error: at com.mysql.cj.jdbc.NonRegisteringDriver.(NonRegisteringDriver.java:98)
Agent Manager: Agent error: at sun.misc.Unsafe.ensureClassInitialized(Native Method)
Agent Manager: Agent error: at java.lang.J9VMInternals.initialize(J9VMInternals.java:87)
Agent Manager: Agent error: at java.lang.Class.forName(Class.java:347)
Agent Manager: Agent error: at JavaAgent.NotesMain(Unknown Source)
Agent Manager: Agent error: at lotus.domino.AgentBase.runNotes(Unknown Source)
Agent Manager: Agent error: at lotus.domino.NotesThread.run(Unknown Source)
Agent Manager: Agent error: Caused by:
Agent Manager: Agent error: java.security.AccessControlException: Access denied ("java.lang.RuntimePermission" "setContextClassLoader")
Agent Manager: Agent error: at java.security.AccessController.throwACE(AccessController.java:176)
Agent Manager: Agent error: at java.security.AccessController.checkPermissionHelper(AccessController.java:238)
Agent Manager: Agent error: at java.security.AccessController.checkPermission(AccessController.java:385)
Agent Manager: Agent error: at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
Agent Manager: Agent error: at lotus.notes.AgentSecurityManager.checkPermission(Unknown Source)
Agent Manager: Agent error: at java.lang.Thread.setContextClassLoader(Thread.java:840)
Agent Manager: Agent error: at com.mysql.cj.jdbc.AbandonedConnectionCleanupThread.lambda$static$0(AbandonedConnectionCleanupThread.java:77)
Agent Manager: Agent error: at com.mysql.cj.jdbc.AbandonedConnectionCleanupThread$$Lambda$8/0x0000000000000000.newThread(Unknown Source)
Agent Manager: Agent error: at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619)
Agent Manager: Agent error: at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932)
Agent Manager: Agent error: at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367)
Agent Manager: Agent error: at java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:668)
Agent Manager: Agent error: at com.mysql.cj.jdbc.AbandonedConnectionCleanupThread.(AbandonedConnectionCleanupThread.java:80)
Agent Manager: Agent error: … 10 more
AMgr: Agent 'driver' in 'driver.nsf' completed execution
A little causal research, and the reason was found. It is an error in the code of the driver.Here is the source https://bugs.mysql.com/bug.php?id=88172. To solve the problem I have added the following lines to the java.policy in DominoPrgmDir/jvm/lib/security
grant codeBase "file:${notes.binary}/Traveler/lib/-" {
permission java.lang.RuntimePermission "setContextClassLoader";
};
This fixed the problem in the DOTS plugin, but the error in the Java agent persisted. So i moved the “permission” to the
// default permissions granted to all domains
grant {
section at the top of the file.
This now solved the issue for DOTS and the Java agent. To be more precise. This solved it for the Java agent only, if you have the driver copied to the ndext directory. If you use the updateClasspath() method, you must add additional permissions to make your code work.
Modifying the java.policy file directly is generally not a good idea. The customizations may be lost during an upgrade. HCL Domino uses the parameter javaOptionsFile= to make adjustments to the JVM. Using this file we can now include our own custom.policy file. The content of the file is added to the DEFAULT java.policy at runtime. My custom.policy file has the following content
Save the file and add the following line to the server notes.ini
JavaOptionsFile=<DominoDataDir>/javaoptions.opt
Replace <DominoDataDir> according to your environment. After restarting Domino, you should now be able to load the driver without issues.
But wait. Here is one more thing. When you run the agent to load the driver, it will be loaded but you will see the following on the console.
te amgr run "driver.nsf" 'driver'
JVM: Java Virtual Machine initialized.
AMgr: Start executing agent 'driver' in 'driver.nsf'
Agent Manager: Agent printing: mysql-connector-java-8.0.22.jar
Agent Manager: Agent printing: Driver loaded.
Agent Manager: Agent error: Error cleaning up agent threads
Agent Manager: Agent error: Exception in thread "mysql-cj-abandoned-connection-cleanup"
Agent Manager: Agent error: java.lang.IllegalMonitorStateException
Agent Manager: Agent error: at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:151)
Agent Manager: Agent error: at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1261)
Agent Manager: Agent error: at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
Agent Manager: Agent error: at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:449)
Agent Manager: Agent error: at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
Agent Manager: Agent error: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
Agent Manager: Agent error: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Agent Manager: Agent error: at java.lang.Thread.run(Thread.java:823)
AMgr: Agent 'driver' in 'driver.nsf' completed execution
The errors only occur when the JVM is not yet initialized. Any subsequent agent run does not show the output. Also, running the code in a DOTS plugin does not show any errors.
lo dots
JVM: Java Virtual Machine initialized.
Domino OSGi Tasklet Container started ( profile DOTS )
te dots run de.eknori.driver.main com.mysql.cj.jdbc.Driver C:\\Domino\\Traveler\\lib\\mysql-connector-java-8.0.22.jar
[DOTS] (de.eknori.driver.main) [driver]: Main started.
[DOTS] (de.eknori.driver.main) mysql-connector-java-8.0.22.jar
[DOTS] (de.eknori.driver.main) [driver]: Main disposed.
te amgr run "driver.nsf" 'driver'
AMgr: Start executing agent 'driver' in 'driver.nsf'
Agent Manager: Agent printing: mysql-connector-java-8.0.22.jar
Agent Manager: Agent printing: Driver loaded.
AMgr: Agent 'driver' in 'driver.nsf' completed execution
So far I can’t see why the error doesn’t occur when loading the JDBC MySQL driver in HCL Traveler. HCL uses the exact same code in Notes.jar.
I can only assume that the required adjustments to the java.policy are already in place when initializing the HCL Traveler JVM. I have not yet run my tests in a V11 environment. That is still pending.
Maybe someone is fancy to do this in a V11 environment and give feedback.
When working with Domino Query Language you might need one ore more views in your application. Creating a view is not rocket science at all, but views for DQL have specific requirements. https://help.hcltechsw.com/dom_designer/12.0.0/basic/dql_view_column.html Chances are that you do not recall all of them and you’ll end up with a view that would not work.
Aside from the parameters for viewname and columns/fields, the method also has two optional parameters. Both parameters can be omitted. I even see this as best practice. While one can discuss whether the index for the view should be built immediately or later ( parameter: nobuild ), I don’t see why a view that is only used programmatically should be visible, and thus can be seen by all users of the database. (parameter: visible)
Another problem is the naming of the views. These should be different from the other views to avoid conflicts and to emphasize that the views were created specifically for use with DQL. Currently there is no way to determine this by the properties of the respective design element.
A namespace becomes handy when you have multiple NotesDominoQuery objects in your code and each of those objects use their own views/indexes.
Surely you can manually add a prefix to the viewname. But I think this is error-prone in complex applications. It is much easier to use a namespace, which is automatically passed to the corresponding methods as a property of the respective NotesDominoQuery object. I have created a small LotusScript wrapper class to illustrate the principle.
%REM
Class cNotesDominoQuery
%END REM
Public Class cNotesDominoQuery
m_session As NotesSession
m_db As NotesDatabase
m_dbFilePath As String
m_NotesDominoQuery As NotesDominoQuery
m_namespace As String
m_nobuild As Boolean
%REM
Sub New
%END REM
Public Sub New(dbFilePath As String)
Set m_session = New NotesSession
Set me.m_db = me.m_session.Getdatabase("",dbFilePath, False)
If me.m_db.Isopen Then
Set me.m_NotesDominoQuery = me.m_db.CreateDominoQuery()
me.m_NotesDominoQuery.RefreshDesignCatalog = True
me.m_nobuild = false
End If
End Sub
%REM
Sub createIndex
%END REM
Public Sub createIndex (idxName As String, idxFields As Variant)
Call me.m_NotesDominoQuery.Createindex(me.m_namespace & idxName, _
idxFields,, me.m_nobuild)
End Sub
%REM
Sub updateIndex
%END REM
Public Sub updateIndex (idxName As String, idxFields As Variant)
On Error Resume Next
Call me.removeindex(idxName)
Call me.createIndex(idxName, idxFields)
End Sub
%REM
Sub removeIndex
%END REM
Public Sub removeIndex(idxName As String )
Call me.m_NotesDominoQuery.Removeindex(me.m_namespace & idxName)
End Sub
%REM
Property Set namespace
%END REM
Public Property Set namespace As String
me.m_namespace = namespace
End Property
%REM
Property Set nobuild
%END REM
Public Property Set nobuild As boolean
me.m_nobuild = nobuild
End Property
%REM
Property Get object
%END REM
Public Property Get object As NotesDominoQuery
Set object = me.m_NotesDominoQuery
End Property
End Class
In the example 2 NotesDominoQuery objects are created and assigned to the namespace ns1. and ns2.
Afterwards a view/index can be created for each object, which have the same names. The createIndex() method of the class creates the views considering the assigned namespace. Name conflicts between the two objects are thus avoided as well as conflicts with existing views.
The removeIndex() method works in the same way and deletes only the views/indexes of the respective namespace.
The updateIndex() method also takes the namespace into account.
Dim cNotesDominoQuery As New cNotesDominoQuery("dql.nsf")
cNotesDominoQuery.namespace = "ns1."
Call cNotesDominoQuery.removeIndex("bySubject")
Call cNotesDominoQuery.createIndex("bySubject", "subject")
Dim cNotesDominoQuery2 As New cNotesDominoQuery("dql.nsf")
cNotesDominoQuery2.namespace = "ns2."
Call cNotesDominoQuery2.removeIndex("bySubject")
Call cNotesDominoQuery2.createIndex("bySubject", "subject")
Call cNotesDominoQuery2.updateIndex("bySubject", "quantity")
Run the above code in an agent. On the console, you will see an output similar to this.
dql.nsf harvested, 0 catalog documents removed, 2 view designs or aliases cataloged out of 2 total with 1 field-usable columns 33.795 msecs and LastModified of 21.05.2021 07:52:18
DELETE INDEX operation of Index ns1.bySubject on dql.nsf SUCCEEDED
Index (ns1.bySubject) (using hidden view) on dql.nsf successfully populated, and cataloged for field subject - index will be usable for all DQL terms and sorting using the field name subject
dql.nsf harvested, 0 catalog documents removed, 3 view designs or aliases cataloged out of 3 total with 1 field-usable columns 34.354 msecs and LastModified of 21.05.2021 07:52:18
CREATE INDEX operation of Index ns1.bySubject on dql.nsf SUCCEEDED
dql.nsf harvested, 0 catalog documents removed, 2 view designs or aliases cataloged out of 2 total with 1 field-usable columns 33.279 msecs and LastModified of 21.05.2021 07:52:18
DELETE INDEX operation of Index ns2.bySubject on dql.nsf SUCCEEDED
Index (ns2.bySubject) (using hidden view) on dql.nsf successfully populated, and cataloged for field subject - index will be usable for all DQL terms and sorting using the field name subject
dql.nsf harvested, 0 catalog documents removed, 3 view designs or aliases cataloged out of 3 total with 1 field-usable columns 33.625 msecs and LastModified of 21.05.2021 07:52:18
CREATE INDEX operation of Index ns2.bySubject on dql.nsf SUCCEEDED
dql.nsf harvested, 0 catalog documents removed, 2 view designs or aliases cataloged out of 2 total with 1 field-usable columns 34.456 msecs and LastModified of 21.05.2021 07:52:18
DELETE INDEX operation of Index ns2.bySubject on dql.nsf SUCCEEDED
Index (ns2.bySubject) (using hidden view) on dql.nsf successfully populated, and cataloged for field quantity - index will be usable for all DQL terms and sorting using the field name quantity
dql.nsf harvested, 0 catalog documents removed, 3 view designs or aliases cataloged out of 3 total with 2 field-usable columns 35.770 msecs and LastModified of 21.05.2021 07:52:18
CREATE INDEX operation of Index ns2.bySubject on dql.nsf SUCCEEDED
A couple of days ago, I wrote about how you can determine which database on a server has LargeSummary enabled using the show dir command from the Domino console.
You can use LS2CAPI to dertermine, if the flag is set. Here is the code
Public Type DBOPTIONS
options (3) As Long
End Type
Public Const W32_LIB = {nnotes.dll}
Public Const TUX_LIB = {libnotes.so}
Declare Function W32_NSFDbGetOptionsExt Lib W32_LIB Alias {NSFDbGetOptionsExt}_
(ByVal hdb As Long, retDbOptions As DBOPTIONS) As Integer
Declare Function W32_NSFDbOpen Lib W32_LIB Alias {NSFDbOpen}_
(ByVal dbName As String, hDb As Long) As Integer
Declare Function W32_NSFDbClose Lib W32_LIB Alias {NSFDbClose}_
(ByVal hDb As Long) As Integer
Declare Function TUX_NSFDbGetOptionsExt Lib TUX_LIB Alias {NSFDbGetOptionsExt}_
(ByVal hdb As Long, retDbOptions As DBOPTIONS) As Integer
Declare Function TUX_NSFDbOpen Lib TUX_LIB Alias {NSFDbOpen}_
(ByVal dbName As String, hDb As Long) As Integer
Declare Function TUX_NSFDbClose Lib TUX_LIB Alias {NSFDbClose}_
(ByVal hDb As Long) As Integer
Public Function NSFDbGetOptionsExt (hDb As Long, retDbOptions As DBOPTIONS)
If isDefined("WINDOWS") Then
NSFDbGetOptionsExt = W32_NSFDbGetOptionsExt(hDb, retDbOptions)
Else
NSFDbGetOptionsExt = TUX_NSFDbGetOptionsExt(hDb, retDbOptions)
End If
End Function
Function NSFDbOpen( db As string, hDB As Long) As Integer
If isDefined("WINDOWS") Then
NSFDbOpen = W32_NSFDbOpen(db,hDb)
Else
NSFDbOpen = TUX_NSFDbOpen(db,hDb)
End If
End Function
Function NSFDBClose (hDb As Long)
If isDefined("WINDOWS") Then
NSFDbClose = W32_NSFDbClose(hDb)
Else
NSFDbClose = TUX_NSFDbClose(hDb)
End If
End Function
And here is how you can use the code
Const DBOPTBITS_3 = 3
Const DBOPTBIT_LARGE_BUCKETS_ENABLED = &h104
Sub Click(Source As Button)
Dim hDb As Long
Dim rc As Integer
Dim sDb As String
Dim retDbOptions As DBOPTIONS
sDb = "serv04/singultus!!ua.nsf"
rc = NSFDbOpen(sDb, hDb)
If rc = 0 Then
rc = NSFDbGetOptionsExt (hDb, retDbOptions)
If retDbOptions.options(DBOPTBITS_3) _
And DBOPTBIT_LARGE_BUCKETS_ENABLED Then
Msgbox "LargeSummary enabled"
Else
Msgbox "LargeSummary not enabled"
End If
rc = NSFDbClose(hDb)
End If
End Sub
Barely 2 years later, DOTS is back. The first version was available in Domino v 12 Beta3 codedrop. It was not exactly, what I expected, and I had some discussion with Thomas Hampel and HCL Development.
HCL listened and the gave me a preview what they have changed based on the input provided during Beta testing.
In Beta3, DOTS needs an additional notes.ini parameter to locate the launcher.jar and to start the JVM. This requirement has been removed in the final version.
The ndots.exe delivered with Domino 9.x had a fixed value for the max. memory allocation pool size of only 64M. If you build complex DOTS applications that is way to small. DOTS v 12 now launches a JVM with Xmx = 1024M and Xms = 64M. There are 2 new notes.ini parameters (DOTSJavaMaxHeapSize & DOTSJavaMinHeapSize) to give you full control over this settings
DOTS v 12 supports JavaUserOptionsFile for any non DEFAULT JVM parameters.
A very unpleasant behavior of the installer was that it completely deleted the existing DOTS installation under <DominoPrgmDir>/osgi-dots when upgrading Domino to v 12. Thus all custom plugins were gone. This is handled now and the upgrade will not remove existing plugins in <DominoPrgmDir>/osgi-dots/shared/eclipe/plugins.
For developers it is important to know that plugins developed for the existing DOTS version can no longer be executed under Domino v 12. For all tasklets created using previous DOTS versions, you will need to update the package names and recompile all plugins created with a version of DOTS binaries integrated with Domino 12. Preferably using Eclipse SDK 4.6.2.
During my tests with HCL Domino 12 Beta2 I ran into a couple of issues where the Active Directory Password Sync did not work as expected. The problems are not caused by the beta code. However, I am writing down my observations here, since the errors may also appear later in a productive environment. Maybe my lines will save one or the other a long search.
Request Creator replicates with multiple servers
In the console log of a request creator you see replcation events with multiple servers.
There are 2 databases that are replicated on the request creator, Domino Directory (names.nsf) and Directory Assistance (da.nsf). Both databases are replicated with their admin servers to start every config update cycle. Important to know is that the admin server for each database is determined seperately separately. In my test environment, the admin server for da.nsf was different than the one for names.nsf resulting in multiple replication events.
Wrong configuration document
If you have more than 1 Domain Controller that syncs users passwords with your Domino environment, you must install and configure a request creator on each Domain Controller. In addition, you need a configuration document for each of your request creators. Each request creator should have the same configuration settings. Best practice would therefore be a single configuration document that applies to a group of servers.
During my tests I could observe that instead of the assigned configuration document the default configuration document was used.
There was no Statistics output in the console log and I could also not see any DEBUG log output. Also the configuration was not refreshed in the configured interval. I discussed this with HCL Development and it turnd out that the configuration document was not used due to a problem with the view index in names.nsf on the request creator.
I needed to stop the sync process on the request creator, open names.nsf in the Notes Client. After restarting the AD Domain Controller machine, the correct configuration document was used.
Most likely, this issue occurs when you use a configuration document for a single request creator and later switch to a group based configuration document. I would suggest to use a group based configuration document from the start on.
Is Active Directory Password Sync cluster ready?
You can designate as many request processors as you want, and it’s a good way to provide failover.
The decision as to where the AD Domain Controller sends a request is admittedly not overly sophisticated. It works sequentially through an in-memory list it holds of all servers which are designated as request processors, until it finds an available one. Have a look at the ($PWSProcessors) view in the directory. (Mark De Lellis, HCL Development)
This is where the list is built from, using all servers with Role “2” (Request Processor), in the order they appear in the view.
The notes.ini parameter was added in 11.0.1FP2 (SPR# BSPRBSSPWH) to suppress output of Replication Currency alerts for specific databases that were intentionally marked as “replication disabled”.
I used it in my environment for all databases in DominoDataDir/traveler/map. After upgrading to FP3 a couple of days ago, this parameter seemed not to work any longer. On the console I saw the following output
[2D58:0005-2910] ReplCurrency Source="traveler\map\custom\MapDir.nsf", Dest=CN=serv02/O=singultus (Q:0) traveler\map\custom\MapDir.nsf, UpdateToRepl2="03.04.2021 01:01:06", LastRepl="02.04.2020 04:01:25", TimeDiff= 16167 min, TimeDiff2=543026 min Warning="5000+ mins on replication."
[2D58:0005-2910] ReplCurrency Source="traveler\map\iNotesPMap.nsf", Dest=CN=serv02/O=singultus (Q:1) traveler\map\iNotesPMap.nsf, UpdateToRepl2="10.04.2021 01:00:27", LastRepl="01.04.2020 15:26:41", TimeDiff= 6087 min, TimeDiff2=543781 min Warning="5000+ mins on replication."
[2D58:0005-2910] ReplCurrency Source="traveler\map\JournalEntryMap.nsf", Dest=CN=serv02/O=singultus (Q:3) traveler\map\JournalEntryMap.nsf, UpdateToRepl2="10.04.2021 01:00:38", LastRepl="01.04.2020 15:26:41", TimeDiff= 6087 min, TimeDiff2=543781 min Warning="5000+ mins on replication."
[2D58:0005-2910] ReplCurrency Source="traveler\map\MemoMap.nsf", Dest=CN=serv02/O=singultus (Q:3) traveler\map\MemoMap.nsf, UpdateToRepl2="10.04.2021 01:00:45", LastRepl="01.04.2020 15:26:41", TimeDiff= 6087 min, TimeDiff2=543781 min Warning="5000+ mins on replication."
[2D58:0005-2910] ReplCurrency Source="traveler\map\PersonMap.nsf", Dest=CN=serv02/O=singultus (Q:1) traveler\map\PersonMap.nsf, UpdateToRepl2="10.04.2021 01:00:51", LastRepl="01.04.2020 15:26:41", TimeDiff= 6087 min, TimeDiff2=543781 min Warning="5000+ mins on replication."
[2D58:0005-2910] ReplCurrency Source="traveler\map\R6JournalEntryMap.nsf", Dest=CN=serv02/O=singultus (Q:2) traveler\map\R6JournalEntryMap.nsf, UpdateToRepl2="03.04.2021 01:01:20", LastRepl="02.04.2020 04:01:51", TimeDiff= 16166 min, TimeDiff2=543026 min Warning="5000+ mins on replication."
[2D58:0005-2910] ReplCurrency Source="traveler\map\R6PersonMap.nsf", Dest=CN=serv02/O=singultus (Q:1) traveler\map\R6PersonMap.nsf, UpdateToRepl2="10.04.2021 01:00:57", LastRepl="01.04.2020 15:26:42", TimeDiff= 6087 min, TimeDiff2=543781 min Warning="5000+ mins on replication."
I opened a support case with HCL. Here is the answer
“In FP2 there was a bug where it excluded all except the first entry in the INI. The notes.ini does not allow directories to be listed, only databases. In your case, it appeared to work in FP2 because it blocked all databases except for “traveler/map” which is not a database. So, it blocked all databases from currency checks. And, this is now fixed in 1101FP3 (SPR # BSPRBY9RPA).“
The upcoming FP4 and V12 will not list replication disabled databases in the currency alerts (so no notes.ini is required to exclude databases disabled for replication). The notes.ini is still in FP4 and V12, but the need to exclude replication disabled databases is no longer necessary.
So, here, you need to add individual NSF names as comma separated list of dbs in the INI entry to exclude them from the replication currency check.
Active Directory Password Sync applies the Windows passwords of users registered in an Active Directory domain to their Domino HTTP and/or Notes ID passwords.
Password synchronization is supported for
Registered HCL Notes, HCL Nomad, HCL Verse, and HCL iNotes users accessing Domino servers with HTTP passwords or Notes IDs.
HCL Traveler users accessing their mail through the web browser on their mobile devices.
Web users who are not registered in Domino but who have Person documents in the Domino directory accessing Domino web applications with HTTP passwords.
To setup and configure Active Directory Password Sync, you need at least 2 Domino v12 servers. One server acts as Request Creator and the other one has the role Request Processor.
On the “Active Directory Password Sync” tab of the configuration document for each of the servers you can configure which password changes should be processed. The refresh interval specifes the amount of times in minutes after a request is created to allow the request to be processed.
To sync passwords to Notes IDs, the Request Processor servers require password reset authority to the ID vault. I you miss this configuration step, you’ll see an error message on the console of the Request Processor:
[1DFC:0059-1694] 07.04.2021 10:13:23,00 AD Password Sync> PWSyncProcessStoredRequests: Failed to update password in Notes ID for CN=Herbert Feuerstein/O=singultus: Agent containing ResetUserPassword method must be signed by a designated Password Resetter.
[1DFC:0059-1694] 07.04.2021 10:13:23 Password Sync: Active Directory Password Sync failed for objectGUID 6630b191119c8b45b78b77865a37cc70: Agent containing ResetUserPassword method must be signed by a designated Password Resetter.
Request Creator must be installed on the AD Domain Controller. The install type is “Utility Server”. The server must be configured but will never run as a service.
After you have successfully configured the Request Creator, The installer will remove some of the Domino server executables
[0C14:0002-1284] Deleted Executable File 'C:\Domino\nserver.exe' for security reasons.
[0C14:0002-1284] Deleted Executable File 'C:\Domino\nhttp.exe' for security reasons.
[0C14:0002-1284] Deleted Executable File 'C:\Domino\nldap.exe' for security reasons.
[0C14:0002-1284] Deleted Executable File 'C:\Domino\nsmtp.exe' for security reasons.
[0C14:0002-1284] Deleted Executable File 'C:\Domino\npop3.exe' for security reasons.
[0C14:0002-1284] Deleted Executable File 'C:\Domino\nimap.exe' for security reasons.
The installer finally adds the Domino password library “npwsync.dll” to the Windows registry. The entry can be found here
For security reason, the server id that is used for the Request Creator is encrypted during the installation and configuration process. Make sure that the id does not have a password on it. The secured id cannot be used on any other machine than the Request Creator.
A new application is created on the Request Processor. The application stores the requests that are created by the Request Creator. After a request has been successfully processed, the document is removed from the database.
Most of the items in the document are encrypted.
After you have setup and configured Request Creator and Request Processor, you should restart both machines (OS level).
When a user now updates his / her password, LSA processed the change thru the Domino password library, and a new document is created in the Active Directory Password Sync application. The Request Processor reads new documents from the application and processes them according to its configuration.
Here is an example of what you see on the Request Processor console
Many of us Notes developers know the problem. You have developed a Java agent that contains methods from imported .jar files in addition to the actual code. When the agent is executed on a HCL Domino server, it works for a while ( sometimes longer, sometimes shorter ), but then causes problems due to memory leaks. The problem lies in the architecture of Domino. The imported .jar files are reloaded every time the agent is started and over time they use more and more RAM.
The workaround was to dump the .jar files to the server’s file system. This works, but always causes problems where you as a developer do not always have access to the server. Or there is already a .jar in the file directory; but not in the required version. Copying your own .jar into the file system can then lead to undesirable side effects.
I had therefore created a case at HCL to get a fix for the problem. The problem was tracked under SPR # BHUY8PRMKK. Yesterday I received an update from support:
“The Product Development Team had developed a possible fix that mitigates the issue but the fix was of medium risk and it was decided not to submit the fix. As an alternative they have come up with a workaround. Set the following notes.ini, this will cache the agentloader objects and reuse them.“
EnableJavaAgentCache=2
I have set the parameter on my test system. At the moment it looks like it solves the problem. I will continue testing.
Many thanks to HCL Development & HCL Support ( especially to Abhaysingh Shirke ).
Although the existing programming covers a large part of users’ requirements, it would sometimes be desirable to be able to create more flexible rules.
If you look at how the rule created by the user is implemented internally, you will see that the rule is converted into an @formula.
So it is obvious to have a functionality that allows the direct input of a formula.
I have already posted ideas on the HCL portal (https://domino-ideas.hcltechsw.com/); these are also in “Likely to Implement” status.
Version 12 of Notes / Domino is currently in the beta phase. So I looked at programming in the Domino Directory template (pubnames.ntf). Unfortunately, at the moment it looks like the functionality is not yet implemented in V12.
If and when an implementation will take place; I have no information on that. Is that bad? No, I don’t think so.
The big advantage of Notes / Domino is that we can extend the functionality ourselves in many areas. My extension of the rules from 2008 ran without change from version 8.5 to version 11.0.1FP2. I have looked at the current code of Beta2. This has changed only insignificantly. Therefore I have transferred my extensions into this code.
The changes concern 2 design elements.
Both design elements have no dependencies to other design elements. Furthermore my function extensions are comprehensible for every average developer.
The “(RulesDlg)” form has been extended by a section that allows entering an @formula
The “condition” item was extended by the selection “@Formula“
All changes in LotusScript library “Rules” are commented ‘Ulrich Krause, @Formula support. No change was made to the basic functionality.
I came across a little something yesterday related to DirSync.
If you use a value in the notes.ini variable DIRSYNC_DEFAULT_ARGS that is not in the list of allowed values ( load dirsync -? ), then DirSync does not start anymore. More precisely, Dirsync starts, and quits immediately.
No big deal, but for an admin it is certainly helpful if DirSync prints a message on the Domino console indicating that an incorrect value is used in the variable.
There is a SPR #SPPPBYJLZZ for this, but it doesn’t really have a high priority. Rather a nice to have.
HCL recently has released a new version of the C API Toolkit for Notes/Domino 11.0.1. I had already written about it here.
Thomas Hampel reported that there will also be another release of the toolkit after the release of version 12 of Notes / Domino. In this version the makefiles will be included again, which are missing in version 11.0.1.
Basically, there is nothing against using the makefiles from version 9.0.1. However, I think the makefiles are not very user-friendly, especially for beginners. In addition, a separate makefile with the platform-specific characteristics must be created for each platform.
There are a number of tools that do the manual assembly for you ( cmake, boost-build, automake etc ), but the tools themselves must be installed on each platform, and require a more or less extensive training in the handling.
I have worked with boost-build myself for a long time. To make it easier for beginners (and old hands) to work with makefiles, I started a small project over the turn of the year 2020 / 2021.
The goal is to create makefiles in a standardized way, and to be able to use one and the same makefile for the build of a project on different platforms. If possible, no additional software should be necessary.
The result is “OneMK“. The project consists of several files, which contain definitions for compiler and linker settings, and limit the configuration of a makefile to the absolutely necessary things.
The project can be installed within an existing folder structure in the “makefiles” directory. Here is a screenshot of how I installed it.
Additionally an environment variable GIT_REPOSITORY is expected, which points to the root directory. (git)
If you are working on Linux, then you must also create a corresponding directory structure there and export the environment variable GIT_REPOSITORY accordingly.
The notesapi directory can contain one or more versions of the SDK.
There are several files in the makefiles directory.
common.notesapi.VERSION.mk contains the compiler and linker parameters necessary for the respective SDK version.
common.recipes.mk contains the “recipes” for the build of a project.
There are recipes for
clean – cleans the project directory and deletes .o and .pdb files of a previous build.
info – gives information about the source files and object files
make – the actual build process
install/ uninstall – installs or removes the generated binary file in a target directory to be specified
After “installing” “OneMK“, we can now start creating makefiles for our projects.
Simple example
Here is an example of what the makefile for an executable (application) looks like.
#### ######################################################### ####
# MAKEFILE FOR C/C++ PROJECT
# Author: Ulrich Krause
# Date: 2021/02/10
# sample makefile to build /notesapi/11.0.1/samples/basic/intro
#### ######################################################### ####
BASE_DIR = $(GIT_REPOSITORY)/
ifeq ($(BITNESS),)
BITNESS = 32
endif
TYPE = application
TARGET = intro
SUFFIX = c
#------------------------------------------------------------------
include $(BASE_DIR)makefiles/common.notesapi.11.0.1.mk
#------------------------------------------------------------------
SRCDIR =
INCLUDES =
#------------------------------------------------------------------
# Create list of sourcefiles and calculate obj files accordingly
#
SRCS := $(wildcard $(strip $(SRCDIR)*.$(SUFFIX)))
OBJS := $(SRCS:%.$(SUFFIX)=%.$(OBJ))
#------------------------------------------------------------------
# Recipes (included from common.recipes.mk)
#
include $(BASE_DIR)makefiles/common.recipes.mk
#------------------------------------------------------------------
Create a new file in the directory /notesapi/1101/samples/basic/intro/make.mk and copy the code into the file. If the environment variable GIT_REPOSITORY is set, and the necessary files are in the previously described location, then you can start the build process with the following command in the directory /notesapi/1101/samples/basic/intro/.
This works on both Windows and Linux with the same makefile. You do not need to create separate makefiles for the different platforms. On Windows, make sure to open the appropriate command prompt for 32 or 64 bit.
make -f make.mk clean make
will start the build process. you should see output similar to this on the console. For 32 Bit ( wich is the standard BITNESS in the makefile )
For 64Bit build, you can either set this as the default in your makefile or use
make -f make.mk clean make BITNESS=64
If you see WARNINGS during the build process, you can either fix them in your source code ( which is the preferred way to handle warnings and errors) or you can simply ignore them.
To suppress the console output, on Windows you can add #pragma statements into the pragma.h file in the makefiles folder. The file is a forced include and will be added to your project includes at build time.
If you want to install the builded binary, use
make -f make.mk clean make install INSTALLDIR=c:/notes
Keep in mind that INSTALLDIR is case-sensitive
You can use the exact same makefile also on Linux.
More complex example
Here is a more complex example for an extension manager ( shared dll / lib )
#### ######################################################### ####
# MAKEFILE FOR C/C++ PROJECT
# Author: Ulrich Krause
# Date: 2021/01/17
#### ######################################################### ####
BASE_DIR := $(GIT_REPOSITORY)/
ifeq ($(BITNESS),)
BITNESS = 64
endif
TYPE = shared
TARGET = secureid
SUFFIX = cpp
#------------------------------------------------------------------
include $(BASE_DIR)makefiles/common.notesapi.11.0.1.mk
#------------------------------------------------------------------
SRCDIR += src/
INCLUDES += inc
INCLUDES += $(BASE_DIR)hashlib/src
INCLUDES += $(BASE_DIR)undocumented
LDLIBS_ADDITIONAL_PATH += $(BASE_DIR)_static_libs
ifeq ($(DETECTED_OS),$(strip Windows))
LDLIBS += undocumented-w64.lib
LDLIBS += Iphlpapi.lib
LDLIBS += libhl++-64.lib
LDLIBS += /NODEFAULTLIB:libcmt
MDF := /DEF:secureid.def
WINVER = /DWINVER=0xA00
else
LDLIBS += -lhl++-64
LDLIBS += -lstdc++fs
endif
LANGUAGE = c++17
#------------------------------------------------------------------
# Create list of sourcefiles and calculate obj files accordingly
#------------------------------------------------------------------
SRCS := $(wildcard $(strip $(SRCDIR)*.$(SUFFIX)))
OBJS := $(SRCS:%.$(SUFFIX)=%.$(OBJ))
#------------------------------------------------------------------
# Recipes (included from common.recipes.mk)
include $(BASE_DIR)makefiles/common.recipes.mk
#------------------------------------------------------------------
If you need to exclude src files because the will not work on the target platform, you can do this
#------------------------------------------------------------------
# Create list of sourcefiles and calculate obj files accordingly
#------------------------------------------------------------------
SRCS := $(wildcard $(strip $(SRCDIR)*.$(SUFFIX)))
ifneq ($(DETECTED_OS),$(strip Windows))
SRCS:=$(subst src/windowsonly.cpp,,${SRCS})
endif
OBJS := $(SRCS:%.$(SUFFIX)=%.$(OBJ))
The windowsonly.cpp will be removed from the list of source files, if the DETECTED_OS is not “Windows”.
This is only a short introduction to OneMK. The content of the files is mostly self-explanatory ( or so I hope ). You should be able to create makefiles for your existing projects and build them successfully.
There is a lot room for improvements and I would be happy to get some feedback if you like the idea. And I also appreciate any help. I have tested OneMK with several of my projects, simple and complex. It should also work for you, but I do not guarantee it.
