After 2013 & 2014, I again have the honor to speak at SNouG ( Swiss Notes User Group ) in Zurich on 28-Oct-2015
My session is titled “Honey, I shrunk the data!”. This session has been held a couple of times before at various user groups, but it seems that there still is a strong interest in this topic.
I will not only cover data and design compression, DAOS and some new compact features. The session also includes all things DBMT as well.
See you in Zurich!
MWLUG 2015 in Atlanta was a blast. Thanks to all, who have organized the event. Also a big thank you to all, who attended my session.
Here is the presentation and the sample database & XML data
I recently added a great plugin to my TeamCity configuration that makes the deployment of new releases simple. The plugin adds some new deployers to the TeamCity server.
I am using the FTP Deployer.
The deploy-64 build step opens a connection to my FTP server and uploads files that have been built.
In additional steps, I create the template from the database that sits on my Domino Dev server, stamp it with the release and build number and also copy the license files and release notes into the release directory. The directory is automatically created when the files are deployed.
With this configuration at hand, I am able to create a new build for Windows 32/64 and Linux 32/64, add additional files, automagically increment the build number and zip all bits and pieces into a distro.zip / distro.tgz.
My ONTF project “DomBackUp” can be configured and run via the command-line without having the configuration database installed on the Domino server.
DomBackUp is a fork of nKBackup by Tom Lyne et al. I found the command-line interface code a bit hard to maintain when it comes to enhancements.
So I was looking for an alternative way to create the command-line interface in C++ platform-independent.
boost::program_options is a library that makes it easy to parse command-line options, for example, for console applications.
Here is the code snippet from the DomBackUp project
try {
#include<boost/program_options/options_description.hpp>
#include<boost/program_options/parsers.hpp>
#include<boost/program_options/variables_map.hpp>
...
namespace po = boost::program_options;
...
po::options_description description("DomBackUp usage",PO_LINE_LENGTH);
description.add_options()
("help,h", "Display this help message")
("source,s", po::value( &szSource ), "source file/folder")
("dest,d", po::value( &szDestination ), "destination file/folder")
("include-sub-dirs,inc", po::bool_switch( &bIncludeSubDirs )->default_value(false),
"Include subdirectories, applies to folder backup only (optional, default = false)")
("throttle,t", po::bool_switch( &bThrottle )->default_value(false),
"Wait short time between file writes (optional, default = false)")
("zip,z", po::bool_switch( &bZipAfterBackup )->default_value(false),
"Move file to .zip archive after backup (optional, default = false)")
("unique-filename,u", po::bool_switch( &bUniqueFileName )->default_value(false),
"appends a unix timestamp to the archive filename (optional, default = false)")
("application-type,a", po::value( &szAppType )->default_value("nsf"),
"nsf,ntf,both = application type (optional, default = nsf only)")
("input-file,f", po::value( &szInputFile ),
"Backup all the files specified in an .ind file created in the Domino data folder to <dest>")
("version,v", "Display the version number");
po::variables_map vm;
po::store(po::command_line_parser(argc, argv).options(description).run(), vm);
po::notify(vm);
if(vm.count("help")) {
std::cout << description << "\n";
return FALSE;
}
if(vm.count("version")) {
AddInLogMessageText("V %s on %s", NOERROR, szVersionNumber.c_str(), szServer);
AddInLogMessageText("Termination complete.\n", NOERROR);
return FALSE;
}
}
catch(po::error& e) {
AddInLogMessageText((char*)e.what(), NOERROR);
return (ERR(error));
}
First of all, you have to include some header files and (optional) set the namespace to be used. The only downside of using boost::program_options is the fact that you have to build the static .lib. While most parts of the boost libraries are header only, boost::program_options is not. But once you have the library at hand, it is easy to write the code.
The first line of code creates your interface and sets the Title as well as the line-length. The line-length is 80 by default.
po::options_description description("DomBackUp usage",PO_LINE_LENGTH);
The next thing to do is to add all the options you like to control via the command-line
description.add_options()
("help,h", "Display this help message")
("source,s", po::value( &szSource ), "source file/folder")
Each option can have a description and the command, both verbose (–help) and abbreviated (-h).
In addition, you can bind the command to a variable. The variable can be of any type, string, int or boolean.
For boolean variables, you can implement some kind of toggle mechanism.
description.add_options()
("zip,z", po::bool_switch( &bZipAfterBackup )->default_value(false),
"Move file to .zip archive after backup (optional, default = false)")
In this example, &bZipAfterBackup by default is set to false. To enable the feature, simply add –zip or -z to the command-line. This will toggle from false tu true.
The –help (-h) command does not have any value binding. So we have to add a parser and some ther lines of code that are processed, when the –help (-h) command is being used.
po::variables_map vm;
po::store(po::command_line_parser(argc, argv).options(description).run(), vm);
po::notify(vm);
if(vm.count("help")) {
std::cout << "\n" << description << "\n";
return FALSE;
}
This is what you get when you type “load nbackup -h”
Cool, isn’t it?. Just a couple of lines of code and you get a professional command-line interface.
As said before, the downside is the overhead of having the boost libraries included in your project.
But the footprint is minimal, although the boost libraries are many GB of code.
Building binaries for multiple platforms from source is time consuming. For my ONTF project “DomBackUp” I have to build binaries for AIX (not sure, if I can support AIX in future builds ) , LINUX and Windows, both on 32Bit and 64Bit.
Aside from Atlassian JIRA and STASH, a couple of virtual machines are involved. I also started using TEAMCITY to automatically build the binaries without going to each of the build systems and invoke the build process manually.
TeamCity is a Java-based CI server package. The TeamCity installation and configuration is quick and easy. The fact that it is Java-based should not be an impediment to Windows development shops.
The TeamCity server is a main component, but the browser-hosted interface serves as the primary way to administer TeamCity users, agents, projects, and build configurations.
The main advantages are
You can create the build steps manually or let TeamCity search your GIT repository for existing .sln files and propose build steps automatically. All you have to do is to review the steps and select the correct build configuration ( x64 or Win32 ). For the Linux builds, I use boost build scripts. So you only have to tell TeamCity to invoke the script when the build agent runs.
TeamCity will automatically grab the output from the build script. This makes it very easy to identify errors in the script itself or even compile errors.
Using a CI solution makes it very easy to create nightly builds. Due to the flexibility in configuration, you are able to create the binaries aside from any needed template, have CI update and harmonize the version and build numbers across all parts of your project, create release notes from JIRA and put all parts together to build a shippable package.
The process can be triggered manually or scheduled to create nightly builds.
You are also able to rebuild any previous version by simply checkout the correct commit from your repository.
I recently had to build a static library using CMake. In the current CMake version, apparently something has changed in the linker settings.
The build always ended with an error:
LINK : fatal error LNK1123: failure during conversion to COFF: file invalid
or corrupt [D:\0.GIT\libarchive-3.1.2\vc10.32\CMakeFiles\CMakeTmp\cmTC_bbd0c.vcxproj]
Looking at the linker settings, I found that CMake seems to enable incremental linking by default and this is the root cause for the trouble; at least with Visual Studio 2010
Link:
C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\link.exe /ERRORREPORT:QUEUE
/OUT:”D:\0.GIT\libarchive-3.1.2\vc10.32\CMakeFiles\CMakeTmp\Debug\cmTC_bbd0c.exe”
/INCREMENTAL
/NOLOGO kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib
/MANIFEST /ManifestFile:”cmTC_bbd0c.dir\Debug\cmTC_bbd0c.exe.intermediate.manifest”
/MANIFESTUAC:”level=’asInvoker’ uiAccess=’false'”
/DEBUG /PDB:”D:/0.GIT/libarchive-3.1.2/vc10.32/CMakeFiles/CMakeTmp/Debug/cmTC_bbd0c.pdb”
/SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT
/IMPLIB:”D:/0.GIT/libarchive-3.1.2/vc10.32/CMakeFiles/CMakeTmp/Debug/cmTC_bbd0c.lib”
/MACHINE:X86 cmTC_bbd0c.dir\Debug\cmTC_bbd0c.exe.embed.manifest.res
cmTC_bbd0c.dir\Debug\testCCompiler.obj /machine:X86 /debug
I searched the web for a solution, but could not find anything that really works. So here is what I did to fix the issue.
In in [CMakePrgmDir]\share\cmake-3.3\Modules\Platform\windows.cmake
modify the line 242 ( might be different in other versions )
# add /debug and /INCREMENTAL:YES to DEBUG and RELWITHDEBINFO also add pdbtype
# on versions that support it
set( MSVC_INCREMENTAL_YES_FLAG "")
if(NOT WINDOWS_PHONE AND NOT WINDOWS_STORE)
if(NOT MSVC_INCREMENTAL_DEFAULT)
set( MSVC_INCREMENTAL_YES_FLAG "/INCREMENTAL:YES")
else()
#set( MSVC_INCREMENTAL_YES_FLAG "/INCREMENTAL" ) ' comment out here
set( MSVC_INCREMENTAL_YES_FLAG "/INCREMENTAL:NO" )
endif()
endif()
With this modification in place, I was now able to let CMake create the project configuration files.
A couple of days ago, I had to investigate on an issue in a rather old template. The template had been delivered to a customer ages ago. After downloading the template from the repository ( no, not from a GIT or SVN repository ) and after Domino Designer had opened the template, I saw a lot of errors in the Script Libraries. No worries at this time, because all of those issues can be solved.
I then I opened one of the libs that contains the specific function. The contents of that lib looked different to the curent version; no surprise is that.
Then I searched the template for the function. Nothing. The function was not there, althoug I could see from the code, that it IS in the template.
Calls to that function did not show an error, so it had to be somewhere inside the code. But I could not find it.
I then opened the template in YTRIA ScanEz and navigated to the lib that was supposed to contain the function. Aside from the usual $ScriptLib items, the design also had $ScriptLib_error items.
In the end, I found my function in one of the $ScriptLib items. This template had been build in Designer 7 and the current Domino Designer 9.0.1 seems to have a problem with this mix of $ScriptLib and $ScriptLib_error items when opening such a design element in the editor.
As an aside, I found that the “Recompile All LotusScript” apparently does not have any problems finding the correct code; all code compiled without any complaint.
Next, I deleted the $ScriptLib_error items from the design
I omitted the warning ( don’t do that, if you do not know, what you’re doing !!)
and after another recompile and reopen of the template in Domino Designer , the template was fully functional again.
I am using Teamcity on Ubuntu 14.04. Here is a simple script to start / stop TeamCity server.
#!/bin/sh
# /etc/init.d/teamcity – startup script for teamcity
export TEAMCITY_DATA_PATH=”/home/teamcity/.BuildServer”
export TEAMCITY_SERVER_OPTS=-Djava.awt.headless=true # Configure TeamCity for use on a headless OS.case $1 in
start)
start-stop-daemon –start -c teamcity –exec /home/teamcity/TeamCity/bin/runAll.sh start
;;stop)
start-stop-daemon –start -c teamcity –exec /home/teamcity/TeamCity/bin/runAll.sh stop
;;esac
exit 0
Copy the script (teamcity) to etc/init.d.
Register the file as a startup script
sudo update-rc.d teamcity defaults
Make the file executable
sudo chmod +x /etc/init.d/teamcity
Now you can start teamcity with sudo service teamcity start or stop ist with sudo service teamcity stop
sudo zypper addrepo http://download.opensuse.org/repositories/devel:/tools:/scm/SLE_11_SP3/devel:tools:scm.repo
sudo zypper addrepo http://download.opensuse.org/repositories/devel:/languages:/perl/SLE_11_SP3/devel:languages:perl.repo
sudo zypper install git-core
To start an external program from a C / C++ program, I used the following code.
GetStartupInfo( &start_info ); // You must fill in this structure
ret = CreateProcess( NULL, // specify the executable program
(LPSTR)buffer, // the command line arguments
NULL,
NULL,
FALSE,
NORMAL_PRIORITY_CLASS | CREATE_NO_WINDOW,
NULL,
NULL,
&start_info,
&process_info);
// Successfully created the process. Wait for it to finish.
WaitForSingleObject( process_info.hProcess, INFINITE );
// Get the exit code.
result = GetExitCodeProcess(process_info.hProcess, &exitCode);
// Close the handles.
CloseHandle( process_info.hProcess );
CloseHandle( process_info.hThread );
The code starts an external program and waits, until this program has finished before processing the next lines of code.
After using this code for a couple of years now, I found a shorter version today …
ret = system((char *)buffer);
I found only one limitation. It will print any of your fprint statements to the Domino console; using CREATE_NO_WINDOW with CreateProcess, you can suppress the output.
The presentation and sample can be downloaded here
I recently ran into an issue, where my Java SQL and LDAP connectors stopped working after applying FP3 to Domino 9.0.1 / 64Bit.
Thanks to the help of the community, I was pointed to an already existing fix for this issue. IBM had released an JVM patch . I downloaded the patch and tried to install it by simply double-clicking the downloaded file.
On the machine ( a testing environment ), where I tried to install the patch, also the Notes Client is installed.
This obviously confuses the installer and so it tried to apply the patch to the Notes client instead of the Domino server.
There is no way to change the install path in the GUI. I looked into the install log and found an interesting entry
3 Command Line Args:
0: LAX_VM
1: D:\Notes\jvm\bin\java.exe
2: -D$USER_INSTALL_DIR$=D:\Notes
So, I gave it a try and added the path to my Domino installation as parameters to the .exe file
D:\temp\jvmpatch>9.0.1.3_Server_w64_JVM_Patch_20150303_142312.exe LAX_VM d:\Domino\jvm\bin\java.exe -D$USER_INSTALL_DIR$=D:\Domino
Now the patch was successfully applied to the Domino server JVM …
I have not tried the 32Bit installation. Here the file states to be “ClientServer”. So, maybe, you can make a selection in the 32Bit version and the 64Bit version is simply missing this option.
If the command line is documented somewhere, blame me. If not, maybe this post can help saving some time.
A quick tipp for using the ONTF Dependency Documenter from Andrew Magerman.
If you have a lot of libraries in your database, the resulting image might be too large to fit into the page and the result seems to be truncated.
I have talked too Andrew already, and it seems that there is some kind of limitation on the image size for a page design element.
You can work around this issue by resizing the image in the image properties.
We also discussed some options and I am sure that Andrew will provide an updated version of the plugin soon.
While working with the cAPI, I ran into a nasty problem with signing design elemnts in an application on the server with a different ID than the server ID.
This seems only to be a problem with XPages related design elements.
When you try to open an XPage from that database in the browser, you will get a 403 Error message.
When I sign the same database from the client with the same ID using Ytria SignEZ, everything works as expected.
Other elements are not affected. So forms, views, agents and script libraries are signed correctly. Well, don’t get me wrong, even XPages are signed correctly, but they just do not work any longer after signing.
So, how do I sign a database in the background on the server with a different ID?
Here is, what I do. The ID is stored as an attachment in a configuration document. The file is detached at runtime to a temporary directory outside the Domino installation path. The code opens the file and returns a handle to the ID
if ( NOERROR != SECKFMOpen(&hKFC,
(char*)detachedFile.c_str(),
(char*)if_file_pwd.c_str(),
SECKFM_open_All, 0, NULL))
{
...
}
An XPage is defined by NOTE_CLASS_FORM and DESIGN_FLAG_XSPPAGE, so later in the code, we have to open the note EXPANDED
ret = tnote.open1(tdb.hDb, nid, OPEN_EXPAND);
Finally, the design note is signed using NSFNoteSignExt3
ret = NSFNoteSignExt3(
tnote.h, hKFC, NULL, MAXWORD, NULLHANDLE,
SIGN_NOTES_IF_MIME_PRESENT, 0, NULL);
and contracted afterwards
ret = NSFNoteContract( tnote.h );
To verify, if anything is wrong with the signature, I use
ret = NSFNoteVerifySignature (
tnote.h, 0, 0, retSigner, retCertifier);
So far, so good. As I said before, the code runs without any issues and the verification also does not report any errors.
At ConnectED, I went to the “Meet the Developers Lab” and asked for advice. The problem seems not to be in my code, but the Devs had a vague idea, what might be the problem.
I did some tests today and here is what I found out.
When you sign a design element using the client, The last entry in the $UpdatedBy field is the name of the signer.
When you do the same on the server, then the server’s name is the last name in the list.
Hence it is different from the name that is stored in the $Signature field, the error occurs.
This also explains, why I can open XPages that have been signed using my code and the server ID instead of the signer ID. In this case, names in $UpdatedBy and $Signature are identical.
As far as I can see, there is no workaround for that. So the only way to get this solved is to open a PMR with IBM …
Recently, I had to figure out, if a shared library on an AIX system was loaded at Domino server startup. I am not an expert on UNIX, so I asked GOOGLE for some help.
I came across the UNIX command genkld, whic did exactly what I wanted to do
genkld will create a list that is printed to the console that shows all loaded shared libraries
You can find more useful information about “Shared library memory footprints on AIX” here
I recently installed SP1 on a Windows 2008 R2 Server. The partition for the OS only had 10GB space available. After SP1 had been successfully installed, only 7GB of free space was left.
You can delete the backup files if SP1 has been installed without any issues.
Use the following command to free up disk space after the service pack installation:
dism /online /cleanup-image /spsuperseded
The process takes a few minutes to complete, it ends with the sentences “Service Pack Cleanup operation completed. The operation completed successfully”.
Please remember that you cannot uninstall the service pack after you have cleaned up the disk space.
Uploading files to an ESXi datastore ist terrible slow using the build-in upload mechanism. However if you add FTP to your ESXi server, and use an ftp client to upload/download files, the upload/download will be much faster.
ProFTPD runs good in ESXi 5.x and offers high transfer rate over normal transfers.
To add ProFTPD Service, do the following.
Enable SSH in ESXi.
Connect to your ESXi host using any ssh client like putty.
Navigate to your datastore folder cd /vmfs/volumes/datastore/
[Replace “datastore” to match your datastore name]
Create any folder where your FTP Service files can be downloaded from the internet or any other source.
mkdir install cd install
Now download ProFTPD using wget
wget http://esxi-customizer.googlecode.com/files/ProFTPD-1.3.3-8-offline_bundle.zip
install ProFTPD using esxcli
esxcli software vib install --no-sig-check -d /vmfs/volumes/datastore/install/ProFTPD-1.3.3-8-offline_bundle.zip
I needed to restart the ESXi system to activate ProFTPD in the Security Profiles.
My vSphere Client starts with a german User Interface. Obviously, the language depends on the OS language. In my cas, it’s German …
To be able to provide screenshots with english labels and error messages in a way, that I can find results on Google more easily. I was looking for a way to switch the language of my vSphere Client to English.
There is no setting in the UI itself, but here is a little trick, how it can be done without pushing language files around like in the pre WW1 days.
Simply add one of the following parameters to the start parameter of the vSphere Client desktop icon
en_USde_DEja_JPzh_CNfr_FRko_KR"C:\Program Files\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\VpxClient.exe" -locale en_US
You can do the same with your web based client.
Th switch the language to French, simply add the language parameter to the URL
https://client-hostname:9443/vsphere-client/?locale=fr_FR
Ytria’s consoleEZ lets you improve the way you work by giving you access to one or several IBM Domino server consoles in one dashboard.
In fact with consoleEZ, you can:
You can download a time limited beta version here.
TriggerHappy 2.0 for Domino on Linux has been released on OpenNTF. It is still BETA and not intended to be used in production.
For now, I only provide the 32Bit version, because despite the 64Bit Version compiles and links, it freezes my server on startup.
The 32Bit version starts without issues, but invokes NSD on server quit and restart. Will look into that, but with Connect 2014 just around the corner, I do not have much spare time to do so.
If anyone is willing to offer help and could review the sourcecode ( Ben? Nathan?, Daniel? ) … Much appreciated.