Essbase crash on Linux

We noticed several crash of Essbase process (ESSSRVR) on Linux, specially when users are doing restructuring and running calculations. Call Stack in the exception log look like this.


Process Type: Application
Application Name: XXXXX
Database Name: XXXXX
Exception Log File: /essbase/Hyperion/AnalyticServices/app/XXXX/XXXX/log00001.xcp
Current Thread Id: -1499931744
Signal Number: 0x11=Segmentation Violation
Signal Code: 0x-168009176=Unknown



----- Stack Trace -----



[0] /essbase/Hyperion/AnalyticServices/bin/libessutlu.so(adXcptWriteStackInfo+0x3e) [0xf73ad7de]
[1] /essbase/Hyperion/AnalyticServices/bin/libessutlu.so(adXcptCreateRpt+0x224) [0xf73ac8cc]
[2] /essbase/Hyperion/AnalyticServices/bin/libessutlu.so(adXcptHandlerDiag+0x83) [0xf73ac62b]
[3] /essbase/Hyperion/common/JRE/Sun/1.5.0/lib/i386/server/libjvm.so [0xf5e2ae4d]
[4] /essbase/Hyperion/common/JRE/Sun/1.5.0/lib/i386/server/libjvm.so [0xf5e28c7f]
[5] /essbase/Hyperion/common/JRE/Sun/1.5.0/lib/i386/server/libjvm.so(JVM_handle_linux_signal+0x1d8) [0xf5e2b6b8]
[6] /essbase/Hyperion/common/JRE/Sun/1.5.0/lib/i386/server/libjvm.so [0xf5e28ba4]
[7] [0xffffe600]
[8] /essbase/Hyperion/AnalyticServices/bin/libesssvrd.so(adCalmExec+0x64e) [0xf7bf4a5a]
[9] /essbase/Hyperion/AnalyticServices/bin/libesssvrd.so(adCalcCSCR+0x5fe) [0xf7bcb4b6]
[10] /essbase/Hyperion/AnalyticServices/bin/libesssvrd.so(adCalcCSCR+0x2040) [0xf7bccef8]
[11] /essbase/Hyperion/AnalyticServices/bin/libesssvrd.so(adCalcCSCR+0x6eb) [0xf7bcb5a3]
[12] /essbase/Hyperion/AnalyticServices/bin/libesssvrd.so(adCalcScrCalc+0xf5) [0xf7bcad01]
[13] /essbase/Hyperion/AnalyticServices/bin/libesssvrd.so(EpXSvrCalcExec+0x484) [0xf7d987dc]
[14] /essbase/Hyperion/AnalyticServices/bin/libesssvrd.so [0xf7d9ce47]
[15] /essbase/Hyperion/AnalyticServices/bin/libesssvrd.so(EpXSvrDispatch+0xf8) [0xf7d98350]

We noticed that the VM for the ESSSRVR process was very high. Even for Sample application, VM was around 1.5G. That pretty high. Since Essbase is 32-bit application, it can only address 2GB of memory. Since restructuring/calc need more memory, chances are that process can go beyond 2GB, hence crash.

We changed one of the parameter in Essbase.cfg file. Set the JvmModuleLocation to blank.

; JvmModuleLocation /essbase/Hyperion/common/JRE/Sun/1.5.0/lib/i386/server/libjvm.so
JvmModuleLocation

After this, ESSSRVR process will not load the JVM, hence the VM for the process will be low. After this change, VM for sample application was around 350M as compare to 1.5G. This will give sufficient room for the process to grow when users are doing restructuring/calc.

After this change, we did not notice the similar crash.

Wish we had Essbase 64-bit on Linux available so that we will not hit this limitation.