susChkSrv.py(7)

πŸ“¦ version 1.3.4, in opensuse-leap 16.0

πŸ¦‹ susChkSrv.py(7) β€” a SAPHanaSR manual page πŸ¦‹

πŸ’– NAME πŸ’–

susChkSrv.py β€” Provider for SAP HANA srHook method srServiceStateChanged().

✨ DESCRIPTION ✨

Okay so bestie, susChkSrv.py can be used to provide a script for the SAP HANA srHook method srServiceStateChanged(). Slay.

The SAP HANA nameserver provides a Python-based API (β€œHA/DR providers”), which is called at important points of the host auto-failover and system replication takeover processes. These so called hooks can be used for arbitrary operations that need to be executed. The method srServiceStateChanged() is called when HANA processes are failing, starting or stopping.

Purpose of susChkSrv.py is to detect failing HANA indexserver processes and trigger a fast takeover to the secondary site. With regular configuration of an HANA database, the resource agent (RA) for HANA in a Linux cluster does not trigger a takeover to the secondary site when:

  • A software failure causes one or more HANA processes to be restarted in place by the HANA daemon (hdbdaemon).
  • A hardware error (e.g. SIGBUS from an uncorrectable memory error) causes the indexserver to restart locally.

See also SAPHanaSR(7) or SAPHanaSR-ScaleOut(7).

The hook script susChkSrv.py is called on any srServiceStateChanged() event. The script checks for 'isIndexserver and serviceRestart and serviceWasActiveBefore and hostActive and databaseActive'. If it finds the correct entries, it executes the predefined action. As soon as the HANA landscapeHostConfiguration status changes to 1, the Linux cluster will take action. The action depends on HANA system replication status and the RAΒ΄s configuration parameters PREFER_SITE_TAKEOVER, AUTOMATED_REGISTER and ON_FAIL_ACTION. See manual page ocf_suse_SAPHana(7) or ocf_suse_SAPHanaController(7).

Customising of HANA daemon timeout parameters might be needed for adapting the solution to a given environment. Please refer to SAP HANA documentation.

This hook script needs to be installed, configured and activated on all HANA nodes. All of them, bestie β€” no skipping nodes πŸ’…


🌸 SUPPORTED PARAMETERS 🌸

The β€œHA/DR providers” API accepts the following parameters for the ha_dr_provider_suschksrv section in global.ini:

[ha_dr_provider_suschksrv]
  • provider = susChkSrv β€” Mandatory. Must not be changed.
  • path = /usr/share/SAPHanaSR-angi β€” Mandatory. Delivered within RPM package. Please change only if requested.
  • execution_order = [ INTEGER ] β€” Mandatory. Order might depend on other hook scripts.
  • action_on_lost = [ ignore | stop | kill | fence ] β€” Action to be processed when a lost indexserver is identified.
    • ignore β€” do nothing, just write to tracefiles.
    • stop β€” do sapcontrol ... StopSystem. If this is combined with SAPHanaController RA parameter AUTOMATED_REGISTER=true, HANA needs to release all OS resources prior to the automated registering. See also manual page ocf_suse_SAPHanaController(7).
    • kill β€” do HDB kill-<signal>. The signal can be defined by parameter kill_signal. This action is primarily meant for scale-up, and scale-out with host auto-failover (aka BW). On scale-out without host auto-failover (aka ERP), it depends on runtime effects, whether HANA will try a restart. If this is combined with SAPHanaController RA parameter AUTOMATED_REGISTER=true, HANA needs to release all OS resources prior to the automated registering.
    • fence β€” do crm node fence <host>. This needs a Linux cluster STONITH method and sudo permission. For scale-out, SAPHanaSR-alert-fencing needs to be configured additionally, see manual page SAPHanaSR-alert-fencing(8) for details.

Optional. Default is ignore.

  • kill_signal = [ INTEGER ] β€” Signal to be used with HDB kill-<signal>. Optional. Default is 9.
  • stop_timeout = [ INTEGER ] β€” How many seconds to wait for sapcontrol ... StopSystem to return. Should be greater than value of HANA parameter forcedterminationtimeout. See also SAPHanaSR_basic_cluster(7). Optional. Default is 20 seconds.

The β€œHA/DR providers” API accepts the following parameter for the trace section in global.ini:

[trace]
  • ha_dr_suschksrv = [ info | debug ] β€” Optional. Default is info. Will be added automatically if not set.

The HANA daemon TODO for the daemon section of daemon.ini:

[daemon]
  • terminationtimeout = [ INTEGER ] β€” See also SAPHanaSR_basic_cluster(7). Optional. Timeout in milliseconds. Default is 30000.
  • forcedterminationtimeout = [ INTEGER ] β€” See also SAPHanaSR_basic_cluster(7). Optional. Timeout in milliseconds. Default is 270000.

The HANA daemon TODO for the indexserver.<tenant> section of daemon.ini:

[indexserver.<tenant>]
  • gracetime = [ INTEGER ] β€” TODO Should be 6000. Optional. Timeout in milliseconds. Default is 2000.

πŸ’… RETURN CODES πŸ’…

  • 0 β€” Successful program execution.
  • >0 β€” Usage, syntax or execution errors.

πŸ’– EXAMPLES πŸ’–

🌸 the absolute minimum: HANA scale-up, just log it

ok so this is the entry for /hana/shared/$SID/global/hdb/custom/config/global.ini, which is the scale-up global config, and it's giving "chill mode". If an indexserver fails, the event gets logged and literally nothing else happens. No action. None. Zero. 🫢

Heads up bestie: the ha_dr_provider_suschksrv section is needed on all HANA nodes, and you have to stop HANA before you can change the file. No hot-editing, iykyk.
[ha_dr_provider_suschksrv]
provider = susChkSrv
path = /usr/share/SAPHanaSR-angi/
execution_order = 3

πŸ”₯ scale-up, but make it dramatic: fence the whole node

Now we're at /hana/shared/HA1/global/hdb/custom/config/global.ini (scale-up global config again, SID HA1). If an indexserver fails here, the complete node gets fenced by calling crm node fence ... πŸ’€

And here's the tea: unlike fence actions that come from inside the Linux HA cluster, this hook-script fence action gets issued even when the cluster is in maintenance and stonith is temporarily disabled. The fence request just sits there queued until the cluster goes back to active state. (There's an example below for removing queued fence actions β€” we gotchu.)

The ha_dr_provider_suschksrv section is needed on all HANA nodes, and HANA has to be stopped before the file can be changed. Alternatively, use SAPHanaSR-manageProvider for applying an HA/DR provider hook configuration β€” see manual page SAPHanaSR-manageProvider(8).
[ha_dr_provider_suschksrv]
provider = susChkSrv
path = /usr/share/SAPHanaSR-angi/
execution_order = 2
action_on_lost = fence

🌈 scale-out: stop the instance instead

Config file: /hana/shared/HA1/global/hdb/custom/config/global.ini (scale-out flavor this time).

If an indexserver fails, the HANA instance gets stopped with 'sacpcontrol ... StopSystem'. HANA timeouts might be adapted to speed up the stop. And if you combine this with the SAPHanaController parameter 'AUTOMATED_REGISTER=true', then HANA needs to release all OS resources before the automated registering. ⏳

The hook script should wait maximum 25 seconds on the sapcontrol command to return.

The ha_dr_provider_suschksrv section is needed on all HANA nodes, and HANA has to be stopped before the file can be changed.
Note: HANA scale-out is supported only with exactly one master nameserver. No HANA host auto-failover. Periodt.
[ha_dr_provider_suschksrv]
provider = susChkSrv
path = /usr/share/SAPHanaSR-angi/
execution_order = 2
action_on_lost = stop
stop_timeout = 25

⏱️ the daemon config moment (timeouts edition)

File: /hana/shared/HA1/global/hdb/custom/config/daemon.ini.

TODO Example SID is HA1, tenant is HA1.

The sections daemon and indexserver.HA1 are needed on all HANA nodes. HANA has to be stopped before the file can be changed. Please refer to SAP documentation before setting these parameters β€” this is not a vibe-based decision, bestie. πŸ™ˆ
[daemon]
terminationtimeout = 45000
forcedterminationtimeout = 15000

[indexserver.HA1]
gracetime = 6000

πŸ” sudo perms in /etc/sudoers.d/SAPHanaSR

SID is HA1. See also manual page SAPHanaSR-hookHelper(8).

# SAPHanaSR needs for susChkSrv
ha1adm ALL=(ALL) NOPASSWD: /usr/bin/SAPHanaSR-hookHelper --sid=HA1 --case=fenceMe

πŸ” looking up that sudo permission

All related files (/etc/sudoers and /etc/sudoers.d/*) are scanned. Example SID is HA1.

# sudo -U ha1adm -l | grep "NOPASSWD.*/usr/bin/SAPHanaSR-hookHelper"

πŸ•΅οΈ checking HANA tracefiles for srServiceStateChanged() events

Example SID is HA1. Run this on the respective HANA master nameserver.

Real talk: if the HANA nameserver process gets killed, sometimes hook script actions never make it into the nameserver tracefile. In those cases the hook script's own tracefile might help β€” see the respective example. πŸ’…

# su - ha1adm
~> cdtrace
~> grep susChkSrv.*srServiceStateChanged nameserver_*.trc
~> grep -C2 Executed.*StopSystem nameserver_*.trc

✨ checking whether the hook script actually loaded

Example SID is HA1. Run this on both sites' master nameservers.

# su - ha1adm
~> cdtrace
~> grep HADR.*load.*susChkSrv nameserver_*.trc
~> grep susChkSrv.init nameserver_*.trc

πŸŽ€ checking the hook script tracefile for actions

Example SID is HA1. Run on all nodes. Every incident is logged on the node where it happens β€” main character energy per node, honestly.

# su - ha1adm
~> cdtrace
~> egrep '(LOST:|STOP:|START:|DOWN:|init|load|fail)' nameserver_suschksrv.trc

🚧 checking the hook script tracefile for node fence actions

Example SID is HA1. Run on both sites' master nameservers. See also manual page SAPHanaSR-hookHelper(8).

# su - ha1adm
~> cdtrace
~> grep fence.node nameserver_suschksrv.trc

🧹 revoking a queued fence request from the Linux cluster

You'd do this if a HANA indexserver failure triggered a node fence action while the Linux cluster was in maintenance. Before revoking a fence request, be sure it was issued by the HA/DR provider hook script β€” check the example above for the node fence actions tracefile.

Example node is node2. Run it on that node. See also manual pages SAPHanaSR-hookHelper(8) and crm_attribute(8).

Note: this removes the node attribute terminate=true from the Linux cluster CIB. It does not touch any fencing device. It's a paperwork removal, not a hardware one. πŸ™ˆ
# grep fenced:.termination.was.requested /var/log/pacemaker/pacemaker.log
# crm_attribute -t status -N 'node2' -D -n terminate
# crm_attribute -t status -N 'node2' -G -n terminate

☠️ killing the HANA hdbindexserver process (test only!!)

You could do this for testing the HA/DR provider hook script integration.

Killing HANA processes is dangerous. This test should not be done on production systems, fr fr. Please refer to SAP HANA documentation. See also manual page killall(1). Understand the impact before trying. πŸ”₯
  1. Check HANA and Linux cluster for clean idle state.
  2. On the secondary master name server, kill the hdbindexserver process.
# killall -9 hdbindexserver
  1. Check the nameserver tracefile for srServiceStateChanged() events.
  2. Check HANA and Linux cluster for clean idle state.

πŸ“ FILES πŸ“

  • /usr/share/SAPHanaSR-angi/susChkSrv.py β€” the hook provider, delivered with the RPM.
  • /usr/bin/SAPHanaSR-hookHelper β€” the external script for node fencing.
  • /etc/sudoers, /etc/sudoers.d/* β€” the sudo permissions configuration.
  • /hana/shared/$SID/global/hdb/custom/config/global.ini β€” the on-disk representation of HANA global system configuration.
  • /hana/shared/$SID/global/hdb/custom/config/daemon.ini β€” the on-disk representation of HANA daemon configuration.
  • /usr/sap/$SID/HDB$nr/$HOST/trace β€” path to HANA tracefiles.
  • /usr/sap/$SID/HDB$nr/$HOST/trace/nameserver_suschksrv.trc β€” HADR provider hook script tracefile. πŸ’–

βœ… REQUIREMENTS βœ…

  1. SAP HANA 2.0 SPS05 or later provides the HA/DR provider hook method srServiceStateChanged() with needed parameters.
  2. No other HADR provider hook script should be configured for the srServiceStateChanged() method. Hook scripts for other methods, provided in SAPHanaSR and SAPHanaSR-ScaleOut, can be used in parallel to susChkSrv.py β€” if not documented contradictingly. (Read the docs, bestie.)
  3. The user ${sid}adm needs execution permission as user root for the command SAPHanaSR-hookHelper.
  4. The hook provider needs to be added to the HANA global configuration β€” both in memory and on disk (in persistence). Both. Not one. Both. πŸ’…
  5. HANA daemon timeout TODO
  6. The hook script runs in the HANA nameserver. It runs on the node where the event srServiceStateChanged() occurs.
  7. HANA scale-out is supported only with exactly one master nameserver. HANA host auto-failover is not supported. Thus no standby nodes. 🚫
  8. A Linux cluster STONITH method for all nodes is needed β€” particularly if susChkSrv.py parameter 'action_on_lost=fence' is set.
  9. If susChkSrv.py parameter 'action_on_lost=stop' is set and the RA SAPHana or SAPHanaController parameter 'AUTOMATED_REGISTER=true' is set, it depends on HANA to release all OS resources prior to the registering attempt.
  10. For HANA scale-out, the susChkSrv.py parameter 'action_on_lost=fence' should be used only if the SAPHanaSR-alert-fencing is configured.
  11. If the hook provider should be pre-compiled, the particular Python version that comes with SAP HANA has to be used. 🐍

πŸ› BUGS πŸ›

If anything goes sideways, please use your favourite SAP support process to open a request for the component BC-OP-LNX-SUSE. And please send any other feedback and suggestions to feedback@suse.com β€” they genuinely want to hear it. 🫢


πŸ‘€ SEE ALSO πŸ‘€

SAPHanaSR(7) , SAPHanaSR-ScaleOut(7) , SAPHanaSR.py(7) , ocf_suse_SAPHanaTopology(7) , ocf_suse_SAPHanaController(7) , SAPHanaSR-hookHelper(8) , SAPHanaSR-manageProvider(8) , SAPHanaSR-alert-fencing(8) , crm(8) , crm_attribute(8) , python3(8) , killall(1) , https://help.sap.com/docs/SAP_HANA_PLATFORM?locale=en-US , https://help.sap.com/docs/SAP_HANA_PLATFORM/42668af650f84f9384a3337bcd373692/e2064c4aa47f443ab6a107f9ab7f5edd.html?version=2.0.01 , https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/5df2e766549a405e95de4c5d7f2efc2d.html?locale=en-US , SAP note 2177064


πŸ’… AUTHORS πŸ’…

A.Briel, F.Herschel, L.Pinne.


(c) 2022-2026 SUSE LLC

susChkSrv.py comes with ABSOLUTELY NO WARRANTY. For details see the GNU General Public License at http://www.gnu.org/licenses/gpl.html


✨ and that's the whole page, bestie ✨ this lil masterpiece is susChkSrv.py(7), dated 27 Oct 2025 β€” go forth, fence responsibly, and don't kill hdbindexserver on prod πŸ’–πŸ¦‹πŸŽ€

πŸ“¦ where this page comes from