SAPHanaSR-Scal..._basic_cluster(7)

📦 version 1.3.4, in opensuse-leap 16.0

✨ NAME ✨

SAPHanaSR-Scal..._basic_cluster(7) — yes bestie, that's the SAPHanaSR manual page we're in. The actual NAME entry is SAPHanaSR-ScaleOut_basic_cluster — SAP HANA System Replication scale-out basic cluster configuration. Cluster lore time. 💅

🌸 DESCRIPTION 🌸

Okay so the SAP HANA System Replication scale-out scenario needs a certain basic cluster configuration. Besides those necessary settings, some additional configurations might match specific needs. Adapting a few SAP HANA settings might be beneficial too. No cap. 🦋

🧚 CRM basics

  • no-quorum-policy = freeze — The crm basic parameter no-quorum-policy in the CIB defines how the cluster should act in case of quorum loss. With more than two nodes, the cluster must not ignore the quorum loss. For SAPHanaSR-ScaleOut, an odd number of nodes is required. Setting no-quorum-policy to 'freeze' won't allow the partition to shoot any other node when it doesn't have quorum. Cluster will not be able to add and start new resources, but running will stay alive.
  • default-resource-stickiness = 1000 — The crm basic parameter default-resource-stickiness in the CIB defines the 'stickiness' score a resource gets on the node where it is currently running. This prevents the cluster from moving resources around without an urgent need during a cluster transition. The correct value depends on number of resources, colocation rules and resource groups. Particularly additional groups colocated to the HANA primary master resource can affect cluster decisions. Too high value might prevent not only unwanted but also useful actions. This is because SAPHanaSR uses an internal scoring table for placing the HANA roles on the right nodes.
  • concurrent-fencing = true — The crm basic parameter concurrent-fencing in the CIB allows the cluster to fence more than one node at a time. This helps to reduce the time needed for a take over in case a whole data center is lost. If nodes are fenced one by one, the time needed would be equal to the number of nodes * stonith timeout. With concurrent-fencing enabled the time needed is in the range of 2 * stonith timeout, independent of the number of nodes. See also pcmk_action_limit below.
  • failure-timeout = 86400 — The crm basic parameter failure-timeout in the CIB defines how long failed actions will be kept in the CIB. After that time the failure record will be deleted. The time is measured in seconds. See also migration-threshold below.
  • migration-threshold = 50 — The crm basic parameter migration-threshold in the CIB defines how many errors on a resource can be detected before this resource will be migrated to another node. See also failure-timeout.
  • record-pending = false — The op_default record-pending in the CIB defines whether the intention of an action upon the resource is recorded in the Cluster Information Base (CIB). Setting this parameter to 'true' allows the user to see pending actions like 'starting' and 'stopping' in crm_mon and Hawk.

🛡️ SBD STONITH basics

  • pcmk_action_limit = -1 — The sbd stonith parameter pcmk_action_limit in the CIB defines the maximum number of concurrent fencing actions. It allows parallel fencing of multiple nodes. A value of '-1' means virtually unlimited. See also concurrent-fencing above.
  • pcmk_delay_max = 1s — The sbd stonith parameter pcmk_delay_max in the CIB defines an upper limit for waiting before a fencing/stonith request will be triggered. This parameter should prevent the cluster from unwanted double fencing in case of split-brain. A value around 30 seconds is required in two-node clusters. It is not needed in usual SAPHanaSR-ScaleOut setups.
  • SBD_STARTMODE=clean — The sbd daemon parameter SBD_STARTMODE in /etc/sysconfig/sbd defines whether the sbd daemon starts automatically after a node has been fenced. The value "clean" tells sbd to start only if the node was not previously fenced. This does not work for diskless sbd. See manual page sbd(8) and stonith_sbd(7) for details.

🧚 systemd basics

saphostagent.service enabled
SAP${SID}_${INO}.service enabled

In case systemd-style init is used for the HANA database, the services saphostagent and SAP${SID}_${INO} need to be enabled and running inside the SAP slice. The instance profile Autostart feature needs to be off. The service saptune is highly recommended, see manual page saptune(8).

🔗 pacemaker service dependency to SAP instance service

[Unit]
Wants=SAP${SID}_${INO}.service
After=SAP${SID}_${INO}.service

In case systemd-style init is used for the HANA database, it might be desired to have the SAP instance service stopping after pacemaker at system shutdown. Therefore a drop-in file for the pacemaker service might help. See examples below.

⚙️ pacemaker service basics

  • PCMK_fail_fast = yes — The parameter PCMK_fail_fast in /etc/sysconfig/pacemaker specifies how pacemaker reacts on failures of its subdaemons. Default "no" means to restart failed subdaemons, while "yes" means fencing the node. Setting "yes" might help to avoid undefined situations. See also SAPHanaSR-alert-fencing(8). Optional, default no.

🍓 SAP HANA basics

/usr/sap/${SID}/SYS/global/hdb/custom/config/global.ini

[memorymanager]
final_memory_release_shutdown = [ auto | on | off ]
final_memory_release_crash = [ auto | on | off ]

Starting with SAP HANA 2.0 SPS06, the database shutdown can be accelerated by optimising memory de-allocation.

⚠️ Please refer to SAP documentation before setting this parameters.

/usr/sap/${SID}/SYS/global/hdb/custom/config/daemon.ini

[daemon]
terminationtimeout =  [ millisec ]
forcedterminationtimeout = [ millisec ]

The first parameter defines the timeout from sending SIGTERM to finally terminating child processes when HANA is shutting down by the STOP event. Used also as maximal delay in system restart if 'restartterminationtimeout' parameter is not set. The second defines the timeout from sending the SIGTERM to finally terminating child processes when HANA is shutting down by the QUIT event. See also manual page susChkSrv.py(7).

⚠️ Please refer to SAP documentation before setting this parameters.

/hana/shared/${SID}/global/hdb/custom/config/nameserver.ini

[landscape]
master = node1:31nr1
worker = node1 node2
active_master = node1:31nr1
roles_node1 = worker
roles_node2 = worker

For two-node scale-out HANA without standby nodes, this entry is needed at both sites that are managed by the Linux cluster. The HANA has to be stopped before the files can be edited.

🚨 Do not copy the file between nodes.

It might be necessary to un-register and re-register the secondary to make the change effective on the secondary site. In the example, node1 should be the master namerserver´s hostname, node2 the worker´s hostname, nr the instance number. Crucial is to define only one master namerserver per site, but no candidates. Please check SAP HANA documentation for details. 🫶

✨ EXAMPLES ✨

Okay bestie, this is the example energy section — configs, checks, and little cluster rituals. 💖

🎀 crm basic configuration

Below is an example crm basic configuration for SAPHanaSR-ScaleOut. These are the specific parameters you need; some general parameters are left out. This example is for SLE-HA 15 SP1 with disk-based SBD:

property cib-bootstrap-options: \
 have-watchdog=true \
 cluster-infrastructure=corosync \
 cluster-name=hacluster \
 stonith-enabled=true \
 placement-strategy=balanced \
 stonith-timeout=180 \
 no-quorum-policy=freeze \
 concurrent-fencing=true
rsc_defaults rsc-options: \
 resource-stickiness=120 \
 migration-threshold=50 \
 failure-timeout=86400
op_defaults op-options: \
 timeout=600 \
 record-pending=true

🦋 crm SBD stonith configuration for SLE 15

To complete the SBD setup, you need to activate SBD as the STONITH/fencing mechanism in the CIB. For SAPHanaSR-ScaleOut, SBD is normally used instead of any other fencing/stonith mechanism. Example for a basic disk-based SBD resource on SLE 15:

primitive rsc_stonith_sbd stonith:external/sbd \
 params pcmk_action_limit="-1" pcmk_delay_max="1"

🌸 crm SBD stonith configuration for SLE 16

Same SBD setup plot, new SLE era. To complete the SBD setup, you need to activate SBD as the STONITH/fencing mechanism in the CIB. For SAPHanaSR-ScaleOut, SBD is normally used instead of any other fencing/stonith mechanism. Example for a basic disk-based SBD resource on SLE 16:

primitive rsc_stonith_sbd stonith:fence_sbd \
 params pcmk_action_limit="-1" pcmk_delay_max="1"

🫶 crm simple IP address resource configuration

Let the Linux cluster manage one IP address and move that address along with the HANA primary master nameserver. If the HANA primary master nameserver is in transition, the IP address might be placed somewhere temporarily.

primitive rsc_ip_SLE_HDB00 IPaddr2 \
 op monitor interval=10s timeout=20s \
 params ip=192.168.178.188
colocation col_ip_with_SLE_HDB00 \
 2000: rsc_ip_SLE_HDB00:Started mst_SAPHanaCon_SLE_HDB00:Promoted

💅 crm pedantic IP address resource configuration

Let the Linux cluster manage one IP address and move that address along with the HANA primary master nameserver. While the HANA primary master nameserver is in transition, the IP address is stopped.

primitive rsc_ip_SLE_HDB00 IPaddr2 \
 op monitor interval=10s timeout=20s \
 params ip=192.168.178.188
colocation col_ip_with_SLE_HDB00 \
 2000: rsc_ip_SLE_HDB00:Started mst_SAPHanaCon_SLE_HDB00:Promoted
location loc_ip_hdb_not1 rsc_ip_SLE_HDB00 \
 rule -inf: master-rsc_SAPHanaCon_SLE_HDB00 lt 145
location loc_ip_hdb_not2 rsc_ip_SLE_HDB00 \
 rule -inf: not_defined master-rsc_SAPHanaCon_HA1_HDB00

🔥 crm protective IP address resource configuration

Let the Linux cluster manage one IP address. The resource should tolerate short absence of (virtual) network cards. If a monitor and the resulting start fails, the node should get fenced. This protects against risk of HANA dual-primary. A colocation constraint between this IP address resource and the HANA primary master nameserver is needed as well, see respective examples.

primitive rsc_ip_SLE_HDB00 IPaddr2 \
 op monitor interval=30 timeout=60 \
 op start timeout=90 on-fail=fence \
 params ip=192.168.178.188 cidr_netmask=32

This configuration might be used in public cloud environments where virtual NICs might be expected to disappear temporarily. Of course, it would be better to not let NICs disappear in production. I.e. by adding redundancy to the network or by setting the cluster into maintenance.

🦋 crm IP address for active/active read-enabled resource configuration

Let the Linux cluster manage an additional IP address and move that address along with the HANA secondary master nameserver. Example SID is SLE, instance number 00. The name of the node attribute is all lowercase, e.g. hana_sle_roles.

Note: This example works for two-node HANA scale-out. 💖
primitive rsc_ip_ro_SLE_HDB00 IPaddr2 \
 op monitor interval=10s timeout=20s \
 params ip=192.168.178.199
colocation col_ip_ro_secondary_SLE_HDB00 \
 2000: rsc_ip_ro_SLE_HDB00:Started mst_SAPHanaCon_SLE_HDB00:Unpromoted
location loc_ip_ro_master_SLE_HDB00 \
 rsc_ip_ro_SLE_HDB00 \
 rule -inf: hana_sle_roles ne master1:master:worker:master

🌈 alternative location constraint for active/active read-enabled configuration, SAPHanaSR-angi 1.2.12 and newer

Same use case as the example above. No colocation needed and the location constraint is simplified.

location loc_ip_ro_HA1_HDB00 \
 rsc_ip_ro_SLE_HDB00 \
 rule 2000: master-rsc_SAPHanaCon_SLE_HDB00 eq 100

🎀 crm grouped IP address resource configuration

Let the Linux cluster manage one IP address and move that address along with the HANA primary master nameserver. An auxiliary resource is needed for a specific public cloud purpose.

primitive rsc_ip_SLE_HDB00 IPaddr2 \
 op monitor interval=10s timeout=20s \
 params ip=192.168.178.188 cidr_netmask=32
primitive rsc_lb_SLE_HDB00 azure-lb \
 params port=62502
group grp_ip_SLE_HDB00 rsc_lb_SLE_HDB00 rsc_ip_SLE_HDB00 \
 meta resource-stickiness=1
colocation col_ip_with_SLE_HDB00 \
 8000: grp_ip_SLE_HDB00:Started mst_SAPHanaCon_SLE_HDB00:Promoted

💌 crm MailTo resource configuration

You could define a MailTo resource that informs you as soon as the score for the promoted instance of the SAPHanaController resource changes. See also manual pages SAPHanaSR-showAttr(8) and ocf_heartbeat_MailTo(7). Example SID is HA1, instance number is 10.

primitive rsc_mailto_HA1_HDB10 MailTo \
 params email="root@localhost" subject="master-
rsc_SAPHanaCon_HA1_HDB10 changed." \
 op monitor timeout=10 interval=30 depth=0 \
location loc_mailto_HA1_HDB10_with_primary rsc_mailto_HA1_HDB10 \
 rule 2000: master-rsc_SAPHanaCon_HA1_HDB10 eq 150

👑 check how resource stickiness affects promotion scoring

SAPHanaSR uses an internal scoring table. The promotion scores for HANA primary and secondary master are in a certain range. The scores used by the Linux cluster should be in the same range.

# SAPHanaSR-showAttr | grep master.:master
# crm_simulate -Ls | grep promotion

🧹 clean up SDB stonith resource after write failure

In rare cases the SBD stonith resource fails writing to the block device. After the root cause has been found and fixed, the failure message can be cleaned.

# stonith_admin --cleanup --history=<originator_node>

🛑 disable automatic start of Linux cluster at system boot

This might be done if the Linux cluster should never start automatically at system boot. Useful with diskless sbd, where SBD_STARTMODE does not work.

# systemctl disable sbd
# systemctl disable corosync
# systemctl disable pacemaker

🩺 check saphostagent and show SAP instances

Basic check for the saphostagent.

# /usr/sap/hostctrl/exe/saphostctrl -function Ping
# /usr/sap/hostctrl/exe/saphostctrl -function ListInstances

🌟 check systemd services for the HANA database

In case systemd-style init is used for the HANA database, the services can be checked. Example SID is HA1, instance number is 10.

# systemctl list-unit-files | grep -i sap
# systemctl status SAPHA1_10.service
# systemd-cgls -u SAP.slice
# systemd-cgls -u SAPHA1_10.service
# systemctl show SAPHA1_10.service

🎀 show pacemaker service drop-in file

In case systemd-style init is used for the HANA database, it might be desired to have the SAP instance service stopping after pacemaker at system shutdown. A drop-in file might help. Example SID is S07, instance number is 00.

# cat /etc/systemd/system/pacemaker.service.d/00-pacemaker.conf
[Unit]
Description=pacemaker needs SAP instance service
Documentation=man:SAPHanaSR-ScaleOut_basic_cluster(7)
Wants=SAPS07_00.service
After=SAPS07_00.service

🦋 check for pacemaker dependency to SAP instance service

Example SID is S07, instance number is 00.

# systemctl show pacemaker.service | grep SAPS07_00
# systemd-delta | grep pacemaker
# systemd-analyze dot | grep "pacemaker.*SAPS07_00"

🔍 check running pacemakerd daemon for PCMK_ settings

No output means default, which is f.e. PCMK_fail_fast=no. See /etc/sysconfig/pacemakerd for details.

# cat /proc/$(pgrep pacemakerd)/environ | tr '\0' '\n' | grep PCMK_

💜 check what SBD_ settings pacemakerd knows from running sbd daemon

No output means default. See /etc/sysconfig/sbd for details.

# cat /proc/$(pgrep pacemakerd)/environ | tr '\0' '\n' | grep SBD_

😭 check for pacemaker IPC timeouts

# grep "pacemaker-.*unresponsive.to.ipc" /var/log/messags

🗂️ FILES 🗂️

  • /var/lib/pacemaker/pengine/pe-*.bz — pacemaker CIB snapshot files
  • /var/log/crmsh/crmsh.log — crm shell log
  • /var/log/messages — general system log
  • /var/log/pacemaker/pacemaker.log — pacemaker log

💀 BUGS 💀

In case of any problem, please use your favourite SAP support process to open a request for the component BC-OP-LNX-SUSE. Please report any other feedback and suggestions to feedback@suse.com.

👑 SEE ALSO 👑

🌸 AUTHORS 🌸

F.Herschel, L.Pinne.

(c) 2018 SUSE Linux GmbH, Germany.

(c) 2019-2026 SUSE LLC

For details see the GNU General Public License at http://www.gnu.org/licenses/gpl.html

Okay bestie, that’s the whole page — mwah 💋. Page date: 16 Jan 2026; source/version: SAPHanaSR-Scal..._basic_cluster(7). Go slay your cluster. 💖✨

📦 where this page comes from