SAPCMCONTROLZONE_MAINTENANCE_EXAMPLES(7)

📦 version 1.0.0, in opensuse-leap 16.0

💖 SAPCMControlZone_maintenance_examples(7) 💖

okay bestie, so this is the maintenance examples page for SAPCMControlZone — section 7 of the manual, if you're keeping score 🎀. It's the page you open when you're like "how do I actually touch this thing without breaking the cluster," and honestly it ate.

✨ NAME ✨

SAPCMControlZone_maintenance_examples — maintenance examples for SAPCMControlZone. literally just that, no cap 💅

🌸 DESCRIPTION 🌸

Maintenance examples for ControlZone resources in Linux HA clusters. that's the whole vibe 🦋

Please see ocf_suse_SAPCMControlZone(7) for more examples and read the REQUIREMENTS section there — like, actually read it, don't just skim it and come back confused 🙈

🔥 EXAMPLES 🔥

⭐ Example for checking clean and idle state

These steps should be performed before doing anything with the cluster, and after something has been done. Example user is mzadmin.

# su - mzadmin -c "mzsh status"
# crm_mon -1r
# crm configure show | grep cli-
# cibadmin -Q | grep fail-count
# cs_clusterstate -i

like, vibe check the whole cluster first. every single time. 👑

⭐ Example for manual takeover using Linux cluster

ControlZone application and Linux cluster are checked for clean and idle state. The ControlZone resources are moved to the other node. The related location rule is removed after the takeover took place. ControlZone application and Linux cluster are checked for clean and idle state. Resource group is grp_C11, user is mzadmin.

# su - mzadmin -c "mzsh status"
# crm_mon -1r
# crm configure show | grep cli-
# cibadmin -Q | grep fail-count
# cs_clusterstate -i

# crm resource move grp_C11 force
# cs_wait_for_idle -s 9; crm_mon -1r
# crm resource clear grp_C11

# cs_wait_for_idle -s 6; crm_mon -1r
# crm configure show | grep cli-
# su - mzadmin -c "mzsh status"

see how it starts and ends with the same check? that's the ✨ bookend ✨ of cluster hygiene 🧚

⭐ Example for generic maintenance procedure

ControlZone application and Linux cluster are checked for clean and idle state. The ControlZone resource group is set into maintenance mode. This is needed to allow manual actions on the resources. After the manual actions are done, the resource group is put back under cluster control. It is neccessary to wait for each step to complete and to check the result. ControlZone application and Linux cluster are checked for clean and idle state. Resource group is grp_C11, user is mzadmin. See also example above.

# su - mzadmin -c "mzsh status"
# crm_mon -1r
# crm configure show | grep cli-
# cibadmin -Q | grep fail-count
# cs_clusterstate -i
# crm resource maintenance grp_C11

<do maintenance>

# crm resource refresh grp_C11
# cs_wait_for_idle -s 6; crm_mon -1r
# crm resource maintenance grp_C11 off
# cs_wait_for_idle -s 6; crm_mon -1r
# su - mzadmin -c "mzsh status"
lowkey the <do maintenance> line is doing so much heavy lifting here 😭 that's you, doing the thing, in the middle of the block. main character energy fr fr.

⭐ Set whole Linux cluster into maintenance

This disables all resource management as well as node fencing. However, fence requests from outside will be queued and executed once the cluster leaves maintenance mode. See manual page crm(8) and stonith_admin(8).

# crm maintenance on
# crm_attribute --query -t crm_config -n maintenance-mode
heads up bestie — fencing is off while you're in here, but outside fence requests don't get dropped, they get queued. so they'll slam through the moment you leave maintenance mode. plan accordingly 💀

⭐ Remove left-over maintenance attribute from overall Linux cluster

This could be done to avoid confusion caused by different maintenance procedures. Before doing so, check for cluster attribute maintenance-mode="false".

# crm_attribute --query -t crm_config -n maintenance-mode
# crm_attribute --delete -t crm_config -n maintenance-mode

look before you delete, always. the query is not optional, it's the whole point 🫶

⭐ Remove left-over standby attribute from Linux cluster nodes

This could be done to avoid confusion caused by different maintenance procedures. Before doing so for all nodes, check for node attribute standby="off" on all nodes. Example node is node1.

# crm_attribute --query -t nodes -N node1 -n standby
# crm_attribute --delete -t nodes -N node1 -n standby

all nodes means all nodes, don't get cute and do one and dip 🙈

⭐ Remove left-over maintenance attribute from resource

This should usually not be needed. Resource group is grp_C11.

# crm_resource --resource grp_C11 --delete-parameter maintenance --meta

"should usually not be needed" is manpage-speak for "if you're here, something got messy" 😭 no judgement tho

⭐ Disable Linux cluster on all cluster nodes

On any cluster node the cluster will not start automatically on boot anymore. Nevertheless a currently running cluster will keep running. Needs password-less ssh between cluster nodes.

# crm cluster run "crm cluster disable"
# crm cluster run "systemctl status pacemaker" | grep pacemaker.service

highlight: it stops auto-starting, it does not stop the cluster that's running right now. two totally different things, don't mix them up 💅

⭐ Start Linux cluster on all cluster nodes

Needs password-less ssh between cluster nodes.

# crm cluster start --all
# crm cluster run "systemctl status pacemaker" | grep pacemaker.service

the lil grep pacemaker.service at the end is your proof it actually worked. love a receipt ✨

⭐ Start Linux cluster after node has been fenced

It is recommended to not configure the Linux cluster for always starting autmatically on boot. Better is to start automatically only, if cluster and/or node have been stopped cleanly. If the node has been rebooted by STONITH, the cluster should not start automatically. If the cluster is configure that way, some steps are needed to start the cluster after a node has been rebooted by STONITH. STONITH via SBD is used in this example.

# cs_clear_sbd_devices --all
# cs_show_sbd_devices
# crm cluster start
# crm_mon -r
ok this one's important: if STONITH rebooted the node, you do not want the cluster just waltzing back up on its own. auto-start only for clean stops. otherwise, do the steps above. it's giving responsible adult 🧚

⭐ Overview on maintenance procedure for Linux cluster or OS

ControlZone instance remains running. But it will not automatically be moved or restarted while the Linux cluster is inactive. It is neccessary to wait for each step to complete and to check the result. See examples above for details.

  1. Check status of Linux cluster and ControlZone instance, see above.
  2. Set the Linux cluster into maintenance mode.
  3. Stop Linux Cluster on all nodes.
  4. Perform maintenance on Linux cluster or OS.
  5. Start Linux cluster on all nodes.
  6. Let Linux cluster detect status of ControlZone resources.
  7. Set cluster ready for operations.
  8. Check status of Linux cluster and ControlZone instance, see above.

eight steps. no skipping. the way this list starts and ends with "check status" — that's not a coincidence bestie, that's the whole philosophy 🎀

💀 BUGS 💀

Please report feedback and suggestions to feedback@suse.com.

(no cap, they want to hear from you 🌸)

🦋 SEE ALSO 🦋

ocf_suse_SAPCMControlZone(7) , SAPCMControlZone_basic_cluster(7) , crm(8) , crm_simulate(8) , crm_report(8) , cibadmin(8) , crm_mon(8) , sbd(8) , stonith_admin(8) , corosync-cfgtool(8) , cs_clusterstate(8) , cs_wait_for_idle(8) , cs_show_cluster_actions(8) , cs_clear_sbd_devices(8) , cs_show_sbd_devices(8) , ha_related_sap_notes(7) , ha_related_suse_tids(7)

the reading list is long but iykyk 📚

👑 AUTHORS 👑

F.Herschel, L.Pinne

(c) 2024 SUSE LLC SAPCMControlZone comes with ABSOLUTELY NO WARRANTY. For details see the GNU General Public License at http://www.gnu.org/licenses/gpl.html

k that's the page bestie 💖 this was SAPCMControlZone_maintenance_examples(7) from the SAPCMControl...ance_examples(7) manual, dated 15 Apr 2024 — go forth and maintain responsibly, mwah 🫶✨🍓

📦 where this page comes from