CMAP_KEYS(7)

💖 CMAP_KEYS(7) 💖

okay so bestie, welcome to cmap_keys — a whole tour of the keys chillin' inside the Configuration Map. it's giving "backstage pass to corosync's brain" and honestly? iconic. 🌈

---

✨ NAME ✨

cmap_keys — Overview of keys stored in the Configuration Map 🗺️💅

---

🌸 OVERVIEW 🌸

so there are 3 main types of keys stored in CMAP, and they're all serving different vibes:

• Mapping of values stored in the config file. 📄
• Runtime statistics. 📊
• Other user created values. 🧚

and just so we're clear, in this man page, wild-cards have the usual meaning. no cap. * does * things. 🦋

---

🎀 ICMAP KEYS 🎀

these keys are in the icmap (default) map, bestie. the main character map. 👑

• internal_configuration.* — Internal configuration data. every single key in this prefix is read only 🚫✍️. it's only useful for getting a list of loaded services. that's it, that's the whole job.
• logging.* — Values read from the configuration file. it's possible to change them at runtime 💫. if you need subsystem specific configuration, the key must be in the form logging.logger_subsys.SERVICE.key, where SERVICE is the upper case name of the service and key is same as in the configuration file. all values are of string type. 🧵
• nodelist.* — Values are read from the configuration file only (dynamic updates are <b>not</b> allowed, sorry bestie 😭). Each node element in the configuration file gets assigned its position starting from zero. So the first node from the config file has nodelist.node.0. prefix. To be a valid entry, each node must have ring0_addr key. To change the nodeid key, use a u32 data type. 🔢

Local node position is stored in local_node_pos key (RO), so it's easy to find out nodeid/ring addresses of the local node directly from cmap. so convenient, we love her 🫶

• runtime.blackbox.* — Trigger keys for storing fplay data. it's recommended that you use the corosync-blackbox command to change keys in this prefix. let the tool do it, pls 🙈
• runtime.force_gather — Set to 'yes' to force the processor to move into the GATHER state.

⚠️ bestie this operation is dangerous and is <b>not recommended</b>. like, genuinely. not me warning you twice. 💀

• runtime.config.* — Contains the values actually in use by the totem membership protocol. Values here are either taken from the Corosync configuration file, defaults or computed from entries in the config file. For information on individual keys please refer to the man page corosync.conf(5). 📚
• runtime.services.* — Prefix with statistics for service engines. Each service has its own service_id key in the prefix with the name runtime.services.SERVICE., where SERVICE is the lower case name of the service. Inside the service prefix is the number of messages received and sent by the corosync engine in the format runtime.services.SERVICE.EXEC_CALL.rx and runtime.services.SERVICE.EXEC_CALL.tx, where EXEC_CALL is the internal id of the service call (so for example 3 in cpg service is receive of multicast message from other nodes). 🔥
• runtime.totem.members.* — Prefix containing members of the totem single ring protocol. Each member keys has format runtime.totem.members.NODEID.KEY, where key is one of:
  • config_version — Config version of the member node. 📌
• resources.process.PID.* — Prefix created by applications using SAM with CMAP integration. It contains the following keys:
  • recovery — Recovery policy of the process. Can be one of quit or restart. 🔁
  • poll_period — Value passed in sam_initialize as a time_interval. ⏱️
  • last_updated — Last time SAM received a heartbeat from the client. 💓
  • state — State of the client. Can be one of failed, stopped, running and waiting for quorum. 🌡️
• uidgid.* — Information about users/groups which are allowed to make IPC connections to corosync. Entries loaded from configuration file are stored with uidgid.config.* prefix and are pruned on configuration file reload. Dynamic entries has uidgid.* prefix and a configuration file reload doesn't affect them. two whole species of keys, living their own lives 🦋
• quorum.cancel_wait_for_all — Tells votequorum to cancel waiting for all nodes at cluster startup. Can be used to unblock quorum if notes are known to be down. For pcs use only. 🔒
• cfg.shutdown_timeout — Sets the timeout within which daemons that are registered for cfg callbacks must respond to a corosync_cfg_try_shutdown() request. the default is 5000 mS. ⏳
• config.reload_in_progress — This value will be set to 1 (or created) when a corosync.conf reload is started, and set to 0 when the reload is completed. This allows interested subsystems to do atomic reconfiguration rather than changing each key. Note that individual add/change/delete notifications will still be sent during a reload. 📝✨
• config.totemconfig_reload_in_progress — This key is similar to config.totemconfig_reload_in_progress but changed after the totem config trigger is processed. It is useful (mainly) for situations when nodelist.local_node_pos must be correctly reinstated before anything else. 🎯

---

to be continued, bestie… 💜

Okay so stats map time bestie — and the vibe check here is important: every single key in this map is read-only. You're not writing, you're observing. 💅

Tracking modifications of individual keys is supported in the stats map, but not prefixes. However — plot twist — Add/Delete operations ARE supported on prefixes, so you can totally track new ipc connections or knet interfaces. ✨

---

STATS KEYS

stats.srp.*

Prefix containing statistics about totem. Typical key prefixes:

• commit_entered — Number of times the processor entered COMMIT state.
• commit_token_lost — Number of times the processor lost token in COMMIT state.
• consensus_timeouts — How many times the processor timed out forming a consensus about membership.
• continuous_gather — How many times the processor was not able to reach consensus.
• firewall_enabled_or_nic_failure — Set to 1 when processor was not able to reach consensus for long time. The usual reason is a badly configured firewall or connection failure.
• gather_entered — Number of times the processor entered GATHER state.
• gather_token_lost — Number of times the processor lost token in GATHER state.
• mcast_retx — Number of retransmitted messages.
• mcast_rx — Number of received multicast messages.
• mcast_tx — Number of transmitted multicast messages.
• memb_commit_token_rx — Number of received commit tokens.
• memb_commit_token_tx — Number of transmitted commit tokens.
• memb_join_rx — Number of received join messages.
• memb_join_tx — Number of transmitted join messages.
• memb_merge_detect_rx — Number of received member merge messages.
• memb_merge_detect_tx — Number of transmitted member merge messages.
• orf_token_rx — Number of received orf tokens.
• orf_token_tx — Number of transmitted orf tokens.
• recovery_entered — Number of times the processor entered recovery.
• recovery_token_lost — Number of times the token was lost in recovery state.
• rx_msg_dropped — Number of received messages which were dropped because they were not expected (as example multicast message in commit state).
• token_hold_cancel_rx — Number of received token hold cancel messages.
• token_hold_cancel_tx — Number of transmitted token hold cancel messages.
• mtt_rx_token — Mean transit time of token in milliseconds. In other words, time between two consecutive token receives.
• avg_token_workload — Average time in milliseconds of holding time of token on the current processor.
• avg_backlog_calc — Average number of not yet sent messages on the current processor.

---

stats.knet.nodeX.linkY.*

Statistics about the network traffic to and from each node and link when using tke kronosnet transport

• connected — Whether the link is connected or not
• up_count — Number of times this link has changed state to UP
• down_count — Number of times this link has changed state to DOWN
• latency_ave / latency_max / latency_max — Calculated latencies of this link. Note that if there has been no traffic on the link then latency_min will show a very large number.
• latency_samples — The number of samples used to calculate the latency figures, so you have some idea of their precision.
• rx_data_packets / tx_data_packets — The number of packets sent/received on this link
• rx_data_bytes / tx_data_bytes — The number of bytes sent/received on this link
• rx_pmtu_packets / tx_pmtu_packets — The number of packets sent/received by the PMTUd subsystem
• rx_pmtu_bytes / tx_pmtu_bytes — The number of bytes sent/received by the PMTUd subsystem
• rx_ping_packets / tx_ping_packets — The number of packets sent/received as pings
• rx_ping_bytes / tx_ping_bytes — The number of bytes sent/received as pings
• rx_pong_packets / tx_pong_packets — The number of packets sent/received as pongs
• rx_pong_bytes / tx_pong_bytes — The number of bytes sent/received as pongs
• rx_total_packets / tx_total_packets — The total number of packets sent/received. The aggregate of all of the above packet stats
• rx_total_bytes / tx_total_bytes — The total number of bytes sent/received. The aggregate of all of the above bytes stats
• tx_data_retries / tx_pmtu_retries / tx_ping_retries / tx_pong_retries / tx_total_retries — Number of times a transmit operation had to be retried due to the socket returning EAGAIN

---

stats.ipcs.*

There is information about total number of active connections from client programs at the time the request was made. active number of closed connections during whole runtime of corosync closed Total number of connections that have been made since corosync was started

---

stats.ipcs.ID.*

Each IPC connection has a unique ID. This is in the form [[serviceX:][PID:]internal_id.

Typical keys in this prefix are:

• proc_name — process name of connected process (unavailable on some platforms)
• dispatched — number of dispatched messages.
• invalid_request — number of requests made by IPC which are invalid (calling non-existing call, ...).
• name — contains short name of the IPC connection (unavailable on some platforms).
• overload — is number of requests which were not processed because of overload.
• queue_size — contains the number of messages in the queue waiting for send.
• recv_retries — is the total number of interrupted receives.
• requests — contains the number of requests made by IPC.
• responses — is the number of responses sent to the IPC client.
• send_retries — contains the total number of interrupted sends.
• service_id — contains the ID of service which the IPC is connected to.

---

stats.schedmiss.<n>.*

If corosync is not scheduled after the required period of time it will log this event and also write an entry to the stats cmap under this key. There can be up to 10 entries (0..9) in here, when an 11th event happens the earliest will be removed. 😭

These events will always be in reverse order, so stats.schedmiss.0.* will always be the latest event kept and 9 the oldest. If you want to listen for notifications then you are recommended to listen for changes to stats.schedmiss.0.timestamp or stats.schedmiss.0.delay. 👑

• timestamp — The time of the event in ms since the Epoch (ie time_t * 1000 but with valid milliseconds).
• delay — The time that corosync was paused (in ms, float value).

---

stats.clear.*

These are write-only keys used to clear the stats for various subsystems

• totem — Clears the pg & srp totem stats.
• knet — Clears the knet stats
• ipc — Clears the ipc stats
• schedmiss — Clears the schedmiss stats
• all — Clears all of the above stats

---

DYNAMIC CHANGE USER/GROUP PERMISSION TO USE COROSYNC IPC

Is the same as in the configuration file. eg: to add UID 500 use

# corosync-cmapctl -s uidgid.uid.500 u8 1

GID is similar, so to add a GID use

# corosync-cmapctl -s uidgid.gid.500 u8 1

For removal of permissions, simply delete the key 🗑️

# corosync-cmapctl -d uidgid.gid.500

---

SEE ALSO

corosync_overview(7), corosync.conf(5), corosync-cmapctl(8)

---

and that's the whole page bestie, you made it to the end 🫶 this lil masterpiece is a corosync Man Page, dated 2018-10-08, filed under CMAP_KEYS(7) — go forth and cmapctl responsibly, ily 💖✨