LOCK_PROFILING(9)

๐Ÿ“ฆ version 15.1, in freebsd 14.5, freebsd 15.1, nixpkgs unstable

๐Ÿ‘‘ LOCK_PROFILING โ€” kernel lock profiling support

ok so bestie, this is a page from the kernel developer's manual (section 9, kernel-land, very main character energy) and it's literally all about spying on your locks. no cap. let's get into it ๐Ÿ’…


โœจ SYNOPSIS โœจ

options LOCK_PROFILING

that's it. one line. that's the whole vibe check. ๐ŸŽ€


๐ŸŒธ DESCRIPTION ๐ŸŒธ

okay so the LOCK_PROFILING kernel option adds support for measuring and reporting lock use and contention statistics. think of it as a little detective following your locks around the kernel, taking notes. ๐Ÿ•ต๏ธโ€โ™€๏ธ

these stats get collated by "acquisition point". and an acquisition point is just a distinct place in the kernel source code โ€” identified by source file name and line number โ€” where a lock gets acquired. so like, kern/foo.c line 427 is one point, and it keeps its own lil receipts. iykyk. ๐Ÿงพ

for each acquisition point, these are the things it accumulates โ€” all six, bestie, we're not skipping any:

  • the longest time the lock was ever continuously held after being acquired at this point. ๐Ÿ’–
  • the total time the lock was held after being acquired at this point.
  • the total time that threads have spent waiting to acquire the lock. ๐Ÿ˜ญ
  • the total number of non-recursive acquisitions.
  • the total number of times the lock was already held by another thread when this point was reached โ€” meaning it needed a spin or a sleep. ๐Ÿ’€
  • the total number of times another thread tried to acquire the lock while it was held after having been acquired at this point.

and on top of all that, the average hold time and average wait time get derived from the total hold time and total wait time respectively, plus the number of acquisitions. math girlies rise. ๐Ÿงฎโœจ


๐Ÿฆ‹ the sysctl variables ๐Ÿฆ‹

the LOCK_PROFILING kernel option also adds these sysctl(8) variables to control and monitor the profiling code. slay:

  • debug.lock.prof.enable โ€” enables or disables the lock profiling code. it defaults to 0 (off), so it's giving "asleep until you say go" ๐Ÿ˜ด
  • debug.lock.prof.reset โ€” resets the current lock profiling buffers. fresh start energy. ๐Ÿซง
  • debug.lock.prof.stats โ€” the actual profiling statistics, in plain text. the columns go like this, from left to right:
    • max โ€” the longest continuous hold time in microseconds.
    • wait_max โ€” the longest continuous wait time in microseconds.
    • total โ€” the total (accumulated) hold time in microseconds.
    • wait_total โ€” the total (accumulated) wait time in microseconds.
    • count โ€” the total number of acquisitions.
    • avg โ€” the average hold time in microseconds, derived from the total hold time and the number of acquisitions.
    • wait_avg โ€” the average wait time in microseconds, derived from the total wait time and the number of acquisitions.
    • cnt_hold โ€” the number of times the lock was held and another thread attempted to acquire the lock.
    • cnt_lock โ€” the number of times the lock was already held when this point was reached.
    • name โ€” the name of the acquisition point, derived from the source file name and line number, followed by the name of the lock in parentheses. iconic naming convention tbh. ๐Ÿท๏ธ
  • debug.lock.prof.rejected โ€” the number of acquisition points that got ignored after the table filled up. rip to those, they didn't make the cut. ๐Ÿ™ˆ
  • debug.lock.prof.skipspin โ€” disables or enables the lock profiling code for the spin locks. it defaults to 0, which means "do profiling for the spin locks". spin spin spin ๐ŸŒ€
  • debug.lock.prof.skipcount โ€” do sampling approximately every N lock acquisitions. we're not measuring everything, we're doing a lil vibe sample. ๐Ÿ“

๐Ÿ’œ SEE ALSO ๐Ÿ’œ

sysctl(8), mutex(9)

go read those too, bestie, they're the supporting cast. ๐Ÿซถ


๐ŸŒˆ HISTORY ๐ŸŒˆ

  • Mutex profiling support appeared in FreeBSD 5.0.
  • Generalized lock profiling support appeared in FreeBSD 7.0.

glow-up arc, honestly. ๐Ÿ“ˆโœจ


๐ŸŽ€ AUTHORS ๐ŸŽ€

the MUTEX_PROFILING code was written by Eivind Eklund <eivind@FreeBSD.org>, Dag-Erling Smรธrgrav <des@FreeBSD.org> and Robert Watson <rwatson@FreeBSD.org>.

the LOCK_PROFILING code was written by Kip Macy <kmacy@FreeBSD.org>.

and this manual page was written by Dag-Erling Smรธrgrav <des@FreeBSD.org>.

give them their flowers, fr fr. ๐Ÿ’๐Ÿ‘‘


โญ NOTES โญ

okay this is the serious part, bestie, pay attention, i'm not being dramatic (i'm being a little dramatic):

the LOCK_PROFILING option increases the size of struct lock_object, so a kernel built with that option will not work with modules built without it. no mixing, no matching. the sizes literally don't agree. ๐Ÿ’€

the LOCK_PROFILING option also prevents inlining of the mutex code, which can result in a fairly severe performance penalty. this is, however, not always the case. LOCK_PROFILING can introduce a substantial performance overhead that's easily monitorable using other profiling tools โ€” so combining profiling tools with LOCK_PROFILING is not recommended. ๐Ÿ˜ญ (the "not recommended" is doing a lot of heavy lifting there.)

measurements are made and stored in nanoseconds using nanotime(9) (on architectures without a synchronized TSC), but they're presented in microseconds. and honestly? this should still be sufficient for the locks you'd be most interested in profiling โ€” the ones that are held long and/or acquired often. long-holders and frequent-flyers, that's your target audience. ๐ŸŽฏ

LOCK_PROFILING should generally not be used in combination with other debugging options, because the results may be strongly affected by interactions between the features. and in particular:

  • LOCK_PROFILING will report higher than normal uma(9) lock contention when run with INVARIANTS, due to the extra locking that occurs when INVARIANTS is present.
  • likewise, using it in combination with WITNESS will lead to much higher lock hold times and contention in profiling output.

so yeah โ€” don't stack the debug options and then act surprised when the numbers look wild. the numbers are correct, your setup is just chaotic. ๐Ÿซ ๐Ÿ”ฅ


okay that's the whole page, mwah ๐Ÿ’‹ โ€” this one came from the Debian edition and it's dated March 7, 2012, so if you're citing it in your little kernel essay, you know the date. go forth and profile responsibly โœจ๐Ÿฆ‹๐ŸŽ€

๐Ÿ“ฆ where this page comes from