scout(1)

💅 scout(1) — the Scalasca Trace Tools analyzer, bestie edition ✨

okay so. you opened scout(1) from the Scalasca Trace Tools and honestly? iconic. let me tell you everything, no gatekeeping.

---

🌸 NAME 🌸

scout — the Scalasca parallel trace analyzer. she analyzes traces. that's the whole vibe.

---

🎀 SYNOPSIS 🎀

scout.ser [OPTION] (ANCHOR_FILE | EXPERIMENT_DIR)
scout.omp [OPTION] (ANCHOR_FILE | EXPERIMENT_DIR)
scout.mpi [OPTION] (ANCHOR_FILE | EXPERIMENT_DIR)
scout.hyb [OPTION] (ANCHOR_FILE | EXPERIMENT_DIR)

four flavors, one queen. pick your fighter 💖

---

🦋 DESCRIPTION 🦋

so scout is the scalable automatic trace-analysis component of the Scalasca Trace Tools. she's the one who does the homework. and specifically, she's serving:

• identifying wait states in communication and synchronization operations that pop up, like, for example when workloads are unevenly distributed (the classic 😭),
• pinpointing the root causes of those wait states — yes, the actual delays, the origin story,
• identifying the activities on the critical path of the target application, which highlights the routines that literally determine how long the program runs, aka the best candidates for optimization. main character routines, honestly.

usually, scout gets launched automatically by the Scalasca measurement collection and analysis nexus scan(1) after a successful measurement, but only if event tracing is configured. but she can also be run manually on an existing event trace measurement, so go off.

scout currently supports trace experiments in two different event trace formats: OTF2 traces generated by the Score-P instrumentation and measurement system, and legacy traces in EPILOG format generated by the measurement system of the Scalasca 1.x release series. so for OTF2 event traces you gotta hand scout the corresponding ANCHOR_FILE (like, e.g., traces.otf2), and for EPILOG traces you give her the experiment directory name EXPERIMENT_DIR. periodt.

Depending on the build configuration and the capabilities of the target platform, the scout analyzer may be available in up to four forms:

scout.ser : is always built. she's used to analyze event traces generated by serial applications. she can also be used to analyze event traces from multi-threaded applications, however, she will then only provide information about the master thread. so, like, partial info, but valid.

scout.omp : is built whenever the Scalasca Trace Tools are configured with OpenMP support. she's used to analyze event traces generated by pure multi-threaded applications (e.g., using OpenMP or POSIX threads). she can also be used to analyze event traces from serial applications, though analysis incurs a higher overhead than using scout.ser.

scout.mpi : is built whenever the Scalasca Trace Tools are configured with MPI support. she's used to analyze event traces generated by pure MPI applications. she can also be used on traces from multi-threaded MPI applications, however, she will then only provide information about the master thread of each process and its MPI activities.

scout.hyb : is built if the Scalasca Trace Tools are configured with both MPI and OpenMP support. she's used to analyze event traces generated by multi-threaded MPI applications (e.g., MPI+OpenMP or MPI+Pthreads), providing information about all OpenMP/POSIX threads of each MPI process. she can also be used on traces from pure MPI applications, though analysis incurs a slightly higher overhead than using scout.mpi.

💀 heads up bestie: scout.mpi and scout.hyb are implemented as MPI programs, so they have to be executed using appropriate MPI launch commands and flags. Also, the number of MPI processes for scout must be identical to the number of MPI processes used for the target application execution. no cap.

If successful, scout produces the following output files in the measurement archive directory:

• scout.cubex: the trace analysis result
• trace.stat: trace analysis pattern statistics

---

✨ OPTIONS ✨

scout accepts a bunch of command-line options to enable/disable particular analysis features. When scout is launched automatically from the Scalasca measurement collection and analysis nexus scan(1), these options can be passed to the analyzer via the SCAN_ANALYZE_OPTS environment variable. iykyk 💅

🌈 Common options

• --statistics — Enables most-severe instance tracking and wait-state statistics. This is the default.
• --no-statistics — Disables most-severe instance tracking and wait-state statistics.
• --critical-path — Enables critical-path analysis. This is the default.
• --no-critical-path — Disables critical-path analysis.
• --rootcause — Enables root-cause analysis. This is the default.
• --no-rootcause — Disables root-cause analysis.
• --single-pass — Use single-pass forward analysis only. Disables both critical-path and root-cause analysis, as well as the detection of Late Receiver wait states.
• -v, --verbose — Increase verbosity. (say more, queen 🗣️)
• --help — Print a brief usage summary, then exit.

🧚 MPI options (scout.mpi/scout.hyb only)

• --time-correct — Enables enhanced timestamp correction. Event traces collected on clusters without a synchronized clock may contain logical clock condition violations (such as a receive completing before the corresponding send is initiated). When scout detects such situations, she issues a warning that the analysis may be inconsistent and recommends (re-)running trace analysis with her integrated timestamp correction algorithm activated. so helpful of her fr fr.
• --no-time-correct — Disables enhanced timestamp correction. This is the default.

---

🍓 EXIT STATUS 🍓

scout exits with status 0 if automatic trace analysis was successful, and greater than 0 if errors occur. simple as that. 💖

---

🫶 NOTES 🫶

scout poses a number of requirements on the input event trace data, which are documented in the OPEN_ISSUES file installed as part of the Scalasca Trace Tools documentation. It is also available online at https://www.scalasca.org. If those requirements are not met, scout may abort, deadlock, or crash. like. not ideal. 😭

If scout crashes or deadlocks even though the documented requirements are met (which usually indicates a bug), restricting the scope of the analysis by disabling certain features (e.g., critical-path and/or root-cause analysis) may help as a workaround. In any case, please report such issues for further investigation (see BUGS below). we need the receipts.

---

💀 BUGS 💀

Please report bugs to scalasca@fz-juelich.de. Make sure to include at least the following information in your bug report:

• The Scalasca Trace Tools version reported by scalasca -V.
• The Scalasca Trace Tools configuration reported by scalasca -c.
• The Score-P version reported by scorep --version.
• The Score-P configuration reported by scorep-info config-summary.
• The exact command line of the failing command.
• The exact failure/error message.

Also, please archive a copy of the entire experiment archive directory including the event trace data, as this may be required to aid in debugging. However, ONLY PROVIDE TRACE DATA IF EXPLICITLY REQUESTED, as the data volume may be excessive.

🔥 that all-caps is not me being dramatic, that's the actual rule. do not send the trace data unless they ask. the volume is, and I quote, excessive.

---

⭐ EXAMPLES ⭐

scout.omp scorep_foo_Ox4_trace/traces.otf2

Perform the Scalasca OpenMP event trace analysis on the OTF2 event trace with anchor file scorep_foo_Ox4_trace/traces.otf2.

mpiexec -n 16 scout.mpi --time-correct scorep_bar_16_trace/traces.otf2

Apply the enhanced timestamp correction and perform the Scalasca MPI event trace analysis on the OTF2 event trace with anchor file scorep_bar_16_trace/traces.otf.

mpiexec -n 4 scout.hyb epik_foobar_4x4_trace

Perform the hybrid Scalasca MPI+OpenMP event trace analysis on the EPILOG event trace in the experiment archive epik_foobar_4x4_trace generated by the Scalasca 1.x release series.

---

🌈 SEE ALSO 🌈

scalasca(1), scan(1), square(1)

The full Scalasca Trace Tools documentation is available online at https://www.scalasca.org.

---

and that's the whole page, bestie 💖 go analyze some traces, you're gonna eat. mwah 🫶✨

Version 2.6.2 — dated Wed Apr 2 2025 — straight from the Scalasca Trace Tools, scout(1) out. 💅