SISO(1)

πŸ“¦ version 1.5.22, in ubuntu stonking

πŸ’… siso β€” SISO(1) πŸ’…

General Commands Manual Β· periodt, bestie, let's get into it


πŸŽ€ NAME πŸŽ€

siso β€” Ninja-compatible build system optimized for remote execution.


✨ SYNOPSIS ✨

siso [global flags] command [arguments]

πŸ¦‹ DESCRIPTION πŸ¦‹

Ok so picture this: siso is a build tool whose whole entire personality is making Chromium and Android builds go zoomie. It's built as a drop-in replacement for Ninja, which means you literally do not have to touch your workflow, change a single habit, or text your team "hey we're migrating" β€” it just slots right in. Slay.

siso runs build actions natively on Remote Build Execution (RBE) and squeezes out performance by dodging stat, disk, and network I/O as much as it physically can. It keeps resource consumption low by sharing a single process memory space, and it hoards detailed performance metrics for every build action so you can go analyze them with cloud tracing and profiling tools. Metrics girlie behavior fr fr. 🌈


🌈 GLOBAL FLAGS 🌈

  • -alsologtostderr=false β€” log to standard error as well as files
  • -blockprof_rate=0 β€” block profile rate
  • -blockprofile β€” write block profile to this file
  • -cpuprofile β€” write cpu profile to this file
  • -credential_helper=google-application-default β€” path to a credential helper. The lore lives here: https://github.com/EngFlow/credential-helper-spec/blob/main/spec.md
    • luci-auth uses luci-auth.
    • gcloud uses gcloud.
    • google-application-default or "" uses Google Application Default Credentials.
    • mTLS disables per RPC credentials.
    • environment variable SISO_CREDENTIAL_HELPER sets the default value.
  • -log_backtrace_at β€” when logging hits line file:N, emit a stack trace
  • -log_dir β€” If non-empty, write log files in this directory
  • -log_link β€” If non-empty, add symbolic links in this directory to the log files
  • -logbuflevel=0 β€” Buffer log messages logged at this level or lower (-1 means don't buffer; 0 means buffer INFO only; ...). Has limited applicability on non-prod platforms.
  • -logtostderr=false β€” log to standard error instead of files
  • -memprofile β€” write memory profile to this file
  • -memprofile_gc=true β€” run runtime.GC() before each heap profile dump for a cleaner inuse view
  • -memprofile_interval=0s β€” if non-zero, dump heap profiles at this interval to rolling files derived from -memprofile (e.g. siso-memory.0001.pprof); the bare -memprofile path is not written. yes she said rolling files, she's extra like that
  • -mutexprof_frac=0 β€” mutex profile fraction
  • -mutexprofile β€” write mutex profile to this file
  • -pprof_addr β€” listen address for go tool pprof. e.g. localhost:6060
  • -stderrthreshold=2 β€” logs at or above this threshold go to stderr
  • -trace β€” go trace output for go tool trace
  • -v=0 β€” log level for V logs
  • -version=false β€” print version
  • -vmodule β€” comma-separated list of pattern=N settings for file-filtered logging

🧚 SUBCOMMANDS 🧚

πŸ‘‘ MAIN COMMANDS πŸ‘‘

πŸ₯· ninja β€” build the requested targets as ninja

build the requested targets as ninja. She does exactly what it says on the tin, no notes. πŸ’…

$ siso ninja [-C <dir>] [options] [targets...]

Flags:

  • -C=. β€” ninja running directory (chdir before run)
  • -action_salt β€” action salt
  • -batch=true β€” batch mode. prefer thoughput over low latency for build failures. disable -fast_nop, -fast_local -fast_last_failure -fast_exit
  • -build_id=00000000-0000-0000-0000-000000000000 β€” ID for the build. used for invocation_id of remote-apis-sdks and tool_invocation_id of remote-apis, and Cloud logging resource build_id label.
  • -build_pprof=siso_build.pprof β€” build pprof filename (relative to -log_dir)
  • -cache_dir=$HOME/.cache/siso β€” cache directory
  • -cache_enable_read=true β€” cache enable read
  • -cartfs_endpoint β€” cartfs server endpoint. e.g. localhost:65001
  • -cleandead=false β€” clean built files that are no longer produced by the manifest
  • -clobber=false β€” clobber build
  • -cloud_profiler_service_name=siso β€” cloud profiler service name
  • -collector_address β€” address to dial the collector. Can be path for unix socket unix:///path/to/socket or host:port.
  • -config β€” config name passed to starlark
  • -config_repo_dir=build/config/siso β€” config repo directory (relative to workspace)
  • -d β€” enable debugging (use -d list to list modes)
  • -deps_log=.siso_deps β€” deps log filename (relative to -C, -state_dir)
  • -enable_build_ninja_files_upload=false β€” enable Build Ninja files upload to RBE-CAS
  • -enable_cloud_logging=false β€” enable cloud logging
  • -enable_cloud_monitoring=false β€” enable cloud monitoring
  • -enable_cloud_profiler=false β€” enable cloud profiler
  • -enable_cloud_trace=false β€” enable cloud trace
  • -enable_collector=false β€” enable OTEL collector. Effectively not active, will be removed later. TODO: b/455433899.
  • -enable_resultstore=false β€” enable resultstore
  • -explain_log=siso_explain β€” explain log filename (relative to -log_dir)
  • -f=build.ninja β€” input build manifest filename (relative to -C)
  • -failed_commands=siso_failed_commands.sh β€” script file to rerun the last failed commands
  • -failure_summary β€” filename for failure summary (relative to -log_dir)
  • -fast_exit=false β€” enable fast exit
  • -fast_last_failure=false β€” enable fast last failure check
  • -fast_local=false β€” enable fast local
  • -fast_nop=false β€” enable fast nop check
  • -frontend_file β€” frontend FIFO file to report build status to soong ui, or - to report to stdout.
  • -fs_defer_digest=false β€” defer digest calculation
  • -fs_digest_xattr β€” xattr for sha256 digest; empty enables the default on Cog/CartFS only
  • -fs_keep_tainted=false β€” keep manually modified generated file
  • -fs_min_flush_timeout=30s β€” minimum timeout for flush. ignored if it is shorter than 10s
  • -fs_state=.siso_fs_state β€” fs state filename
  • -fs_state_compression_level=1 β€” fs state compression level (1 = fastest, 10 = best)
  • -fs_state_compression_threads=8 β€” number of threads to use for data compression
  • -fs_state_mmap=true β€” use memory-mapped I/O for state file reads/writes
  • -heartbeat_period=0s β€” print a heartbeat with this frequency on the console when --quiet is set and this value is non zero.
  • -invocation_json=siso_metadata.json β€” invocation metadata JSON filename (relative to -log_dir)
  • -j=-1 β€” not supported. use -remote_jobs and -local_jobs instead
  • -job_id=00000000-0000-0000-0000-000000000000 β€” ID for a grouping of related builds such as a Buildbucket job. used for correlated_invocations_id of remote-apis and remote-apis-sdks, and Cloud logging resource job_id label.
  • -k=1 β€” keep going until N jobs fail (0 means inifinity)
  • -l=-1 β€” not supported.
  • -load=@config//main.star β€” config filename (@config// is --config_repo_dir)
  • -local_cache_enable=false β€” local cache enable
  • -local_jobs=0 β€” run N local jobs in parallel. when the value is no positive, the default will be computed based on # of CPUs.
  • -localexec_log=siso_localexec β€” localexec log filename (relative to -log_dir)
  • -log_dir β€” log directory (relative to -C) [default: same dir as build.ninja]
  • -metrics_json=siso_metrics.json β€” metrics JSON filename (relative to -log_dir)
  • -metrics_labels β€” comma-separated arbitrary key value pairs in the form key=value, which are added to cloud monitoring metrics and siso_metadata.json.
  • -metrics_project β€” override Cloud Monitoring GCP project where Siso sends action and build metrics.
  • -n=false β€” dry run
  • -namespace β€” namespace for cloud logging's resource label
  • -o=false β€” alias of -offline
  • -offline=false β€” offline mode.
  • -output_local_strategy=full β€” strategy for output_local. "full": download all outputs. "greedy": downloads most outputs except intermediate objs. "minimum": downloads as few as possible. "graph": downloads only remote outputs that a local step will consume, decided at schedule time from the dep graph, spreading downloads over the compile burst. "pathfilter:<path_filter>": use path filter. "patterns:<pattern-list>": colon-separated pathfilter's patterns
  • -output_log=siso_output β€” output log filename (relative to -log_dir)
  • -prepare=false β€” build inputs of targets, but not build target itself.
  • -project β€” cloud project ID. can set by $SISO_PROJECT
  • -quiet=false β€” don't show progress status, just command output
  • -re_cache_enable_read=true β€” remote exec cache enable read
  • -re_cache_enable_write=false β€” remote exec cache allow local trusted uploads
  • -re_exec_enable=true β€” remote exec enable
  • -reapi_address=remotebuildexecution.googleapis.com:443 β€” reapi address
  • -reapi_batch_compress_blob=0 β€” use compressed blobs in BatchUpdateBlobs if server supports it and size is bigger than this. specify 0 to disable.
  • -reapi_byte_stream_read_threshold=2097152 β€” if blob size >= threshold, use ByteStream API (compression-aware)
  • -reapi_cas_address β€” reapi cas address (if empty, share conn with reapi_address)
  • -reapi_compress_blob=1024 β€” use compressed blobs if server supports compressed blobs and size is bigger than this. specify 0 to disable blob-level compression.
  • -reapi_digest_function=sha256 β€” which digest function we use for content. pick from: sha256, sha1, gitsha1, blake3, md5, sha384, sha512, murmur3, vso, or sha256tree ✨
  • -reapi_enable_grpc_compression=false β€” turn on grpc compression. heads up bestie: if you enable this, blob-level compression gets forcibly disabled. no cap 🫢
  • -reapi_grpc_conn_pool=25 β€” the grpc connection pool πŸ’…
  • -reapi_grpc_keepalive_permit_without_stream=false β€” grpc keepalive permit without stream
  • -reapi_grpc_keepalive_time=30s β€” grpc keepalive time
  • -reapi_grpc_keepalive_timeout=20s β€” grpc keepalive timeout
  • -reapi_insecure=false β€” reapi insecure mode. the default can be set by $RBE_service_no_security πŸ‘€
  • -reapi_instance=default_instance β€” the reapi instance name. (for Google RBE: if the instance is fully qualified (starts with projects/), the project ID gets inferred from it. otherwise, the project ID from -project (or $SISO_PROJECT) is used to construct the instance name. if both are provided, -project is used for other cloud services) πŸ’–
  • -reapi_keep_exec_stream=false β€” keep the Execute stream open as long as possible
  • -reapi_max_retries=10 β€” max retries for remote execution
  • -reapi_priority=0 β€” reapi priority for action executions. the semantics and supported values depend on the backend πŸ€·β™€οΈ
  • -reapi_tls_ca_cert β€” load TLS CA certificates from this file to connect to the RE api service. default can be set by $RBE_tls_ca_cert
  • -reapi_tls_client_auth_cert β€” certificate to use when using mTLS to connect to the RE api service. default can be set by $RBE_tls_client_auth_cert
  • -reapi_tls_client_auth_key β€” key to use when using mTLS to connect to the RE api service. default can be set by $RBE_tls_client_auth_key
  • -reapi_trace_cookie β€” if set, it's sent as the cookie gRPC metadata header on every RPC to this backend, to force server-side trace sampling (e.g. Dapper) πŸ”
  • -reapi_version_to_use β€” specify the re api version to use, in the format v<major>.<minor>. e.g. v2.0
  • -remote_jobs=0 β€” run N remote jobs in parallel. when the value is not positive, the default gets computed based on # of CPUs πŸ”₯
  • -spawn_helper β€” command name of external spawn helper. use builtin spawn helper if empty.
  • -state_dir β€” state directory (relative to -C) [default: same dir as build.ninja]
  • -strict_remote=false β€” don't use local for remote step. i.e. no fastlocal, no local fallback 🚫
  • -t β€” run a subtool (use -t list to list subtools)
  • -trace_json=siso_trace.json β€” trace JSON filename (relative to -log_dir)
  • -trace_span_threshold=100ms β€” theshold for trace span record
  • -trace_threshold=1m0s β€” threshold for trace record
  • -v=false β€” show all command lines while building (alias of --verbose)
  • -verbose=false β€” show all command lines while building
  • -verbose_failures=true β€” show failed command lines
  • -w β€” adjust warnings. not supported b/288807840 😭
  • -write_reclient_metrics_logs=false β€” write Reclient's RBE build metrics to rbe_metrics.{txt, pb} under -log_dir.

🌈 REAPI 🌈

πŸ¦‹ fetch β€” fetch contents

babes, this one fetches contents from CAS or ActionResult. it prints contents to stdout, or extracts into <dir> for -type *-extract πŸ’–

$ siso fetch [-project <project>] [-reapi_instance <instance>] \
         [-type <type>] \
         <digest> [<dir>]
$ siso fetch [-type <type>] \
 bytesteam://<endpoint>/projects/<project>/instances/<instance>/blobs/<digest>

Flags ✨:

  • -project β€” cloud project ID. can be set by $SISO_PROJECT
  • -reapi_address=remotebuildexecution.googleapis.com:443 β€” reapi address
  • -reapi_batch_compress_blob=0 β€” use compressed blobs in BatchUpdateBlobs if the server supports it and the size is bigger than this. specify 0 to disable.
  • -reapi_byte_stream_read_threshold=2097152 β€” if blob size >= threshold, use the ByteStream API (compression-aware)
  • -reapi_cas_address β€” reapi cas address (if empty, share conn with reapi_address)
  • -reapi_compress_blob=1024 β€” use compressed blobs if the server supports compressed blobs and the size is bigger than this. specify 0 to disable blob-level compression.
  • -reapi_digest_function=sha256 β€” content digest function: sha256, sha1, gitsha1, blake3, md5, sha384, sha512, murmur3, vso, or sha256tree
  • -reapi_enable_grpc_compression=false β€” enable grpc compression. if enabled, blob-level compression will be forcibly disabled. πŸ’€
  • -reapi_grpc_conn_pool=25 β€” grpc connection pool
  • -reapi_grpc_keepalive_permit_without_stream=false β€” grpc keepalive permit without stream
  • -reapi_grpc_keepalive_time=30s β€” grpc keepalive time
  • -reapi_grpc_keepalive_timeout=20s β€” grpc keepalive timeout
  • -reapi_insecure=false β€” reapi insecure mode. default can be set by $RBE_service_no_security
  • -reapi_instance=default_instance β€” reapi instance name. (for Google RBE: if instance is fully qualified (starts with projects/), project ID is inferred from it. Otherwise, project ID from -project (or $SISO_PROJECT) is used to construct the instance name. If both are provided, -project is used for other cloud services)
  • -reapi_keep_exec_stream=false β€” keep Execute stream open as long as possible
  • -reapi_max_retries=10 β€” max retries for remote execution
  • -reapi_priority=0 β€” reapi priority for action executions. The semantics and supported values depend on the backend
  • -reapi_tls_ca_cert β€” Load TLS CA certificates from this file to connect to the RE api service. default can be set by $RBE_tls_ca_cert
  • -reapi_tls_client_auth_cert β€” Certificate to use when using mTLS to connect to the RE api service. default can be set by $RBE_tls_client_auth_cert
  • -reapi_tls_client_auth_key β€” Key to use when using mTLS to connect to the RE api service. default can be set by $RBE_tls_client_auth_key
  • -reapi_trace_cookie β€” if set, sent as the cookie gRPC metadata header on every RPC to this backend, to force server-side trace sampling (e.g. Dapper)
  • -reapi_version_to_use β€” specify re api version to use, in format of v<major>.<minor>. e.g. v2.0
  • -type=raw β€” data type. here's the whole menu πŸ“:
    • raw: raw content
    • command: command message in text proto format
    • action: action message in text proto format
    • result: action result message in text proto format
    • dir: directory message in text proto format
    • tree: tree message in text proto format
    • dir-extract: directory message extract to <dir> (if <dir> is specified) or list (if <dir> is not specified)
    • tree-extract: tree message extract to <dir> (if <dir> is specified) or list (if <dir> is not specified)
    • action-list: actions associated with two phase cache key.

🧚 isolate β€” isolate uploads and computes tree digests

isolate uploads and computes tree digest for each targets πŸ’…

$ siso isolate [-project <project>] [-src_cas_instance <instance>] \
    -C <dir> \
    -dst_cas_instance projects/<cas project>/instances/<instance> \
    -dump_json <output json path> \
    <target> ...

Flags ✨:

  • -C=. β€” ninja running directory (chdir before run)
  • -cas_instance β€” alias of -dst_cas_instance for backward compatibility πŸ‘‘
  • -config_repo_dir=build/config/siso β€” config repo directory (relative to workspace)
  • -dst_cas_address=remotebuildexecution.googleapis.com:443 β€” reapi address (for dst_cas)
  • -dst_cas_batch_compress_blob=0 β€” use compressed blobs in BatchUpdateBlobs if server supports it and size is bigger than this. specify 0 to disable. (for dst_cas)
  • -dst_cas_byte_stream_read_threshold=2097152 β€” if blob size >= threshold, use ByteStream API (compression-aware) (for dst_cas)
  • -dst_cas_compress_blob=1024 β€” use compressed blobs if server supports compressed blobs and size is bigger than this. specify 0 to disable blob-level compression. (for dst_cas)
  • -dst_cas_digest_function=sha256 β€” content digest function: sha256, sha1, gitsha1, blake3, md5, sha384, sha512, murmur3, vso, or sha256tree (for dst_cas)
  • -dst_cas_enable_grpc_compression=false β€” enable grpc compression. if enabled, blob-level compression will be forcibly disabled. (for dst_cas)
  • -dst_cas_grpc_conn_pool=25 β€” grpc connection pool
  • -dst_cas_grpc_keepalive_permit_without_stream=false β€” grpc keepalive permit without stream (for dst_cas)
  • -dst_cas_grpc_keepalive_time=30s β€” grpc keepalive time (for dst_cas)
  • -dst_cas_grpc_keepalive_timeout=20s β€” grpc keepalive timeout (for dst_cas)
  • -dst_cas_insecure=false β€” reapi insecure mode. default can be set by $RBE_service_no_security
  • -dst_cas_instance=default_instance β€” reapi instance name (for dst_cas)
  • -dst_cas_keep_exec_stream=false β€” keep Execute stream open as long as possible
  • -dst_cas_max_retries=10 β€” max retries for remote execution
  • -dst_cas_tls_ca_cert β€” Load TLS CA certificates from this file to connect to the RE api service. default can be set by $RBE_tls_ca_cert
  • -dst_cas_tls_client_auth_cert β€” the certificate you hand over when you're doing mTLS to talk to the RE api service. default can be set by $RBE_tls_client_auth_cert.
  • -dst_cas_tls_client_auth_key β€” the key that matches that cert for mTLS to the RE api service. default can be set by $RBE_tls_client_auth_key.
  • -dst_cas_trace_cookie β€” if you set it, it gets sent as the cookie gRPC metadata header on every single RPC to this backend, which forces server-side trace sampling (think Dapper). for dst_cas, bestie πŸ“
  • -dst_cas_version_to_use β€” pick which re api version you wanna use, format is v<major>.<minor>. like v2.0. yes the angle brackets are literally part of the shape, don't @ me πŸ’…
  • -dump_json β€” dump it into a json file πŸ“„
  • -enable_cloud_logging=true β€” turn cloud logging on ✨
  • -fs_defer_digest=false β€” defer the digest calculation
  • -fs_digest_xattr β€” the xattr for the sha256 digest. leave it empty and the default only kicks in on Cog/CartFS
  • -fs_keep_tainted=false β€” keep that generated file you manually modified (we see you πŸ‘€)
  • -fs_min_flush_timeout=30s β€” minimum timeout for flush. heads up: this one is ignored if it's shorter than 10s
  • -fs_state=.siso_fs_state β€” the fs state filename
  • -fs_state_compression_level=1 β€” fs state compression level, where 1 = fastest and 10 = best πŸ†
  • -fs_state_compression_threads=8 β€” how many threads to throw at data compression
  • -fs_state_mmap=true β€” use memory-mapped I/O for state file reads/writes
  • -job_id=00000000-0000-0000-0000-000000000000 β€” an ID for a group of related builds, like a Buildbucket job
  • -namespace β€” namespace for cloud logging's resource label
  • -project β€” the cloud project ID. can be set by $SISO_PROJECT
  • -src_cas_address=remotebuildexecution.googleapis.com:443 β€” the reapi address. for src_cas, obviously πŸ’
  • -src_cas_batch_compress_blob=0 β€” use compressed blobs in BatchUpdateBlobs if the server supports it and the size is bigger than this. specify 0 to disable. for src_cas
  • -src_cas_byte_stream_read_threshold=2097152 β€” if the blob size is >= the threshold, use the ByteStream API (compression-aware). for src_cas
  • -src_cas_compress_blob=1024 β€” use compressed blobs if the server supports compressed blobs and the size is bigger than this. specify 0 to disable blob-level compression. for src_cas
  • -src_cas_digest_function=sha256 β€” the content digest function, and your options are: sha256, sha1, gitsha1, blake3, md5, sha384, sha512, murmur3, vso, or sha256tree. for src_cas 🌈
  • -src_cas_enable_grpc_compression=false β€” enable grpc compression. but wait: if you enable this, blob-level compression gets forcibly disabled. for src_cas
  • -src_cas_grpc_conn_pool=25 β€” grpc connection pool
  • -src_cas_grpc_keepalive_permit_without_stream=false β€” grpc keepalive permit without stream. for src_cas
  • -src_cas_grpc_keepalive_time=30s β€” grpc keepalive time. for src_cas
  • -src_cas_grpc_keepalive_timeout=20s β€” grpc keepalive timeout. for src_cas
  • -src_cas_insecure=false β€” reapi insecure mode. default can be set by $RBE_service_no_security
  • -src_cas_instance=default_instance β€” reapi instance name. for src_cas
  • -src_cas_keep_exec_stream=false β€” keep the Execute stream open for as long as humanly possible 🫢
  • -src_cas_max_retries=10 β€” max retries for remote execution
  • -src_cas_tls_ca_cert β€” load TLS CA certificates from this file when connecting to the RE api service. default can be set by $RBE_tls_ca_cert
  • -src_cas_tls_client_auth_cert β€” the certificate you use when doing mTLS to the RE api service. default can be set by $RBE_tls_client_auth_cert
  • -src_cas_tls_client_auth_key β€” the key for that mTLS connection to the RE api service. default can be set by $RBE_tls_client_auth_key
  • -src_cas_trace_cookie β€” if set, it goes out as the cookie gRPC metadata header on every RPC to this backend to force server-side trace sampling (like Dapper). for src_cas
  • -src_cas_version_to_use β€” specify which re api version to use, in the format v<major>.<minor>. e.g. v2.0

🧚 proxy β€” proxy RE-API service

okay so this one literally proxies the RE API service. iconic behaviour honestly πŸ‘‘

 $ siso proxy [-project <project>] [-reapi_address <addr>] [-reapi_instance <instance>] \
     -addr unix:///<path>

 $ siso ninja -reapi_address unix:///<path> --reapi_insecure=true ...

✨ Flags ✨

  • -addr β€” the address to listen on
  • -project β€” the cloud project ID. can be set by $SISO_PROJECT
  • -reapi_address=remotebuildexecution.googleapis.com:443 β€” the reapi address
  • -reapi_batch_compress_blob=0 β€” use compressed blobs in BatchUpdateBlobs if the server supports it and the size is bigger than this. specify 0 to disable.
  • -reapi_byte_stream_read_threshold=2097152 β€” if the blob size is >= the threshold, use the ByteStream API (compression-aware)
  • -reapi_cas_address β€” the reapi cas address. if it's empty, it shares the conn with reapi_address
  • -reapi_compress_blob=1024 β€” use compressed blobs if the server supports compressed blobs and the size is bigger than this. specify 0 to disable blob-level compression.
  • -reapi_digest_function=sha256 β€” the content digest function: sha256, sha1, gitsha1, blake3, md5, sha384, sha512, murmur3, vso, or sha256tree
  • -reapi_enable_grpc_compression=false β€” enable grpc compression. but: if you enable it, blob-level compression gets forcibly disabled.
  • -reapi_grpc_conn_pool=25 β€” grpc connection pool
  • -reapi_grpc_keepalive_permit_without_stream=false β€” grpc keepalive permit without stream
  • -reapi_grpc_keepalive_time=30s β€” grpc keepalive time
  • -reapi_grpc_keepalive_timeout=20s β€” grpc keepalive timeout
  • -reapi_insecure=false β€” reapi insecure mode. default can be set by $RBE_service_no_security
  • -reapi_instance=default_instance β€” reapi instance name. for Google RBE specifically: if the instance is fully qualified (starts with projects/), the project ID gets inferred from it. otherwise, the project ID from -project (or $SISO_PROJECT) is used to construct the instance name. and if you give both, -project is used for the other cloud services πŸ’«
  • -reapi_keep_exec_stream=false β€” keep the Execute stream open as long as possible
  • -reapi_max_retries=10 β€” max retries for remote execution
  • -reapi_priority=0 β€” reapi priority for action executions. the semantics and the supported values are up to the backend, so like... ask your backend 🫠
  • -reapi_tls_ca_cert β€” load TLS CA certificates from this file to connect to the RE api service. default can be set by $RBE_tls_ca_cert
  • -reapi_tls_client_auth_cert β€” the certificate to use when connecting to the RE api service with mTLS. default can be set by $RBE_tls_client_auth_cert
  • -reapi_tls_client_auth_key β€” the key to use when connecting to the RE api service with mTLS. default can be set by $RBE_tls_client_auth_key
  • -reapi_trace_cookie β€” if set, it's sent as the cookie gRPC metadata header on every RPC to this backend, forcing server-side trace sampling (e.g. Dapper)
  • -reapi_version_to_use β€” specify which re api version to use, in the format v<major>.<minor>. e.g. v2.0

πŸ¦‹ recall β€” recall action

recall an action by digest, or do a remote exec call to run it. it's giving time travel but for builds βͺ

to recall the action identified by <digest> in <dir>:

 $ siso recall [-project <project>] [-reapi_instance <instance>] \
     <dir> <digest>

inside <dir> you'll get "action.txt", "command.txt" and "root/".

and yes bestie, you may modify *.txt or the files in root/ 🌸

if you don't specify <digest>, it just runs the action in <dir>. and you can totally omit <dir> if it's the current directory ".". so convenient πŸ’…

  • to issue a remote exec call to run the command:
 $ siso recall [-project <project>] [-reapi_instance <instance>] <dir>
  • to re-run the command on a remote worker without fetching cache:
 $ siso recall [-project <project>] [-reapi_instance <instance>] -re_cache_enable_read=false <dir>
  • to use local docker to run the command:
 $ siso recall [-local] <dir>

✨ Flags ✨

  • -cpus β€” how much of the available CPU resources the action can use (e.g. '1.5' for at most one and a half of the CPUs)
  • -exec_timeout=0s β€” set the exec call timeout
  • -execute_request β€” execute request proto
  • -hdd=false β€” run the action with slowed-down I/O that resembles a hard-disk with 12MB/s throughput, 75 read IOPS and 150 write IOPS 🐌
  • -local=false β€” force running the action locally using Docker, even if REAPI is configured
  • -memory β€” the maximum amount of memory the action can use (e.g. 512m or 2g)
  • -project β€” the cloud project ID. can be set by $SISO_PROJECT
  • -re_cache_enable_read=true β€” remote exec cache enable read
  • -reapi_address=remotebuildexecution.googleapis.com:443 β€” the reapi address
  • -reapi_batch_compress_blob=0 β€” use compressed blobs in BatchUpdateBlobs if the server supports it and the size is bigger than this. specify 0 to disable.
  • -reapi_byte_stream_read_threshold=2097152 β€” if the blob size is >= the threshold, use the ByteStream API (compression-aware)
  • -reapi_cas_address β€” the reapi cas address. if it's empty, it shares the conn with reapi_address
  • -reapi_compress_blob=1024 β€” use compressed blobs if the server supports compressed blobs and the size is bigger than this. specify 0 to disable blob-level compression.
  • -reapi_digest_function=sha256 β€” the content digest function: sha256, sha1, gitsha1, blake3, md5, sha384, sha512, murmur3, vso, or sha256tree
  • -reapi_enable_grpc_compression=false β€” turn grpc compression on. heads up bestie: the moment you flip this to enabled, blob-level compression gets forcibly disabled. it's giving either/or, no cap. 😀
  • -reapi_grpc_conn_pool=25 β€” grpc connection pool size.
  • -reapi_grpc_keepalive_permit_without_stream=false β€” grpc keepalive permit-without-stream toggle.
  • -reapi_grpc_keepalive_time=30s β€” grpc keepalive time. πŸ•°οΈ
  • -reapi_grpc_keepalive_timeout=20s β€” grpc keepalive timeout.
  • -reapi_insecure=false β€” reapi insecure mode. your default can be set by $RBE_service_no_security instead of typing it every time. πŸ’…
  • -reapi_instance=default_instance β€” the reapi instance name. for Google RBE there's a lil logic going on here: if the instance is fully qualified (starts with projects/), the project ID is inferred from it. otherwise, the project ID from -project (or $SISO_PROJECT) is used to build the instance name. and if you give both, then -project is what gets used for the other cloud services. okurrr. ✨
  • -reapi_keep_exec_stream=false β€” keep the Execute stream open as long as possible. clingy in the best way. 🫢
  • -reapi_max_retries=10 β€” max retries for remote execution.
  • -reapi_priority=0 β€” reapi priority for action executions. the semantics and supported values are fully up to the backend, so it's giving "iykyk". ✨
  • -reapi_tls_ca_cert β€” load TLS CA certificates from this file to connect to the RE api service. default can be set by $RBE_tls_ca_cert.
  • -reapi_tls_client_auth_cert β€” the certificate to use when you're doing mTLS to connect to the RE api service. default can be set by $RBE_tls_client_auth_cert.
  • -reapi_tls_client_auth_key β€” the key to use when you're doing mTLS to connect to the RE api service. default can be set by $RBE_tls_client_auth_key.
  • -reapi_trace_cookie β€” if you set this, it gets sent as the cookie gRPC metadata header on every RPC to this backend, which forces server-side trace sampling (think Dapper). πŸͺ
  • -reapi_version_to_use β€” pick which re api version to use, formatted like v<major>.<minor>, e.g. v2.0.
  • -stats=false β€” run the command under /usr/bin/time and print detailed resource stats once it's done. note: this may fail if the container's glibc is incompatible with the host. πŸ’€

✨ INVESTIGATION ✨

πŸ” fs β€” command group to access siso hashfs data

command group to access siso hashfs data. 🌸

just run siso fs to see all the subcommands. and for the tea on any one of them, use siso fs help [subcommand]. 🫢

πŸ“Š metrics β€” command group to analyze siso_metrics.json

command group to analyze siso_metrics.json.

run siso metrics to display the subcommands, and siso metrics help [subcommand] for more info about a specific subcommand. ✨

πŸ‘€ ps β€” display running steps of ninja build

display the running steps of a ninja build. literally the main character of "what is my build doing rn". πŸ‘‘

for a local build:

$ siso ps [-C dir]

for builders build:

$ siso ps --stdout_url <compile-step-stdout-URL>

flags:

  • -C=. β€” ninja running directory (chdir before run).
  • -config_repo_dir=build/config/siso β€” config repo directory (relative to the workspace).
  • -interval=-1ns β€” query interval, but only if it's positive. by default it's 1s on a terminal. ⏱️
  • -n=0 β€” limit the number of steps, if it's positive.
  • -state_dir=. β€” state directory (relative to -C).
  • -stdout_url β€” stdout streaming URL. 🌈

🧐 query β€” command group to query ninja build graph

command group to query ninja build graph.

use siso query to display subcommands, and siso query help [subcommand] when you need more info about one of them. πŸ’œ

πŸ“ report β€” report siso logs

report siso logs. it collects siso logs in <dir>. πŸŽ€

$ siso report -C <dir>

flags:

  • -C=. β€” ninja running directory (chdir before run).
  • -config_repo_dir=build/config/siso β€” config repo directory (relative to the workspace).
  • -fs_digest_xattr β€” xattr for sha256 digest; an empty value enables the default on Cog/CartFS only. πŸ“

🌐 webui β€” starts the experimental webui

starts the experimental webui. bestie, listen: it is not ready for wide use yet, it requires static files to work, and it is subject to breaking changes at any moment. this one is giving "come back later" energy fr. 🚧

flags:

  • -C=. β€” ninja running directory (chdir before run).
  • -config_repo_dir=build/config/siso β€” config repo directory (relative to the workspace).
  • -f=build.ninja β€” input build manifest filename (relative to -C).
  • -local_development=false β€” whether to use local files instead of embedded files.
  • -metrics_file β€” optional path to siso_metrics.json to load (experimental, -C is still required for now).
  • -port=8080 β€” port to use (defaults to 8080). πŸšͺ

πŸ” AUTH πŸ”

βœ… auth-check β€” prints current auth status

prints current auth status. let's see who you are bestie. πŸ’…

flags:

  • -check_reapi=true β€” check reapi call.
  • -project β€” cloud project ID. can be set by $SISO_PROJECT.
  • -read_only=false β€” check for read only access.
  • -reapi_address=remotebuildexecution.googleapis.com:443 β€” reapi address.
  • -reapi_batch_compress_blob=0 β€” use compressed blobs in BatchUpdateBlobs if the server supports it and the size is bigger than this. specify 0 to disable.
  • -reapi_byte_stream_read_threshold=2097152 β€” if blob size is >= this threshold, use the ByteStream API (compression-aware).
  • -reapi_cas_address β€” reapi cas address (if it's empty, share the conn with reapi_address).
  • -reapi_compress_blob=1024 β€” use compressed blobs if the server supports compressed blobs and the size is bigger than this. specify 0 to disable blob-level compression.
  • -reapi_digest_function=sha256 β€” content digest function: sha256, sha1, gitsha1, blake3, md5, sha384, sha512, murmur3, vso, or sha256tree.
  • -reapi_enable_grpc_compression=false β€” enable grpc compression. if enabled, blob-level compression will be forcibly disabled.
  • -reapi_grpc_conn_pool=25 β€” grpc connection pool.
  • -reapi_grpc_keepalive_permit_without_stream=false β€” grpc keepalive permit without stream.
  • -reapi_grpc_keepalive_time=30s β€” grpc keepalive time.
  • -reapi_grpc_keepalive_timeout=20s β€” grpc keepalive timeout.
  • -reapi_insecure=false β€” reapi insecure mode. default can be set by $RBE_service_no_security.
  • -reapi_instance=default_instance β€” reapi instance name. (for Google RBE: if instance is fully qualified (starts with projects/), project ID is inferred from it. otherwise, project ID from -project (or $SISO_PROJECT) is used to construct the instance name. if both are provided, -project is used for other cloud services.)
  • -reapi_keep_exec_stream=false β€” keep Execute stream open as long as possible.
  • -reapi_max_retries=10 β€” max retries for remote execution.
  • -reapi_priority=0 β€” reapi priority for action executions. the semantics and supported values depend on the backend.
  • -reapi_tls_ca_cert β€” load TLS CA certificates from this file to connect to the RE api service. default can be set by $RBE_tls_ca_cert.
  • -reapi_tls_client_auth_cert β€” certificate to use when using mTLS to connect to the RE api service. default can be set by $RBE_tls_client_auth_cert.
  • -reapi_tls_client_auth_key β€” key to use when using mTLS to connect to the RE api service. default can be set by $RBE_tls_client_auth_key.
  • -reapi_trace_cookie β€” if set, sent as the cookie gRPC metadata header on every RPC to this backend, to force server-side trace sampling (e.g. Dapper).
  • -reapi_version_to_use β€” specify re api version to use, in format of v<major>.<minor>. e.g. v2.0.

πŸͺͺ login β€” login to siso system

login to siso system. get in bestie, we're going in. ✨

πŸšͺ logout β€” logout from siso system

logout from siso system. bye babes! πŸ‘‹


πŸ› DEBUGGING πŸ›

🧡 ninjafrontend β€” ninjafrontend reader

ninjafrontend reader. parses the ninjafrontend proto. πŸ”¬

$ siso ninja -frontend_file=- .. | siso ninjafrontend -template '...'

e.g. show message only:

--template='{{- with .Message}}{{with .Message}}{{.}}{{end}}{{end}}'

show edge output only:

--template='{{- with .EdgeFinished}}{{with .Output}}{{.}}{{end}}{{end}}'

see the proto definition:

https://chromium.googlesource.com/infra/infra/go/src/infra/+/refs/heads/main/build/siso/toolsupport/soongutil/proto/frontend.proto

see template syntax: https://pkg.go.dev/text/template

flags:

  • -template={{.}} β€” template for frontend.Status message. 🧚

πŸ“¦ sandbox β€” run in sandbox

run in sandbox. πŸ–οΈ

$ siso sandbox -nsjail '<json nsjail request>' -- args ...

you can manually construct the json string of go.chromium.org/build/siso/toolsupport/nsjail.Request.

flags:

  • -C=. β€” ninja running directory (chdir before run).
  • -cleanup=true β€” cleanup sandbox after execution.
  • -config_repo_dir=build/config/siso β€” config repo directory (relative to the workspace).
  • -fs_defer_digest=false β€” defer digest calculation.
  • -fs_digest_xattr β€” xattr for sha256 digest; empty enables the default on Cog/CartFS only.
  • -fs_keep_tainted=false β€” keep manually modified generated file.
  • -fs_min_flush_timeout=30s β€” minimum timeout for flush. ignored if it's shorter than 10s.
  • -fs_state=.siso_fs_state β€” fs state filename.
  • -fs_state_compression_level=1 β€” fs state compression level (1 = fastest, 10 = best). πŸƒβ€β™€οΈ
  • -fs_state_compression_threads=8 β€” number of threads to use for data compression.
  • -fs_state_mmap=true β€” use memory-mapped I/O for state file reads/writes.
  • -nsjail β€” json format of nsjail request. πŸ”’

πŸ”Ž scandeps β€” run scandeps

run scandeps. πŸ•΅οΈ

...and the scandeps examples keep going, bestie, no cap:

$ siso scandeps -C <dir> -req '<json scandeps request>'
$ siso scandeps -C <dir> -target <build target name>
$ siso scandeps -C <dir> -- <command line>

That <json scandeps request>? Iconic shortcut alert πŸ’– you can find it sitting right there in the siso.INFO log, under "scandeps failed Request" β€” literally just copy-and-paste the json string straight out of the log. Chef's kiss. Or, if you're feeling main character energy, you can manually construct the json string of infra/build/siso/scandeps.Request. Your call, gorgeous 🫢

And the flags for this one:

  • -C=. β€” the ninja running directory (chdir happens before it runs)
  • -config_repo_dir=build/config/siso β€” config repo directory (and yeah, it's relative to the workspace)
  • -req β€” json format of a scandeps request
  • -state_dir=. β€” state directory (relative to -C, so pay attention ✨)
  • -target β€” build target name

πŸ› οΈ INTERNAL-HELPER

These are the behind-the-scenes girlies. They do the work so you don't have to πŸ’…

man

generates a man page for siso

man [-out <file>]:
  Generates a man page for siso and its subcommands in troff
format.

So it manifests a whole man page, in troff format, for siso and all its subcommands. Meta, right? πŸ¦‹

Flags:

  • -out β€” output file path (default is stdout, so if you don't say anything it just yeets it at you)

spawn-helper

internal: fork/exec server for local actions

It's an internal helper that spawns local build actions on behalf of siso. 🐣

Real talk: users do not run this directly, okay? No cap. siso launches it automatically for local actions, specifically so it doesn't have to fork() its big ol' large heap. She's efficient like that 😌

Flags:

  • -conn_fd=0 β€” inherited socketpair fd to serve the spawn protocol on
  • -log_file β€” file for the helper's diagnostics (default: stderr)

πŸ“– COMMAND-HELP

flags

describe all known top-level flags

flags [<subcommand>]:

With an argument, print all flags of that <subcommand>. Else, print a description of all known top-level flags. (And bestie, heads up: the basic help information only discusses the most generally important top-level flags. So flags is the full tea β˜•)

help

describe subcommands and their syntax

help [<subcommand>]:

With an argument, it prints detailed information on the use of the specified subcommand. With no argument, print a list of all commands and a brief description of each. It's giving ✨directory of everything✨

version

prints the executable version

Prints the executable version and the CIPD package the executable was installed from (if it was installed via CIPD, obviously β€” if it wasn't, don't expect it to manifest out of thin air πŸ’€).

Flags:

  • -cipd_url β€” show version info for this cipd URL.
  • -online=true β€” get information from CIPD and Git repo.

🌍 ENVIRONMENT

All the little env vars that can secretly be your bestie or your villain, depending on the day:

  • SISO_CREDENTIAL_HELPER β€” sets the default value of -credential_helper. It specifies a path to a credential helper program. See https://github.com/EngFlow/credential-helper-spec/blob/main/spec.md
  • SISO_PROJECT β€” sets the default value of --project, to specify Google Cloud Project.
  • SISO_REAPI_INSTANCE β€” sets the default value of -reapi_instance, to specify RE API instance name.
  • SISO_REAPI_ADDRESS β€” sets the default value of -reapi_address, to specify RE API service address.
  • SISO_REAPI_CAS_ADDRESS β€” sets the default value of -reapi_cas_address, to specify RE API content addressable storage service address.
  • SISO_LIMITS β€” can control siso's resource semaphore. Comma separated key value pairs in the form of key=value.
    • step: maximum number of concurrent steps.
    • preproc: maximum number of concurrent preprocesses.
    • scandeps: maximum number of concurrent scandeps.
    • local: maximum number of concurrent local executions.
    • fastlocal: maximum number of local executions when local is idle even if step is remote executable.
    • startlocal: maximum initial steps to run locally even if step is remote executable.
    • remote: maximum number of concurrent remote executions.
    • rewrap: maximum number of concurrent use_remote_exec_wrapper steps.
    • cache: maximum number of concurrent exec cache lookups.
    • thread: maximum number of threads.
    • e.g. SISO_LIMITS=local=10,remote=100
  • SISO_FSMONITOR β€” a path of file system monitoring tool. Currently watchman is supported. (experimental).
  • SISO_EXPERIMENTS β€” comma separated feature names.
  • SISO_BUILD_ID β€” sets the default value of --build_id, to identify the build process. Used for tool_invocation_id of remote apis, and build_id label of Cloud logging resources.
  • SISO_COLLECTOR_ADDRESS β€” sets the default value of --collector_address.
  • NO_COLOR β€” disable colored text output. See https://no-color.org/ (sad, but valid πŸ™ˆ)
  • TERM β€” used to detect smart terminal. "dumb" or "dumb-emacs-ansi" are not smart terminal.
  • RBE_service_no_security β€” sets the default value of -reapi_insecure, when using RE API with no authentication. (reclient compat)
  • RBE_tls_client_auth_cert, RBE_tls_client_auth_key β€” sets the default values of -reapi_tls_client_auth_cert and -reapi_tls_client_auth_key, when to use mTLS authentication. (reclient compat)
  • RBE_tls_ca_cert β€” sets the default value of -reapi_tls_ca_cert to specify Root CA Certificates file. (reclient compat)
  • NINJA_CORE_MULTIPLIER, NINJA_CORE_LIMIT β€” used to define default of rewrap's limit. (reclient compat)
  • RBE_remote_disabled β€” used to decide offline mode or not. (reclient compat)
  • RBE_metrics_project β€” sets the default value of -metrics_project, to specify Google Cloud Project for cloud monitoring. (reclient compat)
  • RBE_metrics_labels β€” sets the default value of -metrics_labels, which is comma separated arbitrary key value pairs in the form key=value to be added for cloud monitoring metrics. (reclient compat)

🐧 DEBIAN-SPECIFIC NOTES

Okay so the Debian package is not identical to the upstream CIPD-distributed binary β€” it differs in two ways, and we're being fully transparent about it because that's the vibe 🌈

Removed: OTel Collector subcommand

The collector subcommand (an internal-helper for Google's metrics infrastructure) is not included. It requires 84 Go modules not packaged in Debian and is only useful within Google Cloud infrastructure.

Removed: Direct Google Cloud Monitoring export

The direct export to Google Cloud Monitoring is not available. Build metrics can still be exported to any OpenTelemetry-compatible collector using the --collector_address flag with the ninja subcommand. This is the standard, vendor-neutral approach to metrics collection and works with Grafana, Jaeger, Prometheus, and other OTLP-compatible tools. ✨

Good news though: all core functionality (Ninja-compatible builds, local execution, Remote Build Execution via RBE, Starlark configuration) is fully preserved. She ate and left no crumbs πŸ’…

πŸ‘€ SEE ALSO

Full documentation is available at https://chromium.googlesource.com/build/+/refs/heads/main/siso/ 🌸


πŸ’– AUTHORS

The Chromium Authors.


And that's a wrap, bestie! πŸŽ€ This whole page is the Siso Manual, page SISO(1) β€” from the source at https://chromium.googlesource.com/build/+/refs/heads/main/siso/. Go build something iconic with it πŸ¦‹βœ¨πŸ’œ

πŸ“¦ where this page comes from