IPCONTROLLER(1)

✨ ipcontroller — the manual page bestie 💖

okay so this is IPCONTROLLER(1), from the User Commands section of the manual (section 1, the "you can literally just run this" section). buckle up, we're starting a whole parallel-computing cluster together 🫶

🌸 NAME 🌸

ipcontroller — starts an IPython cluster controller. that's it, that's the whole vibe. she's the one who starts the party 💅

🦋 DESCRIPTION 🦋

okay so you wanna do parallel computing? then you start the IPython controller. periodt.

The IPython controller provides a gateway between the IPython engines and clients. so think of her as the group chat admin who relays messages between everybody, no cap. The controller needs to be started before the engines — she's the main character, everyone else shows up after her 👑

and you can configure her using command line options, or using a cluster directory. Cluster directories contain config, log and security files and are usually located in your ipython directory and named as "profile_name". See the profile and profile-dir options for details — iykyk ✨

💅 Options 💅

Arguments that take values are actually convenience aliases to full Configurables, whose aliases are listed on the help line. For more information on full configurables, see --help-all. (yes bestie, there's a whole extended universe of configurables hiding behind that flag 🌈)

• --log-to-file — send log output to a file 📝
• --debug — set log level to logging.DEBUG (maximize logging output). she will NOT stop talking and honestly? we love that for her 💀
• --quiet — set log level to logging.CRITICAL (minimize logging output). the introvert setting 🙈
• --init — Initialize profile with default config files. This is equivalent to running ipython profile create <profile> prior to startup. like a lil starter pack 🎀
• --usethreads — Use threads instead of processes for the schedulers 🧵
• --sqlitedb — use the SQLiteDB backend
• --mongodb — use the MongoDB backend
• --dictdb — use the in-memory DictDB backend
• --nodb — use dummy DB backend, which doesn't store any information. This is the default as of IPython 0.13. ⭐ To enable delayed or repeated retrieval of results from the Hub, select one of the true db backends. (so if you need to come back for your results later, --nodb is not gonna eat, bestie 😭)
• --reuse — reuse existing json connection files ♻️
• --restore — Attempt to restore engines from a JSON file. For use when resuming a crashed controller. she's literally doing CPR on your cluster 🚑
• --secure — Use HMAC digests for authentication of messages. Setting this flag will generate a new UUID to use as the HMAC key. 🔐
• --no-secure — Don't authenticate messages. (lowkey unhinged of you but okay 🫣)
• --ssh=<Unicode> (IPControllerApp.ssh_server) — Default: '' ssh url for clients to use when connecting to the Controller processes. It should be of the form: [user@]server[:port]. The Controller's listening addresses must be accessible from the ssh server 🔑
• --enginessh=<Unicode> (IPControllerApp.engine_ssh_server) — Default: '' ssh url for engines to use when connecting to the Controller processes. It should be of the form: [user@]server[:port]. The Controller's listening addresses must be accessible from the ssh server 🔑
• --location=<Unicode> (IPControllerApp.location) — Default: 'pesto' The external IP or domain name of the Controller, used for disambiguating engine and client connections. 🌍 (and yes the default is literally pesto, we don't make the rules)
• --url=<Unicode> (HubFactory.url) — Default: '' The 0MQ url used for registration. This sets transport, ip, and port in one variable. For example: url='tcp://127.0.0.1:12345' or url='epgm://*:90210' 🔗
• --ip=<Unicode> (HubFactory.ip) — Default: '' The IP address for registration. This is generally either '127.0.0.1' for loopback only or '*' for all interfaces.
• --transport=<Unicode> (HubFactory.transport) — Default: 'tcp' The 0MQ transport for communications. This will likely be the default of 'tcp', but other values include 'ipc', 'epgm', 'inproc'. 🚚
• --port=<Int> (HubFactory.regport) — Default: 0 The port on which the Hub listens for registration.
• --ping=<Int> (HeartMonitor.period) — Default: 3000 The frequency at which the Hub pings the engines for heartbeats (in ms) 💓 (yes, that's milliseconds, bestie, she is checking in CONSTANTLY)
• --scheme=<Enum> (TaskScheduler.scheme_name) — Default: 'leastload' Choices: ('leastload', 'pure', 'lru', 'plainrandom', 'weighted', 'twobin') select the task scheduler scheme [default: Python LRU] Options are: 'pure', 'lru', 'plainrandom', 'weighted', 'twobin','leastload' 🎲
• --hwm=<Int> (TaskScheduler.hwm) — Default: 1 specify the High Water Mark (HWM) for the downstream socket in the Task scheduler. This is the maximum number of allowed outstanding tasks on each engine. The default (1) means that only one task can be outstanding on each engine. Setting TaskScheduler.hwm=0 means there is no limit, and the engines continue to be assigned tasks while they are working, effectively hiding network latency behind computation, but can result in an imbalance of work when submitting many heterogenous tasks all at once. Any positive value greater than one is a compromise between the two. ⚖️
• --profile-dir=<Unicode> (ProfileDir.location) — Default: '' Set the profile location directly. This overrides the logic used by the profile option.
• --profile=<Unicode> (BaseIPythonApplication.profile) — Default: 'default' The IPython profile to use.
• --ipython-dir=<Unicode> (BaseIPythonApplication.ipython_dir) — Default: '' The name of the IPython directory. This directory is used for logging configuration (through profiles), history storage, etc. The default is usually $HOME/.ipython. This option can also be specified through the environment variable IPYTHONDIR. 📁
• --log-level=<Enum> (Application.log_level) — Default: 30 Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL') Set the log level by value or name.
• --config=<Unicode> (BaseIPythonApplication.extra_config_file) — Default: '' Path to an extra config file to load. If specified, load this config file in addition to any other IPython config. 🗂️
• --work-dir=<Unicode> (BaseParallelApplication.work_dir) — Default: '/home/joe' Set the working dir for the process.
• --log-to-file=<Bool> (BaseParallelApplication.log_to_file) — Default: False whether to log to a file
• --clean-logs=<Bool> (BaseParallelApplication.clean_logs) — Default: False whether to cleanup old logfiles before starting 🧹
• --log-url=<Unicode> (BaseParallelApplication.log_url) — Default: '' The ZMQ URL of the iplogger to aggregate logging.
• --cluster-id=<Unicode> (BaseParallelApplication.cluster_id) — Default: '' String id to add to runtime files, to prevent name collisions when using multiple clusters with a single profile simultaneously. When set, files will be named like: 'ipcontroller-<cluster_id>-engine.json' Since this is text inserted into filenames, typical recommendations apply: Simple character strings are ideal, and spaces are not recommended (but should generally work). 🏷️
• --ident=<CUnicode> (Session.session) — Default: '' The UUID identifying this session.
• --user=<Unicode> (Session.username) — Default: 'joe' Username for the Session. Default is your system username.
• --keyfile=<Unicode> (Session.keyfile) — Default: '' path to file containing execution key.

To see all available configurables, use --help-all. 👀

🔥 EXAMPLES 🔥

ipcontroller --ip=192.168.0.1 --port=1000
       # listen on ip, port for engines

ipcontroller --scheme=pure
       # use the pure zeromq scheduler

bestie reminder: the controller goes first, always. engines and clients are the opening act, she is the headliner 💅

and that's the whole page!! this was ipyparallel 6.2.4, written up back in September 2020, and it's page IPCONTROLLER(1) in the User Commands corner of the manual. mwah, go start your cluster, you've got this 🫶💖✨