clmformat(1)
π clmformat β the manual, but make it make sense π
okay bestie, welcome to clmformat(1), aka USER COMMANDS territory. this is clm_format β the girl whose whole job is displaying cluster results in readable form (and optionally with labels and/or cohesion and stickiness measures attached). she's serving β¨legibilityβ¨.
real talk though: unless you run her with -dump fname or --dump, clm format is dependent on the macro processor zoem being present, as described further down. no zoem, no fancy output, periodt.
also the -icl fname input clustering option is always required β always, no exceptions, she will not move without it. the -imx fname input matrix option is required in fancy mode. and the tab file option -tab fname is what you need if you want label information in the output instead of mcl identifiers. π
β¨ SYNOPSIS β¨
clm format has two whole modes of output: dump and fancy. if you specify neither, fancy is what you get. in this mode, clm format generates a large array of performance measures related to nodes and clusters in both interlinked html output and plain text files. the files will be contained in an output directory that is newly created if not yet existing. in fancy mode the -imx option is required and the macro processor zoem must be available (http://micans.org/zoem). π¦
if dump is specified (see below for how to do that) clm format just generates a dump file where each line contains a cluster in the form of tab-separated indices, or tab-separated labels in case the -tab option is used. this dump is easy to parse with a simple or even quick-and-dirty script. you can include some very simple performance measures in this dump file by supplying --dump-measures. use -dump fname to specify the name of the file to dump to, rather than having clm format construct a file name by itself.
clm format can combine both modes by using either --dump or -dump fname and --fancy. in this case the dump file will be created in the output directory that is used by fancy mode. iconic behaviour honestly π
clm format
-icl fname (input cluster file) -imx fname (input matrix/graph file) [-tf spec (apply tf-spec to input matrix)] [-pi num (apply pre-inflation to matrix)] [-tab fname (read tab file)] [--lazy-tab (allow mismatched tab-file)] [-lump-count n (node threshold)] [--dump (write dump to dump.<icl-name>)] [-dump fname (write dump to file)] [--dump-pairs (write cluster/node pair per line)] [--dump-measures (write simple performance measures)] [-dump-node-sep str (separate entries with str)] [--fancy (spawn information blizzard)] [-dir dirname (write results to directory)] [-infix str (use after base name/directory)] [-nsm fname (output node stickiness file)] [-ccm fname (output cluster cohesion file)] [--adapt (allow domain mismatch)] [--subgraph (take subgraph with --adapt)] [-zmm fname (assume macro definitions are in fname)] [-fmt fname (write to encoding file fname)] [-h (print synopsis, exit)] [--apropos (print synopsis, exit)] [--version (print version, exit)]
consult the option descriptions and the introduction above for interdependencies of options. they're all tangled up together, iykyk π«Ά
clm format generates in fancy mode a logical description of the to-be-formatted content in a very small vocabulary of format-specific zoem macros. the appearance of the output can be easily changed by adapting a zoem macro definition file (also output by clm format) that is used by the zoem interpreter to interpret the logical elements.
the output format is apt to change over subsequent releases, as a result of user feedback. such changes will most likely be confined to the zoem macro definition file. so likeβ¦ don't get too attached to the exact look, she's evolving πΈ
the OUTPUT EXPLAINED section further below is likely to be of interest. (yes, bestie, you're gonna want to read it.)
π DESCRIPTION π
the primary function of clm format is to display cluster results and associated confidence measures in a readable form, by listing clusters in terms of the labels associated with the indices that are used in the mcl matrix. the labels must be stored in a so called tab file; see the -tab option for more information. π
π NOTE π
clm format output is in the form of zoem macros. you need to have zoem installed in your system if you want clm format to be of use. Zoem will not be necessary if you are using the -dump option. π
the -imx mx option is required unless the -dump option is used. the latter option results in special behaviour described under the -dump fname entry.
output is by default written in a directory that is newly created if it does not yet exist (normally several files will be created, for which the directory acts as a natural container). it is possible to simply output to the current directory β for that you need to specify -dir ./. if -dir is not specified, the output directory fmt.<clname> will be used, where <clname> is the argument to the -cl option. in the output directory, clm format will normally write two files. one contains zoem macros encoding formatted output (the encoding file), and the second (the definition file) contains zoem macro definitions which are used by the former. π
the encoding file is by default called fmt.azm (cf. the -fmt fname option). it contains zoem macros. it imports the macro definition file called clmformat.zmm that is normally also written by clm format. another macro definition file can be specified by using the -zmm <defsname> option. in this case clm format will refrain from writing the definition file and replace mentions of clmformat.zmm in the encoding file by <defsname>.
the encoding file needs to be processed by issuing one of the following commands from within the directory where the file is located.
zoem -i fmt -d html
zoem -i fmt -d txt
the first will result in HTML formatted output, the second in plain text format. obviously, you need to have installed zoem (e.g. from http://micans.org/zoem/src/) for this to work. π§
for each cluster a paragraph is output. first comes a listing of other clusters (in order of relevance, possibly empty) for which a significant amount of edges exists between the other and the current cluster. second comes a listing of the nodes in the current cluster. for each node a small sublist is made (in order of relevance, possibly empty) of other clusters in which the node has neighbours and for which the total sum of corresponding edge weights is significant. several quantities are output for each node/cluster pair that is deemed relevant. these are explained in the section OUTPUT EXPLAINED. π
clusters will by default be output to file until the total node count has exceeded a threshold (refer to the -lump-count option).
clm format also shows how well each node fits in the cluster it is in and how cohesive each cluster is, using simple but effective measures (described in section OUTPUT EXPLAINED). this enables you to compare the quality of the clusters in a clustering relative to each other, and may help in identifying both interesting areas and areas for which cluster structure is hard to find or perhaps absent. so basically she's giving you a vibe check on your own data π
π OPTIONS π
-icl fname(input cluster file) β Name of the clustering file. this one is mandatory, don't even try it without her.-imx fname(input matrix/graph file) β Name of the graph/matrix file.-tf spec(apply tf-spec to input matrix) β Transform the input matrix values according to the syntax described inmcxio(5).-tab fname(read tab file) β The filefnameshould be in tab format. Refer tomcxio(5).--lazy-tab(allow mismatched tab-file) β Allow missing and spurious entries in the tab file. she's being chill about it π-dump fname(write dump to file) β Clusters are written tofile. For each cluster a single line is written containing all indices of all nodes in that cluster. The indices are separated by tabs. If a tab file is specified, the indices are replaced by the corresponding tab file entry.--dump(write dump to file) β As-dump fnameexcept thatclm formatwrites to the file nameddump.<icl-name>where<icl-name>is the argument to the-icloption.-infix str(incorporate in base name) βstris included in the output file names. This can be used to store the results of differentclm formatruns (e.g. with differing-tfarguments) in the same directory. π--fancy(force fancy mode) β This enforces fancy mode if either of-dumpor--dumpis given. The dump file will be created in the output directory.--dump-pairs(write cluster/node pair per line) β Rather than writing a single cluster on each line, write a single cluster index/node (either tab entry or index) pair per line. Works in conjunction with the-taband-imxoptions.--dump-measures(write simple performance measures) β If an input matrix is specified with-imx fname, three measures of efficiency are prepended, respectively the simple projection score, efficiency or coverage, and the max-efficiency or max-coverage.-dump-node-sep str(separate entries with str) β Separate entries in the dump file withstr.-pi num(apply pre-inflation to matrix) β Apply pre-inflation to the matrix specified with the-imxoption. This will cause the efficiency scores to place a higher reward on high-weight edges being covered by a clustering (assuming thatnumis larger than one).
This option is also useful when mcl itself was instructed to use pre-inflation when clustering a graph.
-lump-count n(node threshold) β The zoem file is created such that during zoem processing clusters are formatted and output within a single file until the node threshold has been exceeded. A new file is then opened and the procedure repeats itself. π--adapt(allow domain mismatch) β Allow the cluster domain to differ from the graph domain. Presumably the clustering is a clustering of a subgraph. The cohesion and stickiness measures will pertain to the relevant part of the graph only.--subgraph(use restriction) β If the cluster domain is a subset of the graph domain, the cohesion and stickiness measures will by default still pertain to the entire graph. By setting this option, the measures will pertain to the subgraph induced by the cluster domain.-dir dirname(write results to directory) β Usedirnameas output directory. It will be created if it does not exist already.-fmt fname(write to encoding file fname) β Write to encoding filefnamerather than the defaultfmt.azm. It is best to supplyfnamewith the standard zoem suffix.azm. Zoem will process file of any name, but those lacking the.azmsuffix must be specified using the zoem-I fnameoption.-zmm defsname(assume macro definitions are in fname) β If this option is used,clm formatwill not output the definition file, and mentions of the definition file in the encoding file will use the file namedefsname. This option assumes that a valid definition file by the name ofdefsnamedoes exist.
heads up bestie: that last one is a trust fall β she's assuming defsname is a real, valid definition file that exists. if it isn't, that's on you, not her. π
-nsmfname β output node stickiness file. Wanna keep that (optional) node stickiness matrix? Name the file right here, bestie. π Its structure is: the columns range over all elements in the graph, exactly as specified by the-imxoption. The rows range over the clusters, exactly as specified by the-icloption. And the entries hold the projection value of that particular node onto that particular cluster β which is literally the sum of the weights of all arcs going out from the node to some node in that cluster, written as a fraction relative to the sum of weights of all outgoing arcs. π-ccmfname β output cluster cohesion file. This one names the file where the (optional) cluster cohesion matrix gets stored. πΈ Structure check: both columns and rows range over all clusters in the clustering, as specified by the-icloption. An entry specifies the projection of one cluster onto another cluster β which is simply the average of the projection value onto the second cluster of all nodes in the first cluster. Periodt. π
β¨ OUTPUT EXPLAINED β¨
Okay so, what follows is the whole explanation of the output that the standard zoem macros give you. It comes in a pretty terse number-packed format. The decision was made not to include headers and captions in the output, just to keep it readable. So you might wanna print out the annotated examples below. π¨οΈ And ngl, at the same side of the equation, the following is probably tough reading unless you have an actual example of clmformatted output right there in front of you. π
π«Ά Bestie, if you're reading this in a terminal, you might need to resize it so it's wider than 80 columns, because the examples below are formatted in verbatim mode. Do it, thank me later. β¨
Below, mention is made of the projection value for a node/cluster pair. This is simply the total amount of edge weights for that node in that cluster (corresponding to neighbours of the node in the cluster) relative to the overall amount of edge weights for that node (corresponding to all its neighbours). The coverage measure (referred to as cov) is also used. It's similar to the projection value, except that:
- the coverage measure rewards the inclusion of large edge weights (and penalizes the inclusion of insignificant edge weights), and
- it rewards node/cluster pairs for which the neighbour set of the node is very similar to the cluster.
The maximum coverage measure (referred to as maxcov) is similar to the normal coverage measure, except that it rewards inclusion of large edge weights even more. The cov and maxcov performance measures have several nice continuity and monotonicity properties and are described in [1]. π₯
πΈ Example cluster header
Cluster 0 sz 15 self 0.82 cov 0.43-0.26
10: 0.11
18: 0.05
12: 0.02
π¬ explanation
Cluster 0 sz 15 self 0.82 cov 0.43-0.26
| | | | |
clid count proj cov covmax
10: 0.11
| |
clidx1 projx1
18: 0.05
| |
clidx2 projx2
clidβ Numeric cluster identifier, (arbitrarily) assigned by MCL.countβ The size of clusterclid.projβ Projection value for clusterclid[d].covβ Coverage measure for clusterclid[d].maxcovβ Max-coverage measure for clusterclid[d].clidx1β Index of other cluster sharing relatively many edges.projx1β Projection value for theclid/clidx1pair of clusters [e].clidx2β :projx2β : asclidx1andprojx1
π¦ Example inner node
An inner node is listed under a cluster, and it is simply a member of that cluster. The name is as opposed to 'outer node', described below. π
[foo bar zut]
21 7-5 0.73 0.420-0.331 0.282-0.047 0.071-0.035 <3.54>
10 6/3 0.16 0.071-0.047 0.268-0.442
12 4/2 0.11 0.071-0.035 0.296-0.515
π¬ explanation
[label]
21 7-5 0.73 0.420-0.331 0.282-0.047 0.071-0.035 <3.54>
| | | | | | | | | | |
idx nbi nbo proj cov covmax max_i min_i max_o-min_o SUM
10 6/3 0.16 0.268-0.442 0.071-0.047
| | | | | | | |
clusid sz nb proj cov covmax max_i min_i
labelβ Optional; with-tab<tabfile> option.idxβ Numeric (mcl) identifier.nbiβ Count of the neighbours of nodeidxwithin its cluster.nboβ Count of the neighbours of nodeidxoutside its cluster.projβ Projection value [a] ofnbiedges.covβ Skewed projection [b], rewards inclusion of large edge weights.covmaxβ Ascovabove, rewarding large edge weights even more.max_iβ Largest edge weight in thenbiset, normalized [c].min_iβ Smallest edge weight in thenbiset [c].max_oβ Largest edge weight outside thenbiset [c]min_oβ Smallest edge weight outside thenbiset [c].SUMβ The sum of all edges leaving nodeidx.clusidβ Index of other cluster that is relevant for nodeidx.szβ Size of that cluster.nbβ Count of neighbours of nodeidxin clusterclusid.projβ Projection value of edges from nodeidxto clusterclusid.covβ Skewed projection of edges from nodeidxto clusterclusid.covmaxβ Maximally skewed projection, as above.max_oβ Largest edge weight for nodeidxto clusterclusid[c].min_oβ Smallest edge weight for nodeidxto clusterclusid[c].
π§ Example outer node
An outer node is listed under a cluster. The node is not part of that cluster, but it seems to have substantial connections to that cluster. It's giving "we're not together but we're together". π
[zoo eek few]
29 18#2 2-5 0.65 0.883-0.815 0.436-0.218 0.073-0.055
/4 0.27 0.070-0.109 0.073-0.055
π¬ explanation
[label]
29 18#2 2-5 0.65 0.883-0.815 0.436-0.218 0.073-0.055
| | | | | | | | | | | |
idx cl sz nbi nbo proj cov maxcov max_i min_i max_o min_o
id
/4 0.27 0.070-0.109 0.073-0.055 <2.29>
| | | | | | |
nb proj cov maxcov max_i min_i SUM
labelβ Optional; with-tab<tabfile> option.idxβ Numeric (mcl) identifierclidβ Index of the cluster that nodeidxbelongs toszβ Size of the cluster that nodeidxbelongs toprojβ :covβ : All these entries are the same as described abovecovmaxβ : for inner nodes, pertaining to clusterclid,max_iβ : i.e. the native cluster for nodeidxmin_iβ : (it is a member of that cluster).max_oβ :min_oβ :nbβ The count of neighbours of nodeidxin the current clusterprojβ Projection value for nodeidxrelative to current cluster.covβ Skewed projection (rewards large edge weights), as above.covmaxβ Maximally skewed projection, as above.max_oβ Largest edge weight for nodeidxin current cluster [c].min_oβ smallest edge weight for nodeidxin current cluster [c].SUMβ The sum of all edges leaving nodeidx.
π [a]
The projection value for a node relative to some subset of its neighbours is the sum of edge weights of all edges to that subset. The sum is witten as a fraction relative to the sum of edge weights of all neighbours.
π [b]
cov and covmax stand for coverage and maximal coverage. The coverage measure of a node/cluster pair is a generalized and skewed projection value [a] that rewards the presence of large edge weights in the cluster, relative to the collection of weights of all edges departing from the node. The maxcov measure is a projection value skewed even further, correspondingly rewarding the inclusion of large edge weights. The cov and maxcov performance measures have several nice continuity properties and are described in [1].
π [c]
All edge weights are written as the fraction of the sum SUM of all edge weights of edges leaving node idx.
π [d]
For clusters the projection value and the coverage measures are simply the averages of all projection values [a], respectively coverage measures [b], taken over all nodes in the cluster. The cluster projection value simply measures the sum of edge weights internal to the cluster, relative to the total sum of edge weights of all edges where at least one node in the edge is part of the cluster.
π [e]
The projection value for start cluster x and end cluster y is the sum of edge weights of edges between x and y as a fraction of the sum of all edge weights of edges leaving x.
π AUTHOR
Stijn van Dongen.
π REFERENCES
[1] Stijn van Dongen. Performance criteria for graph clustering and Markov cluster experiments. Technical Report INS-R0012, National Research Institute for Mathematics and Computer Science in the Netherlands, Amsterdam, May 2000. http://www.cwi.nl/ftp/CWIreports/INS/INS-R0012.ps.Z
π SEE ALSO
mclfamily(7) for an overview of all the documentation and the utilities in the mcl family.
And that's a wrap on clmformat! π This was version 14-137, dated 16 May 2014, and yes bestie it is indeed clmformat(1). Go forth and cluster, mwah π
π¦ where this page comes from
- ubuntu jammy: mcl 1:14-137+ds-9build2 Β· GPL-3.0-or-later
- Copyright (c) 2014 Stijn van Dongen
- π girlified by deepseek-ai/DeepSeek-V4.1-Flash on : a language model's rewrite of the original, under its terms