PARTIALATOR(1)

📦
version 0.13.0 · 2 versions
, in nixpkgs unstable

💖 NAME 💖

okay so bestie, welcome to the partialator manual page — that's PARTIALATOR(1), from the General Commands Manual, and she's literally the main character of merging partial reflections 💅

partialator — scaling and post-refinement of partial reflections ✨

🦋 SYNOPSIS 🦋

partialator -i input.stream -o output.hkl -y pointgroup [options] ...

partialator --help

🌸 DESCRIPTION 🌸

okay so partialator merges reflections by scaling and post refinement, and the whole vibe is that it accounts for the partialities of the reflections. That means it models the geometry of diffraction for each pattern — crystal orientation, unit cell parameters, X-ray bandwidth and so on — and then it tries to optimise those geometrical parameters so that the fully integrated intensities calculated using the model agree as closely as possible between the many patterns. no cap, that's the whole gig 🫶

see USAGE CASES below for examples of commands to merge reflections in different ways, for example with and without partiality or scaling. iykyk.

in addition to the output reflection list, partialator will write a file called partialator.params which contains the scaling factors determined for each crystal. after each iteration, a file will be written called pgraph-itern.dat which contains the observed and calculated partialities for a randomly chosen set of "free" reflections which were not included in the refinement. lowkey obsessed with the bookkeeping 💖

partialator will also write datasets merged using two halves of the total data using the same filename as you give for -o or --output, with 1 and 2 appended. for example, partialator.hkl1 and partialator.hkl2. compare these two files with each other to calculate figures of merit such as Rsplit and CC1/2. see the section on CUSTOM DATASET SPLITTING for more discussion on this topic. slay, half-datasets are iconic 🌈

✨ OPTIONS ✨

  • -i filename, --input=filename — give the name of the input stream, bestie. that's the tea ☕
  • -o filename, --output=filename — give the name of the output file. the default is --output=partialator.hkl.
  • -y pointgroup, --symmetry=pointgroup — merge according to symmetry pointgroup. periodt.
  • -n n, --iterations=n — run n cycles of scaling and post refinement 🔁
  • --no-scale — disable the scaling part of the refinement calculation.
  • --no-Bscale — disable the Debye-Waller part of the scaling calculation.
  • --no-pr — disable the orientation/physics model part of the refinement calculation.
  • --no-deltacchalf — disable rejection based on deltaCChalf.
  • -m model, --model=model — specify the partiality model. see the list below for possible choices 👑
  • -j n — run n analyses in parallel. multitasking icon energy ⚡
  • --polarisation=type — specify the polarisation of the incident radiation. type can be horiz or vert to indicate 100% polarisation of the electric field in the horizontal plane or vertical plane respectively. setting type to none completely disables the polarisation correction (see the note below). alternatively, type can be a direction followed by a percentage polarisation fraction. for example, 45deg90 means that 90% of the radiation is polarised with its electric field in a direction 45 degrees from horizontal, and 10deg100 means that all the radiation is polarised at 10 degrees from horizontal. the angle is specified clockwise from horizontal as viewed along the beam direction, i.e. as shown by the CrystFEL GUI. the beam is unpolarised when the fraction is 50% (equal parts of the radiation have their electric field in the specified plane). if the polarisation fraction is 100%, it can be omitted. for example 10deg or horiz.
💀 okay important note, bestie: --polarisation=none is not the same as, for example, --polarisation=vert50. in the first case, the polarisation correction will be completely disabled. in the other case, the incident beam will be unpolarised, but the polarisation of the diffracted radiation will still be corrected for (the factor of (1+cos^2(2theta))/2 or, equivalently, (2-sin^2(2theta))/2).

the default is --polarisation=horiz.

  • --no-polarisation — synonym for --polarisation=none.
  • --max-adu=n — include reflections only if their peak values were less than n. that means, n is the saturation value of the detector. the default is infinity, i.e. no cutoff. no saturation drama allowed 🙈
  • --min-res=n — merge crystals only if they diffract to beyond n Angstroms resolution. the default is infinity, i.e. all crystals are included. the resolution is taken from the diffraction_resolution_limit line in the stream.
  • --min-measurements=n — include a reflection in the output only if it appears at least least n times. the default is --min-measurements=2. (yes, the original says "least least" — we keep the lore fr fr 😭)
  • --push-res=n — merge reflections which are up to n nm^-1 higher than the apparent resolution limit of each individual crystal. n can be negative to merge lower than the apparent resolution limit. the default is --push-res=inf, which means no resolution cutoff at all. push it real good 💅
  • --start-after=n — ignore the first n crystals in the input. the default is --start-after=0, i.e. start at the beginning.
  • --stop-after=n — stop processing after n crystals have been successfully read. the default is --stop-after=0, which means to process all the patterns from the start point to the end of the input (see --start-after). know when to leave the party 🎀
  • --no-free — disable cross-validation (for testing only). testing-only behaviour, don't make it your whole personality 🔥
  • --custom-split=filename — read a list of filenames, event IDs and dataset IDs from filename. see the section on CUSTOM DATASET SPLITTING below.
  • --max-rel-B=n — reject crystals if the absolute values of their relative Debye-Waller ("B") factors are more than n square Angstroms. the default is --max-rel-B=100.
  • --output-every-cycle — write out the per-crystal parameters and reflection lists after every cycle of refinement, instead of only at the end. the intermediate reflection lists and parameter filenames will be prefixed with iterN_ (note the trailing underscore), where N is the iteration number. if you use --custom-split, intermediate results will also be output for each custom dataset. literally everything, every cycle, no crumbs 🍓
  • --no-logs — do not write the extensive log files needed for plotting contour maps and spectrum graphs. this makes the process a lot faster, but you probably do want these logs to check that post-refinement is working reasonably. fast is cute but logs are your bestie for vibe checks 👑
  • --log-folder=folder — specify the location of the log folder (see --no-logs). the default is --log-folder=pr-logs.
  • -w pg — set the apparent symmetry of the crystals. the ambiguity operator will be determined by comparing this to the actual symmetry.

if you prefer (or the scenario demands it), you can specify the ambiguity operator directly using --operator.

  • --operator=op — specify the indexing ambiguity operator. example: --operator=k,h,-l.

if you prefer, you can specify the ambiguity operator by specifying the apparent symmetry using -w.

  • --force-bandwidth=bw, --force-radius=R, --force-lambda=l — set the X-ray bandwidth, initial profile radius or wavelength for all crystals before proceeding, overriding the values from the stream. bandwidth is given as a fraction, i.e. --force-bandwidth=0.0013 means 0.13 percent (approximate FWHM). radius is given in nm^-1. wavelength is given in Angstroms. forcing the issue but make it fashion 💜
  • --harvest-file=fn — write a list of parameters to fn, in JSON format. this is intended to be used for harvesting data into a database system.
  • --unmerged-output=fn — write unmerged, but otherwise fully corrected, intensities to fn.
  • --output-cell=fn — write the average unit cell to fn. the average is taken over a,b,c,al,be,ga from the crystals that contributed to the final merged intensity data, including any changes made to the parameters by post-refinement. crystals will be omitted from the average if their lattice type, unique axis or centering don't match the first accepted crystal. gatekeeping the unit cell, but respectfully 🌸

🧚 PARTIALITY MODELS 🧚

the available partiality models are, and yes this is the moment:

  • xsphere — the volume of intersection between a sphere centered on each reciprocal lattice point and the part of reciprocal space excited by the Ewald sphere taking into account the finite bandwidth of the incident X-rays. a "source coverage factor" is included to take into account the spectral brightness of the effective source for the reflection. the X-ray spectrum is modelled as a super-Gaussian with exponent 1.5, and the overlap integral is evaluated numerically.

this model is the same as that described in Acta Cryst. D71 (2015) p1400.

  • unity — fix all partialities at 1. simple. iconic. no notes ✨
  • offset — treat the radiation as perfectly monochromatic, and calculate the partiality as exp(-t^2/R^2), where t is the excitation error and R is the reflection radius (taking mosaicity into account). this model is similar to that used by XDS.
  • ggpm — an analytical overlap integral modelling the reflection profiles as Gaussian functions, and the radiation spectrum as a sum of Gaussians. math girlie behaviour 🧮💅

✨ USAGE CASES ✨

Okay bestie, here's the tea on all the different ways you can run partialator. Pick your fighter 💅

Merging without scaling, partialities, or post-refinement

partialator -i my.stream -o my.hkl -y mypointgroup --model=unity
--iterations=0

Bare minimum, zero glow-up. Just merge and go 🫶

Merging without partialities or post-refinement, but with scaling

partialator -i my.stream -o my.hkl -y mypointgroup --model=unity
--iterations=1
Ngl, crank up the iterations if you want the scaling to actually be accurate — but that costs you more CPU time and possibly more rejected crystals 😭

Merging with partialities, but no post-refinement and no scaling

partialator -i my.stream -o my.hkl -y mypointgroup
--model=xsphere --iterations=0

Merging with partialities, with scaling, but no post-refinement

partialator -i my.stream -o my.hkl -y mypointgroup
--model=xsphere --iterations=1 --no-pr

Merging with partialities, post-refinement, AND scaling

partialator -i my.stream -o my.hkl -y mypointgroup
--model=xsphere --iterations=1
Same deal as before: more iterations = more accurate scaling and post-refinement, but it'll eat more CPU time and might reject more crystals 💀

Merging with partialities and post-refinement, but without scaling

Lowkey a weird thing to want to do, but like… here you go:

partialator -i my.stream -o my.hkl -y mypointgroup
--model=xsphere --iterations=1 --no-scale
Again with the iterations thing — higher number means more accurate post-refinement, but it costs more CPU time and possibly more rejected crystals ✨

🦋 CUSTOM DATASET SPLITTING 🦋

So picture this: you're doing a time-resolved experiment (or whatever). The iconic move is to make sure all your time points got processed exactly the same way. Like, identical. Rather than running partialator separately for each time point, you should process the whole thing together and only do the splitting into time points right before the final output. Trust me on this one 💖

Here's why, okay? Say you're doing simple scaling (no B factor). If you merge each dataset independently, they'd probably all end up with different overall scaling factors, and then when you compare the results you'd have to factor that difference in. In practice most programs can handle that job pretty easily — but what about when a B factor is included? And what if partialities are included? Like… how unique is the solution even? 🙈

With partialator --custom-split, you hand it a separate text file containing a list of filenames, event numbers and dataset names — one event (detector frame) per line, with each field separated by exactly one space. For every unique dataset name, you get a separate reflection list output. All the crystals get refined together, but they get merged according to the dataset names you gave. The parameters (scaling factors, partialities, etc.) determined during the joint refinement get applied. And for each dataset you also get a separate pair of split half-datasets written out, so you can calculate figures of merit like Rsplit and CC1/2 for each one 👑

So if your overall output filename (the one you gave with -o or --output) were merged.hkl, then a dataset named dataset would be written to merged-dataset.hkl. The matching half-datasets would go to merged-dataset.hkl1 and merged-dataset.hkl2. Slay 🎀

Heads up: the filenames and event names must match exactly what's written into the stream as the Image filename and Event — so you gotta account for options like indexamajig --prefix and --basename. Check that the numbers of crystals in each dataset (which partialator prints on the terminal) match what you expect, and that no patterns have been "lost" 👀

There's no requirement for every event in the list to appear in the stream, and no requirement for every event in the stream to belong to one of the datasets. But if an event is listed for more than one dataset, the results are "undefined" 💀 Like, genuinely undefined. Don't do it.

If you don't have event IDs for your data — i.e. if you've got one detector frame per input file — then just leave the event IDs out of the custom split file entirely. Easy 🍓

And finally: note that the main dataset, all the custom split datasets, and all the half-datasets are all subject to --min-measurements. Every single one. No exceptions ✨

🌸 AUTHOR 🌸

This page was written by Thomas White. Shoutout to the icon behind the manual, we love and appreciate you 🫶

💜 REPORTING BUGS 💜

Found something cursed? Report bugs to <taw@physics.org>, or go visit <http://www.desy.de/~twhite/crystfel>. Bestie will sort you out 🧚

Copyright © 2012-2021 Deutsches Elektronen-Synchrotron DESY, a research centre of the Helmholtz Association.

partialator, and this manual, are part of CrystFEL.

CrystFEL is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. Bestie, that's the whole vibe 💖

CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. No warranty, bestie. None. Zero. Not a crumb. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with CrystFEL. If not, see <http://www.gnu.org/licenses/>. ✨

⭐ SEE ALSO ⭐

and that's the whole page bestie, go forth and merge responsibly 💅✨

This was partialator(1) from CrystFEL, copyright © 2012-2021 Deutsches Elektronen-Synchrotron DESY, a research centre of the Helmholtz Association — mwah, love u, bye 🎀🦋💜

📦 where this page comes from