TRANSLATETOOLKIT(1)
💅 TRANSLATETOOLKIT(1) — Translate Toolkit
okay so welcome to Translate Toolkit’s documentation, bestie — this covers both the user’s and programmer’s perspective, and we’re gonna get through it together.
🌸 NAME
translatetoolkit — Translate Toolkit Documentation
🫶 USER’S GUIDE
This part has the user’s documentation for the tools included in the Translate Toolkit. Periodt.
✨ Features
- Work with ONE localisation format. You’ll no longer be editing DTD files in one tool,
.propertiesin another, OpenOffice GSI in a third. Simply do all your localisation in a PO or XLIFF editor — iconic behaviour only. - Converters for a number of formats:
- OpenOffice.org SDF/GSI
- Mozilla:
.properties, DTD, XHTML,.inc,.ini, etc - Others: Comma Separated Value, TMX, XLIFF, TBX, PHP, WordFast TXT, Qt
.ts, txt,.ini, Windows.rc, ical, subtitles, Mac OS X strings
- File access to localization files through the format API in all the above formats, as well as
.qph,.qm,.mo - Output valid target file types. We make sure that your output files (e.g.
.properties) contain all comments from the original file and preserves the layout of the original as far as possible. If your PO entry is marked as fuzzy we use the English text, not your half complete translation. The converters for OpenOffice.org and Mozilla formats will also perform simple checks and corrections to make sure you have none of those hard to find localisation bugs. - Our checker has over 42 checks to find errors such as: missing or translated variables, missing accelerator keys, bad escaping, start capitalisation, missing sentences, bad XML and much more.
- Language awareness, taking language conventions for capitalisation, quotes and other punctuation into account.
- Find conflicting translations easily, cases where you have translated a source word differently or used a target word for 2 very different English concepts.
- Extract messages using simple text or a regular expression allowing you to quickly find and extract words that you need to fix due to glossary changes.
- Merge snippets of PO files into your existing translations.
- Create word, string and file counts of your files. Making it much easier to budget time as string counts do not give you a good indication of expected work.
- Create a set of PO files with debugging entries to allow you to easily locate the source of translations. Very useful in OpenOffice.org which provides scant clues as to where the running application has sourced the message.
The Translate Toolkit is also a powerful API for writing translation and localisation tools, already used by our own and several other projects. See the base class section for more information.
🛠 Installation
This is a guide to installing the Translate Toolkit on your system. If the Translate Toolkit is already packaged for your system, this is probably the easiest way to install it. For several Linux distributions, the package might be available through your package manager. On Windows, we recommend using a virtual environment.
If your system already has the toolkit prepackaged, then please let us know what steps are required to install it.
🧚 Building
For build instructions, see the Building page.
🦋 Download
Download a stable released version. Or if you have a python environment, run pip install translate-toolkit. For those who need problems fixed, or who want to work on the bleeding edge, get the latest source from Git.
If you install through your distribution’s package manager, you should automatically have all the dependencies you need. If you are installing a version from Version Control, or from a source release, you should check the README file for information on the dependencies that are needed. Some of the dependencies are optional. The README file documents this.
🎀 Installing packaged versions
Get the package for your system:
- RPM — If you want to install easily on an RPM based system
.tar.gz— for source based installing on Linux.deb— for Debian GNU/Linux (etch version)
The RPM package can be installed by using the following command:
$ rpm -Uvh translate-toolkit-1.0.1.rpmTo install a tar.bz2:
$ tar xvjf translate-toolkit-1.1.0.tar.bz2
$ cd translate-toolkit-1.1.0
$ su
$ ./setup.py installOn Debian (if you are on etch), just type the following command:
$ aptitude install translate-toolkitIf you are using an old Debian stable system, you might want to install the .tar.bz2 version. Be sure to install python and python development first with:
$ apt-get install python python-devAlternatively newer packages might be in testing.
🪟 Installing on Windows
On Windows we recommend that you install Translate Toolkit using a virtual environment. This makes installation clean and isolated.
Use the latest Python 3.9. Install virtualenvwrapper-win to simplify handling of virtualenvs.
- Install latest Python 3.9
- Open
cmd.exeor similar pip install virtualenvwrapper-winmkvirtualenv ttkwhere “ttk” is the name for the new virtualenvpip install translate-toolkit[recommended]to install latest stable orpip install –pre translate-toolkit[recommended]to try a pre-releasepo2prop –versionto double check you have the right version
Next times you need to use Translate Toolkit just remember to:
- Open
cmd.exeor similar workon ttkto enable the virtualenv again- Run the Translate Toolkit commands you want
🌈 Installing from Git
If you want to try the bleeding edge, or just want to have the latest fixes from a stabilising branch then you need to use Git to get your sources:
$ git clone https://github.com/translate/translate.gitThis will retrieve the master branch of the Toolkit. Further Git instructions are also available.
Once you have the sources you have two options, a full install:
$ su
$ ./setup.py installor, running the tools from the source directory:
$ su
$ pip install -e .⭐ Verify installed version
To verify which version of the toolkit you have installed run:
$ prop2po --version
prop2po 3.12.1🧹 Cleaning up existing installation
To remove old versions of the toolkit which you might have installed without a virtual environment or without your package manager.
The following advice only applies to manual installation from a tarball.
- Find location of your python packages:
$ python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"- Delete toolkit package from your Python site-packages directory e.g.:
$ rm -R /usr/local/lib/python3.9/dist-packages/translate🔥 Converters
💬 General Usage
The tools follow a general usage convention which is helpful to understand.
📥 Input & Output
The last two arguments of your command are the input and output files/directories:
moz2po <input> <output>You can of course still use the -i and -o options which allows you to reorder commands:
moz2po -o <output> -i <input>😭 Error Reporting
All tools accept the option --errorlevel. If you find a bug, add this option and send the traceback to the developers.
moz2po <other-options> --errorlevel=traceback🫶 Templates
If you are working with any file format and you wish to preserve comments and layout then use your source file as a template.
po2dtd -t <source-file> <input> <output>This will use the files in <source-file> as a template, merge the PO files in <input>, and create new DTD files in <output>.
If you ran this without the templates you would get valid DTD files but they would not preserve the layout or all the comments from the source DTD file.
The same concept of templates is also used when you merge files.
pomerge -t <old> <fixes> <new>This would take the <old> files merge in the <fixes> and output new PO files, preserving formatting, into <new>. You can use the same directory for <old> and <new> if you want the merges to overwrite files in <old>.
🎯 source2target
The converters all follow this convention:
- source = the format from which you are converting e.g. in
oo2powe are converting from OpenOffice.org SDF/GSI - target = the format into which you are converting e.g. in
oo2powe are converting to Gettext PO
🆘 Getting Help
The --help option will always list the available commands for the tool.
moz2po --help📱 android2po
Converts Android resource files to Gettext PO format.
🌸 Usage
android2po [options] <android> <po>Where:
<android>— is a valid Android resource file or directory of those files<po>— is a directory of PO or POT files
🎀 Options:
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read fromINPUTinxmlformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths-o OUTPUT,--output=OUTPUT— write toOUTPUTinpo,potformats-t TEMPLATE,--template=TEMPLATE— read fromTEMPLATEinxmlformat-S,--timestamp— skip conversion if the output file has newer timestamp--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default: ‘msgctxt’)
💖 Examples
These examples demonstrate the use of android2po:
android2po -i strings-en.xml -o en.poto simply convert strings-en.xml to en.po.
To convert a source and target Android resources files into a PO file:
android2po -t strings-en.xml -i strings-ca.xml -o ca.postrings-en.xml contains the template resource and strings-ca.xml the localised resource strings.
🦋 csv2po
Convert between CSV (Comma Separated Value) files and the PO format. This is useful for those translators who can only use a Spreadsheet, a modern spreadsheet can open CSV files for editing. It is also useful if you have other data such as translation memory in CSV format and you wish to use it with your PO translations.
If you are starting out with your own CSV files (not created by po2csv), take note of the assumptions of the column layout explained below.
🌸 Usage
csv2po [options] <csv> <po>
po2csv [options] <po> <csv>Where:
<csv>— is a file or directory containing CSV files<po>— is a file or directory containing PO files
🎀 Options (csv2po)
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read fromINPUTincsvformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths-o OUTPUT,--output=OUTPUT— write toOUTPUTinpo,potformats-t TEMPLATE,--template=TEMPLATE— read fromTEMPLATEinpo,pot,potformats-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--charset=CHARSET— set charset to decode fromcsvfiles--columnorder=COLUMNORDER— specify the order and position of columns (location,source,target)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default: ‘msgctxt’)
🌈 Options (po2csv)
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read fromINPUTinpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths-o OUTPUT,--output=OUTPUT— write toOUTPUTincsvformat-S,--timestamp— skip conversion if the output file has newer timestamp--columnorder=COLUMNORDER— specify the order and position of columns (location,source,target)
🍓 CSV file layout
The resultant CSV file has the following layout:
- Column A — Location: All the PO
#:location comments. These are needed to reconstruct or merge the CSV back into the PO file. - Column B — Source Language: The
msgidor source string. - Column C — Target Language: The
msgstror target language.
The layout can be customized by --columnorder, you can add, or remove columns using that as well, for example --columnorder=context,source,target.
✨ Examples
These examples demonstrate the use of csv2po:
po2csv -P pot csvWe use the -P option to recognise POT files found in pot and convert them to CSV files placed in csv:
csv2po csv poConvert CSV files in csv to PO files placed in po:
csv2po --charset=windows-1250 -t pot csv poUser working on Windows will often return files encoded in everything but Unicode. In this case we convert CSV files found in csv from windows-1250 to UTF-8 and place the correctly encoded files in po. We use the templates found in pot to ensure that we preserve formatting and other data. Note that UTF-8 is the only available destination encoding.
csv2po --columnorder=location,target,source fr.csv fr.poIn case the CSV file has the columns in a different order you may use --columnorder.
💀 Bugs
- Translation comments
#[space]and KDE comments_:are not available in CSV mode which effects the translators effectiveness - Locations
#:that are not conformant to PO (i.e. have spaces) will get messed up by PO tools.
🧚 csv2tbx
Convert between CSV (Comma Separated Value) files and the TBX format for terminology exchange.
🌸 Usage
csv2tbx [--charset=CHARSET] [--columnorder=COLUMNORDER] <csv> <tbx>Where:
<csv>— is a CSV file<tbx>— is the target TBX file
🎀 Options (csv2tbx)
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read fromINPUTincsvformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths-o OUTPUT,--output=OUTPUT— write toOUTPUTintbxformat-S,--timestamp— skip conversion if the output file has newer timestamp--charset=CHARSET— set charset to decode fromcsvfiles--columnorder=COLUMNORDER— specify the order and position of columns (comment,source,target)
🍓 CSV file layout
The CSV file is expected to have three columns (separated by commas, not other characters like semicolons), of which the default layout is:
- Column A — Comment: All the PO
#:location comments. These are not used in the TBX files, and can be left empty, but could be generated bypo2csv. - Column B — Source Language: The
msgidor source string. - Column C — Target Language: The
msgstror target language.
✨ Examples
These examples demonstrate the use of csv2tbx:
csv2tbx terms.csv terms.tbxto simply convert terms.csv to terms.tbx.
To convert a directory recursively to another directory with the same structure of files:
csv2tbx csv-dir tbx-target-dirThis will convert CSV files in csv-dir to TBX files placed in tbx-target-dir.
csv2tbx --charset=windows-1250 csv tbxUsers working on Windows will often return files in encodings other than Unicode-based encodings. In this case we convert CSV files found in csv from windows-1250 to UTF-8 and place the correctly encoded files in tbx. Note that UTF-8 is the only available destination encoding.
💬 Two column CSV
When your CSV is just two columns, you tell it the order and go:
csv2tbx --columnorder=source,target foo.csv foo.tbx📝 Notes
Wanna make sure everything's actually standards-compliant and see which features made the cut? Peek at CSV and TBX, bestie — they've got the receipts.
🧚 flatxml2po
Okay so this one converts flat XML (.xml) files into Gettext PO format — think of it as XML that's simple, monolingual, and only one level deep. Very minimal, very demure ✨
🛠️ Usage
flatxml2po [options] <xml> <po>
po2flatxml [options] <po> <xml> [-t <base-xml>]Where, like, what even are those:
<xml>— a valid.xmlfile, or a whole directory full of them 🗂️<po>— a directory of PO or POT files<base-xml>— a template, or the original file from before translation. required if you want roundtrips that preserve extraneous data. don't skip it, fr fr
⚙️ Options (flatxml2po)
--version— show program's version number and dip 👋-h,--help— show this help message and exit--manpage— output a manpage based on the help (very meta of it)--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inxmlformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths 🚫-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-S,--timestamp— skip conversion if the output file has a newer timestamp ⏰-r ROOT,--root=ROOT— name of the XML root element (default:“root”)-v VALUE,--value=VALUE— name of the XML value element (default:“str”)-k KEY,--key=KEY— name of the XML key attribute (default:“key”)-n NS,--namespace=NS— XML namespace uri (default:None)
⚙️ Options (po2flatxml)
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inxmlformat-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inxmlformat-S,--timestamp— skip conversion if the output file has a newer timestamp-r ROOT,--root=ROOT— name of the XML root element (default:“root”)-v VALUE,--value=VALUE— name of the XML value element (default:“str”)-k KEY,--key=KEY— name of the XML key attribute (default:“key”)-n NS,--namespace=NS— XML namespace uri (default:None)-w INDENT,--indent=INDENT— indent width in spaces,0for no indent (default:2) 🎀
📦 Formats Supported
Go check out the flat XML format document to see exactly how much of the XML format is supported — it's all laid out there, iykyk.
🌈 Examples
Bestie, this example walks through a roundtrip of flat XML translations and recovering your existing ones. Two birds, one slay.
First up, we gotta create a set of POT files:
flatxml2po -P lang/en pot/Every .xml file chilling in the lang/en directory gets converted into Gettext POT files and dropped into the pot directory. Easy.
Translating for the very first time? Then you can skip the next step, lucky you. But if you need to recover existing translations, we do this:
flatxml2po -t lang/en lang/zu po-zu/So we take the English XML files in lang/en plus your existing Zulu translation in lang/zu, and we create a set of PO files in po-zu. Those now have your translations in them. 🫶
Real talk though: for this to work 100%, your English and Zulu need to be at the same revision. If they're not, you're gonna have to review all the translations. No shortcuts, sorry.
Now you're in a position to translate either your recovered translations or your brand-new POT files. Main character energy.
Once translated, you convert back like so:
po2flatxml -t lang/en po-zu/ lang/zuYour translations in the Zulu PO directory po-zu get converted to XML, using the files in lang/en as templates, and your shiny new translations land in lang/zu. 💅
And to update your translations? Just redo the POT creation step and use pot2po to bring your translation up-to-date. That's it, that's the whole vibe.
🚧 Limitations
Indentation is spaces-only (set with --indent greater than zero) or fully flattened — no indentation at all, everything on a single line, which is --indent set to zero. Tabs are not supported when using po2flatxml. Not even a little bit. 😭
🌐 html2po, po2html
These two convert the translatable bits in HTML into PO format, then shove your translated text back into the HTML templates. Glow-up for your website, basically ✨
🛠️ Usage
html2po [options] <html-src> <po>
po2html [options] -i <po> -t <html-src> -o <html-dest>Where, decoder ring time:
<html-src>— an HTML file, or a directory of HTML files, in the source language<html-dest>— an HTML file, or a directory of HTML files, translated into the target language<po>— a PO file, or a directory of PO files
⚙️ Options (html2po)
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inhtm,html,xhtmlformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-S,--timestamp— skip conversion if the output file has a newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)-u,--untagged— include untagged sections 🙈--keepcomments— preserve html comments as translation notes in the output--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:‘msgctxt’)--multifile=MULTIFILESTYLE— how to split po/pot files (single,topleveloronefile) (default:‘single’; if set to‘onefile’, a single po/pot file will be written.‘toplevel’not used.)
⚙️ Options (po2html)
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inhtm,html,xhtmlformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inhtm,html,xhtmlformats-S,--timestamp— skip conversion if the output file has a newer timestamp--threshold=PERCENT— only convert files where the translation completion is above PERCENT 📊--fuzzy— use translations marked fuzzy--nofuzzy— don't use translations marked fuzzy (default)
🌈 Examples
html2po -P site potThis finds every HTML file (.htm, .html, .xhtml) in site, converts them to POT files, and drops them in pot. Look at it go.
You can create and update PO files for different languages using the pot2po command. Like, say you want PO files for a Xhosa translation — do this:
pot2po -i pot -t site -o xhThat merges the POT files in pot into the PO files in xh (if any exist). Merging era.
And then, after you've edited the PO files in xh, generate the translated version of the web site like so:
po2html -i xh -t site -o site-xhAll the PO translations in xh get converted to HTML using the HTML files in site as templates, and the new translated HTML files land in site-xh. Chef's kiss. 👑
Prefer a single PO/POT file for your whole web site? Sure, you can create one like this:
html2po -P --multifile=onefile site file.potAnd when po2html gets invoked with a single PO file as input plus a directory of template HTML files, it produces one output file per template file. So to generate translated output files from a single PO file, invoke po2html like this:
po2html -i xh.po -t site -o site-xhIn this example, xh.po is the translation file for Xhosa, site is the directory where the source-language HTML files live, and site-xh is where the translated HTML files end up. 🌸
📝 Notes
The HTML format description spills all the tea on the format of localisable HTML content and what this converter can actually do.
💀 Bugs
Some items end up in the msgid's that should not be translated. Yeah. It's giving "known issue," we don't talk about it 🙈
🗓️ ical2po
Added in version 1.2. ✨
Converts iCalendar (*.ics) files to Gettext PO format. Calendar girlies, this one's for you. 🎀
Usage
okay so here’s the vibe: you run one of these two commands, depending on which direction you’re going, bestie 💅
ical2po [options] <ical> <po>
po2ical [options] -t <ical> <po> <ical>Where:
<ical>— is a valid.icsfile or directory of those files<po>— is a directory of PO or POT files
✨ Options (ical2po) ✨
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inicsformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inicsformat-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default: ‘msgctxt’)
🦋 Options (po2ical) 🦋
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inicsformat-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inicsformat-S,--timestamp— skip conversion if the output file has newer timestamp--threshold=PERCENT— only convert files where the translation completion is above PERCENT--fuzzy— use translations marked fuzzy--nofuzzy— don’t use translations marked fuzzy (default)
🎀 Examples 🎀
This example is all about the roundtrip of iCalendar translations. Yes, you can do recovery of translations, but it’s unlikely that you will ever need to do that.
First we need to create a set of POT files.
ical2po -P ical.ics ical.potThe ical.ics file is converted to Gettext POT files called ical.pot. Directories of iCalendar files can also be processed.
Begin translating the ical.pot file by first copying it to make a PO file.
cp ical.pot ical-af.poYou are now in a position to translate the file ical-af.po in your favourite translation tool.
Once translated you can convert back as follows:
po2ical -t ical.ics ical-af.po ical-af.icsYour translations found in the Afrikaans PO file, ical-af.po, will be converted to .ics using the file ical.ics as a template and creating your newly translated .ics file ical-af.ics.
To update your translations simply redo the POT creation step and make use of pot2po to bring your translation up-to-date.
🌸 Notes 🌸
The converter will only process events in the calendar file, the file itself can contain many other things that could be localisable. Please raise a bug if you want to extract additional items.
The converter does not make use of the LANGUAGE attribute which is permitted in the format. The LANGUAGE attribute does not aid multilingualism in this context so is ignored.
The converter could conceivably also process vCard files, but this has not been implemented for lack of a clear need. Please raise a bug with an example if you have such a file that could benefit from localisation.
🧚 ini2po 🧚
Converts .ini files to Gettext PO format.
Usage
ini2po [options] <ini> <po>
po2ini [options] -t <ini> <po> <ini>Where:
<ini>— is a valid.inifile or directory of those files<po>— is a directory of PO or POT files
💖 Options (ini2po) 💖
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inini,isl,issformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inini,isl,issformats-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default: ‘msgctxt’)
🌈 Options (po2ini) 🌈
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inini,islformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inini,islformats-S,--timestamp— skip conversion if the output file has newer timestamp--threshold=PERCENT— only convert files where the translation completion is above PERCENT--fuzzy— use translations marked fuzzy--nofuzzy— don’t use translations marked fuzzy (default)
🧚 Formats Supported 🧚
INI files need to be organized into separate languages per file and in the following format:
[Section]
; a comment
a = a stringComment marked with the hash symbol (#) are also allowed, and the colon (:) is also accepted as key-value delimiter:
[Section]
# another comment
b : a stringThis variants in comment marks and key-value delimiters can be mixed in one single INI file:
[Section]
; a comment
a = a string
# another comment
b : a string
c:'other example with apostrophes'
d:"example with double quotes"The spacing between the key-value delimiter and the key, and the between the value and the key-value delimiter is not important since the converter automatically strips the blank spaces.
NOTE: A section must be present at the file beginning in order to get ini2po working properly. You may add it by hand at the file beginning.
NOTE: Strings marked with double quotes and/or apostrophes will carry these quotation marks to the generated .po file, so they will appear like:
#: [Section]c
msgid "'other example with apostrophes'"
msgstr ""
#: [Section]d
msgid "\"example with double quotes\""
msgstr ""🍓 Examples 🍓
This example looks at roundtrip of .ini translations as well as recovery of existing translations.
First we need to create a set of POT files.
ini2po -P ini/ pot/All .ini files found in the ini/ directory are converted to Gettext POT files and placed in the pot/ directory.
If you are translating for the first time then you can skip the next step. If you need to recover your existing translations then we do the following:
ini2po -t lang/ zu/ po-zu/Using the English .ini files found in lang/ and your existing Zulu translation in zu/ we create a set of PO files in po-zu/. These will now have your translations. Please be aware that in order for the to work 100% you need to have both English and Zulu at the same revision. If they are not, you will have to review all translations.
You are now in a position to translate your recovered translations or your new POT files.
Once translated you can convert back as follows:
po2ini -t lang/ po-zu/ zu/Your translations found in the Zulu PO directory, po-zu/, will be converted to .ini using the files in lang/ as templates and placing your newly translated .ini files in zu/.
To update your translations simply redo the POT creation step and make use of pot2po to bring your translation up-to-date.
💀 Issues 💀
We do not extract comments from .ini files. These are sometimes needed as developers provide guidance to translators in these comments.
⭐ json2po ⭐
Converts .json files to Gettext PO format.
Usage
json2po [options] <json> <po>
po2json [options] -t <json> <po> <json>Where:
<json>— is a valid.jsonfile or directory of those files<po>— is a directory of PO or POT files
👑 Options (json2po) 👑
--version— show program’s version number and exit
okay so, still in the flag zone, bestie — here’s the rest of the option stack, no skips, no cap:
-h,--help— show this help message and exit ✨--manpage— output a manpage based on the help 💅--progress=PROGRESS— show progress as:dots,none,bar,names,verbose🦋--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback🎀-i INPUT,--input=INPUT— read fromINPUTin JSON format 🌸-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths 🚫-o OUTPUT,--output=OUTPUT— write toOUTPUTinpo,potformats 💖-t TEMPLATE,--template=TEMPLATE— read fromTEMPLATEin JSON format 🧚-S,--timestamp— skip conversion if the output file has newer timestamp ⏰-P,--pot— output PO Templates (.pot) rather than PO files (.po) 👑--filter=FILTER— leaves to extract e.g.‘name,desc’: (default: extract everything) 🔍--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:‘msgctxt’) 💀
💅 Options (po2json): 💅
--version— show program’s version number and exit 🌈-h,--help— show this help message and exit ✨--manpage— output a manpage based on the help 💅--progress=PROGRESS— show progress as:dots,none,bar,names,verbose🦋--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback🎀-i INPUT,--input=INPUT— read fromINPUTinpo,potformats 🌸-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths 🚫-o OUTPUT,--output=OUTPUT— write toOUTPUTin JSON format 💖-t TEMPLATE,--template=TEMPLATE— read fromTEMPLATEin JSON format 🧚-S,--timestamp— skip conversion if the output file has newer timestamp ⏰--threshold=PERCENT— only convert files where the translation completion is abovePERCENT🔥--fuzzy— use translations marked fuzzy 🙈--nofuzzy— don’t use translations marked fuzzy (default) 👑--removeuntranslated— remove untranslated strings from output 🧹
✨ Examples ✨
This example looks at roundtrip of .json translations as well as recovery of existing translations. 🫶
First we need to create a set of POT files.
json2po -P json/ pot/All .json files found in the json/ directory are converted to Gettext POT files and placed in the pot/ directory. 🤝
If you are translating for the first time then you can skip the next step. If you need to recover your existing translations then we do the following:
json2po -t lang/ zu/ po-zu/Using the English .json files found in lang/ and your existing Zulu translation in zu/ we create a set of PO files in po-zu/. These will now have your translations. Please be aware that in order for the to work 100% you need to have both English and Zulu at the same revision. If they are not, you will have to review all translations. 😭
You are now in a position to translate your recovered translations or your new POT files. 💪
Once translated you can convert back as follows:
po2json -t lang/ po-zu/ zu/Your translations found in the Zulu PO directory, po-zu/, will be converted to .json using the files in lang/ as templates and placing your newly translated .json files in zu/. 🌟
To update your translations simply redo the POT creation step and make use of pot2po to bring your translation up-to-date. 🔁
🦋 md2po, po2md 🦋
Convert translatable items in Markdown text to the PO format. Insert translated text into Markdown templates. ✨
Usage
md2po [options] <md-src> <po>
po2md [options] -i <po> -t <md-src> -o <md-dest>Where:
<md-src>— is a Markdown file or a directory of Markdown files, source language 🌸<md-dest>— is a Markdown file or a directory of Markdown files, translated to the target language 💖<po>— is a PO file or directory of PO files 🎀
🎀 Options (md2po): 🎀
--version— show program’s version number and exit 🌈-h,--help— show this help message and exit ✨--manpage— output a manpage based on the help 💅--progress=PROGRESS— show progress as:dots,none,bar,names,verbose🦋--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback🎀-i INPUT,--input=INPUT— read fromINPUTin Markdown format 🌸-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths 🚫-o OUTPUT,--output=OUTPUT— write toOUTPUTinpo,potformats 💖-S,--timestamp— skip conversion if the output file has newer timestamp ⏰-P,--pot— output PO Templates (.pot) rather than PO files (.po) 👑--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:‘msgctxt’) 💀--multifile=MULTIFILESTYLE— how to split po/pot files (single,topleveloronefile) (default:‘single’; if set to‘onefile’, a single po/pot file will be written.‘toplevel’not used.) 🔥
🎀 Options (po2md): 🎀
--version— show program’s version number and exit 🌈-h,--help— show this help message and exit ✨--manpage— output a manpage based on the help 💅--progress=PROGRESS— show progress as:dots,none,bar,names,verbose🦋--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback🎀-i INPUT,--input=INPUT— read fromINPUTinpo,potformats 🌸-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths 🚫-o OUTPUT,--output=OUTPUT— write toOUTPUTin Markdown format 💖-t TEMPLATE,--template=TEMPLATE— read fromTEMPLATEin Markdown format 🧚-S,--timestamp— skip conversion if the output file has newer timestamp ⏰-m MAXLENGTH,--maxlinelength=MAXLENGTH— reflow (word wrap) the output to the given maximum line length. set to0to disable 🎀--threshold=PERCENT— only convert files where the translation completion is abovePERCENT🔥--fuzzy— use translations marked fuzzy 🙈--nofuzzy— don’t use translations marked fuzzy (default) 👑
🌟 Examples 🌟
md2po -P source-md-dir potThis will find all Markdown files (.md, .markdown, .txt, .text) in source-md-dir, convert them to POT files (that is, extract all translatable content) and place them in pot. 🫶
See the pot2po command for more information about how to create PO files from the POT files, and to update existing PO files when the POT files have changed. 👑
Suppose you have created PO files for translation to Xhosa and placed them in the directory xh. You can then generate the translated version of the Markdown documents like so:
po2md -i xh -t source-md-dir -o xh-md-dirAll the PO translations in xh will be converted to Markdown using Markdown files in source-md-dir as templates and outputting new translated Markdown files in xh-md-dir. 🌸
Should you prefer a single PO/POT file for your collection of Markdown files, this is how to do it:
md2po -P --multifile=onefile source-md-dir file.potAnd similarly, to generate multiple translated output files from a single PO file, invoke po2md like so:
po2md -i xh.po -t source-md-dir -o xh-md-dirIn this example, xh.po is the translation file for Xhosa, source-md-dir is the directory where the Markdown files in the source language can be found, and xh-md-dir is the directory where the translated Markdown files will end up. 💖
🌱 Notes 🌱
The Markdown format description gives more details on the format of the localisable Markdown content and the capabilities of this converter. 📝
🐚 moz2po 🐚
moz2po converts Mozilla files to PO files. It wraps converters that handle .properties, .dtd and some strange Mozilla files. The tool can work with files from Mozilla’s Mercurial repository. The tools thus provides a complete roundtrip for Mozilla localisation using PO files and PO editors. 🔥
⚠️ NOTE: This page should only be used as a reference to the command-line options formoz2poandpo2moz. For more about using the Translate Toolkit and PO files for translating Mozilla products, please see the page on Mozilla L10n Scripts. 🫶
Usage
moz2po [options] <dir> <po>
po2moz [options] <po> <dir>Where:
<dir>— is a directory containing valid Mozilla files 🦋<po>— is a directory of PO or POT files 🎀
🎀 Options (moz2po): 🎀
--version— show program’s version number and exit 🌈-h,--help— show this help message and exit ✨--manpage— output a manpage based on the help 💅--progress=PROGRESS— show progress as:dots,none,bar,names,verbose🦋--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback🎀-i INPUT,--input=INPUT— read fromINPUTininc,it,*,dtd,propertiesformats 🌸-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths 🚫-o OUTPUT,--output=OUTPUT— write toOUTPUTinit.po,it.pot,manifest,xhtml.po,xhtml.pot,ini.po,ini.pot,rdf,js,*,html.po,html.pot,inc.po,inc.pot,dtd.po,dtd.pot,properties.po,properties.potformats 💖-t TEMPLATE,--template=TEMPLATE— read fromTEMPLATEinit,*,properties,dtd,incformats 🧚-S,--timestamp— skips the conversion if the output file already has a newer timestamp 👑-P,--pot— outputs PO Templates (.pot) instead of PO files (.po) 🎀--duplicates=DUPLICATESTYLE— tells it what to do about duplicate strings (aka identical source text):mergeormsgctxt(default: ‘msgctxt’) 💅
✨ Options (po2moz) ✨
--version— shows the program’s version number and bails out 👋-h,--help— shows this help message and bails out--manpage— spits out a manpage built from the help 📄--progress=PROGRESS— shows progress as:dots,none,bar,names,verbose📊--errorlevel=ERRORLEVEL— shows errorlevel as:none,message,exception,traceback🚨-i INPUT,--input=INPUT— reads from INPUT indtd.po,dtd.pot,ini.po,ini.pot,inc.po,inc.pot,manifest,it.po,it.pot,*,html.po,html.pot,js,rdf,properties.po,properties.pot,xhtml.po,xhtml.potformats 🫶-x EXCLUDE,--exclude=EXCLUDE— keeps names matching EXCLUDE out of the input paths 🙈-o OUTPUT,--output=OUTPUT— writes to OUTPUT indtd,*,inc,it,propertiesformats-t TEMPLATE,--template=TEMPLATE— reads from TEMPLATE indtd,*,inc,it,propertiesformats-S,--timestamp— skips the conversion if the output file has a newer timestamp ⏰-l LOCALE,--locale=LOCALE— sets the output locale (required, bestie — it’s literally what sets the directory names) 📁--removeuntranslated— removes untranslated strings from the output 🧹--threshold=PERCENT— only converts files where the translation completion is above PERCENT 📈--fuzzy— uses translations marked fuzzy 🌸--nofuzzy— doesn’t use translations marked fuzzy (default) 🚫
🎀 Examples 🎀
🌟 Creating POT files
SEE ALSO: Creating Mozilla POT files.
So after you’ve extracted the en-US l10n files, you can run this:
moz2po -P l10n/en-US potThat creates a set of POT (-P) files in the pot directory, pulled from the Mozilla files in l10n/en-US, ready to be used as PO Templates 💖
And if you wanna make a set of POT files with a different base language? Try this:
moz2po -P l10n/fr-FR fr-potThis one creates a set of POT files in fr-pot that have French as your source language. slay 🇫🇷
🌟 Creating PO files from existing non-PO translations
Got existing translations already (Mozilla-related or other Babelzilla files) and you want them in PO so you can keep translating? These generic instructions will do the trick:
moz2po -t en-US af-ZA af-ZA_pofilesThis combines the untranslated template en-US files from en-US with your existing translations in af-ZA, and outputs PO files to af-ZA_pofiles ✨
moz2po -t l10n/fr l10n/xh po/xhFor those who aren’t English-fluent, you can do the same with another language. In this case the msgid will hold the French text from l10n/fr. This is genuinely useful when the translator’s other language isn’t English but French, Spanish or Portuguese. Just make sure the source languages — i.e. the msgid language — are fully translated as against en-US, okay? 🫶
🌟 Creating Mercurial ready translations
po2moz -t l10n/en-US po/xh l10n/xhCreates Mozilla files using the template files in l10n/en-US (peep above for how to create them) with the PO translations in po/xh, and outputs everything to l10n/xh. The files now sitting in l10n/xh are ready to submit to Mozilla and can be used to build a language pack or a translated version of Mozilla 👑
🌟 Issues
You can do the bulk of your work (like 99%) with moz2po 💪
Localisation of XHTML is not yet perfect — you might wanna work with the files directly. ngl, it is what it is 😭
Issue 203 tracks the outstanding features that would allow complete localisation of Mozilla, including all help, start pages, rdf files, etc. It also tracks some bugs 🐛
Accesskeys don’t yet work in .properties files, and in several cases where the Mozilla .dtd files don’t follow the normal conventions — for example in security/manager/chrome/pippki/pref-ssl.dtd.po. You might also wanna check the files mentioned in this Mozilla bug 329444, where mistakes in the DTD-definitions cause problems matching accelerators with the text 🙃
You might wanna give special attention to the following files, since they contain customisations that are not really translations:
mail/chrome/messenger/downloadheaders.dtd.potoolkit/chrome/global/intl.properties.po
Also, all width, height and size specifications need to be edited with feedback from testing the translated interfaces 📐
There are some constructed strings in the Mozilla code that we can’t do much about. Take good care to read the localisation notes. For an example, see mail/chrome/messenger/downloadheaders.dtd.po. In that specific file, the localisation note from the DTD file is lost, so take good care of those. ⚠️
The file extension of the original Mozilla file is required to tell the Toolkit how to do the conversion. So a file like foo.dtd must be named foo.dtd.po for po2moz to recognise it as a DTD file 💅
🌈 odf2xliff and xliff2odf 🌈
Converts OpenDocument (ODF) files to XLIFF localization files. And it creates translated ODF files by combining the original ODF files with XLIFF files containing translations of strings in the original document. Two-for-one, iconic ✨
XLIFF is the XML Localization Interchange File Format developed by OASIS (The Organization for the Advancement of Structured Information Standards) to allow translation work to be standardised no matter what the source format is, and to let the work be freely moved from tool to tool 🦋
If you’re more used to software translation or l10n, you might wanna read a bit about Document translation. That should help you get the most out of translating ODF with XLIFF 💜
💫 Usage
odf2xliff [options] <original_odf> <xliff>
xliff2odf [options] -t <original_odf> <xliff> <translated_odf>Where, bestie:
<original_odf>— is an ODF document whose strings have to be translated<xliff>— is an XLIFF file<translated_odf>— is an ODF file to generate by replacing the strings in<original_odf>with the translated strings in<xliff>
💫 Options (odf2xliff)
--version— shows the program’s version number and bails out 👋-h,--help— shows this help message and bails out--manpage— spits out a manpage built from the help 📄--progress=PROGRESS— shows progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— shows errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— reads from INPUT in ODF format 📥-o OUTPUT,--output=OUTPUT— writes to OUTPUT in XLIFF format 📤-S,--timestamp— skips the conversion if the output file has a newer timestamp ⏰
💫 Options (xliff2odf)
--version— shows the program’s version number and bails out 👋-h,--help— shows this help message and bails out--manpage— spits out a manpage built from the help 📄--progress=PROGRESS— shows progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— shows errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— reads from INPUT in XLIFF formats 📥-o OUTPUT,--output=OUTPUT— writes to OUTPUT in ODF format 📤-t TEMPLATE,--template=TEMPLATE— reads from TEMPLATE in ODF format-S,--timestamp— skips the conversion if the output file has a newer timestamp ⏰
💫 Examples
odf2xliff english.odt english_français.xlfCreates an XLIFF file from an ODT file (and the source ODF file could also be any of the other ODF files too, including ODS, ODG, etc., no cap) 🎀
xliff2odf -t english.odt english_français.xlf français.odtUses english.odt as the template document and english_français.xlf as the file of translations, and creates the translated file français.odt 🇫🇷💖
💫 Bugs
This filter is not yet extensively used — we appreciate your feedback, fr fr 🫶 For more info on conformance to standards, see the XLIFF or OpenDocument Format pages.
👑 oo2po 👑
Converts between OpenOffice.org GSI/SDF files and the PO format. This tool gives you a complete roundtrip: it preserves the structure of the GSI file and creates completely valid PO files 💅
Heads up: oo2xliff will convert the SDF files to XLIFF format 🌈
⭐ Usage
oo2po [options] <sdf> <output>
po2oo [options] [-t <en-US.sdf>] -l <targetlang> <input> <sdf|output>Or, for XLIFF files:
oo2xliff [options] -l <targetlang> <sdf> <output>
xliff2oo [options] [-t <en-US.sdf>] -l <targetlang> <input> <sdf|output>Where:
<sdf>— a valid OpenOffice.org GSI or SDF file ✨<output>— a directory for the resultant PO/POT/XLIFF files<input>— a directory of translated PO/XLIFF files<targetlang>— the ISO 639 language code used in the sdf file, e.g.af
🎀 Options (oo2po and oo2xliff)
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inoo,sdfformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,pot,xlfformats-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po) (only available inoo2po)-l LANG,--language=LANG— set target language to extract from oo file (e.g.af-ZA) (required foroo2xliff)--source-language=LANG— set source language code (defaulten-US)--nonrecursiveinput— don't treat the input oo as a recursive store--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:'msgctxt')--multifile=MULTIFILESTYLE— how to split po/pot files (single,topleveloronefile)
💅 Options (po2oo and xliff2oo)
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,pot,xlfformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inoo,sdfformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inoo,sdfformats-S,--timestamp— skip conversion if the output file has newer timestamp-l LANG,--language=LANG— set target language code (e.g.af-ZA) [required]--source-language=LANG— set source language code (defaulten-US)-T,--keeptimestamp— don't change the timestamps of the strings--nonrecursiveoutput— don't treat the output oo as a recursive store--nonrecursivetemplate— don't treat the template oo as a recursive store--skipsource— don't output the source language, but fallback to it where needed--filteraction=ACTION— action onpofilterfailure:none(default),warn,exclude-serious,exclude-all--threshold=PERCENT— only convert files where the translation completion is above PERCENT--fuzzy— use translations marked fuzzy--nofuzzy— don't use translations marked fuzzy (default)--multifile=MULTIFILESTYLE— how to split po/pot files (single,topleveloronefile)
✨ Examples ✨
Ok bestie, these examples show off basically every useful way to call oo2po — let's go 🫶
Creating POT files
oo2po -P en-US.sdf potThis one grabs the messages out of en-US.sdf and drops them into a directory called pot. The -P is doing the heavy lifting here — it's what makes us create POT files instead of PO files. Iconic behaviour, honestly 💅
oo2po -P --source-language=fr fr-FR.sdf french-potNow instead of making English POT files, we're making POT files that have French sitting in the msgid. This is literally so useful for translators who aren't English-literate. Fair warning though: you need to have a fully translated sdf in the source language for this to work, no cap.
Creating PO files from existing work
oo2po --duplicates=merge -l zu zu-ZA.sdf zuluExtracts all the existing Zulu (zu) messages out of zu-ZA.sdf and puts them in a directory called zulu. And if you find duplicate messages in a file, --duplicates=merge squishes them into one single message (this is the default behaviour for traditional PO files, fyi). You might wanna run pomigrate2 after this to make sure your PO files match the latest POT files:
cat GSI_af.sdf GSI_xh.sdf > GSI_af-xh.sdf
oo2po --source-language=af -l xh GSI_af-xh.sdf af-xh-poHere we're making PO files with your existing translations but with a different source language, which is kind of a flex ngl. First we combine the two SDF files. Then oo2po creates a set of PO files in af-xh-po using Afrikaans (af) as the source language and Xhosa (xh) as the target language, all pulled from the combined SDF file GSI_af-xh.sdf.
Creating a new GSI/SDF file
po2oo -l zu zulu zu_ZA.sdfUses the PO files found in zulu to create an SDF file called zu_ZA.sdf for language zu 🦋
po2oo -l af -t en-US.sdf --nofuzzy --keeptimestamp --filteraction=exclude-serious afrikaans af_ZA.sdfThis creates an Afrikaans (af) SDF file called af_ZA.sdf, using en-US.sdf as a template, preserving the timestamps inside the SDF file, and also eliminating any serious errors in the translation. Let me break down why each bit eats:
Using templates ensures the resultant SDF file has exactly the same format as the template SDF file. In an SDF file each translated string can have a timestamp attached, and that creates a large amount of unuseful traffic when you're comparing versions of the SDF file — by preserving the timestamp we ensure it doesn't change, so we can see the translation changes clearly. We've included the nofuzzy option (on by default) which prevents fuzzy PO messages from getting into the SDF file. And lastly filteraction is set to exclude-serious: variable failures and translated XML will be excluded from the final SDF. 🎀
🧚 helpcontent2
The escaping of helpcontent2 from SDF files was very confusing — issue 295 implemented a fix that appeared in version 1.1.0 (all known issues were fixed in 1.1.1). Translators are now able to translate helpcontent2 with clean escaping. Slay 💜
🌸 php2po
Converts PHP localisable string arrays to Gettext PO format.
Usage
php2po [options] <php> <po>
po2php [options] <po> <php>Where:
<php>— a valid PHP localisable file or a directory of those files<po>— a directory of PO or POT files
🌈 Options (php2po)
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inphpformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inphpformat-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:'msgctxt')
💖 Options (po2php)
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inphpformat-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inphpformat-S,--timestamp— skip conversion if the output file has newer timestamp--threshold=PERCENT— only convert files where the translation completion is above PERCENT--fuzzy— use translations marked fuzzy--nofuzzy— don't use translations marked fuzzy (default)
⭐ Formats Supported
Go check the PHP format document to see to which extent the PHP format is supported 👑
🍓 Examples
This example looks at the roundtrip of PHP translations as well as recovery of existing translations.
First we need to create a set of POT files.:
php2po -P lang/en pot/All .php files found in the lang/en directory get converted to Gettext POT files and placed in the pot directory. Easy 💅
If you're translating for the first time then you can skip the next step. But if you need to recover your existing translations, here's what we do:
php2po -t lang/en lang/zu po-zu/Using the English PHP files found in lang/en and your existing Zulu translation in lang/zu, we create a set of PO files in po-zu. Those are gonna have your translations in them now 💖 but bestie, listen — for that to work 100% you need both English and Zulu at the same revision. If they're not? You're gonna have to review all your translations. No skipping, iykyk 😭
You're now in a position to translate your recovered translations or your new POT files. Slay.
Once translated you can convert back as follows:
po2php -t lang/en po-zu/ lang/zuYour translations found in the Zulu PO directory, po-zu, will be converted to PHP using the files in lang/en as templates and placing your new translations in lang/zu. Chef's kiss 🫶
To update your translations, simply redo the POT creation step and make use of pot2po to bring your translation up-to-date. Periodt.
po2tmx
Convert Gettext PO files to a TMX translation memory file. TMX is the Translation Memory eXchange format developed by OSCAR. ✨
- OSCAR (Open Standards for Container/Content Allowing Re-use), a special interest group of the now defunct LISA (Localization Industry Standards Association). The Gala LISA OSCAR Standards page has more details on the possible future for the standards.
If you are interested in po2tmx, you might also be interested in posegment that can be used to perform some automated segmentation on sentence level. 👀
Usage
po2tmx [options] --language <target> <po> <tmx>Where:
┌───────┬───────────────┐
│ <po> │ is a PO file │
├───────┼───────────────┤
│ <tmx> │ is a TMX file │
└───────┴───────────────┘Options
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT intmxformat-S,--timestamp— skip conversion if the output file has newer timestamp-l LANG,--language=LANG— set target language code (e.g.af-ZA) [required]--source-language=LANG— set source language code (default:en)--comments=COMMENT— set default comment import:none,source,typeorothers(default:none)
Examples
po2tmx -l xh browser.po browser.tmxUse the Xhosa (xh) translations in the PO file browser.po to create a TMX file called browser.tmx. Boom, done 💅
Bugs and issues
Markup stripping
po2tmx conforms to TMX v1.4 without stripping markup. See the TMX conformance page for more details.
It has not been widely tested so your mileage may vary. 🙈
TMX and PO in OmegaT
In some tools, like OmegaT, PO files are parsed without expanding escaped sequences, even though such tools use TMX for translation memory. Keep this in mind when using po2tmx, because po2tmx converts \n and \t to newlines and tabs in the TMX file. If such a TMX file is used while translating PO files in OmegaT, matching will be less than 100%. 😬
In other tools, such as Swordfish, the PO comment "no-wrap" is interpreted in the same way as the equivalent function in XML, which may also lead to mismatches if TMXes from po2tmx are used.
There is nothing wrong with po2tmx, but if used in conjunction with tools that handle PO files differently, it may lead to less than perfect matching. Not a villain arc, just a compatibility note 🧚
Tips
TMX with only unique segments
To create a TMX with no duplicates (in other words, only unique strings), use msgcat to first create a large PO file with non-uniques removed. Big brain move ⭐
po2wordfast
Convert Gettext PO files to a Wordfast Translation Memory translation memory file.
Wordfast is a popular Windows based computer-assisted translation tool. 🪟
Usage
po2wordfast [options] --language <target> <po> <wordfast>Where:
┌────────────┬────────────────────────┐
│ <po> │ a PO file or directory │
├────────────┼────────────────────────┤
│ <wordfast> │ a Wordfast translation │
│ │ memory file │
└────────────┴────────────────────────┘Options
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT intmxformat-S,--timestamp— skip conversion if the output file has newer timestamp-l LANG,--language=LANG— set target language code (e.g.af-ZA) [required]--source-language=LANG— set source language code (default:en)
Examples
po2wordfast -l xh-ZA browser.po browser.txtUse the Xhosa (xh-ZA) translations in the PO file browser.po to create a Wordfast translation memory file called browser.txt. Obsessed with how clean that is 🌸
pot2po
Convert a Gettext PO Template file to a PO file and merge in existing translations if they are present. A translation memory (compendium) can also be used for fuzzy matching. This corresponds to a large extent with the program msgmerge from the gettext package. ✨
NOTE: This tool also works with translation formats other than Gettext PO, for example XLIFF. 🦋
Usage
pot2po [options] <pot> <po>Where:
┌───────┬──────────────────────────┐
│ <pot> │ is a PO Template (POT) │
│ │ file or directory of POT │
│ │ files │
├───────┼──────────────────────────┤
│ <po> │ is a PO file or a │
│ │ directory of PO files │
└───────┴──────────────────────────┘Options
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT incatkeys,lang,pot,ts,xlf,xliffformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT incatkeys,lang,po,pot,ts,xlf,xliffformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE incatkeys,lang,po,pot,ts,xlf,xliffformats (old translations)-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--tm=TM— The file to use as translation memory when fuzzy matching-s MIN_SIMILARITY,--similarity=MIN_SIMILARITY— The minimum similarity for inclusion (default:75%)--nofuzzymatching— Disable all fuzzy matching
Examples
pot2po -t zu-1.0.1 pot-2.0.2 zu-2.0.2Here we are initialising the PO files in zu-2.0.2 based on the POT files in pot-2.0.2. We are using the old translations in zu-1.0.1 as templates so that we can reuse our existing translations in the new files. Reuse era 👑
pot2po can also be used to update against newer templates an existing translation file in a format different than Gettext PO, for example XLIFF:
pot2po -t af.xlf -i templates.xlf -o updated-af.xlfIf the POT files have undergone major reshuffling then you may want to use pomigrate2 which can now use pot2po as its merging backend. pomigrate2 will do its best to migrate your files to the correct locations before merging. It will also make use of a compendium if requested. 🫶
pot2po --tm=compendium.po --similarity=60 -t xh-old pot xh-newWith this update we are using compendium.po as a translations memory (you can make use of other files such as TMX, etc). We will accept any match that scores above 60%.
Merging
It helps to understand when and how pot2po will merge. The default is to follow msgmerge's behaviour but we add some extra features with fuzzy matching:
- If everything matches we carry that across
- We can resurrect obsolete messages for reuse
- Messages no longer used are made obsolete
- If we cannot find a match we will first look through the current and obsolete messages and then through any global translation memory
- Fuzzy matching makes use of the Levenshtein distance algorithm to detect the best matches
Performance
Fuzzy matches are usually of good quality. Installation of the python-Levenshtein package will speed up fuzzy matching. Without this a Python based matcher is used which is considerably slower. ⚡
Bugs
pomergeandpot2poshould probably become one.
prop2po
Convert between Java property files (.properties) and Gettext PO format.
Note: this tool completely eliminates the need fornative2asciiaspo2propdoes the correct escaping to the Latin1 encoding that is needed by Java. 💅
The following other formats are also supported via the --personality parameter:
- Adobe Flex
- Skype
.lang - Mac OS X
.strings - Mozilla
.properties
Usage
prop2po [options] <property> <po>
po2prop [options] -t <template> <po> <property>Where:
<property>— is a directory containing property files or an individual property file 🗂️<po>— is a directory containing PO files and an individual property file<template>— is a directory of template property files or a single template property file
✨ Options (prop2po) ✨
--version— show program’s version number and exit 💜-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT in properties format-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE in properties format-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--personality=TYPE— override the input file format:flex,java,mozilla,java-utf8,skype,gaia,strings(for.propertiesfiles, default:java)--encoding=ENCODING— override the encoding set by the personality--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default: ‘msgctxt’)
💅 Options (po2prop) 💅
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT in properties format-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE in properties format-S,--timestamp— skip conversion if the output file has newer timestamp--personality=TYPE— override the input file format:flex,java,mozilla,java-utf8,skype,gaia,strings(for.propertiesfiles, default:java)--encoding=ENCODING— override the encoding set by the personality (since1.8.0) ⭐--removeuntranslated— remove untranslated strings from output--threshold=PERCENT— only convert files where the translation completion is above PERCENT--fuzzy— use translations marked fuzzy--nofuzzy— don’t use translations marked fuzzy (default)
🌈 Examples
These examples demonstrate most of the useful invocations of prop2po:
🌸 Creating POT files
prop2po -P properties potExtract messages from properties directory and place them in a directory called pot. The -P option ensures that we create POT files instead of PO files. 🫶
prop2po -P file.properties file.potExtract messages from file.properties and place them in file.pot.
🦋 Creating PO files from existing work
prop2po --duplicates=msgctxt -t reference zu zu-poExtract all existing Zulu messages from zu directory and place the resultant PO files in a directory called zu-po. If you find duplicate messages in a file then use Gettext’s mgsctxt to disambiguate them. During the merge we use the .properties files in reference as templates and as the source of the English text for the msgid. Once you have your PO files you might want to use pomigrate2 to ensure that your PO files match the latest POT files.
🎀 Creating .properties files from your translations
po2prop -t reference zu-po zuUsing our translations found in zu-po and the templates found in reference we create a new set of property files in zu. These new property files will look exactly like those found in the templates, but with the text changed to the translation. Any fuzzy entry in our PO files will be ignored and any untranslated item will be placed in zu in English. The .properties file created will be based on the Java specification and will thus use escaped Unicode. Where:
ṽḁḽṻḝWill appear in the files as:
\u1E7D\u1E01\u1E3D\u1E7B\u1E1DTo get output as used by Mozilla localisation do the following:
po2prop --personality=mozilla -t reference zu-po zuThis will do exactly the same as above except that the output will now appear as real Unicode characters in UTF-8 encoding. 👑
🧚 Doing away with native2ascii
The native2ascii command is the traditional tool of property file localisers. With prop2po there is no need to use this command or to ever work directly with the escaped Unicode. 💀
If you are working mostly with Gettext PO files then this is a double benefit as you can now use your favourite PO editor to translate Java applications. Your process would now look like this:
prop2po some.properties some.poFirstly create a PO file that you can translate. Now translate it in your favourite PO editor. 🔥
po2prop -t some.properties some.po some-other.propertiesUsing the original properties file as a template we preserve all layout and comments, combined with your PO translation we create a new translate properties file. During this whole process we have not needed to understand or process any escaping prop2po and po2prop handle that all automatically.
If you have existing translations you can recover them as follows:
prop2po -t some.properties translations.properties translations.poThis takes the default English properties file and combines it with your translate properties file and created a PO file. You now continue translating using your PO file. 💖
🔥 rc2po
Converts Windows Resource .rc files to Gettext PO format.
⭐ Usage
rc2po [options] <rc> <po>
po2rc [options] -t <rc> <po> <rc>Where:
<rc>— is a valid Windows Resource file or directory of those files<po>— is a directory of PO or POT files
🌟 Options (rc2po)
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inrcformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inrcformat-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--charset=CHARSET— charset to use to decode the RC files (default:cp1252)-l LANG,--lang=LANG— LANG entry (default:LANG_ENGLISH)--sublang=SUBLANG— SUBLANG entry (default:SUBLANG_DEFAULT)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default: ‘msgctxt’)
💜 Options (po2rc)
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inrcformat-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inrcformat-S,--timestamp— skip conversion if the output file has newer timestamp--charset=CHARSET— charset to use to decode the RC files (default:utf-8)-l LANG,--lang=LANG— LANG entry--sublang=SUBLANG— SUBLANG entry (default:SUBLANG_DEFAULT)--threshold=PERCENT— only convert files where the translation completion is above PERCENT--fuzzy— use translations marked fuzzy--nofuzzy— don’t use translations marked fuzzy (default)
🎀 Formats Supported
NOTE: This implementation is based mostly on observing WINE.rcfiles, these should mimic other non-WINE.rcfiles.
🌈 Examples
This example looks at roundtrip of Windows Resource translations as well as recovery of existing translations.
First we need to create a set of POT files. 🫶
rc2po -P lang/ pot/All .rc files found in the lang/ directory are converted to Gettext POT files and placed in the pot/ directory.
If you are translating for the first time then you can skip the next step. If you need to recovery your existing translations then we do the following:
rc2po -t lang zu po-zu/Using the English .rc files found in lang and your existing Zulu translation in zu we create a set of PO files in po-zu. These will now have your translations. Please be aware that in order for the to work 100% you need to have both English and Zulu at the same revision, if they are not you will have to review all translations. Also the .rc files may be in different encoding, we cannot at the moment process files of different encodings and assume both are in the same encoding supplied.
You are now in a position to translate your recovered translations or your new POT files.
Once translated you can convert back as follows:
po2rc -t lang/ po-zu/ zu/Your translations found in the Zulu PO directory, po-zu, will be converted to .rc using the files in lang/ as templates and placing your new translations in zu/.
To update your translations simply redo the POT creation step and make use of pot2po to bring your translation up-to-date.
Issues
If you are recovering translation using rc2po -t en.rc xx.rc xx.po then both en.rc and xx.rc need to be in the same encoding.
There might be problems with MENUs that are deaply nested. 💀 (yes, that's "deeply" — the manpage said it like that, and we're honoring it, no cap)
resx2po
Converts .Net Resource (.resx) files to Gettext PO format — that monolingual file format used in Microsoft .Net Applications. It's giving crossover episode energy fr fr 💅
Usage
resx2po [options] <resx> <po>
po2resx [options] <po> <resx> -t <resx>Where:
<resx>— is a valid.resxfile or directory of those files<po>— is a directory of PO or POT files
Options (resx2po)
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT in RESX format-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE in RESX format-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--filter=FILTER— leaves to extract e.g.'name,desc': (default: extract everything)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:'msgctxt')
Options (po2resx)
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT in RESX format-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE in RESX format-S,--timestamp— skip conversion if the output file has newer timestamp--fuzzy— use translations marked fuzzy--nofuzzy— don’t use translations marked fuzzy (default)
Examples
This example looks at roundtrip of .resx translations as well as recovery of existing translations. Iconic behavior honestly ✨
First we need to create a set of POT files:
resx2po -P resx/ pot/All .resx files found in the resx/ directory are converted to Gettext POT files and placed in the pot/ directory. Slay 🎀
If you are translating for the first time then you can skip the next step. If you need to recover your existing translations then we do the following:
resx2po zu/ po-zu/ -t lang/Using the English .resx files found in lang/ and your existing Zulu translation in zu/ we create a set of PO files in po-zu/. These will now have your translations. Please be aware that in order for the to work 100% you need to have both English and Zulu at the same revision. If they are not, you will have to review all translations.
You are now in a position to translate your recovered translations or your new POT files.
Once translated you can convert back as follows:
po2resx po-zu/ zu/ -t lang/Your translations found in the Zulu PO directory, po-zu/, will be converted to .resx using the files in lang/ as templates and placing your newly translated .resx files in zu/.
To update your translations simply redo the POT creation step and make use of pot2po to bring your translation up-to-date.
sub2po
sub2po allows you to use the same principles of PO files with Subtitles. In PO only items that change are marked fuzzy and only new items need to be translated, unchanged items remain unchanged for the translation. It's honestly such a clean system, obsessed 😭👑
Usage
sub2po [options] <foo.srt> <foo.po>
po2sub [options] [-t <foo.srt>] <XX.po> <foo-XX.srt>Where:
foo.srt— is the input subtitle filefoo.po— is an empty PO file that may be translatedXX.po— is a PO file translated into the XX languagefoo-XX.srt— is thefoo.srtfile translated into language XX
Options (sub2po)
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT in.srtformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inass,srt,ssa,subformats-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:'msgctxt')
Options (po2sub)
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT insrtformat-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE intxtformat-S,--timestamp— skip conversion if the output file has newer timestamp--threshold=PERCENT— only convert files where the translation completion is above PERCENT--fuzzy— use translations marked fuzzy--nofuzzy— don’t use translations marked fuzzy (default)
Examples
To create the POT files is simple:
sub2po -P SUBTITLE_FILE subtitles.potA translator would copy the POT file to their own PO file and then create translations of the entries. If you wish to create a PO file and not a POT file then leave off the -P option. Easy peasy 🌸
To convert back:
po2sub -t SUBTITLE_FILE subtitles-XX.po subtitles-XX.srtTranslating
Translate as normal. However, see the issues mentioned at Subtitles. 💜
Bugs
There might be some issues with encodings, since the srt files don’t specify them. We assume files to be encoded in UTF-8, so a conversion should solve this easily. Note that most of the handling of the srt files come from gaupol.
symb2po
Added in version 1.3. 🦋
Converts Symbian-style translation files to PO files and vice versa. The Symbian translation files currently have a strong Buddycloud flavour, but the tools will be made more general as the need arises. Ngl that's kind of a vibe 👑
Usage
symb2po [options] [-t <target_lang_symb>] <source_lang_symb> <po>
po2symb [options] -t <target_lang_symb> <po> <target_lang_symb>Where: 🌸
<target_lang_symb>— is a valid Symbian translation file or directory of those files<source_lang_symb>— is a valid Symbian translation file or directory of those files<po>— is a PO or POT file or a directory of PO or POT files
✨ Options (symb2po) ✨
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT in php format-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE in the Symbian translation format-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:‘msgctxt’)
💅 Options (po2symb) 💅
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT in php format-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE in the Symbian translation format-S,--timestamp— skip conversion if the output file has newer timestamp
🎀 Examples 🎀
symb2po
Okay so the most common use of symb2po is to generate a POT (PO template) file from the English translation — and heads up, the tool currently expects the Symbian translation file to end with the extension .r01, which is the code for English translation files. 💖 That file then serves as the source document from which all translations will be derived. Iconic behaviour, honestly.
To create a POT file called my_project.pot from the source Symbian translation file my_project.r01, the following is executed:
symb2po my_project.r01 my_project.potAnd when you want to re-use existing translations in the Symbian translation format, symb2po can merge that translation into the source Symbian translation to produce a translated PO file. The existing Symbian translation file is specified with the -t flag. Slay.
To create a file called my_project-en-fr.po (this is not the recommended PO naming convention, just so you know 🫶) from the source Symbian translation file my_project.r01 and its French translation my_project.r02, execute:
symb2po -t my_project.r02 my_project.r01 my_project-en-fr.poNOTE: Ensure that the English and French files are well aligned — in other words, no changes to the source text should have happened since the translation was done. fr fr, this one matters 😭
po2symb
The po2symb tool is used to extract the translations in a PO into a template Symbian translation file. The template Symbian translation file supplies the "shape" of the generated file (formatting and comments). It's giving scaffolding, honestly.
So in order to produce a French Symbian translation file using the English Symbian translation file my_project.r01 as a template and the PO file my_project-en-fr.po (again, this is not the recommended PO naming convention) as the source document, execute:
po2symb -t my_project.r01 my_project-en-fr.po my_project.r02🌈 Notes 🌈
The tools won't touch anything appearing between lines marked as:
// DO NOT TRANSLATEThe string r_string_languagegroup_name is used to set the Language-Team PO header field. 👑
The Symbian translation header field Author is used to set the Last-Translator PO header field.
💀 Issues 💀
The file format is heavily tilted towards the Buddycould implementation.
The tools do nothing with the Name and Description Symbian header fields. Which means po2symb will just copy the values in the supplied template. So you might see something such as:
Description : Localisation File : English...sitting there in a generated French translation file. Not me seeing that and screaming 🙈 it is what it is.
🍓 Bugs 🍓
Probably many, since this software hasn't been tested much yet. 💀
🦋 tbx2po 🦋
Convert between TermBase eXchange (.tbx) glossary format and Gettext PO format.
Usage
tbx2po <tbx> <po>Where:
<tbx>— is a TBX file<po>— is the target PO file
✨ Options (tbx2po) ✨
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT in csv format-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT in tbx format-S,--timestamp— skip conversion if the output file has newer timestamp
💜 Examples 💜
These examples demonstrate the use of tbx2po:
tbx2po terms.tbx terms.po...to simply convert terms.tbx to terms.po. Easy.
To convert a directory recursively to another directory with the same structure of files:
tbx2po tbx-dir po-target-dirThis will convert TBX files in tbx-dir to PO files placed in po-target-dir. Recursion ate, no crumbs. 🔥
🌸 Notes 🌸
For conformance to the standards and to see which features are implemented, see PO Files and TBX.
🌟 tiki2po 🌟
Converts TikiWiki language.php files to Gettext PO format.
Usage
tiki2po [options] <tiki> <po>
po2tiki [options] <po> <tiki>Where:
<tiki>— is a validlanguage.phpfile for TikiWiki<po>— is a PO file
✨ Options (tiki2po) ✨
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT in php format-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-S,--timestamp— skip conversion if the output file has newer timestamp--include-unused— When converting, include strings in the "unused" section?
💅 Options (po2tiki) 💅
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT in php format-S,--timestamp— skip conversion if the output file has newer timestamp
🎀 Examples 🎀
These examples demonstrate the use of tiki2po:
tiki2po language.php language.poConvert the tiki language.php file to .po:
po2tiki language.po language.phpConvert a .po file to a tiki language.php file. Round trip, bestie. 🔁
🌈 Notes 🌈
- Templates are not currently supported.
🧚 ts2po 🧚
Convert Qt .ts localization files to Gettext .po format files using ts2po, and convert the translated PO Files files back to Qt .ts using po2ts.
The Qt toolkit comes with a localization application, Qt Linguist — however you might wish to standardise on one localization tool. ts2po allows you to standardise on the PO format and PO related tools. Main character energy for the PO ecosystem, honestly. 👑
NOTE: Virtaal and Pootle can edit .ts files directly without the need for any conversion.
WARNING:po2tsuses our older.tssupport. Thus many of the newer features in.tsare not supported. To support those features rather edit directly in Virtaal or Pootle. 😭 Please don't come for me, I'm just the messenger.
Usage
ts2po [options] <ts> <po>
po2ts [options] <po> <ts>Where:
<ts>— is a Qt.tsfile or a directory that contains.tsfiles<po>— is a PO file or a directory of PO files
✨ Options (ts2po) ✨
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT intsformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:msgctxt)
🌸 Options (po2ts) 🌸
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT intsformat-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE intsformat-S,--timestamp— skip conversion if the output file has newer timestamp-c CONTEXT,--context=CONTEXT— use supplied context instead of the one in the.pofile comment
🎀 Examples 🎀
ts2po -P psi.ts psi.potThis will create a POT file called psi.pot from the Qt .ts file called psi.ts. Obsessed with how clean that is 💅
po2ts af.po psi_af.tsNow take your translated PO files af.po and convert it into a translated Qt .ts file, psi_af.ts. Slay.
NOTE: You need to use the tools from the Qt toolkit to create the compiled.qmlanguage files for the application. Don’t skip this step bestie, the.tsfile alone is not it 😭
💀 Bugs 💀
There are probably still some bugs related to migrating the various attributes across for the different formats. The converters don’t support all the newer features of the TS format, whereas the native support of Virtaal and Pootle is much better. Ngl, it’s giving "works but know the limits" — lowkey be careful with the shinier TS stuff 🫶
🦋 txt2po 🦋
txt2po allows you to use the same principles of PO files with normal text files. In PO only items that change are marked fuzzy and only new items need to be translated, unchanged items remain unchanged for the translation. Literally the whole point — your unchanged bits stay put, no cap ✨
🌈 Usage 🌈
txt2po [options] <foo.txt> <foo.po>
po2txt [options] [-t <foo.txt>] <XX.po> <foo-XX.txt>Where:
foo.txt— is the input plain text filefoo.po— is an empty PO file that may be translatedXX.po— is a PO file translated into the XX languagefoo-XX.txt— is thefoo.txtfile translated into language XX
👑 Options (txt2po) 👑
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT in*,txtformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--encoding=ENCODING— The encoding of the input file (default:UTF-8)--flavour=FLAVOUR— The flavour of text file:plain(default),dokuwiki,mediawiki--no-segmentation— Don’t segment the file, treat it like a single message--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:msgctxt)
🍓 Options (po2txt) 🍓
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT intxtformat-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE intxtformat-S,--timestamp— skip conversion if the output file has newer timestamp--encoding=ENCODING— The encoding of the template file (default:UTF-8)-w WRAP,--wrap=WRAP— set number of columns to wrap text at--threshold=PERCENT— only convert files where the translation completion is above PERCENT--fuzzy— use translations marked fuzzy--nofuzzy— don’t use translations marked fuzzy (default)
🧚 A roundtrip example 🧚
💜 Preparing input files 💜
With txt2po a text file is broken down into sections. Each section is separated by a line of whitespace. Each section will appear as a msgid in the PO file. Because of this simple method of breaking up the input file it might be necessary to alter the layout of your input file. For instance you might want to separate a heading from a paragraph by using whitespace. Like, actually think about your blank lines, they’re doing the heavy lifting here 😭
For steps in a process you would want to leave a blank line between each step so that each step can be translated independently.
For a list of items you might want to group them together so that a translator could for example place them in alphabetic order for their translation.
Once the input file is prepared you can proceed to the next step.
🌟 Creating the POT files 🌟
This is simple:
txt2po -P TEXT_FILE text_file.potA translator would copy the POT file to their own PO file and then create translations of the entries. If you wish to create a PO file and not a POT file then leave off the -P option.
You might want to manually edit the POT file to remove items that should not be translated. For instance if part of the document is a license you might want to remove those if you do not want the license translated for legal reasons. Periodt — sometimes you literally have to hand-edit, it’s fine 💅
🦄 Translating 🦄
Translate as normal. However translators should be aware that writers of the text file may have used spaces, dashes, equals, underscores and other aids to indicate things such as:
- Headings and sub-headings
- Code examples, command lines examples
- Various lists
- etc
They will need to adapt these to work in their language being aware of how they will appear once they are merged with the original text document. Iykyk — the vibes of the formatting have to survive translation too 🌸
🎀 Creating a translated text file 🎀
With the translations complete you can create a translated text file like this:
po2txt -w 75 -t TEXT_FILE translated.po TEXT_FILE.translatedThis uses the original text file as a template and creates a new translated text file using the translations found in the PO file.
The -w command allows you to reflow the translated text to N number of characters, otherwise the text will appear as one long line. So yeah, use -w unless you want one cursed mega-line 💀
🌈 Help with Wiki syntax 🌈
🫶 dokuwiki 🫶
To retrieve the raw syntax for your dokuwiki page add ?do=export_raw to you URL. The following would retrieve the DokuWiki home page in raw dokuwiki format https://www.dokuwiki.org/dokuwiki?do=export_raw
wget https://www.dokuwiki.org/dokuwiki?do=export_raw -O txt2po.txt
txt2po --flavour=dokuwiki -P txt2po.txt txt2po.pot
# edit txt2po.pot
po2txt -t txt2po.txt fr.po fr.txtFirst we retrieve the file in raw dokuwiki format, then we create a POT file for editing. We created a French translation and using po2txt plus the original file as a template we output fr.txt which is a French version of the original txt2po.txt. This file can now be uploaded to the wiki server. Full circle, iconic 🔥
⭐ MediaWiki ⭐
To retrieve the raw media wiki syntax add ?action=raw to you wiki URL. The following retrieves the Translate Toolkit page from Wikipedia in raw MediaWiki format Translate_Toolkit?action=raw.
To process follow the instructions above but substituting the MediaWiki retrieval method.
🦋 web2py2po 🦋
Converts web2py translation files to PO files and vice versa.
Web2py, formerly known as Gluon) is an open-source, Python-based web application framework by Massimo Di Pierro (inspired by Django and Rails).
Web2py uses an internal localization engine based on Python dictionaries, which is applied with the T() lookup function. Web2py provides a built-in translation interface for the T()-engine, which is excellent for rapid application development. Honestly? Chef’s kiss for getting your app localized fast 🫶✨
On the flip side, if you're doing collab + workflow control in a bigger community, you'll probably rather want to use Pootle, Launchpad or similar facilities for translation — which means you need to transform the web2py dictionaries into PO files and vice versa. And that is exactly what the web2py2po converters are for. Bestie, they ate. 💅
Usage
web2py2po [options] <web2py> <po>
po2web2py [options] <po> <web2py>Where:
<web2py>— is a valid web2py translation file<po>— is a PO or POT file or a directory of PO or POT files
Options (web2py2po)
--version— show program's version number and exit ✨-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inphpformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:'msgctxt')
Options (po2web2py)
--version— show program's version number and exit ✨-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inphpformat-S,--timestamp— skip conversion if the output file has newer timestamp--threshold=PERCENT— only convert files where the translation completion is above PERCENT--fuzzy— use translations marked fuzzy--nofuzzy— don't use translations marked fuzzy (default)
Notes
Handling of blanks/untranslated messages:
Untranslated messages in the web2py translation files are usually marked with a leading "*** ", so:
- All target strings from the web2py sources with a leading
"*** "are inserted as blankmsgstr's into the PO result (web2py2po) - Blank
msgstr's from the PO file will get themsgidstring with a leading"*** "as target string in the web2py result (po2web2py)
🦋 xliff2po
Converts XLIFF localization files to Gettext PO files. XLIFF is the XML Localization Interchange File Format developed by OASIS (Organization for the Advancement of Structured Information Standards) to allow translation work to be standardised no matter what the source format, and to allow the work to be freely moved from tool to tool. Literally built different. 🔥
Usage
po2xliff [options] <po> <xliff>
xliff2po [options] <xliff> <po>Where:
<po>— is a PO file or directory of PO files<xliff>— is an XLIFF file or directory of XLIFF files
Options (xliff2po)
--version— show program's version number and exit ✨-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inxliffformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:'msgctxt')
Options (po2xliff)
--version— show program's version number and exit ✨-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inxliffformat-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inxliffformat-S,--timestamp— skip conversion if the output file has newer timestamp
Examples 🍓
xliff2po -P xliff potCreate POT files from the XLIFF files found in directory xliff and output them to the directory pot.
po2xliff xh xh-xlfConvert the Xhosa PO files in xh to XLIFF and place them in xh-xlf.
Bugs 💀
This filter is not yet extensively used… expect bugs. See XLIFF to see how well our implementation conforms to the standard.
The PO plural implementation is still very new and needs active testing.
Ngl, consider yourself warned bestie. 🙈
🌸 yaml2po
Added in version 2.2.6.
Converts YAML localization files to Gettext PO format.
Usage
yaml2po [options] <yml> <po>
po2yaml [options] <po> <yml>Where:
<yml>— is a valid YAML localisable file or directory of those files<po>— is a directory of PO or POT files
Options (yaml2po)
--version— show program's version number and exit ✨-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inyaml,ymlformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inyaml,ymlformats-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)--duplicates=DUPLICATESTYLE— what to do with duplicate strings (identical source text):merge,msgctxt(default:'msgctxt')
Options (po2yaml)
--version— show program's version number and exit ✨-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,potformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inyaml,ymlformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inyaml,ymlformats-S,--timestamp— skip conversion if the output file has newer timestamp--threshold=PERCENT— only convert files where the translation completion is above PERCENT--fuzzy— use translations marked fuzzy--nofuzzy— don't use translations marked fuzzy (default)
Formats Supported
Check YAML format document to see to which extent the YAML format is supported.
Examples 🧚
This example looks at roundtrip of YAML translations as well as recovery of existing translations. Iconic behaviour honestly. 👑
First we need to create a set of POT files:
yaml2po -P lang/en pot/All .yml files found in the lang/en directory are converted to Gettext POT files and placed in the pot directory.
If you are translating for the first time then you can skip the next step. If you need to recover your existing translations then we do the following:
yaml2po -t lang/en lang/zu po-zu/Using the English YAML files found in lang/en and your existing Zulu translation in lang/zu we create a set of PO files in po-zu. These will now have your translations.
Please be aware that in order for that to work 100% you need to have both English and Zulu at the same revision — if they are not, you will have to review all translations. 😭
You are now in a position to translate your recovered translations or your new POT files.
Once translated you can convert back as follows:
po2yaml -t lang/en po-zu/ lang/zuYour translations found in the Zulu PO directory, po-zu, will be converted to YAML using the files in lang/en as templates and placing your new translations in lang/zu.
So like, po-zu is the vibe source, lang/en is the template bestie, and your shiny new stuff lands in lang/zu ✨.
To update your translations simply redo the POT creation step and make use of pot2po to bring your translation up-to-date.
--accelerator=ACCELERATOR
Okay so this one is all about which lil' marker character is used for accelerators — you know, the underlined letter situation in menus 🎀. Here's the whole tea:
&— used by KDE Desktop and Mozilla (when usingmoz2po)_— used by GNOME Desktop and other GTK+-based applications~— used by LibreOffice and Apache OpenOffice
--duplicates=DUPLICATESTYLE
Gettext PO files only allow one message with a common msgid (source string). Many other formats allow duplicate entries. To create a valid PO file you need to merge these duplicate entries into one PO message. However, this often negatively affects the roundtrip or is not what is expected by the user. Thus we have a number of methods of handling duplicates which we call duplicate styles. 💅
Also affected are conversions in which the source format is empty (allowing possible translation). As the header in a PO file is identified by an empty source string, your message will appear to be a duplicate of the header. In this case duplicate removal is critical — like, do not skip it.
Previously the tools used msgid_comment (KDE style comments) to disambiguate text. However, with the release of Gettext 0.15, the new msgctxt disambiguation is now recommended, especially if you wish to use your files with other Gettext tools. Many other pieces of software now also support this feature, and will probably become the best choice for almost all circumstances. It is the default in our converters. 👑
merge — this is the traditional Gettext approach. All messages with the same source string or English string are merged into one PO message.
#: file1.dtd:instruction_manual
#: file1.dtd:manual_process
msgid "Manual"
msgstr ""If however the source text is blank (these are often configuration options in Mozilla) then the merge style will use KDE comments as used in the msgid_comment style in order to create unambiguous entries that can still be used for configuration.
#: file1.dtd:translators_name
msgid "_: file1.dtd:translators_name\n"
msgstr ""
#: file1.dtd:translators_email
msgid "_: file1.dtd:translators_email\n"
msgstr ""msgctxt (default) — this uses the msgctxt feature of Gettext that was introduced with Gettext 0.15. Some tools might not support it 100%. This option is the default in recent releases of the Translate Toolkit. 🫶
#: file1.dtd:instruction_manual
msgctxt "instruction_manual"
msgid "Manual"
msgstr ""
#: file1.dtd:manual_process
msgctxt "manual_process"
msgid "Manual"
msgstr ""--errorlevel=ERRORLEVEL
This is a parameter that can be passed to most of the programs in the translate toolkit in order to choose the level of feedback that you need when errors occur. It is mostly useful for debugging. Please report your errors to the developers with --errorlevel=traceback. 🙈
none — display no error messages.
message — display on the error message.
An error occurred processing PO fileexception — give the error message and name and Python exception.
ValueError: An error occurred processing PO filetraceback — provide a full traceback for debugging purposes.
csv2po: warning: Error processing: nso/readlicense_oo/docs/readme.csv: Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/translate/misc/optrecurse.py", line 415, in recursiveprocess
success = self.processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)
File "/usr/lib/python2.4/site-packages/translate/misc/optrecurse.py", line 468, in processfile
if fileprocessor(inputfile, outputfile, templatefile, **passthroughoptions):
File "/usr/lib/python2.4/site-packages/translate/convert/csv2po.py", line 183, in convertcsv
outputpo = convertor.convertfile(inputcsv)
File "/usr/lib/python2.4/site-packages/translate/convert/csv2po.py", line 159, in convertfile
raise ValueError("An error occurred processing PO file")
ValueError: An error occurred processing PO file--filteraction=ACTION
none (default) — take no action. Messages from failing test will appear in the output file.
warn — print a warning but otherwise include the message in the output file.
exclude-serious — only exclude errors that are listed as serious by the convertor. All other are included.
exclude-all — exclude any message that fails a test.
--multifile=MULTIFILESTYLE
This options determines how the POT/PO files are split from the source files. In many cases you have source files that generate either too many small files or one large files which you would rather see split up into smaller files. 🦋
single — output individual files.
toplevel — split the source files at the top level. I.e., you see a number of top level files.
onefile — one large file instead of many smaller files.
--personality=TYPE
java (default) — create output strictly according to the specification for .properties files. This will use escaped Unicode for any non-ASCII characters. Thus the following string found in a PO file:
ṽḁḽṻḝWill appear as follows in the output .properties file:
\u1E7D\u1E01\u1E3D\u1E7B\u1E1Dmozilla — Mozilla has made slight adjustments to the Java .properties spec. Mozilla will accept UTF-8 encoded strings in the property file and thus does not need escaped Unicode. Thus the above string – ṽḁḽṻḝ – will not be escaped. Mozilla property files are thus more useful for non-Latin languages in that they are actually readable. 🌈
Of course this style of file is only used by Mozilla and should not be used for other projects that follow the Java spec more strictly.
skype — Skype .lang files are .properties files in UTF-16. The & is used as an accelerator (marked in the PO header).
flex — Flex follows the Mozilla approach, a UTF-8 encoded file with no escaped unicode. We include it as its own dialect for ease of use.
strings — much Mac OS X and iPhone software is translated using .strings files. These are quite different from properties files and we treat them here as key value files.
The files are in UTF-16 with a few minor escaping conventions.
--progress=PROGRESS
All of the programs can give visual feedback. This options allows you to select the style of that feedback. 👀
In the examples we are converting an OpenOffice.org 2.0 sdf/gsi file into POT files using oo2po.
none — no visual feedback, this is useful if you want to use any of the scripts as part of another script and don’t want feedback to interfere with the operation.
$ oo2po -P --progress=none en-US.sdf pot
$dots — use visual dots to represent progress. Each dot represent a file that has been processed.
$ oo2po -P --progress=dots en-US.sdf pot
.............................................................................................
.............................................................................................
.........................................
$bar (default) — use a progress bar consisting of hashes (#) to show progress.
$ oo2po -P --progress=bar en-US.sdf pot
processing 227 files...
[############################## ] 69%This is the default mode of operation, therefore this command would create the same output.
$ oo2po -P en-US.sdf potverbose — combine the hash (#) progress bar form the bar option with the actual names of files that have been processed.
$ oo2po -P --progress=verbose en-US.sdf pot
processing 227 files...
so3/src.oo
dbaccess/source/ui/uno.oo
helpcontent2/source/text/shared.oo
wizards/source/formwizard.oo
sch/source/ui/dlg.oo
helpcontent2/source/text/sbasic/shared/01.oo
dbaccess/source/core/resource.oo
svtools/source/sbx.oo
dbaccess/source/ui/relationdesign.oo
scp2/source/writer.oo
filter/source/xsltdialog.oo
[## ] 5%names — prints out only the filenames without any other progress indicator. This is a good option when outputting to a log file rather than a terminal.
$ oo2po -P --progress=names en-US.sdf pot
so3/src.oo
dbaccess/source/ui/uno.oo
helpcontent2/source/text/shared.oo
wizards/source/formwizard.oo
sch/source/ui/dlg.oo
helpcontent2/source/text/sbasic/shared/01.oo
dbaccess/source/core/resource.oo
svtools/source/sbx.oo
dbaccess/source/ui/relationdesign.oo
scp2/source/writer.oo
filter/source/xsltdialog.ooConverters change many different formats to PO and back again. Sometimes only one direction is supported, or conversion is done using non-PO formats. The converters follow a general pattern of usage, understanding that will make the converters much easier to use and understand. 💖
android2po— Android resources to PO convertercsv2po— Comma Separated Value (CSV) converter. Useful for doing translations using a spreadsheet.csv2tbx— create TBX (TermBase eXchange) files from Comma Separated Value (CSV) filesflatxml2po— Flat XML converterhtml2po— HTML converterical2po— iCalendar file converterini2po— Windows INI file converterjson2po— JSON file convertermd2po— Markdown file convertermoz2po— Mozilla.propertiesand.dtdconverter. Works with Firefox and Thunderbirdodf2xliff— convert OpenDocument (ODF) documents to XLIFF and vice-versa.oo2po— OpenOffice.org SDF converter (also works asoo2xliff). 💅php2po— PHP localisable string arrays converter.po2tmx— TMX (Translation Memory Exchange) converter.po2wordfast— Wordfast Translation Memory converter.pot2po— initialise PO Template files for translation.prop2po— Java property file (.properties) converter.rc2po— Windows Resource.rc(C++ Resource Compiler) converter.resx2po— .Net Resource (.resx) file converter.sub2po— Converter for various subtitle files.symb2po— Symbian-style translation to PO converter.tiki2po— TikiWikilanguage.phpconverter.ts2po— Qt Linguist.tsconverter.txt2po— Plain text to PO converter.web2py2po— web2py translation to PO converter.xliff2po— XLIFF (XML Localisation Interchange File Format) converter.yaml2po— YAML (Yet Another Markup Language) converter.
🛠️ Tools
The PO tools let you manipulate and work with PO files. Yes bestie, that’s literally the whole vibe. ✨
🎀 Quality Assurance
🌸 junitmsgfmt
Added in version 3.9.
Added –untranslated flag, to enable reporting of untranslated messages.
Added in version 1.7.
Run msgfmt and provide JUnit type output for use in continuous integration systems like Hudson and Jenkins. 🫶
Usage
junitmsgfmt po/*.po > msgfmt_junit.xml🦋 poconflicts
poconflicts takes a PO file and creates a set of output PO files that contain messages that conflict. During any translation project that involves a large amount of work or a number of translators you will see message conflicts. A conflict is where the same English message has been translated differently (in some languages this may have been intentional). Conflicts occur due to different translation style or a shift in translations as the translators or project mature.
poconflicts allows you to quickly identify these problem messages, investigate and correct them. To merge the files back, they have to be restructured into the correct directory structure using porestructure in order to enable merging using pomerge. 🔥
Usage
poconflicts [options] <po> <conflicts>Where:
<po>— is a directory of existing PO files or an individual PO file.<conflicts>— is a directory containing one PO file for each conflict.
Options:
--version— show program’s version number and exit.-h,--help— show this help message and exit.--manpage— output a manpage based on the help.--progress=PROGRESS— show progress as:dots,none,bar,names,verbose.--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback.-i INPUT,--input=INPUT— read fromINPUTinpoformat.-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths.-o OUTPUT,--output=OUTPUT— write toOUTPUTinpoformat.-I,--ignore-case— ignore case distinctions.-v,--invert— invert the conflicts thus extracting conflicting destination words.--accelerator=ACCELERATORS— ignores the given accelerator characters when matching.
🍓 Examples
Here are some examples that demonstrate the usefulness of poconflict. 😌
poconflicts --accelerator=~ -I xhosa conflictsThis extracts messages from the PO files in the xhosa directory and places a new PO file for each identified conflict in conflicts. We are working with OpenOffice files and we therefore use the tilde (~) as the accelerator marker (with this set F~ile is considered the same as ~File). We are also ignoring the case of the message using -I (thus File is considered the same as file or FILE).
Another useful option is to look at the inverted conflicts. This will detect target words that have been used to translate different source words. 👑
poconflicts --accelerator=~ -I -v xhosa conflictsNow in the conflicts directory we will find PO files based on the Xhosa word. We can now check where a Xhosa word has been used for different source or English words. Often there is no problem but you might find cases where the same Xhosa word was used for Delete and Cancel – clearly a usability issue.
The translator makes the needed corrections to the files and then we can proceed to merge the results back into the PO files. Unchanged entries can be removed.
Now restructure the files to resemble the original directory structure using porestructure:
porestructure -i conflicts -o conflicts_treeNow merge the changes back using pomerge:
pomerge -t xhosa -i conflicts_tree -o xhosaThis takes the corrected files from conflicts_tree and merge them into the files in xhosa using the same files as templates.
🔥 pofilter
pofilter allows you to run a number of checks against your PO, XLIFF or TMX files. These checks are designed to pick up problems with capitalisation, accelerators, variables, etc. Those messages that fail any of the checks are output and marked so that you can correct them. 🎀
Use pofilter -l to get a list of available checks.
Once you have corrected the errors in your PO files you can merge the corrections into your existing translated PO files using pomerge.
Usage
pofilter [options] <in> <out>Where:
<in>— the input file or directory which contains PO or XLIFF files.<out>— the output file or directory that contains PO or XLIFF files that fail the various tests.
Options:
--version— show program’s version number and exit.-h,--help— show this help message and exit.--manpage— output a manpage based on the help.--progress=PROGRESS— show progress as:dots,none,bar,names,verbose.--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback.-i INPUT,--input=INPUT— read fromINPUTinpot,po,xlf,tmxformats.-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths.-o OUTPUT,--output=OUTPUT— write toOUTPUTinpo,pot,xlf,tmxformats.-l,--listfilters— list filters available.--review— include elements marked for review (default).--noreview— exclude elements marked for review.--fuzzy— include elements marked fuzzy (default).--nofuzzy— exclude elements marked fuzzy.--nonotes— don’t add notes about the errors (since version 1.3).--autocorrect— output automatic corrections where possible rather than describing issues.--language=LANG— set target language code (e.g.af-ZA) [required for spell check]. This will help to makepofilteraware of the conventions of your language.--openoffice— use the standard checks for OpenOffice translations.--libreoffice— use the standard checks for LibreOffice translations.--mozilla— use the standard checks for Mozilla translations.--drupal— use the standard checks for Drupal translations.--gnome— use the standard checks for Gnome translations.--kde— use the standard checks for KDE translations.--wx— use the standard checks for wxWidgets translations – identical to--kde.--excludefilter=FILTER— don’t useFILTERwhen filtering.-t FILTER,--test=FILTER— only use testFILTERs specified with this option when filtering.--notranslatefile=FILE— read list of untranslatable words fromFILE(must not be translated).--musttranslatefile=FILE— read list of translatable words fromFILE(must be translated).--validcharsfile=FILE— read list of all valid characters fromFILE(must be in UTF-8).
🌈 Example
Here are some examples to demonstrate how to use pofilter:
pofilter --openoffice af af-checkUse the default settings (accelerator and variables) for OpenOffice.org. Check all PO files in af and output any messages that fail the check in af-check (create the directory if it does not already exist).
pofilter -t isfuzzy -t untranslated zu zu-checkOnly run the isfuzzy and untranslated checks, this will extract all messages that are either fuzzy or untranslated.
pofilter --excludefilter=simplecaps --nofuzzy nso nso-checkRun all filters except simplecaps. You might want to do this if your language does not make use of capitalisation or if the test is creating too many false positives. Also only run the checks against messages that are not marked fuzzy. This is useful if you have already marked problem strings as fuzzy or you know that the fuzzy strings are bad, with this option you don’t have to see the obviously wrong messages.
pofilter --language=fr dir dir-checkTell pofilter that you are checking French translations so that it can take the conventions of the language into account (for things like punctuation, spacing, quoting, etc.) It will also disable some tests that are not meaningful for your language, like capitalisation checks for languages that don’t have capital letters.
pofilter --excludefilter=untranslatedTell pofilter not to complain about your untranslated units.
pofilter -lList all the available checks.
🐛 Bugs
There are minor bugs in the filters. Most relate to false positives, corner cases or minor changes for better fault description.
🎀 Descriptions of all pofilter tests 🎀
Okay bestie, here’s the full rundown of the tests you get in pofilter, Pootle and Virtaal — plus what kinds of errors each one is good at catching and where each one has limits.
Heads up: the software might point at errors that aren’t necessarily wrong. Those are false positives, and they happen. 💅
Right now there are 48 tests. You can always get the current list by running:
pofilter -lAnd if you want the tests specific to a targeted application or group of applications, run:
pofilter --gnome -l🧚 Adding new tests and new language adaptations 🧚
If you have an idea for a new test, or you want to add target-language adaptations for your language, please help us with information about your test idea and the specifics of your language.
💅 Test Classification 💅
Some tests are more important than others, so they’re classified to help you figure out which ones to run first.
- Critical — can break a program
dialogsizes,escapes,newlines,nplurals,printf,pythonbraceformat,tabs,variables,xmltags
- Functional — may confuse the user
accelerators,acronyms,blank,emails,filepaths,functions,gconf,kdecomments,long,musttranslatewords,notranslatewords,numbers,options,purepunc,sentencecount,short,spellcheck,urls,unchanged
- Cosmetic — make it look better
brackets,doublequoting,doublespacing,doublewords,endpunc,endwhitespace,puncspacing,simplecaps,simpleplurals,startcaps,singlequoting,startpunc,startwhitespace,validchars
- Extraction — useful mainly for extracting certain types of string
compendiumconflicts,credits,hassuggestion,isfuzzy,isreview,untranslated
🔥 Test Description 🔥
💖 accelerators 💖
Checks whether accelerators are consistent between the two strings.
Make sure you use the --mozilla, --kde, etc. options so that pofilter knows which type of accelerator it’s looking for. This test picks up accelerators that are missing and ones that shouldn’t be there.
For some languages, this check alters its default behavior in the Mozilla checker — instead it checks that accelerators are not present in the translation. The purpose is to make sure that in languages where accelerators shouldn’t be used, they don’t show up in translations. This is common for Indic languages.
💖 acronyms 💖
Checks that acronyms that appear are unchanged.
If an acronym appears in the original, this test checks that it appears in the translation. Translating acronyms is a language decision, but many languages leave them unchanged. In that case this test is useful for tracking down translations of the acronym and correcting them.
💖 blank 💖
Checks whether a translation is totally blank.
This checks to see if a translation has inadvertently been translated as blank, i.e. as spaces. This is different from untranslated, which is completely empty. This test is useful because if something is translated as " " it will appear to most tools as if it is translated.
💖 brackets 💖
Checks that the number of brackets in both strings match.
If ([{ or }]) appear in the original, this checks that the same number appear in the translation.
💖 compendiumconflicts 💖
Checks for Gettext compendium conflicts (#-#-#-#-#).
When you use msgcat to create a PO compendium, it will insert #-#-#-#-# into entries that are not consistent. If the compendium is used later in a message merge, then these conflicts will appear in your translations. This test quickly extracts those for correction.
💖 credits 💖
Checks for messages containing translation credits instead of normal translations.
Some projects have consistent ways of giving credit to translators by having a unit or two where translators can fill in their name and possibly their contact details. This test allows you to find these units easily to check that they are completed correctly, and it also disables other tests that might incorrectly get triggered for these units (such as urls, emails, etc.).
💖 dialogsizes 💖
Checks that dialog sizes are not translated.
This is a Mozilla specific test. Mozilla uses a language called XUL to define dialogues and screens. This can make use of CSS to specify properties of the dialogue. These properties include things such as the width and height of the box. The size might need to be changed if the dialogue size changes due to longer translations. Thus translators can change these settings. But you are only meant to change the number, not translate the words width or height. This check catches instances where these are translated. It will also catch other types of errors in these units.
💖 doublequoting 💖
Checks whether doublequoting is consistent between the two strings.
Checks on double quotes " to ensure that you have the same number in both the original and the translated string. This test takes into account that several languages use different quoting characters, and will test for them instead.
💖 doublespacing 💖
Checks for bad double-spaces by comparing to the original.
This identifies if you have [space][space] when you don’t have it in the original, or if it appears in the original but not in your translation. Some of these are spurious, and how you correct them depends on the conventions of your language.
💖 doublewords 💖
Checks for repeated words in the translation.
Words that have been repeated in a translation will be highlighted with this test, e.g. "the the", "a a". These are generally typos that need correcting. Some languages may have valid repeated words in their structure — in that case either ignore those instances or switch this test off using the --excludefilters option.
💖 emails 💖
Checks to see that emails are not translated.
Generally you should not be translating email addresses. This check looks to see that email addresses, e.g. info@example.com, are not translated. In some cases of course you should translate the address, but generally you shouldn’t.
💖 endpunc 💖
Checks whether punctuation at the end of the strings match.
This ensures that the ending of your translation has the same punctuation as the original. E.g. if it ends in :[space] then so should yours. It is useful for ensuring that you have ellipses […] in all your translations, not simply three separate full-stops. You may pick up some errors in the original: feel free to keep your translation and notify the programmers. In some languages, characters such as ? ! are always preceded by a space, e.g. [space]? — do what your language customs dictate. Other false positives you will notice are, for example, if through changes in word-order you add “), etc. at the end of the sentence. Do not change these: your language word-order takes precedence.
It must be noted that if you are tempted to leave out [full-stop] or [colon] or add [full-stop] to a sentence, that often these have been done for a reason, e.g. a list where fullstops make it look cluttered. So initially match them with the English, and make changes once the program is being used.
This check is aware of several language conventions for punctuation characters, such as the custom question marks for Greek and Arabic, Devenagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it is not there yet.
💖 endwhitespace 💖
Checks whether whitespace at the end of the strings matches.
Operates the same as endpunc but is only concerned with whitespace. This filter is particularly useful for those strings which will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. The whitespace is an inherent part of the string. This filter makes sure you don’t miss those important but otherwise invisible spaces!
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
💖 escapes 💖
Checks whether escaping is consistent between the two strings.
Checks escapes such as \n \uNNNN to ensure that if they exist in the original string, you also have them in the translation.
💖 filepaths 💖
Checks that file paths have not been translated.
Checks that paths such as /home/user1 have not been translated. Generally you do not translate a file-path, unless it is being used as an example, e.g. [your_user_name/path/to/filename.conf].
💖 functions 💖
Checks to see that function names are not translated.
Checks that function names, e.g. rgb() or getEntity.Name(), are not translated.
💖 gconf 💖
Checks if we have any gconf config settings translated.
Gconf settings should not be translated, so this check makes sure gconf settings such as name or modification_date are not translated in the translation. It allows you to change the surrounding quotes but will ensure that the setting values remain untranslated.
💖 hassuggestion 💖
Checks if there is at least one suggested translation for this unit.
If a message has a suggestion (an alternate translation stored in alt-trans units in XLIFF and .pending files in PO), then these will be extracted. This is used by Pootle and is probably only useful in pofilter when using XLIFF files.
💖 isfuzzy 💖
Checks if the po element has been marked fuzzy.
If a message is marked fuzzy in the PO file, then it is extracted. Note this is different from --fuzzy and --nofuzzy options which specify whether tests should be performed against messages marked fuzzy.
💖 isreview 💖
Checks if the po element has been marked for review.
If you have made use of the ‘review’ flags in your translations:
# (review) reason for review
# (pofilter) testname: explanation for translatorThen if a message is marked for review in the PO file, it will be extracted. Note this is different from --review and --noreview options which specify whether tests should be performed against messages already marked as under review.
💖 kdecomments 💖
Checks to ensure that no KDE style comments appear in the translation.
KDE style translator comments appear in PO files as "_: comment\n". New translators often translate the comment. This test tries to identify instances where the comment has been translated.
💖 long 💖
Checks whether a translation is much longer than the original string.
This is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives.
💖 musttranslatewords 💖
Checks that words configured as definitely translatable don’t appear in the translation.
Like, say you decide that in your language ‘OK’ must be translated — then this test will flag any ‘OK’ that shows up in the translation if it was in the source string. You gotta specify a file with all the must translate words using --musttranslatefile. 💅
newlines 🌸
Checks whether newlines are consistent between the two strings.
Counts the number of \n newlines (and variants such as \r\n) and reports an error if they differ.
nplurals ✨
Checks for the correct number of noun forms for plural translations.
This uses the plural information in the language module of the toolkit. It’s the same as the Gettext nplural value. It’ll check that the number of plurals required is the same as the number supplied in your translation.
notranslatewords 🎀
Checks that words configured as untranslatable appear in the translation too.
Lots of brand names should not be translated, and this test lets you easily make sure words like: Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no translate words using --notranslatefile.
numbers 🔢
Checks whether numbers of various forms are consistent between the two strings.
You’ll see some errors where you either wrote the number in full or converted it to the digit in your translation. Also, changes in order will trigger this error.
Some languages don’t use latin numbers but instead use different numbers. This check will take that into account.
options 🛠️
Checks that command line options are not translated.
In messages that contain command line options, such as --help, this test will check that these remain untranslated. These could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. If the options has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated.
printf 🔥
Checks whether printf format strings match.
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are place holders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. The test can also manage variables-reordering using the %1$s syntax. The variables’ type and details following data are tested to ensure that they are strictly identical, but they may be reordered.
SEE ALSO: pythonbraceformat
SEE ALSO: printf Format String
puncspacing 💖
Checks for bad spacing after punctuation.
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc.
Some languages don’t use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account.
purepunc 🦋
Checks that strings that are purely punctuation are not changed.
This extracts strings like + or - as these usually should not be changed.
pythonbraceformat 🐍
Checks whether Python brace format strings match.
Python supports both a variant of the printf formatting system, and its own formatting language which uses placeholders enclosed in braces. The placeholders can be named, numbered, or anonymous; the former two are filled in from positional args, the latter from keyword arguments. Example:
'the {} {0} hungry {insect}'.format('very', insect='caterpiller')
# --> 'the very very hungry caterpiller'The pythonbraceformat filter checks for the following problems:
- named placeholders that are present in the original, but missing in the translation, and vice versa.
- originals and translations that require different numbers of positional args.
When the translation has variables not in the original, this can lead to program crashes. The translation not using all variables the original uses is safe. Nonetheless, this filter triggers in both cases.
SEE ALSO: PEP 3101 – Advanced String Formatting
sentencecount 🌈
Checks that the number of sentences in both strings match.
Adds the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it’s the meaning of the original you want to convey, not the exact way it was written in the English.
short 🍓
Checks whether a translation is much shorter than the original string.
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives.
simplecaps 👑
Checks the capitalisation of two strings isn’t wildly different.
This will pick up many false positives, so don’t be a slave to it. It is useful for identifying translations that don’t start with a capital letter (upper-case letter) when they should, or those that do when they shouldn’t. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case.
simpleplurals 💜
Checks for English style plural(s) for you to review.
This test will extract any message that contains words with a final (s) in the source text. You can then inspect the message, to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy. In this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives.
For languages with no plural forms (only one noun form) this test will simply test that nothing like (s) was used in the translation.
singlequoting 🧚
Checks whether singlequoting is consistent between the two strings.
The same as doublequoting but checks for the ' character. Because this is used in contractions like it’s and in possessive forms like user’s, this test can output spurious errors if your language doesn’t use such forms. If a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check.
spellcheck 📚
Checks for words that don’t pass a spell-check.
This test will check for misspelled words in your translation. The test first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors e.g. program names, brand names, function names.
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the ‐ Hunspell or aspell dictionaries). This test will only work if you have specified the --language option.
The pofilter error that is created, lists the misspelled word, plus suggestions returned from the spell checker. That makes it easy for you to identify the word and select a replacement.
startcaps 🌟
Checks that the message starts with the correct capitalisation.
After stripping whitespace and common punctuation characters, it then checks to see that the first remaining character is correctly capitalised. So, if the sentence starts with an upper-case letter, and the translation does not, an error is produced.
This check is entirely disabled for many languages that don’t make a distinction between upper and lower case. Contact us if this is not yet disabled for your language.
startpunc 💫
Checks whether punctuation at the beginning of the strings match.
Operates as endpunc but you will probably see fewer errors.
startwhitespace 🌬️
Checks whether whitespace at the beginning of the strings matches.
As in endwhitespace but you will see fewer errors.
tabs 🗂️
Checks whether tabs are consistent between the two strings.
Counts the number of \t tab markers and reports an error if they differ.
unchanged 🪞
Checks whether a translation is basically identical to the original string.
This checks to see if the translation isn’t just a copy of the English original. Sometimes, this is what you want, but other times you will detect words that should have been translated.
untranslated 🚩
Checks whether a string has been translated at all.
This check is really only useful if you want to extract untranslated strings so that they can be translated independently of the main work.
urls 🔗
Checks to see that URLs are not translated.
This checks only basic URLs (http, ftp, mailto etc.) not all URIs (e.g. afp, smb, file). Generally, you don’t want to translate URLs, unless they are example URLs (http://your_server.com/filename.html). If the URL is for configuration information, then you need to query the developers about placing configuration information in PO files. It shouldn’t really be there, unless it is very clearly marked: such information should go into a configuration file.
validchars ✅
Checks that only characters specified as valid appear in the translation.
Often during character conversion to and from UTF-8 you get some strange characters appearing in your translation. This test presents a simple way to try and identify such errors.
This test will only run if you specify the --validcharsfile command line option. This file contains all the characters that are valid in your language. You must use UTF-8 encoding for the characters in the file.
If the test finds any characters not in your valid characters file then the test will print the character together with its Unicode value (e.g. 002B).
variables 🧩
Checks whether variables of various forms are consistent between the two strings.
This checks to make sure that variables that appear in the original also appear in the translation. Make sure you use the --kde, --openoffice, etc flags as these define what variables will be searched for. It does not at the moment cope with variables that use the reordering syntax of Gettext PO files.
xmltags 🏷️
Checks that XML/HTML tags have not been translated.
So basically this check counts up all the tags in your source string and then double-checks that the translation has the exact same number. If the counts don't line up? Girl. 😭 That means either a tag straight-up went missing, or your translator went and translated a tag itself — and both of those are errors. Not one, not the other. Both.
It's got manners though: it ignores tags (or things that merely look like tags) that cover the whole string, like <Error> — no biggie there. But it will throw false positives for stuff like An <Error> occurred, because babes, in that case Error genuinely needs translating. It'll also let you translate the alt attribute in things like <img src=bob.png alt="Image description"> — or similar translatable attributes in OpenOffice.org help files. 🫶
🫶 pogrep
pogrep is that girl who goes and extracts every message matching a regular expression into a brand-new set of PO files, so you can inspect them, edit them, and correct them at your leisure. ✨ Then you take those corrections and merge them all back with pomerge. Iconic little workflow, ngl.
Usage
pogrep [options] <in> <out>Where:
<in>/<out>— In and out are either directories or files.Outwill contain PO/XLIFF files with only those messages that match the regular expression that was you searched for. 💅
Options
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read fromINPUTingmo,mo,po,pot,tmx,xlf,xlff,xliffformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths-o OUTPUT,--output=OUTPUT— write toOUTPUTingmo,mo,po,pot,tmx,xlf,xlff,xliffformats--search=SEARCHPARTS— searches the given parts (source,target,notes,locations)-I,--ignore-case— ignore case distinctions-e,--regexp— use regular expression matching-v,--invert-match— select non-matching lines--accelerator=ACCELERATORS— ignores the given accelerator characters when matching-k,--keep-translations— always extract units with translations
Example
pogrep --accelerator="_" --search msgid -I -e "software|hardware" only-zu only-zu-checkSo this one searches for the words “software” or “hardware” in the msgid field. 🎀 It ignores case (-I) and treats the underscore (_) character as an accelerator key. Then it rummages through every PO file in the directory only-zu and drops any matches into PO files in the directory only-zu-check. Run this when you know the word for software and hardware got changed mid-translation and you wanna check + correct every single instance of it. Literally so satisfying. 💅
pogrep --search=msgid -e '^\w+(\s+\w+){0,3}$' -i templates -o short-wordsFind all messages in the templates directory that have between 1 and 4 words and place them in short-words. Use this when you want quick results by translating the messages that are most likely menu entries or dialogue labels. Lowkey the smartest way to feel productive. ✨
pogrep --search=msgstr -I -e "Ifayile" zu zu-checkSearch all translations for the occurrence of Ifayile. 🦋 You'd use this to check whether words have been used correctly. Super useful when you spot problematic use of the same word for different concepts. And hey — you can use pocompendium to find these conflicts. Teamwork, bestie. 🫶
Notes
Unicode normalization 🌸
pogrep will normalize Unicode strings. This lets you search for strings that contain the same character but are using precomposed Unicode characters, or which are composed using another composition recipe. An individual user will in all likelihood only compose characters in one way, but normalization makes sure data created in a team setting can be shared. No cap, it's a whole team-safety thing. 💖
Further reading
Here's a blog post explaining how pogrep can be used to do more targeted localisation of GNOME: ➖ http://translate.org.za/blogs/friedel/en/content/better-lies-about-gnome-localisation
👑 pomerge
pomerge will merge corrected PO, XLIFF, or TMX files (or snippets!) into your existing PO, XLIFF, TMX files. 💜 Normally the flow is: extract errors using pofilter, make your corrections to those PO (or XLIFF, TMX) snippets, then merge them back in with pomerge. You could also use pogrep to extract a bunch of messages matching a certain string, fix 'em up, then merge the correction back with pomerge. Choose your own adventure, bestie. ✨
It is probably best to run pomerge against files stored in some kind of version control system so that you can monitor what changes were made. 🔥
pomerge will also try to make as small a change as possible to the text, which makes it way easier to see those changes with your version control system. It's giving thoughtful. 🎀
Usage
pomerge [options] [-t <template>] -i <input> -o <output>Where:
<template>— is a set of reference PO, XLIFF, TMX files, either the originals or a set of POT files<input>— contains the corrected files that are to override content in<output><output>— contains the files whose content will be overridden by<input>. This can be the same directory as<template>
Options
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read fromINPUTinpo,pot,xlfformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths-o OUTPUT,--output=OUTPUT— write toOUTPUTinpo,pot,xlfformats-t TEMPLATE,--template=TEMPLATE— read fromTEMPLATEinpo,pot,xlfformats-S,--timestamp— skip conversion if the output file has newer timestamp--mergeblanks=MERGEBLANKS— whether to overwrite existing translations with blank translations (yes/no). Default isyes.--mergefuzzy=MERGEFUZZY— whether to overwrite existing translations with fuzzy translations (yes/no). Default isyes.--mergecomments=MERGECOMMENTS— whether to merge comments as well as translations (yes/no). Default isyes.
Examples
These examples show pomerge in action. 👑
pomerge -t af -i af-check -o afTake corrections from af-check, merge them with the templates in af, and output into af. So you're merging af-check and overriding entries found in af. 😭
Do this only if you're using a version control system so you can check what changes pomerge made — or if you have complete and utter confidence in this tool. Like, absolute faith. No notes. 🙈
pomerge --mergeblanks=yes -t af -i af-check -o af-newMerge the corrections from af-check with templates in af and output to af-new. If an entry is blank in af-check, then make it blank in the output in af-new. Simple as. 💅
Issues
- Seems to have trouble merging KDE style comments back. (Probably not relevant with newest versions any more.) 💀
- Only files found in the input directory will be copied to the output. The template directory is not searched for extra files to copy to the output. Therefore it's always best to have your input directory in version control, and use the same directory as output. That way you can use the diff function of the version control system to double check changes made, with all the files of the input still present. 🫶
🧚 porestructure
porestructure takes the PO files output by poconflicts (a flat structure) and recreates the directory structure according to the poconflict location comments found in each PO message. After being restructured, the messages in the resulting directory structure can be merged back using pomerge. It's giving restoration arc. ✨
Since poconflicts adds conflicting messages — from many different PO files — into a single PO file, the original structure of the files and directories gets lost and the new PO files get output to a single directory. The original structure info is left in (pofilter) comments for each PO element. 🦋
Usage
porestructure [options] <conflicts> <po>Where:
<conflicts>— is a directory containing one the corrected output frompoconflict<po>— is an output directory to write the restructured files to
Options
--version— show program's version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— tells the tool how much error tea to spill:none,message,exception,traceback. 💅-i INPUT,--input=INPUT— reads fromINPUTinpoformat.-x EXCLUDE,--exclude=EXCLUDE— excludes names matchingEXCLUDEfrom input paths.-o OUTPUT,--output=OUTPUT— writes toOUTPUTinpoformat. 🎀
✨ Examples ✨
The documentation for poconflicts has Examples for the complete process using poconflict, porestructure, and pomerge.
These tools are especially useful for measuring and improving translation quality. 👑
junitmsgfmt— runmsgfmtand provide JUnit type output for use in continuous integration systems like Hudson and Jenkins. 🔥poconflicts— extract messages that have conflicting translation. 💀pofilter— filter PO files to find common errors using a number of tests. 🧚pogrep— find strings in your PO files. 🦋pomerge— merge file extracted usingpofilterback into the original files. 🫶porestructure— restructures PO files according topoconflictdirectives. 🌸
🧰 Other tools 🧰
🌈 Levenshtein distance 🌈
The levenshtein distance is used for measuring the “distance” or similarity of two character strings. Other similarity algorithms can be supplied to the code that does the matching.
This code is used in pot2po, tmserver and Virtaal. It is implemented in the toolkit, but can optionally use the fast C implementation provided by python-Levenshtein if it is installed. It is strongly recommended to have python-levenshtein installed. ⭐
To exercise the code the classfile “Levenshtein.py” can be executed directly with:
$ python Levenshtein.py "The first string." "The second string"NOTE: Remember to quote the two parameters. 🙈
The following things should be noted:
- Only the first
MAX_LENcharacters are considered. Long strings differing at the end will therefore seem to match better than they should. A penalty is awarded if strings are shortened. 💅 - The calculation can stop prematurely as soon as it realise that the supplied minimum required similarity cannot be reached. Strings with widely different lengths give the opportunity for this shortcut. This is by definition of the Levenshtein distance: the distance will be at least as much as the difference in string length. Similarities lower than your supplied minimum (or the default) should therefore not be considered authoritative. 😭
💀 Shortcommings 💀
The following shortcommings have been identified:
- Cases sensitivity: ‘E’ and ‘e’ are considered different characters and they differ as much as ‘z’ and ‘e’. This is not ideal, as case differences should be considered less of a difference. 🙃
- Diacritics: ‘ê’ and ‘e’ are considered different characters and they differ as much as ‘z’ and ‘e’. This is not ideal, as missing diacritics could be due to small input errors, or even input data that simply do not have the correct diacritics. 🫠
- Similar but different words: Words that have similar characters, but are different, could increase the similarity beyond what is wanted. The sentences “It is though.” and “It is dough.” differ markedly semantically, but score similarity of almost
85%. A possible solution is to do an additional calculation based on words, instead of characters. 💀 - Whitespace: Differences in tabs, newlines, and space usage should perhaps be considered as a special case. 🌸
🎀 –rewrite=STYLE 🎀
podebug allows you to rewrite the output text in a number of ways. ✨
xxx
The target text is surrounded by xxx as follows:
msgid "English"
msgstr "xxxEnglishxxx"This is useful when you want to identify which text is localisable. There might be text in your application which you cannot localise; this will allow you to quickly identify that text. 👑
en
The source text is copied to the target:
msgid "English"
msgstr "English"In this way you can create translations that contain only the source text. Useful if you are preparing a roundtrip test or want to start an English derived translation such as British English. It produces the same results as msgen but with the advantage that you can add debug markers. 🦋
blank
This simply empties your current translations:
msgid "English"
msgstr ""When you have a set of translation files but no template this allows you to essentially convert a PO into a POT file. This mimics the --empty functionality of msghack. 🫶
bracket
Added in version 1.4.
Places brackets around the translated text.
msgid "English"
msgstr "[English]"This can be used in the same way as xxx to check for translatability. It is also useful with very long strings as it allows you to check that the full string is rendered and has not been cutoff by the application. 💅
chef
Added in version 1.2.
Rewrites the source text using mock Swedish as popularised by the ‐ Swedish Chef.
msgid "English"
msgstr "Ingleesh"This is probably only useful for some fun. It’s not guaranteed that every string will be rewritten as the mock Swedish rules might not apply, thus it’s not ideal for identifying untranslatable strings. 🍓
flipped
Added in version 1.4.
Change the text into a version that uses equivalent Latin characters that are upside down.
msgid "English"
msgstr "Ǝuƃʅısɥ"flipped can give an output that simulates RTL languages. It inserts RTL characters to try to achieve RTL-like results. It’s not perfect but will give you some sense of whether your application can do RTL. Or just use it for fun! 🌈
For really testing right-to-left GUIs, you want to make sure that the whole application is shown in RTL, not just the strings. Test your pseudo-translated file as a translation of an RTL language like Arabic or Hebrew. In case the application relies on other files coming from libraries (like GTK+), you might need to repeat the process for them, or at least ensure that you have the Arabic/Hebrew .mo files for them installed. 🧚
unicode
Added in version 1.2.
Rewrites the source text with Unicode characters that looks like the Latin characters that they are replacing.
msgid "English"
msgstr "Ḗƞɠŀīşħ"This allows a translator or programmer to test a programs ability to use Unicode message strings. By using characters in the Unicode range but that are related to the plain Latin characters that they replace we ensure that the messages are still readable. 🔥
NOTE: Before version1.4, the rewrite rule will also rewrite variables and XML tags, which would cause problems in some situations. Runpofilteras a quick method to fix up incorrect changes, or upgrade to version1.4. 😭
classified
Added in version 3.7.3.
Rewrites the source text with black square (▮) that looks like NSA classified files.
msgid "English"
msgstr "▮▮▮▮▮▮▮"This allows a translator or programmer to visually distinguish what is not part of the translation yet. This way it’s really easy to spot what strings are missing. 👑
🧼 poclean 🧼
This is a rudimentary tool to produce a clean file from an unclean file (Trados/Wordfast) by stripping out the tw4win indicators.
📝 Usage 📝
poclean <input> <output>Where:
<input>— is the text versions of the unclean RTF files.<output>— is the intended output file / directory.
🛠️ Options 🛠️
--version— show program’s version number and exit. 💅-h,--help— show this help message and exit. 🫶--manpage— output a manpage based on the help. 📝--progress=PROGRESS— show progress as:dots,none,bar,names,verbose. ⭐--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback. 💀-i INPUT,--input=INPUT— read fromINPUTinpotformat.-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths.-o OUTPUT,--output=OUTPUT— write toOUTPUTinpo,potformats. 🎀-S,--timestamp— skip conversion if the output file has newer timestamp. ⏰
✨ Examples ✨
To create a text version of the unclean RTF file, you need UnRTF, available here: project site or here (windows).
unrtf translation.rtf --text > translation.poYou might need to convert the encoding of the file, with iconv, for example:
iconv -f latin1 -t utf-8 translation.po > new_translation.poNow you can clean the file with poclean:
poclean new_translation.po clean_translation.po🔥 pocompile 🔥
Compile PO or XLIFF files into MO (Machine Object) files. MO files are installed on your computer and allow a Gettext enabled computer to provide the translations for the application. 👑
📝 Usage 📝
pocompile <po> <mo>Where:
<po/xliff>— is a standard PO file, XLIFF file or directory.<mo>— is the output MO file or directory of MO files.
🛠️ Options 🛠️
--version— show program’s version number and exit. 💅-h,--help— show this help message and exit. 🫶--manpage— output a manpage based on the help. 📝--progress=PROGRESS— show progress as:dots,none,bar,names,verbose. ⭐--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback. 💀-i INPUT,--input=INPUT— read fromINPUTinxlf,po,potformats.-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths.-o OUTPUT,--output=OUTPUT— write toOUTPUTinmoformat. 🎀-S,--timestamp— skip conversion if the output file has newer timestamp. ⏰--fuzzy— use translations marked fuzzy. 🦋--nofuzzy— don’t use translations marked fuzzy (default). 🙈
✨ Examples ✨
okay so pocompile said “watch this” 💅
pocompile --fuzzy file.po file.moThis creates a new MO file called file.mo based on the translation in the PO file file.po. By using the --fuzzy option we use all translations, including the ones marked fuzzy. no cap.
pocompile file.xlf file.moCreate an MO file from an XLIFF file called file.xlf (available from version 1.1 of the toolkit). iconic.
🌸 pocount 🌸
pocount will count the number of strings and words in translatable files.
Supported formats include: PO and XLIFF. Almost all bilingual file formats supported by the Translate Toolkit will work with pocount, including: TMX, TBX, Gettext .mo, Qt .qm, Wordfast .txt TM. 🫶
A number of other formats should be countable as the toolkit develops. Note that only multilingual formats based on the storage base class are supported, but that includes almost all storage formats. iykyk.
💅 Usage 💅
pocount [options] <directory|file(s)>Where:
<directory>— will recurse and count all files in the specified directory<file(s)>— will count all files specified
Options:
-h,--help— show this help message and exit--incomplete— skip 100% translated files
Output format:
--full(default) — statistics in full, verbose format--csv— statistics in CSV format--short— same as--short-strings--short-strings— statistics of strings in short format — one line per file--short-words— statistics of words in short format — one line per file
✨ Examples ✨
pocount makes it easy to count the current state of a body of translations. The most interesting options are those that adjust the output style and decide what to count. obsessed.
🍓 Easy counting 🍓
To count how much work is to be done in your project:
pocount project/This will count all translatable files found in the directory project/ and output the results in --full format.
You might want to be more specific and only count certain files:
pocount *.poThis will count all PO files in the current directory but will ignore any other files that pocount can count.
You can have full control of the files to count by using some of the abilities of the Unix commandline; these may work on Mac OS X but are unlikely to work on Windows:
pocount $(find . -name "*.properties.po")This will first find all files that match *.properties.po and then count them. That would make it easy to count the state of your Mozilla translations of .properties files. slay.
💖 Incomplete work 💖
To count what still needs to be done, ignoring what is 100% complete, you can use the --incomplete option:
pocount --incomplete --short *.xlfWe are now counting all XLIFF files by using the *.xlf expansion. We are only counting files that are not 100% complete and we’re outputting string counts using the --short option. periodt.
🎀 Output formats 🎀
The output options provide the following types of output:
🌸 --full 🌸
This is the normal, or default, mode. It produces the most comprehensive and easy to read data, although the amount of data may overwhelm the user. It produces the following output:
avmedia/source/viewer.po
type strings words (source) words (translation)
translated: 73465 ( 99%) 538598 ( 99%) 513296
fuzzy: 13 ( 0%) 141 ( 0%) n/a
untranslated: 53 ( 0%) 602 ( 0%) n/a
Total: 73531 539341 513296A grand total and file count is provided if the number of files is greater than one.
🌸 --csv 🌸
This format is useful if you want to reuse the data in a spreadsheet. In CSV mode the following output is shown:
Filename, Translated Messages, Translated Source Words, Translated Target Words, Fuzzy Messages, Fuzzy Source Words, Untranslated Messages, Untranslated Source Words, Review Messages, Review Source Words
avmedia/source/viewer.po, 1, 3, 3, 0, 0, 4, 22, 1, 3Totals are not provided in CSV mode.
🌸 --short-strings (alias --short) 🌸
The focus is on easily accessible data in a compact form. This will only count strings and uses a short syntax to make it easy for an experienced localiser to read:
test-po/fuzzy.po strings: total: 1 | 0t 1f 0u | 0%t 100%f 0%uThe filename is followed by a word indicating the type of count; here we are counting strings. The total gives the total string count. While the letters t, f and u represent ‘translated’, ‘fuzzy’ and ‘untranslated’ and here indicate the string counts for each of those categories. The counts are followed by a percentage representation of the same categories.
🌸 --short-words 🌸
The output is very similar to --short-strings above:
test-po/fuzzy.po source words: total: 3 | 0t 3f 0u | 0%t 100%f 0%uBut instead of counting string we are now counting words as indicated by the term ‘source words’.
💀 Bugs 💀
- There are some miscounts related to word breaks.
- When using the short output formats the columns may not be exactly aligned. This is because the number of digits in different columns is unknown before all input files are processed. The chosen tradeoff here was instantaneous output (after each processed file) instead of waiting for the last file to be processed.
🦋 podebug 🦋
Insert pseudo translations or debug markers into target text in XLIFF, Gettex PO and other localization files.
The pseudo translation or debug markers make it easy to reference and locate strings when your translated application is running.
Use it to:
- Target your translations: see what files are being referenced for string appearing in your programs.
- Debug translations: if you know in what file the message occurs then you can quickly find it and fix it.
- Check that everything is translatable: any English only text needs to be analysed so that it can be localised.
- Check for Unicode compliance: by inserting Unicode text outside of the Latin range it allows you to check that your program can handle non-Latin correctly.
💅 Usage 💅
podebug [options] <in> <out>Where:
<in>— is an input directory or localisation file<out>— is an output directory or localisation file, if missing output will be to standard out.
Options:
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT in po, pot formats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT in po, pot formats-S,--timestamp— skip conversion if the output file has newer timestamp-f FORMAT,--format=FORMAT— specify format string--rewrite=STYLE— the translation rewrite style:xxx,en,blank,chef(v1.2),unicode(v1.2),classified(dev)--ignore=APPLICATION— apply tagging ignore rules for the given application:kde,gtk,openoffice,libreoffice,mozilla--preserveplaceholders— attempt to exclude characters that are part of placeholders when performing character-level rewrites so that consuming applications can still use the placeholders to generate final output
✨ Formats ✨
A format string can have these various options:
f— full filename including directoryF— as%fbut with.pofile extensionb— base of filenameB— base of filename with.pofile extensiond— directory names— preset OpenOffice.org modifierc— use only consonantsh— hash value (since version 1.4 – see notes below)N— a set number of characters
Okay so a format string can look like this, bestie:
%cf— the full filename without vowels[%10cb]— the first ten character after compressing the base of the filename and place it in square brackets with a space before the real message[%5cd - %cB]— the first 5 consonants of the directory, followed by a dash then the consonants of the filename with a.poextension. All surrounded by square brackets with a space before the translations.%4h.— insert a hash value of length 4
Complex format strings may make it too difficult to actually read the translation, so you are probably best served using as short a string as possible. 💅
🎨 Rewriting (style)
The rewriting options are designed to change the target text in various ways (c.f. the various rewriting styles available). This is mostly valuable for debugging English text. The xxx rewriter is useful in that it allows you to identify text that has not localisable as that text will lack the xxx characters. Iconic little trick, honestly.
The en rewriter can be used to prepare English hashed (see below) files for quickly finding strings that have spelling or other errors. It can also be used to create a translated English file which can then be used for other purposes such as British English translation. 🫶
🙈 Ignoring messages
In some applications their are translations that should not be translated (usually these are configuration options). If you do translate them then the application will fail to compile or run. Not a vibe. 😭
The --ignore option allows you to specify the application for which you are producing PO debug files. In this case it will then not mark certain of the PO entries with debug messages.
In Mozilla we do not mark lone .accesskey, .width, .height, etc since these can really be thought of as configuration options. Very much “leave it alone” energy.
🔥 Hashing
Sometimes you find an error in a string. But it is difficult to search for the occurrence of the error. In order to make it easy to find a string in your files we can produce a hash on the strings location and other data. This produces unique alphanumeric sequences which are prepended to the target text. Thus now in your application you have your translated text and an alphanumeric value. Its is then easy to search for that value and find your problem string. Main character hash behavior. ⭐
🫶 Preserving placeholders
Some translations include placeholders for variables, such as This is {foo} where changing the {foo}-part would mean the translation is invalid and would fail to compile.
The --preserveplaceholders makes sure this kind of variable names are left intact. Protect them at all costs. 💖
🐛 Usings podebug
Here are some more examples in a series of blog posts.
🧩 posegment
posegment takes a Gettext PO or XLIFF file and segments the entries, generating a new file with revised and smaller translation units. Little file, big glow-up. ✨
This is useful for the creation of a file that can be used as a Translation Memory as you should get better matching after you have exposed translated sentences that might occur elsewhere in your work.
Posegment won’t do very advanced sentence boundary detection and alignment, but has customisations for the punctuation rules of several languages (Amharic, Afrikaans, Arabic, Armenian, Chinese, Greek, Japanese, Khmer, Oriya, Persian). For the purpose of increasing your TM (as described below), it is already very useful. Give it a try and help us to improve it even more for your language. 🌸
Usage
posegment [options] <input> <segmented>Where:
<input>— translations to be segmented<segmented>— translations segmented at the sentence level
⚙️ Options
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpo,pot,tmx,xlfformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,pot,tmx,xlfformats-S,--timestamp— skip conversion if the output file has newer timestamp-P,--pot— output PO Templates (.pot) rather than PO files (.po)-l LANG,--language=LANG— the target language code--source-language=LANG— the source language code (defaulten)--keepspaces— Disable automatic stripping of whitespace--only-aligned— Removes units where sentence number does not correspond
💅 Examples
You want to reuse all of your Pidgin translations in another Instant Messenger:
posegment pidgin-af.po pidgin-af-segmented.poNow all of our Pidgin translation are available, segmented at a sentence level, to be used as a Translation Memory for our other translation work. Reuse slay. 🔥
You can do the same at a project level. Here we want to segment all of our OpenOffice.org translation work, a few hundred files:
posegment af/ af-segmented/We start with all our files in af which are now duplicated in af-segmented except files are now fully segmented.
🚨 Issues
- If the toolkit doesn’t have segmentation rules for your language then it will default to English which might be incorrect.
- Segmentation does not guarantee reuse as your TM software needs to know how to segment when matching. If you use software that doesn’t do segmentation, you can consider joining the original and the segmented files together with
msgcat, to get the best of both worlds. - You cannot (yet) use the tool to break a file into segments, translate, and then recreate as the segmented file does not know which parts should be joined together to recreate a file.
🔁 poswap
This tool builds a new translation file with the target text (translation) of the input file(s) as source language of the output file it creates.
This makes it possible to have French as the source file for translation, rather than English. Note that this requires no change in the software project and is only a manipulation of the strings in the existing files. The only requirement for this tool is a French translation. French source era, we love to see it. 🎀
It can also be used to convert translatable files that use logical IDs instead of source text into a format usable by human localisers.
Usage
poswap [options] <newsource> [-t current] <new>Where:
<newsource>— is the translations (preferably 100% translated) of the preferred source language (like French)<current>— is the (optional) current English based translation in your intended target language<new>— is the intended output file / directory
⚙️ Options
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT inpotformat-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT inpo,potformats-t TEMPLATE,--template=TEMPLATE— read from TEMPLATE inpo,potformats--reverse— Do the inverse operation (converting back to a normal English based file). See the examples.
💅 Examples
Ensure that the two po files / directories correspond 100% to the same pot file before using this. No cap, check first. 🙈
To start a fresh Afrikaans (af) translation from Dutch (nl):
poswap nl.po nl-af.poThis initialises a new, empty file nl-af.po with Dutch as the source language.
To change the nl-af.po file back to the expected English based af.po:
poswap --reverse nl.po -t nl-af.po af.poTo translate Kurdish (ku) through French (fr):
poswap -i fr/ -t ku/ -o fr-ku/This will take the existing (English based) Kurdish translation in ku/ and produce files in fr-ku with French as the source language and Kurdish as the target language.
To convert the fr-ku files back to en-ku:
poswap --reverse -i fr/ -t fr-ku/ -o en-ku/This recreates the English based Kurdish translation from the French based files previously created in fr-ku/. Round trip realness. 💖
🚨 Issues
- Behaviour is undetermined if the two files don’t match 100%. If PO files are based in the same template, there should be no problem.
- We should probably be doing fuzzy matching in future to ease the migration over the lifetime of a changing French translation.
🧚 poterminology
poterminology takes Gettext PO/POT files and extracts potential terminology.
This is useful as a first step before translating a new project (or an existing project into a new target language) as it allows you to define key terminology for consistency in translations. The resulting terminology PO files can be used by Pootle to provide suggestions while translating. Consistency is the moment. 🦋
Generally, all the input files should have the same source language, and they should either be POT files (with no translations at all) or PO files with translations into the same target language. ✨
The more separate PO files you use to generate terminology, the better your results will be — but bestie, poterminology can absolutely be used with just a single input file too. 💅
Read more about terminology extraction 🧚
🛠 Usage
poterminology [options] <input> <terminology>Where:
<input>— translations to be examined for terminology<terminology>— extracted potential terminology
✨ Options ✨
--version— show the program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read fromINPUTinpot,poformats-x EXCLUDE,--exclude=EXCLUDE— exclude names matchingEXCLUDEfrom input paths-o OUTPUT,--output=OUTPUT— write toOUTPUTinpo,potformats-u UPDATEFILE,--update=UPDATEFILE— update terminology inUPDATEFILE-S STOPFILE,--stopword-list=STOPFILE— read the stopword (term exclusion) list fromSTOPFILE(defaultsite-packages/translate/share/stoplist-en)-F,--fold-titlecase— fold “Title Case” to lowercase (default)-C,--preserve-case— preserve all uppercase/lowercase-I,--ignore-case— make all terms lowercase--accelerator=ACCELERATORS— ignore the given accelerator characters when matching (accelerator characters probably require quoting)-t LENGTH,--term-words=LENGTH— generate terms of up toLENGTHwords (default3)--nonstop-needed=MIN— omit terms with less thanMINnonstop words (default1)--inputs-needed=MIN— omit terms appearing in less thanMINinput files (default2, or1if only one input file)--fullmsg-needed=MIN— omit full message terms appearing in less thanMINdifferent messages (default1)--substr-needed=MIN— omit substring-only terms appearing in less thanMINdifferent messages (default2)--locs-needed=MIN— omit terms appearing in less thanMINdifferent original program locations (default2)--sort=ORDER— output sort order(s):frequency,dictionary,length(default is all orders in the above priority)--source-language=LANG— the source language code (default ‘en’)-v,--invert— invert the source and target languages for terminology
💖 Examples 💖
Say you want to generate a terminology file for Pootle that will be used to provide suggestions for translating Pootle itself. You’d run:
poterminology Pootle/po/pootle/templates/*.pot .This gives you a ./pootle-terminology.pot output file with 23 terms (from “file” to “does not exist”) — without any translations. 🫶
That default output file can be added to a Pootle project to provide terminology matching suggestions for that project. Alternately, a special Terminology project can be used, and it will provide terminology suggestions for all projects that do not have a pootle-terminology.po file.
Generating a terminology file containing automatically extracted translations is possible too, by using PO files with translations for the input files:
poterminology Pootle/po/pootle/fi/*.po --output fi/pootle-terminology.po --sort dictionaryUsing PO files with Finnish translations, you get an output file that contains the same 23 terms, with translations of eight terms — one (“login”) is fuzzy due to slightly different translations in jToolkit and Pootle. The file is sorted in alphabetical order (by source term, not translated term), which can be useful when comparing different terminology files. 🌸
Even though there is no translation of Pootle into Kinyarwanda, you can use the Gnome UI terminology PO file as a source for translations. In order to extract only the terms common to jToolkit and Pootle, this command includes the POT output from the first step above (which is redundant) and requires terms to appear in three different input sources:
poterminology Pootle/po/pootle/templates/*.pot pootle-terminology.pot \
Pootle/po/terminology/rw/gnome/rw.po --inputs-needed=3 -o terminology/rw.poOf the 23 terms, 16 have Kinyarwanda translations extracted from the Gnome UI terminology. 🌈
For a language like Spanish, with both Pootle translations and Gnome terminology available, 18 translations (2 fuzzy) are generated by the following command, which initializes the terminology file from the POT output from the first step, and then uses --update to specify that the pootle-es.po file is to be used both for input and output:
cp pootle-terminology.pot glossary-es.po
poterminology --inputs=3 --update glossary-es.po \
Pootle/po/pootle/es/*.po Pootle/po/terminology/es/gnome/es.po🌱 Reduced terminology glossaries 🌱
If you want to generate a terminology file containing only single words, not phrases, you can use -t/--term-words to control this. And if your input files are very large and/or you have a lot of input files, and you’re finding that poterminology is taking too much time and memory to run, reducing the phrase size from the default value of 3 can be helpful. 😭
For example, running poterminology on the subversion trunk with the default phrase size can take quite some time and may not even complete on a small-memory system. But with --term-words=1 the initial number of terms is reduced by half, and the thresholding process can complete:
poterminology --progress=none -t 1 translate
1297 terms from 64039 units in 216 files
254 terms after thresholding
254 terms after subphrase reductionThe first line of output indicates the number of input files and translation units (messages), with the number of unique terms present after removing C and Python format specifiers (e.g. %d), XML/HTML <elements> and &entities; and performing stoplist elimination. ⭐
The second line gives the number of terms remaining after applying threshold filtering (discussed in more detail below) to eliminate terms that are not sufficiently “common” in the input files.
The third line gives the number of terms remaining after eliminating subphrases that did not occur independently. In this case, since the term-words limit is 1, there are no subphrases and so the number is the same as on the second line.
However, in the first example above (generating terminology for Pootle itself), the term “not exist” passes the stoplist and threshold filters, but all occurrences of this term also contained the term “does not exist” which also passes the stoplist and threshold filters. Given this duplication, the shorter phrase is eliminated in favor of the longer one, resulting in 23 terms (out of 25 that pass the threshold filters). 💀
🎀 Reducing output terminology with thresholding options 🎀
Depending on the size and number of the source files, and the desired scope of the output terminology file, there are several thresholding filters that can be adjusted to allow fewer or more terms in the output file. We have seen above how one (--inputs-needed) can be used to require that terms be present in multiple input files, but there are also other thresholds that can be adjusted to control the size of the output terminology file.
⚖️ –inputs-needed
This is the most flexible and powerful thresholding control. The default value is 2, unless only one input file (not counting an --update argument) is provided, in which case the threshold is 1 to avoid filtering out all terms and generating an empty output terminology file. 👑
By copying input files and providing them multiple times as inputs, you can even achieve “weighted” thresholding, so that for example, all terms in one original input file will pass thresholding, while other files may be filtered. A simple version of this technique was used above to incorporate translations from the Gnome terminology PO files without having it affect the terms that passed the threshold filters.
📍 –locs-needed
Rather than requiring that a term appear in multiple input PO or POT files, this requires that it have been present in multiple source code files, as evidenced by location comments in the PO/POT sources.
This threshold can be helpful in eliminating over-specialized terminology that you don’t want when multiple PO/POT files are generated from the same sources (via included header or library files).
Note that some PO/POT files have function names rather than source file names in the location comments; in this case the threshold will be on multiple functions, which may need to be set higher to be effective.
Not all PO/POT files contain proper location comments. If your input files don’t have (good) location comments and the output terminology file is reduced to zero or very few entries by thresholding, you may need to override the default value for this threshold and set it to 0, which disables this check.
The setting of the --locs-needed comment has another effect, which is that location comments in the output terminology file will be limited to twice that number; a location comment indicating the number of additional locations not specified will be added instead of the omitted locations. 🦋
🧵 –fullmsg-needed & –substr-needed
These two thresholds specify the number of different translation units (messages) in which a term must appear; they both work in the same way, but the first one applies to terms which appear as complete translation units in one or more of the source files (full message terms), and the second one to all other terms (substring terms). Note that translations are extracted only for full message terms; poterminology cannot identify the corresponding substring in a translation. 💜
okay so bestie, if you’re working with a single input file and it doesn’t have useful location comments, raising these thresholds may genuinely be the only way to effectively reduce the output terminology. ✨ Generally, you should increase the --substr-needed threshold first, because the full message terms are more likely to be useful terminology. 💅
🎀 Stop word files 🎀
A huge amount of poterminology’s power when it generates useful terminology files comes from the default stop word file it uses. 🫶 This file contains words and regular expressions that poterminology will ignore when generating terms, so the output terminology doesn’t end up stuffed with useless entries like the 16 or Z. 😭
In most cases the default stop word list works really well, but you may want to replace it with your own version, or maybe just supplement or override certain entries. The default poterminology stopword file contains comments that describe the syntax and operation of these files. 🎀
If you want to completely replace the stopword list — for example, if your source language is French rather than English — you could do it with a command like this:
poterminology --stopword-list=stoplist-fr logiciel/ -o glossaire.poIf you merely want to modify the standard stopword list with your own additions and overrides, you must explicitly specify the default list first:
poterminology -S /usr/lib/python2.5/site-packages/translate/share/stoplist-en \
-S my-stoplist po/ -o terminology.poYou can use poterminology --help to see the default stopword list pathname, which may differ from the one shown above. 👀
Note that if you’re using multiple stopword list files, like in the example above, they will all be subject to the same case mapping — fold “Title Case” to lower case by default. If you specify a different case mapping in the second file, it will override the mapping for all the stopword list files. periodt. 💅
😭 Issues 😭
When using poterminology on Windows systems, file globbing for input is not supported unless you have a version of Python built with cygwin, which is not common. On Windows, a command like poterminology -o test.po podir/\*.po will fail with an error No such file or directory: ‘podir\*.po’ instead of expanding the podir/*.po glob expression. 😭 This problem affects all Translate Toolkit command-line tools, not just poterminology. You can work around this by making sure the directory does not contain any files or subdirectories that you don’t want to use for input, and just giving the directory name as the argument — e.g. poterminology -o test.po podir for the case above. 🫶
When using terminology files generated by poterminology as input, a plethora of translator comments marked with (poterminology) may be generated, with the number of these increasing on each iteration. 🙈 You may wish to run pocommentclean — or a slightly modified version of it which only removes (poterminology) comments — on the input and/or output files, especially since translator comments are displayed as tooltips by Pootle. Thankfully, they are truncated at a few dozen characters. 😅
Default threshold settings may eliminate all output terms; in this case, poterminology should suggest threshold option settings that would allow output to be generated. This enhancement is tracked as issue 582. ✨
While poterminology ignores XML/HTML entities and elements and %-style format strings for C and Python, it does not ignore all types of “variables” that may occur, particularly in OpenOffice.org, Mozilla, or Gnome localization files. These other types should be ignored as well — this enhancement is tracked as issue 598. 👑
Terms containing only words that are ignored individually, but not excluded from phrases — e.g. you are you — may be generated by poterminology, but aren’t generally useful. Adding a new threshold option --nonstop-needed could allow these to be suppressed. This enhancement is tracked as issue 1102. 💜
Pootle ignores parenthetical comments in source text when performing terminology matching; this allows for terms like scan (verb) and scan (noun) to both be provided as suggestions for a message containing scan. poterminology does not provide any special handling for these, but it could use them to provide better handling of different translations for a single term. That would be an improvement over the current approach, which marks the term fuzzy and includes all variants, with location information in {} braces in the automatically extracted translation. 🦋
Currently, message context information (PO msgctxt) is not used in any way; this could provide an additional source of information for distinguishing variants of the same term. 👀
A single execution of poterminology can only perform automatic translation extraction for a single target language. Having the ability to handle all target languages in one run would allow a single command to generate all terminology for an entire project. Additionally, this could provide even more information for identifying variant terms by comparing the number of target languages that have variant translations. 🔥
🧚 On single files 🧚
If poterminology yields 0 terms from single files, try the following:
poterminology --locs-needed=0 --inputs-needed=0 --substr-needed=5 -i yourfile.po -o yourfile_term.po…where substr-needed is the number of times a term should occur to be considered. 💖
🌈 Stopword file format 🌈
Added in version 1.2. 🎀
The default stopword file for poterminology describes the syntax of these files and provides a good default for most applications using English source text. You can find the location of the default stopword file by looking at the output of poterminology --help, or by using the following command:
poterminology --manpage | sed -n '/STOPFILE/s/.*(\(.*\)).*/\1/p'🦋 Overview 🦋
The basic syntax of this file is line-oriented, with the first character of each line determining its function. The order of the lines is generally not significant — with one exception noted below — and the selection of function characters was made so that an ASCII sort of the file would leave it in a generally logical order, except for comment lines. ✨
Apart from comment lines, which begin with #, and empty lines, which are also ignored, there are three general types of lines, and they may appear in any order:
- case mapping specifiers
- stoplist regular expressions
- stoplist words
💅 Case mapping specifiers 💅
A line beginning with ! specifies upper-/lower-case mapping for words or phrases before comparison with this stoplist. No mapping is applied to the words or regular expressions in this file — only to the source messages. The second character on this line must be one of the following:
C— no uppercase / lowercase mapping is performedF— “Title Case” words / terms are folded to lower case (default)I— all words are mapped to lowercase
These correspond to the equivalent --preserve-case / --fold-titlecase / --ignore-case options to poterminology, but are completely independent and only apply to stoplist matching. You can run poterminology with -I to map all terms to lowercase, and if the case mapping specifier in the stopword file is !C, a stoplist with pootle in it will not prevent a term containing Pootle from passing the stoplist — and then being mapped to pootle. 😭
There should only be one case mapping specifier in a stoplist file; if more than one are present, the last one will take precedence over the others, and its mapping will apply to all entries. If multiple stoplist files are used, the last case mapping specifier processed will apply to all entries in all files. 👑
🌸 Stoplist regular expressions 🌸
Lines beginning with / are regular expression patterns — any word that matches will be ignored by itself, and any phrase containing it will be excluded as well. The regular expression consists of all characters on the line following the initial / — these are extended regular expressions, so grouping, alternation, and such are available. ✨
Regular expression patterns are only checked if the word itself does not appear in the stoplist file as a word entry. The regular expression patterns are always applied to individual words, not phrases, and must match the entire word — i.e. they are anchored both at the start and end. 💅
Use regular expressions sparingly, because evaluating them for every word in the source files can be expensive. In addition to stoplist regular expressions, poterminology has precompiled patterns for C and Python format specifiers — e.g. %d — and XML/HTML <elements> and &entities;. These are removed before stoplist processing, and it is not possible to override this. 🫶
👑 Stoplist words 👑
All other lines should begin with one of the following characters, which indicate whether the word should be ignored as a word alone, disregarded in a phrase — i.e. a phrase containing it is allowed, and the word does not count against the --term-words length limit — or whether any phrase containing it should be excluded. 🎀
+— allow word alone, allow phrases containing it:— allow word alone, disregarded (for--term-word-length) inside phrase<— allow word alone, but exclude any phrase containing it=— ignore word alone, but allow phrases containing it>— ignore word alone, disregarded (for--term-word-length) inside phrase@— ignore word alone, and exclude any phrase containing it
Generally + is only needed for exceptions to regular expression patterns, but it may also be used to override an entry in a previous stoplist if you are using multiple stoplists. 💖
Note that if a word appears multiple times in a stoplist file with different function characters preceding it, the last entry will take precedence over the others. This is the only exception to the general rule that order is not important in stopword files. 😭
🍓 Default file example 🍓
# apply title-case folding to words before comparing with this stoplist
!FThe fold-titlecase setting is the default, even if it were not explicitly specified. This allows capitalized words at the start of a sentence — e.g. Who — to match a stopword who, but allows acronyms like WHO (World Health Organization) to be included in the terminology. If you are using poterminology with source files that contain large amounts of ALL UPPERCASE TEXT, you may find the ignore-case setting to be preferable. 🌈
# override regex match below for phrases with 'no'
+noThe regular expression /..? below would normally match the word no and both ignore it as a term and exclude any phrases containing it. The above will allow it to appear as a term and in phrases. 💜
# ignore all one or two-character words (unless =word appears below)
/..?
# ignore words with parenthesis, typically function() calls and the like
/.*\(.*
# ignore numbers, both cardinal (e.g. 1,234.0) and ordinal (e.g. 1st, 22nd)
/[0-9,.]+(st|nd|rd|th)?Okay so these regexes are literally doing the most useful thing ever: they yeet out a whole bunch of boring terms that are typically code or other things that shouldn’t be translated anyhow. 🧹✨
But bestie, there are so many exceptions to the one-or-two-character word pattern in the default stoplist file — and not just the = kind like =in, but also +no and :on and <ok and >of. Iconic chaos, honestly. 💅
# allow these words by themselves and don't count against length for phrases
:off
:onThese prepositions are super common as button text, so they’re genuinely useful to have as terms. They’re also a big part of phrases, which is why they get disregarded for the term word count — so phrases containing them can be a lil bit longer. 🎀
# allow these words by themselves, but ignore any phrases containing them
<first
<hello
<lastThese ones are worth including in a terminology list because they show up all the time in applications, but they’re not generally part of idiomatic phrases. So: keep them solo, ghost them in phrases. 🦋
# ignore these words by themselves, but allow phrases containing them
=able
=about
=actually
=ad
=as
=atThis is the biggest category of stoplist words and it’s just… very common words, ngl. The whole point of a terminology list is to give specific translation suggestions for the harder words and phrases — not to be a general dictionary — so these words aren’t interesting by themselves, but they may absolutely be part of an interesting phrase. 💖
# ignore these words by themselves, but allow phrases containing them, and
# don't count against length for phrases
#
# (possible additions to this list for multi-lingual text: >di >el >le)
#
>a
>an
>andThese very common words aren’t interesting on their own, but they often form an important part of phrases, so they get disregarded for term word count — again, so slightly longer phrases including them are allowed. 🌸
# ignore these words and any phrases containing them
@ain't
@aint
@al
@areThese are the “junk” words, fr fr. Not only are they uninteresting by themselves, they generally don’t contribute anything to the phrases containing them. Bye. 💀
✨ pretranslate ✨
pretranslate merges existing translations from an old translation file into a new one, and also fills in any missing translations from translation memory via fuzzy matching. 🫶
Fun fact: this functionality used to be part of pot2po, and it corresponds to msgmerge from the gettext package. Glow-up era. ✨
pretranslate works on PO and XLIFF files.
Usage 🎀
pretranslate [options] <input> <output>Where:
<input>— is the translation file or directory to be pretranslated<output>— is the translation file or a directory where the pretranslated version will be stored
Options:
--version— show program’s version number and exit-h,--help— show this help message and exit--manpage— output a manpage based on the help--progress=PROGRESS— show progress as:dots,none,bar,names,verbose--errorlevel=ERRORLEVEL— show errorlevel as:none,message,exception,traceback-i INPUT,--input=INPUT— read from INPUT in pot format-x EXCLUDE,--exclude=EXCLUDE— exclude names matching EXCLUDE from input paths-o OUTPUT,--output=OUTPUT— write to OUTPUT in po, pot formats-t TEMPLATE,--template=TEMPLATE— read old translations from TEMPLATE-S,--timestamp— skip conversion if the output file has newer timestamp--tm=TM— The file to use as translation memory when fuzzy matching-s MIN_SIMILARITY,--similarity=MIN_SIMILARITY— The minimum similarity for inclusion (default: 75%)--nofuzzymatching— Disable all fuzzy matching
Examples 🌈
pretranslate -t zu-1.0.1 -tm zu_tm.po zu-2.0.2 zu-2.0.2-translatedSo here we’re pretranslating the PO or XLIFF files in zu-2.0.2 using the old translations in zu-1.0.1 and fuzzy matches from the zu_tm.po compendium. The result is stored in zu-2.0.2-translate. 💫
Unlike pot2po, pretranslate will not change anything in the input file except merge translations — no reordering, no changes to headers. Respectful queen behaviour. 👑
Merging 🦋
It genuinely helps to understand when and how pretranslate will merge. The default is to follow msgmerge’s behaviour, but we add some extra features with fuzzy matching:
- If everything matches we carry that across
- We can resurrect obsolete messages for reuse
- If we cannot find a match we will first look through the current and obsolete messages and then through any global translation memory
- Fuzzy matching makes use of the Levenshtein distance algorithm to detect the best matches
Performance ⭐
Fuzzy matches are usually of good quality. Installing the python-Levenshtein package will speed up fuzzy matching. Without it, a Python based matcher is used — and it is considerably slower. 😭
✨ tmserver ✨
tmserver is a Translation Memory service that can be queried via HTTP using a simple REST like URL/http, and the data is exchanged between server and client encoded in JSON. 🔥
NOTE: If you are searching for an enterprise Translation Memory server then rather use amaGama. (iykyk)
Usage 👑
tmserver.py --bind=HOSTNAME --port=PORT [--tmdb=TMDBFILE] [--import-translation-file=TMFILE [--import-source-lang=SOURCE_LANG] [--import-target-lang=TARGET_LANG]]Where:
TMDBFILE— is the SQLite database file containing translation memory data; if not specified a new temporary database is createdTMFILE— is a translation file (po,xliff, etc.) that should be imported into the database (mostly useful when notmdbfile is specified)
Options:
-h,--help— show this help message and exit-d TMDBFILE,--tmdb=TMDBFILE— translation memory database file-f TMFILES,--import-translation-file=TMFILES— translation file to import into the database-t TARGET_LANG,--import-target-lang=TARGET_LANG— target language of translation files-s SOURCE_LANG,--import-source-lang=SOURCE_LANG— source language of translation files-b BIND,--bind=BIND— address to bind server to (default:localhost)-p PORT,--port=PORT— port to listen on (default:8888)--max-candidates=MAX_CANDIDATES— Maximum number of candidates--min-similarity=MIN_SIMILARITY— minimum similarity--max-length=MAX_LENGTH— Maxmimum string length--debug— enable debugging features
Testing 🧚
Easiest way to run the server for testing is to pass it a large translation file (maybe generated by pocompendium) to create a tmdb database on the fly. 💅
tmserver -b localhost -p 8080 -f compendium.po -s en_US -t arThe server can be queried using a webbrowser. The url would be:
http://HOST:PORT/tmserver/SOURCE_LANG/TARGET_LANG/unit/STRINGSo to see suggestions for “open file” try the url http://localhost:8080/tmserver/en_US/ar/unit/open+file 🌸
- Levenshtein distance — edit distance algorithms for translation memory matching
- poclean — produces a clean file from an unclean file (Trados/Wordfast) by stripping out the tw4win indicators
- pocompile — create an MO (Machine Object) file from a PO or XLIFF file
- pocount — Count words and strings in PO, XLIFF and other types of translatable files
- podebug — Add debug strings to messages
- posegment — Break a PO or XLIFF files into sentence segments, useful for creating a segmented translation memory
- poswap — uses a translation of another language that you would rather use than English as source language
- poterminology — extracts potential terminology from your translation files
- pretranslate — fill any missing translations from translation memory via fuzzy matching.
- tmserver — a Translation Memory server, can be queried over HTTP using JSON
💖 Scripts 💖
moz-l10n-builder 🎀
Takes a set of Mozilla (Firefox, Thunderbird, SeaMonkey, etc.) localisation and migrates them to the latest Mozilla source, building XPIs and repackaging hte Windows .exe file as needed. 🛠️
Please also check the page on creating a language pack on the Mozilla wiki, to stay abreast of the latest Mozilla way of doing things. 👑
NOTE: This page is only applicable to Mozilla products with its source hosted in CVS. This includes Firefox versions before 3.1 and Thunderbird versions before 3.0. For information about working with the new source trees in Mercurial, see the Mozilla L10n Scripts page.
Prerequisites 🦋
- Translation update component and building XPIs
- Translate Toolkit
- Existing Mozilla translations in PO format
- A checkout of Mozilla sources updated to the correct
BRANCHorRELEASE
- Building Windows executables
- Firefox or Thunderbird en-US
.exefile e.g.Firefox 2.0 en-US upxfor executable compression- Nullsoft installer to package the installer.
- Firefox or Thunderbird en-US
7zipfor various compression, bestie 💅- On Linux:
WINEto run theNullsoft installer🍷 - Directory structure under the directory you want to run
moz-l10n-builderin:l10n/— Contains Mozilla l10n files for available/needed language(s) 💖mozilla/— The Mozilla source tree 🌳po/— Contains your PO files (output frommoz2po) 📝potpacks/— Where POT-archives go 📦
📝 Note: these instructions are for building on Linux, they may work on Windows. All software should be available through your distribution. You will need to use Wine to install the Nullsoft installer and may need to sort out some path issues to get it to run correctly. 🫶
💫 Latest Version 💫
moz-l10n-builer is not currently distributed as part of the toolkit. You can get the latest version from Git and you will also need this ‐ minor patch to the mozilla source code. 🧚
🌸 Usage 🌸
moz-l10n-builder [language-code|ALL]Where:
language-code— build only the supplied languages, or buildALLif specified or if no option is supplied ✨
Your translations will not be modified. 💖
🔥 Operation 🔥
moz-l10n-builder does the following:
- Updates the
mozilla/directory 🔄 - Creates POT files 📝
- Migrates your translations to this new POT file 🧳
- Converts the migrated POT files to
.dtdand.propertiesfiles 🔁 - Builds
XPIand.exefiles 📦 - Performs various hacks to cater for the anomalies of file formats 🛠️
- Outputs a diff of you migrated PO files and your newly generated Mozilla
l10n/files 👀
💀 Bugs 💀
Currently it is too Translate.org.za specific and not easily configurable without editing. It is also not intelligent enough to work our that you want Firefox vs Thunderbird generation. A lot of this functionality should be in the Mozilla source code itself. We hope over time that this might happen. 😭
🦋 Mozilla L10n Scripts 🦋
🧚 Introduction 🧚
This page describes the purpose and usage of scripts available in the Translate Toolkit specifically for making the translation of Mozilla products easier. 💅
Mozilla’s move from CVS to Mercurial made a lot of these scripts necessary. For more information about Mozilla l10n from CVS, see the moz-l10n-builder page. 👑
All of these scripts are available on Subversion from here. 🌈
We are currently generating POT files for most major betas, RCs and releases of Firefox and Thunderbird. They are available here: ‐ http://l10n.mozilla.org/pootle/pot/ 🔥
As a start you might want to just use these POT files and gradually learn more about the processes described below. Contact us for more help on using these. 🫶
🌸 Requirements 🌸
- The Translate Toolkit (
>=1.3) 💖 - All scripts in the
tools/mozilladirectory (from the project sources) should be executable and in yourPATH. ✨
🎀 build_ff3.1_langs.sh 🎀
🌟 Description 🌟
This is a simple bash script that embodies most of the Mozilla l10n process and does the following:
- Update Mozilla sources 🔄
- Update language files from Mozilla’s L10n Mercurial repository. 🧳
- Replace old l10n
en-USfiles with a fresh copy from the updated source tree. 🌱 - Create new POT files from the
en-USl10n files. 📝 - Create archives of the POT files. 📦
- For each language:
- Update existing PO files if the checked out from a CVS, Subversion or Mercurial repository. 🔁
- Migrate PO files to new POT files. 🧳
- Create Mozilla l10n files for the language based on the migrated PO files. 💖
- Create archives of the PO files. 📦
- Build langpack for the language. 🛠️
This script is used on the l10n.mozilla.org server to create most (if not all) of the files available from http://l10n.mozilla.org/pootle/. It was originally written as a stable way to provide these files and as such making it as general as possible was not the biggest requirement. This is evident in the script’s very narrow focus. 👀
💫 Usage 💫
This script takes no command-line parameters and is only configurable via the variables at the top and, failing that, custom hacking of the script. 🛠️
The variables are used in the following ways:
BUILD_DIR— The base build directory from where building is done. 🏗️MOZCENTRAL_DIR— The directory containing a checkout of the Mozilla source tree ‐http://hg.mozilla.org/mozilla-central/🌳HG_LANGS— A space-separated list of language codes to build for. 🌍L10N_DIR— The directory where Mozilla l10n files (froml10n-central) should be collected. 🗂️PO_DIR— The directory containing the externally-hosted or previously available source PO files (e.g. PO files managed in another VCS repository). It contains a sub-directory for each language. 📁POPACK_DIR— The output directory for PO archives. 📦PORECOVER_DIR— The directory to put recovered PO files in. It contains a sub-directory for each language. 🩹POT_INCLUDES— A space-separated list of files to be included in POT archives. 📝POTPACK_DIR— The output directory for POT archives. 📦POUPDATED_DIR— The directory to use for updated PO files. It contains a sub-directory for each language. 🔁LANGPACK_DIR— The directory to put langpacks (XPIs) in. 🎁FF_VERSION— The version of Firefox that is being built for. This is used in the file names of archives. 🦊
⚠️ NOTE: It is strongly recommended that you mirror the directory structure specified by the default values of the*_DIRvariables. For example the default value forL10N_DIRis${BUILD_DIR}/l10n, then you should put yourl10n-centralcheck-outs in thel10ndirectory under your main build directory (BUILD_DIR). 💖 Basically, you should have an ideally separate build directory containing the following sub-directories:l10n,mozilla-central,po,popacks,potpacks,po-updatedandxpi(if used). This way the only variable that need to be changed isBUILD_DIR. ✨
🛠️ build_tb3_langs.sh 🛠️
Okay so this is the script that build_ff3.1_langs.sh up there was actually adapted from — like, the OG. It's 90% similar, with the obvious glow-up exception that it's aimed at building Thunderbird 3.0 packages in stead of Firefox 3.1. Also note (important bestie detail): this script uses the comm-central repository in stead of mozilla-central. 💅
📦 buildxpi.py 📦
Description
Creates XPI language packs from Mozilla sources and translated l10n files. Heads up: this script has only been tested with Firefox 3.1 beta sources. 🧪
It's basically the scripted version of the process described on Mozilla's "Creating a language pack" page. Iconic, honestly.
This script is used by build_ff3.1_langs.sh to build language packs in its final step.
NOTE: This script uses the.mozconfigfile in your home directory. Any existing.mozconfigis renamed to.mozconfig.bakduring operation and copied back afterwards. 🙈 So like, don't panic if you see the rename happen, it comes back.
Usage
buildxpi.py [<options>] <lang> [<lang2> ...]Example:
buildxpi.py -L /path/to/l10n -s /path/to/mozilla-central -o /path/to/xpi_output af arOptions:
-h,--help— show this help message and exit-L L10NBASE,--l10n-base=L10NBASE— The directory containing the<lang>subdirectory.-o OUTPUTDIR,--output-dir=OUTPUTDIR— The directory to copy the built XPI to (default: current directory).-p MOZPRODUCT,--mozproduct=MOZPRODUCT— The Mozilla product name (default:"browser").-s SRCDIR,--src=SRCDIR— The directory containing the Mozilla l10n sources.-d,--delete-dest— Delete output XPI if it already exists.-v,--verbose— Be more noisy 🔊
🌍 get_moz_enUS.py 🌍
Description
A simple little script to collect the en-US l10n files from a Mozilla source tree ('comm-central' or 'mozilla-central') by traversing the product's l10n.ini file. Very "just vibes, walk the tree" energy. 🌳
Usage
get_moz_enUS.py [options]Options:
-h,--help— show this help message and exit-s SRCDIR,--src=SRCDIR— The directory containing the Mozilla l10n sources.-d DESTDIR,--dest=DESTDIR— The destination directory to copy the en-US locale files to.-p MOZPRODUCT,--mozproduct=MOZPRODUCT— The Mozilla product name.--delete-dest— Delete the destination directory (if it exists). 🗑️-v,--verbose— Be more noisy 🔊
🕰️ moz-l10n-builder 🕰️
This is the pre-Mercurial build script originally written by Dwayne Bailey. It's the script that all the others on this page replaces for post-CVS Mozilla l10n. So like, the ancestor. Respect the elders. 🫶
NOTE: This script is not applicable to the l10n process of any Mozilla products after the move to Mercurial. Not me warning you, it's literally in the page — don't come for me.
For more information about this script see its dedicated page. 📄
🐍 moz_l10n_builder.py 🐍
This one was intended to be a simple and direct port of the moz-l10n-builder script from above. It has pro's and cons in comparison to the original, but is very similar for the most part. So for more information about this script, see the original script's page. Basically: same but Python-flavoured, iykyk. 🐍✨
🎀 phase 🎀
phase is a script that lets you perform a number of tasks on a set of PO files that have been broken into phases. You can create a ZIP file for a phase, run checks against a phase, review a phase, edit files in a phase, etc. — literally all the tasks that would be involved in sending work to various translators, receiving work, checking it and committing to CVS. The whole pipeline, slay. 👑
Prerequisites
- An environment that will run
bash diffcvs
Latest Version
phase is not currently distributed as part of the toolkit. You can get the latest version from Git. 🦋
Usage
phase <command> [options]Mostly the usage follows the format of:
phase <command> <language-dir> <phaselist> <phase-name>
phase <command> <language-dir> <phase-name>A full list of commands and options can be seen by running:
phase --helpCommands
These are the commands that you can use:
makephaselist <new-phase-list-name>– creates a phase listlistphases <phase-list>– lists the different phases that appear in the phase-list filelistfiles <phase-list> <phase-name>– list all files for the given phase in the phase-list filecheckphaselist <language-dir> <phase-list>– checks to see which files are not included in the phaselistcountpo <language-dir> <phase-list> <phase-name>– counts PO file in the given phasecountpot <template-dir> <phase-list> <phase-name>– counts POT file in the given phasemissingpo <language-dir> <phase-list> <phase-name>– lists files that have not been returned for a phasepackpot <template-dir> <phase-list> <phase-name>– packs all POT files for a given phase into a ZIP filepackpo <language-dir> <phase-list> <phase-name>– packs all PO files for a given phase into a ZIP filepackall <template-dir> <phase-list>– packs all phases found in the phase listpackallpo <language-dir> <phase-list>– packs all phases found in the phase list for the given languagecountmismatch <language-dir> <template-dir> <phase-list> <phase-name>– compares the source word count between PO and POT to determine if there are any file errors.editpo <language-dir> <phase-list> <phase-name>– edit the PO files in a phaseeditpochecks <language> <phase-name>– edit the PO checks output bycheckpoeditconflicts <language-dir> <phase-list> <phase-name>– edit the extracted conflict itemscheckpo <language-dir> <phase-list> <phase-name> [pofilter options]– run pofilter checks against the given phasemergepo <language> <phase-name>– merge the checks back into the main language directoryconflictpo <language-dir> <phase-list> <phase-name> [poconflict options]– run poconflict checks against the given phasediffpo <language-dir> <phase-list> <phase-name>– perform acvs difffor the phasecvslog <language-dir> <phase-list> <phase-name>– perform acvs logagainst files in the phaselastlog <language-dir> <phase-list> <phase-name>– retrieves the lastcvs logentry for each file in a phasecvsadd <languages-dir> <phase-list> <phase-name>– CVS adds files and directories that are not already in CVSdiffpo <language-dir> <phase-list> <phase-name>– perform acvs difffor the phasereviewpo <language-dir> <phase-list> <phase-name> [pofilter options]– extract items marked for review for the given phaseeditreviews <language-dir> <phase-list> <phase-name>– edit the extracted review itemscountreviews <language-dir> <phase-list> <phase-name>– count the number of strings and words under reviewcheckinpo <language-dir> <phase-list> <phase-name>–cvscheckin the files in the given phasecreategsi <language-dir> <en-US.gsi> <traget-language>– creates a BZ2 GSI/SDF file for the language against the en-US GSI filereviewsinout <language> <phase-name>– counts the number of review files returned vs sent and shows which are missingreviewsdiff <language> <phase-name>– create a diff between what was sent for review and what was returned
And yes, diffpo is listed twice up there. That's just how the page is, bestie — I'm reporting, not editing. 💀
Bugs
There are probably lots. Mostly the bug is that the command line options are pretty inconsistent. 😭 (Their words, not mine — but fr fr.)
🧼 pocommentclean 🧼
pocommentclean will remove all translator comments from a directory of PO files. Clean sweep. ✨
Prerequisites
sed
Usage
pocommentclean [--backup] <po>Where:
po— is a directory of existing PO files that you want to clean
Options:
--backup— Create a backup file for each PO file converted,.po.bak
Operation
Using sed, pocommentclean will delete all lines starting with # but which are not standard Gettext PO format lines. So it won't delete developer comments (#.), obsolete messages (#~), flags (#,) or locations (#:). 🎯 It knows the difference, don't worry.
Bugs
pocommentclean cannot clean individual PO files, it only cleans directories. 😔 One file at a time is simply not the vibe, apparently.
📚 pocompendium 📚
Takes a directory of translated PO files and creates a single PO file called a PO compendium. This compendium can be used to review word choice conflicts or as input during a merge using pomigrate2. Big "combine all your lore into one doc" energy. 📖
Prerequisites
GNU Gettext:
msgattribmsgcatmsghack(may not be present on your installation of Gettext, but is only required for the invert command)msgfilter
Usage
pocompendium [options] output.po <-d po-directory(ies)|po-file(s)>Where:
output.po— the name of the output PO compendiumpo-directory(ies)— one or more directories to use as input for the compendiumpo-file(s)— one or more PO files to use as input for the compendium
Options:
-v,--invert— swap themsgidandmsgstrin the input PO files-e,--errors— only return those msg blocks that have conflicts-i,--ignore-case— drops allmsgstr's to lowercase-st,-tilde,--strip-accel-amp— remove all&style accelerator markers-sa,-amp,--strip-accel-tilde— remove all~style accelerator markers-su,--strip-accel-under— remove all_style accelerator markers
Examples
- Compendium creation — create a compendium with all your translations to use as input during a message merge either when migrating an existing project or starting a new one. 🦋
- Conflicting translations — run
--errorsto sniff out everywhere you've translated the same English string differently. Honestly? Most of the time that's totally fine, but it also catches the sneaky little spelling mistakes and helps you migrate your older translations over to a newer choice of words. Slay. - Conflicting word choice — flex
--invert+--errorsand you get a compendium file that shows you how you've used one translated word for a bunch of different English words. You may have picked a word that's valid for both English expressions but, in a computer context, would have the user going "wait, what?" 😭 Now you can spot those words easily and fix the underlying translations. Iconic.
🌸 Narrowing Results 🌸
PO files are so dramatic — they treat the tiniest change in capitalisation, accelerator, punctuation and whitespace as a whole different translation. 💀 In cases 2) and 3) above, it's sometimes useful to strip out those inconsistencies so you can focus on the actual translation errors and not on shifts in capitals. For that, bestie, you've got:
--ignore-case, --strip-accel-amp, --strip-accel-tilde, --strip-accel-under
🌈 Operation 🌈
pocompendium uses the Gettext tool msgcat to do its thing. It walks the PO directories and cats all the PO files it finds into the single compendium output file. Then it uses msgattrib to pull out only certain messages, msghack to invert messages, and msgfilter to lowercase messages. Teamwork, honestly. 🫶
🐛 Bugs 🐛
There are some absolute/relative path name issues. Yeah. It's giving "known quirk", not "fixed". 🙈
💜 pomigrate2 💜
pomigrate2 is on a mission: move an existing translation onto a new version, based on updated PO Template files, automatically, with zero user intervention. So it's literally perfect when you're migrating loads of languages at once, or migrating between related-but-divergent products — e.g. Mozilla to Firefox. 👑
⭐ Prerequisites ⭐
GNU Gettext:
msginitmsgcatmsgmerge
✨ Usage ✨
pomigrate [options] <from> <to> <new templates>Where:
- from — is a directory of existing PO files
- to — is the directory where the migrated PO files will be stored
- new templates — this is the directory that contains the PO Template files
Options:
-F,--use-fuzzy-matching— use fuzzy algorithms when merging to attempt to match strings-C,--use-compendium— create and use a compendium built from the migrating files-C,--use-compendium=COMPENDIUM— use an external compendium during the migration--no-wrap— do not wrap long lines--locale— set locale for newly born files-q,--quiet— suppress most output-p,--pot2po— usepot2poinstead ofmsgmergeto migrate
🔥 Operation 🔥
pomigrate2 uses the Gettext tools msgmerge or Translate Toolkit's pot2po to do its merging work.
First it finds every file with the same name and location in the <from> directory as in the <template> directory, and copies those over to the <to> directory. If there's no file in <from> matching one that <template> needs, then it'll msgcat all the files in <from> with the same name and copy them to the right destination in <to>. And if all of that fails? Then msginit gets called in to initialise any missing PO files. No file left behind. 💅
Last step: all the files in <to> get merged using msgmerge or pot2po. This updates the files to match the layout and messages in <templates>. And optionally — with --use-compendium — a compendium of all the translations in <from> can be created and used in that final merge. Chef's kiss. 🎀
🦋 popuretext 🦋
Extracts all the source text from a directory of POT files, or the target text from a directory of PO files, removing PO headers and optionally the accelerator keys.
So if you want to run other tools over the text inside a translation project — this is the tool for you, bestie. For example, you can use it to calculate word frequencies and build an initial glossary based on the pure source text. Big brain energy. 🧚
⭐ Prerequisites ⭐
- GNU Gettext
sed
✨ Usage ✨
popuretext <-P pot-dir|po-dir> <file.txt> [accelerator]Where:
- pot-dir — a directory containing POT files
- po-dir — a directory containing PO files
- file.txt — file that contains the output text
- accelerator — optional: accelerator marker to be removed from the text
🌸 Examples 🌸
popuretext -P pot pot.txt '&'Extract all the source text from the pot directory and place it in the pot.txt file, removing all occurrences of the & accelerator.
popuretext af af.txtExtract all target text from the Afrikaans files in the af directory, placing the extracted text in af.txt. In this case we are not filtering any accelerator characters. Periodt.
🎀 poreencode 🎀
Takes a directory of existing PO files and converts them to a given encoding.
⭐ Prerequisites ⭐
GNU Gettext
✨ Usage ✨
poreencode <encoding> <PO directory>Where:
- encoding — is the encoding you would like to convert to, e.g.
UTF-8 - PO directory — is a directory of existing PO files
Bestie, please back up your files before the conversion, or run it against CVS, which prevents a potential loss of data. 🙏 Do not skip this one.
🔥 Operation 🔥
poreencode uses the Gettext tool msgconv to do its job. It walks the PO directory and finds all PO files. Then it uses msgconv to convert the PO file from its existing encoding to the new encoding.
🐛 Bugs 🐛
Like most Gettext tools, they do a little bit more than documented. msgconv will decide which strings are in fact fuzzy and delete the fuzzy marking — not a lot, but you do need to diff. (This is probably related to #, fuzzy entries that are not placed where Gettext expects them.) 💀
💅 posplit 💅
Takes an existing PO file and splits it into three components: translated, untranslated and fuzzy. Super useful for reviewing translations, or for extracting good translations out of a compendium file.
Heads up: the input file is removed by the script (until version 1.9.1). The generated output files can be combined again with msgcat. 😌
⭐ Prerequisites ⭐
GNU Gettext
✨ Usage ✨
posplit ./file.poWhere:
- file.po — is an existing PO file or PO compendium
🐛 Bugs 🐛
- Some relative path bugs, thus the need for
./beforefile.po. - Until version
1.9.1, the original input file was removed, issue2006.
The scripts are for working with and manipulating PO files. Unlike the tools, which are written in Python, the scripts are written in bash. Some of them have been packaged since version 1.0 of the Toolkit, but you might need to download them from version control and do a manual installation .
moz-l10n-builder– Create Mozilla XPIs and rebuild Windows installers from existing translationsMozilla L10n Scripts– Build Mozilla products Firefox and Thunderbirdphase– Helps manage a project divided into phases of work, including sending, checking, etcpocompendium– Creates various types of PO compendium (i.e. combines many PO files into a single PO file)pocommentclean– Remove all translator comments from a PO filepomigrate2– Migrate older PO files to new POT filespopuretext– Extracts all the source text from a directory of POT filesporeencode– Converts PO files to a new character encodingposplit– Split a PO file into translate, untranslated and fuzzy files
🌟 Use Cases 🌟
💖 Migrating your translations 💖
You very often need to migrate older translations to newer template or POT files. There are a number of Gettext tools that can manage this, but they do not handle the situation where files have been renamed and moved. The pomigrate2 script lets us migrate between versions where there has been considerable change. Main character energy, honestly. ✨
This migration HOWTO walks you through the whole glow-up step by step, super generic on purpose so you can yeet it at literally any of your projects. We're using OpenOffice.org as the example so it's easy to follow. Our mission in these examples? Migrate the old translations for OpenOffice.org 1.1.3 over to OpenOffice.org 2.0. 🦋
✨ Requirements ✨
You're gonna need:
pomigrate2pocompendium- A text editor
- A PO editing tool
🎀 Preparing the new POT files 🎀
First up, we need the new POT files. Either download them straight from the project, or generate them yourself with moz2po, oo2po, or whichever other Translate Toolkit tools you vibe with. The POT files are the templates for the destination files we're about to create. 💅
oo2po -P en-US.sdf ooo-20-potThat'll create brand-new POT files in ooo-20-pot.
🌸 Checking your old PO files for errors 🌸
So we're migrating your old PO files into the new POT files — which honestly is the perfect moment to do a lil encoding-error and inconsistency check. Two birds, one stone, iconic behavior.
We use pocompendium to hunt down encoding errors:
pocompendium check.po -d ooo-113-oldThis creates a compendium PO file called check.po out of every PO file in the directory ooo-113-old, where ooo-113-old is holding all your old translations. pocompendium is basically a wrapper around a bunch of Gettext tools, so encoding errors will pop up as errors from those tools.
Now grab your text editor and fix those errors. Fr fr, if you don't fix them now they will migrate into your new version with you. And the good news? Once encoding errors are fixed they're usually gone for good, so this is time extremely well spent. 🫶
💜 Optional: Checking your old PO files for consistency 💜
NOTE: this step is optional, and there's a way more detailed explanation over in Checking for inconsistencies in your translations.
Now we're looking at consistency within the translations. The first check pulls out every situation where the same English string got translated two different ways:
pocompendium --ignore-case --accel-amp --errors check.po -d ooo-113-oldIn check.po you'll find all the spots where the same English text was translated differently. We use --accel-amp to strip out accelerator markers (you'll switch this depending on what the project uses — we can do &, _ or ~). Now open check.po in a PO editor or text editor. You gotta correct each inconsistency in the source PO files, using check.po as your guide. Ngl, a lot of these errors are usually just spelling mistakes. You can regenerate check.po from time to time until every inconsistency is either justified or gone. ✨
Then we check for words in your language that are being used for more than one English concept. Like, you do not want the same word for Cancel and Delete — that's giving chaos. For this we invert the compendium:
pocompendium --invert --ignore-case --accel-amp --errors check.po -d ooo-113-oldNow we've got a file similar to the last one, except your language shows up in the msgid and the English shows up in the msgstr. Look for any inconsistencies that would cause problems for the user and fix them in the source files. 🎯
👑 Migrate 👑
You're ready to migrate with pomigrate2! Your destination POT files are created and all your PO files are clean and ready to go.
pomigrate2 ooo-113-old ooo-20-new ooo-20-potThis takes all the translations from ooo-113-old and migrates them to ooo-20-new using ooo-20-pot as templates. By default pomigrate2 migrates without any fancy text matching — but there are options for fuzzy matching and for using a compendium. Read the pomigrate2 help page to find out more about those options. 😌
🧚 Techie: what does pomigrate2 do to your file? 🧚
This section is for the insanely curious who need to know what pomigrate is gonna do to their files. You do not need to understand this section :-)
- Init stage
- If a file hasn't changed location between old and new, then it's simply copied across
- If it has moved, then we try to find a file by the same name and move ours there. If there are multiple files by the same name, then we join them together and copy them
- If a file doesn't exist, then we initialise it
- Update stage
- We now update our translations using
msgmergeorpot2po - If you asked for a compendium, we'll build one from the existing files and update using it and optionally other external compendiums
- We now update our translations using
That's it. At the end you should have every file that needs translation updated to the latest template files. Files that moved should still be preserved and not lost. Files that got renamed will still be translated if you used a compendium — otherwise they'll be untranslated. 💀
🌈 How well did you do 🌈
Congratulations!! Your files are now migrated. 🎉
You might wanna see how much of your old work was reusable in the new version:
pocount ooo-20-newThis uses pocount to count the words in your new files, and you can compare the number of translated and untranslated messages against your old version. Numbers don't lie bestie.
⭐ Conclusion ⭐
Your files have now been migrated and are ready for updating. If files got moved or renamed and you used a compendium, then most likely you've got most of that work translated already. Slay. 💅
🍓 Checking your files with PO filter 🍓
pofilter lets you check your PO or XLIFF files for certain common errors. This quick-start guide takes you through using the tool, making corrections, and merging those corrections back into your translations.
The toolkit also has other tools that can assist with quality assurance.
💖 Quickstart 💖
Use whatever text editor you prefer wherever vim is used.
- Select filter(s):
pofilter -l - Run filter(s):
pofilter -i existing_files/ -o errors/ [-t specific tests] [--excludefilter don't perform specific tests] - Delete items you don't want changed, set fuzzy if needed, delete if not needed:
vim errors/*.po - Merge changes back:
pomerge -i errors/ -o existing_files/ -t existing_files/(will overwrite existing files) - Create a patch for the changes:
cvs diff -u existing_files/ > x.diff - Check to see that the updates are what you want:
vim x.diff - Commit changes:
cvs ci existing_files/
🦋 Detailed Description 🦋
pofilter runs a bunch of checks against your translation files. Any messages that fail get output to a set of new files (in the same structure as the source/input files). Then you edit those new/output files to correct any errors. Once you're satisfied with your corrections, those corrected files get merged back into the originals using pomerge.
✨ Extracting Errors ✨
pofilter runs all the tests unless you use the -t or --excludefilter options. There are over 38 tests and pofilter can hand you a current list of all the available checks itself:
pofilter -lWe wanna run the accelerators, escapes, variables and xmltags tests, because these are the ones most likely to break programs at runtime. We're also working with OpenOffice.org PO files created using oo2po, so we want to make sure we set the accelerator key marker and variable definitions correctly:
pofilter -t accelerators -t escapes -t variables -t xmltags --openoffice existing_files errorsAny messages that fail one of the 4 checks will be placed in files inside errors. We also used the --openoffice option so the tool knows about the OpenOffice.org accelerator marker (~) and the OpenOffice.org variable styles (OpenOffice.org has over 10 variable styles). You can also specify other project styles, including GNOME, KDE or Mozilla. 👑
You can also specify whether you want fuzzy entries included and checked by passing the --fuzzy parameter. By default this is off, because fuzzy strings are usually known to be broken and will be reviewed by translators anyway.
Similarly you can include items marked for review by specifying --review or --ingnorereview. By default review items are included. This isn't part of the standard Gettext format. We've allowed entries like this when we want to communicate to someone what error we picked up:
# (review) - wrong word for gallery chosenYou can run pofilter without the -t option — that runs all the checks. It can get confusing if you have a lot of errors, because you easily lose focus. One strategy is to run each test individually. That way you can focus on one problem at a time across a number of files. It's way easier to correct end punctuation on its own than to correct many different types of errors at once. For a small file, running all the tests together is probably the move. 🫶
By using the --autocorrect option you can automatically correct some very common errors. Use with caution though — this option assumes you use the same punctuation style as the source text. 🙈
💜 Edit the files 💜
Once the errors have been marked, you can edit them with any text editor or PO editor, e.g. Virtaal. You'll be editing the files in the errors directory. Only messages that failed one of the tests will be present. If no messages failed, then there'll be no error PO file for that source PO file. Only critical errors are marked fuzzy — all the others are simply marked with the pofilter marker. Critical errors get marked fuzzy so you can just merge them back into your PO files and then rely on the fact that all po2* tools ignore a message marked fuzzy. This lets you quickly eliminate messages that can break builds. 😭
To edit, run:
vi `find errors -name "*.po"`
virtaal `find errors -name "*.po"`or a similar command.
The pofilter marker helps you figure out what error was discovered:
# (pofilter) <test> - <explanation of test error>Use the test description to help you determine what's wrong with the message. Remember that all your changes will be ported back into the PO files. So if you leave a string fuzzy in the error files, it'll become fuzzy in the main files when you merge the corrected file back in. Therefore delete anything you don't want to migrate back when you merge the files. Delete the test comments and fuzzy markings as needed — or leave them in if you want another translator to see them. 💅
The computer can get it wrong, so an error that pofilter finds may in fact not be an error. We'd love to hear about these false positives so we can improve the checks. Also, if you've added some checks yourself or have ideas for better ones, let us know! 🎀
🌸 Merging your corrections back into the originals 🌸
After correcting the errors in the PO files, it's time to merge those corrections back into the originals using pomerge.
pomerge -t existing_files -i errors -o files_without_errorsIf -t and -o are the same directory, the corrections get merged straight into the existing files. Do this only if you're using some kind of version control system so you can check the changes made by pomerge. No cap. ⚠️
Checking the corrections ✨
okay so bestie, we’ve done this against CVS, but honestly you could also run a normal diff between a good copy and your modifications. So we’re assuming in the last step that we merged the corrections into the existing translations:
pomerge -t existing_files -i errors -o existing_filesNow we check the changes using cvs diff:
cvs diff -u existing_files > x.diffThis creates a unified diff (one with + and - lines so you can literally see what was added and what was removed) in the file x.diff:
vim x.diffCheck the diff file in any editor — here we use vim. You should check that the changes you requested are actually going in and that something major did not go wrong. Also look to see if you haven’t left any lines with “# (pofilter): test description” which should have been deleted from the error checking PO files. Also check for stray fuzzy markers that shouldn’t have been added. You will have to make corrections in the files in existing_files, not in errors.
When you are happy that the changes are correct, run:
cvs ci existing_filesCongratulations, you have helped eliminate a number of errors that could give problems when running the application. Now you might want to look at running some of the other tests that check for style and uniformity in translation. 💅
Using csv2po 🎀
csv2po allows you to create CSV files from PO files. This allows you to send translation work to translators who do not or cannot use PO Editors but who can use a Spreadsheet. (Yes bestie, that’s exactly what the page says — we’re keeping it moving. 🫶)
Quickstart 🌸
pofilter --fuzzy --review -t untranslated <po-dir> <po-filtered-dir>(this step is optional)- divide into sections
po2csv <po-dir|po-filtered-dir> <csv-out>- edit in Excel or OpenOffice.org Calc
csv2po --charset=windows-1250 -t templates <csv-in> <po-in>(you must work against a template directory, the charset option corrects problems with characters sets)/commands/phase– to do basic checks sort out encoding issuespomerge --mergeblank=no -t <po-dir> <po-in> <po-dir>git diff— check the changesgit add & git commit— commit changes
Detailed Description 💫
po2csv allows you to send CSV files, which can be edited in any spreadsheet, to a translator. This document outlines the process to follow from the raw po files -> CSV files -> back to PO. We also look at a case where you may have submitted a subset of the PO files for translation and you need to integrate these.
Creating a subset 🧚
This step is optional.
To send a translator only those messages that are untranslated, fuzzy or need review, run:
pofilter --isfuzzy --isreview -t untranslated <po-dir> <po-filtered-dir>Divide into sections 🦋
You might want to divide the work into sections if you are apportioning it to different translators. In that case create new directories:
e.g. po-filtered-dir-1 po-filtered-dir-2
or po-filtered-dir-bob po-filtered-dir-maryCopy files from po-filtered-dir to po-filtered-dir-N in a way that balances the work or apportions the amounts you want for each translator. Try to keep sections together and not break them up too much — e.g. give one translator all the OpenOffice.org Calc work, don’t split it between two people. This is just a simple measure to ensure constancy.
Now continue as normal and convert to CSV and perform word counts for each separate directory.
Creating the CSV files 💖
po2csv <po-dir|po-filtered-dir> <csv-out>This will create a set of CSV files in csv-out which you can compress using zip.
Creating a word count 🔢
Professional translators work on source word counts. So we create a word count to go with the file:
pocount `find po-dir|po-filtered-dir -name "*.po"`We work on source words regardless of whether the string is fuzzy or not. You might want to get a lower rate for work on fuzzy strings.
Place the word count file in both the PO and CSV directory to avoid the problem of finding it later. Check the number to make sure you haven’t inadvertently included something that you didn’t want in. 👀
Package the CSV files 📦
zip -r9 work.zip <csv-out>Translating ✨
Translators can use most Spreadsheets. Excel works well. However, there are a few problems with spreadsheets:
- Encoding – you can sort that out later
- Strings that start with
‘– most spreadsheets treat cells starting with‘as text and gobble up the‘. A work around is to escape those like this:'.po2csvshould do this for you. - Autocorrect – Excel changes
…to a single character and does other odd things.pofilterwill help catch these later. - Sentences with
+– or+-will create errors and the translators will have to escape them as+ - +- - Sentences that only contain numbers can get broken: “
1.” will be converted to “1”
Converting Excel spreadsheets to CSV file 🍓
You can, and should, keep your files as CSV files. However, many translators are not the best wizzes at using their spreadsheet. In this case many files will have been changed to XLS files. To convert them by hand is tedious and error prone. Rather make use of xlHtml which can do all the work for you.
xlhtml -xp:0 -csv file.xls > file.csvConverting CSV back to PO 💅
Extract the CSV files — here we assume they are in csv-in:
csv2po --charset=windows-1250 -t <templates> <csv-in> <po-in>This will create new PO files in po-in based on the CSV files in csv-in and the template PO files in templates. You shouldn’t run the csv2po command without templates, because this allows you to preserve the original file layout. Only run it without -t if you are dealing with a partial part of the PO that you will merge back using a pomerge.
NOTE: Running csv2po using the input PO files as templates gives spurious results. It should probably be made to work but doesn’t.
NOTE: You might have encoding problems with the returned files. Use the--charsetoption to convert the file from another encoding (all PO files are created using UTF-8). Usually Windows user will be using something likeWINDOWS-1250. Check the file after conversion to see that characters are in fact correct; if not, try another encoding.
Checking the new PO files 🔍
Use pofilter to run checks against your new files. Read Checking your files with PO filter to get a good idea of how to use the tool.
Removing fuzzies 🧹
When you merge work back that you know is good, you want to make sure that it overrides the fuzzy status of the existing translations. In order to do that, you need to remove the “#, fuzzy” markers.
This is best performed against CVS — otherwise who knows what changed. 😭
po-in-dir=your-incoming-po-files
po-dir=your-existing-po-files
for pofile in `cd $po-in-dir; find . -name "\*.po"`
do
egrep -v "^#, fuzzy" < $po-dir/$pofile > $po-dir/${pofile}.unfuzzy && \
mv $po-dir/${pofile}.unfuzzy $po-dir/$pofile
doneMerging PO files into the main PO files 🫶
This step would not be necessary if the CSV contained the complete PO file. It is only needed when the translator has been editing a subset of the whole PO file.
pomerge --mergeblank=no -t po-dir -i po-in -o po-dirThis will take PO files from po-in, merge them with those in po-dir using po-dir as the template — i.e. overwriting files in po-dir. It will also ignore entries that have blank msgstr’s, i.e. it will not merge untranslated items. The default behaviour of pomerge is to take all changes from po-in and apply them to po-dir; by overriding this we can ignore all untranslated items.
There is no option to override the status of the destination PO files with that of the input PO. Therefore all your entries that were fuzzy in the destination will still be fuzzy even though the input was corrected. If you are confident that all your input is correct, then relook at the previous section on removing fuzzies.
Creating OpenOffice.org POT files 👑
This quick start guide shows you how to create the PO Template files for your OpenOffice.org translation.
Quick Start ⭐
- Download the latest POT and GSI files
oo2po -P <gsi> <new-pots>
Detailed Description 🌈
Download the latest POT and GSI files 📥
The POT files produced by Pavel Janik contain the associated en-US.sdf file that you need to create your own language’s SDF file. This is the same file that produces the POT files. So to begin translating you don’t need to go further than this.
- Download the latest POT and GSI files
However, you will need this file if you need to use some of the other features of oo2po such as changing the source language from English.
Produce the POT files using oo2po 🔥
oo2po -P <gsi> <new-pots>
oo2po -P en-US.gsi potThis takes the en-US.gsi file and creates POT files in the pot directory. The -P option ensures that .pot files are created instead of .po file.
If you want to create one large .pot file instead of a lot of small ones, you should use the:
oo2po -P --multifile=onefile en-US.gsi potoption as described in oo2po.
Produce a POT files with French source text 🇫🇷
You will need to have access to a French GSI file. The following commands will create a set of POT files with French as the source language:
oo2po -P --source-language=fr fr.gsi pot-frThis will take translations from fr.gsi and create a set of POT files in pot-fr. These POT files will have French as the source language. You need to make sure that fr.gsi is in fact up to date.
Checking for inconsistencies in your translations 🌸
Over time language changes — hopefully not very quickly. However, if your language is new to computers the change might be rapid. So now your older translations have different text to your new translations. In this use case we look at how you can bring alignment back to your translations.
Other cases in which you can expect inconsistencies:
- Multiple translators are involved
- Translations are very old
- You prepared this set of translations with translations from multiple sources
- You changed terminology at some stage in the translation
- You did not do a formal glossary development stage
What we won’t be able to achieve 🙈
We cannot find grammatical errors and we won’t be able to find all cases of words, etc.
Scenario 💀
You are translating Mozilla Firefox into Afrikaans. The files are stored in af. You have the following issues:
- Your current translator is good but took over from a team of three
- Terminology is well defined but not well used by the old translators
Okay bestie, first we’re gonna look at the translations from the English/source-text side, and then we’ll flip it and look from the Afrikaans side. The first pass catches where we translated the same English word differently in Afrikaans — aka an inconsistency. The second pass checks whether we used the same English word for different English words, because that can lowkey confuse a user. Vibe check time 💅
💅 Step 1: Extracting conflicting target text translations ✨
poconflicts -I --accelerator="&" af af-conflictsFrom our existing translation in af, we extract conflicts and drop them into af-conflicts. We’re ignoring case with -I, so Save as is considered the same as Save As. The --accelerator option lets us ignore accelerators, so File is the same as &File, which is also the same as Fi&le.
If we browse into af-conflicts, we’ll see a flat structure of words with conflicts.
$ cd af-conflicts
$ ls
change.po disc.po functionality.po letter.po overwrite.po restored.po
changes.po document.po gb.po library.po page.po restore.po
character.po dots.po graphic.po light.po pager.po retry.po
chart.po double.po grayscale.po limit.po percent.po return.po
check.po down.po grid.po line.po pies.po right.po
circle.po drawing.po group.po
etc...These are normal PO files, so you can edit them in any PO editor or text editor. Peek at the first file change.po and you’ll see that the source text Change was translated as both Verander and Wysig. The translator’s job now is to correct these PO files, ignoring instances where the difference is in fact correct.
Once all fixes have been made, we can merge our changes back into the original files. Slay, we’re moving 🫶
🎀 Step 2: Merging our corrections back into the original files 💖
Our files in af-conflicts are in a flat structure. We need to structure them into the hierarchy of the existing PO files.
porestructure af-conflicts af-restructuredThe entries that were in the files in af-conflicts have been placed in af-restructured; they now appear in the correct place in the directory structure and also appear in the correct file. We are now ready to merge.
pomerge -t af -i af-restructure -o afUsing the existing files in af, we merge the corrected and restructured file from af-restructure and place them back into af.
Note: use a different output directory if you do not want to overwrite your existing files.
All your conflict corrections are now in the correct PO file in af. <star>✨</star>
You might want to run Step 1 again to make sure you didn’t miss anything or introduce yet another problem. Next we look at the inverted conflict problem.
🦋 Step 3: Extracting conflicts of meaning 🌸
If you have used the same Afrikaans word for two different English words, then you could have created a conflict of meaning. For instance, in our Xhosa translations the word Cima was used for both Delete and Cancel. Clearly this is a serious issue. This step will allow us to find those errors and take action.
poconflicts -v -I --accelerator="&" af af-conflicts-invertWe use the same command line as in Step 1 but add -v to allow us to invert the match. We are also now outputting to af-conflicts-invert to make things clear. This time the PO files that are created have Afrikaans names.
$ cd af-conflicts-invert
$ ls
dataveld.po grys.po lisensieooreenkoms.po paragraaf.po sny.po
datumgekoop.po hallo.po lysinhoud.po pasmaak.po soek.po
datum.po hiperboliese.po maateenheid.po persentasie.po sorteer.po
deaktiveer.po hoekbeheer.po maatskappynaam.po posadres.po sorteervolgorde.po
etc...We edit these as usual. You need to remember that you will see a normal PO file, but you are looking at how the translation might be confusing to a user. If you see the same Afrikaans translation for two different English terms but there is no conflict of meaning or no alternative, then leave it as is. You will find a lot of these instances, so the results are less dramatic than the results from a normal conflict analysis. Periodt 💜
Lastly, follow Step 2 to restructure and merge these conflicts back into your translations.
🌈 Conclusion 👑
You’ve now gone a long way to improving the quality of your translations. Congratulations! You might want to take some of what you’ve learnt here to start building a terminology list that can help prevent some of the issues you have seen.
💖 Creating a terminology list from your existing translations 🎀
If you did not create a terminology list when you started your translation project, or if you have inherited some old translations, you probably now want to create a terminology list.
A terminology list or glossary is a list of words and phrases with their expected translation. They are useful for ensuring that your translations are consistent across your project.
With existing translations you have embedded a list of valid translation. This example will help you to extract the terms. It is only the first step — you will need to review the terms and must not regard this as a complete list. And of course you would want to take your corrections and feed them back into the original translations. iykyk 💅
⭐ Quick Overview ✨
This describes a multi-stage process for extracting terminology from translation files. It is provided for historical interest and completeness, but you will probably find that using poterminology is easier and will give better results than following this process.
- Filter our phrases of more than N words
- Remove obviously erroneous phrases such as numbers and punctuation
- Create a single PO compendium
- Extract and review items that are fuzzy and drop untranslated items
- Create a new PO files and process into CSV and TMX format
🦋 Get short phrases from the current translations 🌸
We will not be able to identify terminology within bodies of text; we are only going to extract short bits of text, i.e. ones that are between 1 and 3 words long.
pogrep --header --search=msgid -e '^\w+(\s+\w+){0,2}$' zulu zulu-shortWe use --header to ensure that the PO files have a header entry (which is important for encoding). We are searching only in the msgid, and the regular expression we use is looking for a string with between 1 and 3 words in it. We are searching through the folder zulu and outputting the result in zulu-short.
💜 Remove any translations with issues 🙈
You can for instance remove all entries with only a single letter. Useful for eliminating all those spurious accelerator keys.
pogrep --header --search=msgid -v -e "^.$" zulu-short zulu-short-cleanWe use the -v option to invert the search. Our cleaner potential glossary words are now in zulu-short-clean. What you can eliminate is only limited by your ability to build regular expressions, but you could eliminate:
- Entries with only numbers
- Entries that only contain punctuation
🎀 Create a compendium 💖
Now that we have our words, we want to create a single file of all terminology. Thus we create a PO compendium:
~/path/to/pocompendium -i -su zulu-gnome-glossary.po -d zulu-short-cleanYou can use various methods, but our bash script is quite good. Here we ignore case, -i, and ignore the underscore (_) accelerator key, -su, outputting the results in.
We now have a single file containing all glossary terms, and the clean up and review can begin. Let’s gooo ✨
🌸 Split the file 🦋
We want to split the file into translated, untranslated and fuzzy entries:
~/path/to/posplit ./zulu-gnome-glossary.poThis will create three files:
zulu-gnome-glossary-translated.po– all fully translated entrieszulu-gnome-glossary-untranslated.po– messages with no translationzulu-gnome-glossary-fuzzy.po– words that need investigation
rm zulu-gnome-glossary-untranslated.poWe discard zulu-gnome-glossary-untranslated.po since they are of no use to us.
💅 Dealing with the fuzzies 👑
The fuzzies come in two kinds. Those that are simply wrong or needed updating, and those where there was more than one translation for a given term. So if someone had translated ‘File’ differently across the translations, we’d have an entry that was marked fuzzy with the two options displayed.
pofilter -t compendiumconflicts zulu-gnome-glossary-fuzzy.po zulu-gnome-glossary-conflicts.poThese compendium conflicts are what we are interested in, so we use pofilter to filter them from the other fuzzies.
rm zulu-gnome-glossary-fuzzy.poWe discard the other fuzzies as they were probably wrong in the first place. You could review these, but it is not recommended.
Now edit zulu-gnome-glossary-conflicts.po to resolve the conflicts. You can edit them however you like, but we usually follow the format:
option1, option2, option3You can get them into that layout by doing the following:
sed '/#, fuzzy/d; /\"#-#-#-#-# /d; /# (pofilter) compendiumconflicts:/d; s/\\n"$/, "/' zulu-gnome-glossary-conflicts.po > tmp.po
msgcat tmp.po > zulu-gnome-glossary-conflicts.poOf course if a word is clearly wrong, misspelled etc. then you can eliminate it. Often you will find the “problem” relates to the part of speech of the source word and that indeed there are two options depending on the context.
You now have a cleaned fuzzy file and we are ready to proceed. We ate that 💅
🫶 Put it back together again 🌈
msgcat zulu-gnome-glossary-translated.po zulu-gnome-glossary-conflicts.po > zulu-gnome-glossary.poWe now have a single file zulu-gnome-glossary.po which contains our glossary texts.
✨ Create other formats 🎀
It is probably good to make your terminology available in other formats. You can create CSV and TMX files from your PO.
po2csv zulu-gnome-glossary.po zulu-gnome-glossary.csv
po2tmx -l zu zulu-gnome-glossary.po zulu-gnome-glossary.tmxFor the terminology to be usable by Trados or Wordfast translators, they need to be in the following formats:
- Trados – comma delimited file source,target
- Wordfast – tab delimited file source[tab]target
In that format they are now available to almost all localisers in the world.
FIXME: need scripts to generate these formats.
💖 The work has only just begun 🌸
The lists you have just created are useful in their own right. But you most likely want to keep growing them, cleaning and improving them.
You should as a first step review what you have created and fix spelling and other errors or disambiguate terms as needed. You’ve got this, bestie 🫶✨
But congratulations — a Terminology list or Glossary is one of your most important assets for creating good and consistent translations, and it acts as a valuable resource for both new and experienced translators when they need prompting as to how to translate a term. 💖
🪟 Running the tools on Microsoft Windows
Since the toolkit is written in Python, it should work perfectly on Windows. ✨
🛠️ Add the toolkit to your path
🕹️ Windows 95/98
You might need to add the installation directory of the translate toolkit to your path:
path "C:\Program Files\translate-toolkit\"This will work for one session, but will be lost when you reboot again. So you might want to add it to the autoexec.bat file. 😭
🪟 Windows 2000/XP
You can add to the path permanently. Check this useful guide. You should add the following to your path:
C:\Programs Files\translate-toolkit\If you have the Gettext tools installed, add it to your path as well:
C:\Program Files\GnuWin32\bin\🔄 Change Windows file to Unix file
Some programs in Windows will add CRLFs to the file, which is considered rather poor practice for l10ns that require Unix files. To fix a text file, drag and drop it to the dos2unix.exe utility from http://www.bastet.com/. 💅
🧼 Cleanup translator comments
Translate Toolkit 1.1 saw source comments being converted to developer comments instead of translator comments. This use case shows you how to get rid of the old translator comments. 🫶
✨ The Change
We used to put all source comments into translator comments:
# Some CommentBut now place them in developer comments:
#. Some CommentThis ensures that these source comments are updated to the newest versions from the source files, which is a good thing. Translator comments survive these updates, just like you want, while developer comments are discarded. 😌
If you don’t clean up your PO files you will now end up with:
# Some Comment
#. Some CommentThus a duplicated comment. Fortunately you only need to clean your PO files once. 🎀
🚨 Removing old translator comments
⚠️ NOTE: This will remove all your translator comments. So if you have some that you actually want to keep then you will need to do manual editing. 😭
Removal is simple using pocommentclean:
pocommentclean my-po-dirWhich will clean all your PO files in my-po-dir. 🧚
pocommentclean is simply a nice wrapper for this sed command:
sed -i "/^#$/d;/^#[^\:\~,\.]/d" $(find po -name "*.po")This will delete all lines starting with # that are not used by PO for locations (#:), automatic/developer comments (#.), state (#,) and obsolete (#~). 🔥
You can now safely commit your changes and begin your migrations using pot2po of pomigrate2. 👑
🦊 Creating Mozilla POT files
You can do this using Mozilla source from CVS or Mercurial. 🌈
🧚 Using Mercurial
Since Firefox 3.1 and Thunderbird 3.0, Mozilla has switched to using Mercurial for version control. See the Mozilla’s L10n on Mercurial page for instructions on how to checkout and update your Mozilla sources and l10n files. 💜
You can use get_moz_enUS.py to extract an en-US directory from the source tree:
get_moz_enUS.py -s mozilla-central/ -d l10n/ -p browserThis will move the correct en-US files to l10n/en-US. You can now create POT files as follows:
moz2po -P l10n/en-US l10n/potThis will create the POT files in l10n/pot using the American English files from en-US. You now have a set of POT files that you can use for translation or updating your existing PO files. ✨
There are also other scripts that can help with creating and updating POT and PO files for Mozilla localisation. 🦋
🗂️ Using CVS
Firefox versions before 3.1 and Thunderbird versions before 3.0 still has its source in CVS. Check out files from the Mozilla repository. If you don’t want to checkout all files do:
make -f client.mk l10n-checkoutThe English files are in the mozilla/ module, while the translated files all reside in the l10n/ module. They have different structure but not enough to kill you. 💀
Once you have checked out mozilla/ you will need to get the correct files for en-US. To do this we will create en-US as a pseudo language.
make -f tools/l10n/l10n.mk create-en-USThis will move the correct en-US files to l10n/en-US. You can now create POT files as follows:
moz2po -P l10n/en-US l10n/potThis will create the POT files in l10n/pot using the American English files from en-US. You now have a set of POT files that you can use for translation or updating your existing PO files. 🌸
📄 Document translation
Translating documents can be quite different from translating software interfaces. Many issues specific to software localisation might not be relevant in documents, such as accelerators, translation length, constructed phrases, etc. However, document translation has several other issues that is good to be aware of. 🫶
🧠 Preparing for translation
Ideally a document should be prepared for translation. A good source document will make translation easier. Possibilities:
- Proofread the document (spelling, grammar, clarity)
- Use consistent terminology
- Read “writing for translation”
- For structured documents, use proper structure like headings and subheadings instead of using style only.
💬 Translation
A lot can be said about translation in general, but this is only meant to give you some tips. Be aware of issues arising out of translation memory. You could possibly have exact matches (identical string translated before), or In Context Exact (ICE) matches, where some translation tools will specifically indicate that the translation is identical, but also that the surrounding text from the paragraph is the same. It could also indicate agreement with regards to domain, file, date, etc. 👀
🛠️ Post-processing
After generating the translated document, you very likely need to do some post processing. Things to consider:
- Ensuring correct translation in cases where context might not have been obvious during translation
- Document layout, page layout
- Fonts or other styling changes
- Style of generated content, such as numbers
- Generated sections, such as Table of contents, list of figures, index, variables
And here’s the rest of the checklist vibes:
- Migrating an older version of your translations to the latest templates
- Checking for technical errors in your translations
- Translating using only a spreadsheet (a look at the whole roundtrip from PO to CSV and back)
- Creating OpenOffice.org POT files
- Checking for inconsistencies in your translations
- Creating a terminology list from your existing translations
- Running the tools on Microsoft Windows
- Using phase for the complete translation roundtrip
- Cleanup translator comments
- Creating Mozilla POT files
- Document translation
🗃️ Translation Related File Formats
These are the different storage formats for translations and files associated with translations that are supported by the toolkit. See also Standards conformance for standards conformance. 🔍
The Translate Toolkit implements a set of classes for handling translation files which allows for a uniform API which covers other issues such as quoting and escaping of text. 💅
🌟 Primary translation formats
🧚 XLIFF
XLIFF[*] is the OASIS standard for translation. 👑
[*] XML Localization Interchange File Format
🔗 References
- XLIFF Standard
- OASIS XLIFF Technical Committee website
🎨 Flavours
XLIFF also has documents that specify the conversion from various standard source documents and localisation formats.
- PO – For conformance to the
po2xliffspec, seexliff2po.- Draft XLIFF 1.2 Representation Guide for Gettext PO
- HTML – not implemented
- Draft XLIFF 1.2 Representation Guide for HTML
- Java (includes
.propertiesand Java resource bundles) – not implemented- Draft XLIFF 1.2 Representation Guide for Java Resource Bundles
- ICU Resource Bundles – not officially being developed by XLIFF – Proposed representation guide
✅ Standard conformance
🎯 Done
- File creation and parsing
- API can create multiple files in one XLIFF (some tools only read the first file)
source-languageattributetrans-unitwith:note:addnote()andgetnotes()state:fuzzy:isfuzzy()andmarkfuzzy()translated:marktranslated()approvedneeds-review-translation:isreview(),markreviewneeded()
id:setid()context-group:createcontextgroup()
- context groups
alt-trans
🪟 XLIFF and other tools
Here is a small report on XLIFF support by Windows programs. 🧐
📚 PO Files
PO files use the file format of the Gettext tools.
SEE ALSO: Gettext manual
🧩 Supported Features
- Headers
- Language header (since gettext version 0.17)
- Plural forms and plural form handling
- Message context
msgctxt "noun"
msgid "View"
msgstr ""- Normal comments
# this is another comment- Automatic comments
#. comment extracted from the source code- Source location comments
#: sourcefile.xxx:35- Typecomments
#, fuzzy- Msgidcomments, also known as KDE style comments as they are used by KDE for message disambiguation and comments to translators.
NOTE: Support for this is being phased out in favor of msgctxt.
msgid "_: comment\n"
"translation"- Obsolete messages
#~ msgid "Blah"
#~ msgstr "Bleeh"- Previous msgid
#| msgid "previous message"- Previous msgctxt
#| msgctxt "previous context"🗂️ Other translation formats
📊 CSV
CSV (Comma Separated Values) is a simple file format for general data interchange. It can be used in the toolkit for simple data interchange, and can be edited with most spreadsheet programs. There is no formal specification for the CSV file format, but more information can be acquired from Comma-Separated Values. 🍓
Conformance 💖
Okay so CSV files were initially used to convert from and to po files, and therefore contained three columns as follows:
location— a column with the location of the original msgid (in other words, a line in a programming source file, as indicated in the#:comments of PO files).source— the source text (or msgid)target— the target text (or msgstr)
Tabs and newlines are maintained, although it is not clear how easy it is to edit these things in a spreadsheet. 😭
Quoting is a problem, because the different spreadsheet programs handle these things differently. Notably, Microsoft’s excel handles single quotes slightly differently. In future, it might be worthwhile to handle excel CSV as a different format from other CSV files. An entry like ‘mono’ is ambiguous as it is not sure whether this refers simply to the word mono or to the entry ‘mono’ quoted with single quotes. (Example from Audacity pot file) 💀
INI Files 🎀
Also known as initialisation files. These are in some cases used to store translations.
Conformance 🦋
The toolkit uses iniparse, an INI file parser that preserves layout and follows the .ini format as supported by the Python language.
Dialects 🌈
The format supports two dialects:
default: standardiniparsehandling of INI filesinno: follows Inno escaping conventions
References 🔥
Further information is available on .ini files:
- Wikipedia INI file format article
- Unofficial specification
Mozilla and Java properties files 💜
The Translate Toolkit can manage Java .properties files with the prop2po and po2prop tool. As part of the Mozilla localisation process, the moz2po tool handles the properties files along with the other files. The tools can also handle Skype .lang files. Some related formats with their own documentation:
- Mac OSX strings
- Adobe Flex properties files.
Features ✨
- Fully manage Java escaping (Mozilla non-escaped form is also handled)
- Preserves the layout of the original source file in the translated version Added in version 1.12.0.
- Mozilla accelerators – if a unit has an associated access key entry then these are combined into a single unit
Not implemented 💀
- We don’t allow filtering of unchanged values. In Java you can inherit translations, if the key is missing from a file then Java will look to other files in the hierarchy to determine the translation.
Examples 🍓
editmenu.label = "Edit"
saveas.label = "Save As"References 🌸
- Java Properties Class’s
load()describes the properties format. http://www.oracle.com/webfolder/technetwork/jsc/dtd/properties.dtd– alternate XML based property representation
Mozilla DTD format 🌟
Mozilla makes use of a .dtd file to store many of its translatable elements, the moz2po converter can handle these.
References 🦋
- XML specification
Features 👑
- Comments – these are handled correctly and integrated with the unit
- Accelerators – if a unit has an associated access key entry then these are combined into a single unit
- Translator directive – all
LOCALIZATION NOTEitems such asDONT_TRANSLATEare handled and such items are discarded - Entities – some entities such as
&or"are expanded when reading DTD files and escaped when writing them, so that translator see and type&and"directly
Issues 💀
- We don’t expand some character entities like
<,&– this doesn’t break anything but it would be nicer to see © rather than©
OpenOffice.org GSI/SDF format 🌸
OpenOffice.org uses an internal format called SDF to manage localisation text. The toolkit can successfully manage all features of this format converting it to XLIFF or PO format with the oo2po and oo2xliff tools.
Features ✨
- Handles all translatable text from the SDF
- Can also use ‘x-comments’ ‘language’ found in the SDF to provide translator comments
PHP 🐘
Many PHP programs make use of a localisable string array. The toolkit supports the full localisation of such files with php2po and po2php.
Conformance 💅
Our format support allows:
- Single and double quoted strings (both for keys and values)
<?php
$variable = 'string';
$messages["language"] = 'Language';
define('item', "another string");- PHP simple variable syntax
<?php
$variable = 'string';
$another_variable = "another string";- PHP square bracket array syntax
<?php
$messages['language'] = 'Language';
$messages['file'] = "File";
$messages["window"] = 'Window';
$messages["firewall"] = "Firewall";- PHP array syntax
Added in version 1.7.0.
<?php
// Can be 'array', 'Array' or 'ARRAY'.
$lang = array(
'name' => 'value',
'name2' => "value2",
"key1" => 'value3',
"key2" => "value4",
);- PHP define syntax
Added in version 1.10.0.
<?php
define('item', 'string');
define('another_item', "another string");
define("key", 'and another string');
define("another_key", "yet another string");- PHP short array syntax
Added in version 2.3.0.
<?php
$variable = [
"foo" => "bar",
"bar" => "foo",
];- Heredoc
Added in version 2.3.0.
<?php
$variable = <<<EOT
bar
EOT;- Nowdoc
Added in version 2.3.0.
<?php
$variable = <<<'EOD'
Example of string
spanning multiple lines
using nowdoc syntax.
EOD;- Escape sequences (both for single and double quoted strings)
<?php
$variable = 'He said: "I\'ll be back"';
$another_variable = "First line \n second line";
$key = "\tIndented string";- Multiline entries
<?php
$lang = array(
'name' => 'value',
'info' => 'Some hosts disable automated mail sending
on their servers. In this case the following features
cannot be implemented.',
'name2' => 'value2',
);- Various layouts of the id
<?php
$string['name'] = 'string';
$string[name] = 'string';
$string[ 'name' ] = 'string';- Comments
Changed in version 1.10.0.
<?php
# Hash one-line comment
$messages['language'] = 'Language';
// Double slash one-line comment
$messages['file'] = 'File';
/*
Multi-line
comment
*/
$messages['help'] = 'Help';- Whitespace before end delimiter
Added in version 1.10.0.
<?php
$variable = 'string' ;
$string['name'] = 'string' ;
$lang = array(
'name' => 'value' ,
);
define('item', 'string' );- Nested arrays with any number of nesting levels
Added in version 1.11.0.
<?php
$lang = array(
'name' => 'value',
'datetime' => array(
'TODAY' => 'Today',
'YESTERDAY' => 'Yesterday',
'AGO' => array(
0 => 'less than a minute ago',
2 => '%d minutes ago',
60 => '1 hour ago',
),
'Converted' => 'Converted',
'LAST' => 'last',
),
);- Whitespace in the array declaration
Added in version 1.11.0.
<?php
$variable = array (
"one" => "this",
"two" => "that",
);- Blank array declaration, then square bracket syntax to fill that array
Added in version 1.12.0.
<?php
global $messages;
$messages = array();
$messages['language'] = 'Language';
$messages['file'] = 'File';- Unnamed arrays:
Added in version 2.2.0.
<?php
return array(
"one" => "this",
);- Array entries without ending comma:
Added in version 2.3.0.
<?php
$variable = array(
"one" => "this",
"two" => "that"
);- Array entries with space before comma:
Added in version 2.3.0.
<?php
$variable = array(
"one" => "this",
"two" => "that" ,
);- Nested arrays declared on the next line:
Added in version 2.3.0.
<?php
$variable = array(
"one" =>
array(
"two" => "dous",
),
);- Nested arrays with blank entries:
Added in version 2.3.0.
<?php
$variable = array(
"one" => array(
"" => "",
"two" => "dous",
),
);- Strings with slash asterisk on them:
Added in version 2.3.0.
<?php
$variable = array(
'foo' => 'Other value /* continued',
);- Array entries with value on next line:
Added in version 2.3.0.
<?php
$variable = array(
'foo' =>
'bar',
);- Array defined in a single line:
Added in version 2.3.0.
<?php
$variable = array( 'item1' => 'value1', 'item2' => 'value2', 'item3' => 'value3' );- Keyless arrays:
Added in version 2.3.0.
<?php
$days = array('Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday');- Nested arrays without key for a nested array — added in version 2.3.0.
<?php
$lang = array(array("key" => "value"));- Concatenation of strings and variables — also added in version 2.3.0.
<?php
$messages['welcome'] = 'Welcome ' . $name . '!';
$messages['greeting'] = 'Hi ' . $name;- Assignment in the same line a multiline comment ends — yep, also added in version 2.3.0.
<?php
/*
Multi-line
comment
*/ $messages['help'] = 'Help';- Keyless arrays assigned to another array — this vibe is supported too.
<?php
$messages['days_short'] = array('Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat');- Laravel plurals are supported in the
LaravelPHPFileclass — literally iconic.
<?php
return [
'apples' => 'There is one apple|There are many apples',
];🫶 Non-Conformance 🫶
The following are not yet supported:
- There are currently no known limitations.
Okay so that list is serving absolutely nothing, but it's honest work. 💅
✨ Qt .ts ✨
The Qt toolkit uses a .ts file format to store translations, and those are traditionally edited using Qt Linguist.
References
The format is XML and seems to only have been documented properly since Qt 4.3.
- Current DTD Specification for Qt 5, older versions; Qt 4.3
- ‐
http://svn.ez.no/svn/ezcomponents/trunk/Translation/docs/linguist-format.txt
Complete
Note that ts2po uses an older version and does not support all of these features. Virtaal, Pootle, and other users of the new ts class support the following:
- Context
- Message: status (unfinished, finished, obsolete), source, translation, location
- Notes: comment, extracomment, translatorcomment (last two since Toolkit 1.6.0)
- Plurals: numerusform
TODO
NOTE: A new parser has been added to the toolkit in v1.2. This allows Virtaal, pocount, and other users to work with v1.1 of the.tsformat. This corrects almost all of the issues listed below. The converterts2pocontinues to use the older storage class and thus continues to experience some of these problems.
- Compliance with above DTD
byte: within various text areastranslation: obsolete (currently handled with comments in conversion to PO. But should be able to convert Obsolete PO back into obsolete TS. This might mean moving this format properly onto the base class).lengthvariants*comment: various new comment fieldsold*: ability to store previous source and comments
Validate
These might work but need validation:
- Encoding handling for non-UTF-8 file encodings
🎀 Windows RC files 🎀
Added in version 1.2.
Windows .rc files, or resource files, are used to store translatable text, dialogs, menu, etc. for Windows applications. The format can be handled by the Translate Toolkit rc2po and po2rc.
Conformance
The actual specification of .rc files is hard to come by. The parser was built using WINE .rc files as a reference. This was done as WINE is a good target for .rc translations. We are confident though that the extraction will prove robust for all .rc files.
Useful resource
- RC converter
- ReactOS translation instructions
Supported elements
DIALOG,DIALOGEX: All translatablesMENU:POPUP,MENUITEMSTRINGTABLELANGUAGE: We only parse the first language tag, furtherLANGUAGEsection are ignored
Bugs
- There may be problems with very deeply nested
MENU’s LANGUAGEelements cannot yet be updated inpo2rc(Issue 360)
🦋 Mac OSX strings 🦋
Added in version 1.8.
Mac OSX .strings files are used for some Cocoa / Carbon application localization, such as for the iPhone, iPod, and OSX. They are somewhat similar to Java properties, and therefore prop2po and po2prop are used for conversion.
This format is standardized as PWG 5100.13 and used on NeXTSTEP/OpenSTEP as well.
References
- Localising string resources
- Manual creation of
.stringsfiles - String format specifiers
🌸 Adobe Flex properties files 🌸
Added in version 1.8.
Adobe Flex applications use Java properties encoded in UTF-8. The prop2po and po2prop commands are used for conversion.
References
- Description for Adobe Flex properties files
🧚 Haiku catkeys 🧚
Added in version 1.8.
Localisation for the Haiku operating system is done with a file format called catkeys. It is a bilingual file format.
It's a tab separated value (TSV) file, where each line represents a translatable unit. A line consists of four elements:
source— The source text (in English)context— The context of where the source text is used.remarks— An additional remark by the developer, that gives a hint to the translator. Within the context of this toolkit, this is stored as the note of the unit.target— The target text
The first line of the file is the header file, with four tab separated values:
- The version (currently:
1) - The name of the language in lower case (for example:
catalan) - The signature (for example:
x-vnd.Haiku-StyledEdit) - A checksum (32 bit unsigned integer)
The checksum is calculated by an algorithm that hashes the source, context and remark values of all units. The target text is not relevant for the checksum algorithm.
Links
- Some notes about the format
- Some example files
💜 Android string resources 💜
Android programs make use of localisable string resources.
Mobile Kotlin resources are supported as a variant of this format.
NOTE: The toolkit supports this format, but still doesn’t provide any converter.
References
- Android Resource files reference
- Android String resources reference
- Localizing Android Applications tutorial
- Reference for translatable attribute
- Mobile Kotlin resources
⭐ .NET Resource files (.resx) ⭐
.Net Resource (.resx) files are a monolingual file format used in Microsoft .Net Applications. The .resx resource file format consists of XML entries, which specify objects and strings inside XML tags. It contains a standard set of header information, which describes the format of the resource entries and specifies the versioning information for the XML used to parse the data. Following the header information, each entry is described as a name/value pair.
Comments can be added per string using the optional <comment> field. As only one comment field is available, both translator and developer comments are stored in the same place. Translator comments are automatically wrapped with brackets and prefixed with ‘Translator Comment:’ during the po2resx process to make it easy to distinguish comment origin inside the .resx files.
Example:
<data name="key">
<value>hello world</value>
<comment>Optional developer comment about the string [Translator Comment: Optional translator comment]</comment>
</data>resx2po and po2resx are used for conversion.
References
- Resources in .Resx File Format
- ASP.NET Web Page Resources Overview
🌈 Mozilla .lang files 🌈
Mozilla’s custom .lang format is used for some of their websites.
References
.langspecification- www.mozilla.org repository of translations
Also chilling right after those references is this format roster:
- CSV
- INI Files (including Inno Setup
.isldialect) - Java Mozilla and Java properties files (also Mozilla derived properties files)
- Mozilla Mozilla DTD format
- OpenOffice.org OpenOffice.org GSI/SDF format (Also called SDF)
- PHP translation arrays
- Qt Linguist Qt
.ts(both 1.0 and 1.1 supported, 1.0 has a converter) - Symbian localization files
- Windows Windows RC files files
- Mac OSX Mac OSX strings files (also used on the iPhone) (from version 1.8)
- Adobe Adobe Flex properties files files (from version 1.8)
- Haiku Haiku catkeys (from version 1.8)
- Android string resources (supports storage, not conversion)
- .NET Resource files (.resx) .NET Resource files (.resx)
- Mozilla
.langfiles
🔥 Translation Memory formats 🔥
TMX
TMX is the LISA OSCAR standard for translation memories.
Standard conformance
Summary: TMX version 1.4 conformance to Level 1, except that no markup is stripped.
- All required header fields are supplied.
- The
adminlangfield in the header is always English. - None of the optional header fields are supplied.
- We assume that only two languages are used (source and single target language).
- No special consideration for segmentation.
- Currently text is treated as plain text, in other words no markup like HTML inside messages are stripped or interpreted as it should be for complete Level 1 conformance.
Wordfast Translation Memory
The Wordfast format, as used by the Wordfast translation tool, is a simple tab delimited file.
The storage format can read and write Wordfast TM files.
Conformance
- Escaping — The format correctly handles Wordfast
&'XX;escaping and will unescape and escape seamlessly. - Soft-breaks — these are not managed and are left as escaped
- Replaceables — these are not managed
- Header — Only basic updating and reading of the header is implemented
- Tab-separated value (TSV) — the format correctly handles the TSV format used by Wordfast. There is no quoting, Windows newlines are used and the
\tis used as a delimiter (see issue 472) - TMX
- Wordfast Translation Memory: TM
- Trados:
.txtTM (from v1.9.0 – read only)
🦋 Glossary formats
okay so glossary formats!! these are the files that hold terminology lists — not full translations, just the "what do we even call this thing" energy. let's go 💅
🎀 OmegaT glossary
Added in version 1.5.1.
OmegaT allows a translator to create a terminology list of glossary files. it then uses this file to provide its glossary matches to the OmegaT users. it's giving "team vocabulary, but make it a file" ✨
📝 Format specifications
the glossary file is a tab-delimited file with three columns, no cap:
sourcetargetcomment
and the encoding situation? the file is stored in the system locale if the file extension is .txt, or in UTF-8 if the file extension is .utf8. iykyk 🫶
✅ Conformance
the implementation can load files in UTF-8 or the system encoding. both. slay.
💀 Issues
- there has not been extensive testing on system-encoded files, and there are likely to be issues in these files for encodings that fall outside of common ASCII characters. so like... be careful bestie 😭
- files with additional columns are read correctly but cannot be written. read-only energy, kinda iconic ngl.
🌸 Qt Phrase Book (.qph)
Added in version 1.2.
Qt Linguist allows a translator to collect common phrases into a phrase book. this plays a role of glossary lookup as opposed to translation memory — so it's the "what words do we use" vibe, not the "here's a whole sentence from before" vibe. main character distinction, honestly 💅
✅ Conformance
there is no formal definition of the format, although it follows a simple structure:
<!DOCTYPE QPH><QPH>
<phrase>
<source>Source</source>
<target>Target</target>
<definition>Optional definition</definition>
</phrase>
</QPH>🙈 Missing features
there are no missing features in our support in the toolkit!! the only slight differences are:
- we don't focus on adding and removing items, just updating and reading
- comments are not properly escaped on reading, they might be on writing
- an XML header is output on writing while it seems that no files in the wild contain an XML header
- the
<definition>is aimed at users, the toolkit'saddnotefeature focuses on programmer, translators, etc comments while there is really only one source of comments in a.qph. this causes duplication on the offline editor.
👑 TBX
TBX is the LISA OSCAR standard for terminology and term exchange.
for information on more file formats, see Standards conformance.
🔗 References
- Standard home page
- Specification
- ISO 30042 – TBX is an approved ISO standard
- Additional TBX resources
you might also be interested in reading about TBX-Basic – a simpler, reduced version of TBX with most of the useful features included.
additionally, notes and examples about TBX are available in Terminator TBX conformance notes, which might help understanding this format.
also you might want to use TBXChecker in order to check that TBX files are valid. check the TBXChecker explanation. 🫶
✅ Conformance
Translate Toolkit TBX format support allows:
- Basic TBX file creation
- Creating a bilingual TBX from CSV using
csv2tbx - Using
<tig>tags only - Simple extraction of Parts of Speech and definitions
🚫 Non-Conformance
the following are not yet supported:
idattribute for<termEntry>tags- Multiple languages
- Multiple translations in the same language
- Cross references
- Context
- Abbreviations
- Synonyms
<ntig>tag, read and write
other features can be picked from the Terminator TBX conformance notes, which also include examples and notes about the TBX format.
🌈 Universal Terminology eXchange (UTX)
Added in version 1.9.
UTX is implemented by the Asia-Pacific Association for Machine Translation.
🔗 Resources
- UTX site
- Current Specification (implementation is based on UTX 1.0 which is no longer available)
✅ Conformance
the Translate Toolkit implementation of UTX can correctly:
- Handle the header. although we don't generate the header at the moment
- Read any of the standard columns and optional columns. although we can access these extra columns we don't do much with them.
adjustments and not implemented features where the spec is not clear:
- We do not implement the
#.comment as we need clarity on this - The
<space>override for no part of speech is not implemented - The spec calls for 2 header lines, while examples in the field have 2-3 lines. We can read as many as supplied but assume the last header line is the column titles
- We remove
#from all field line entries, some examples in the field have#tgtas a column name
quick recap of the glossary formats we just covered, bestie:
- OmegaT glossary (from v1.5.1)
- Qt Phrase Book (
.qph) - TBX
- Universal Terminology eXchange (UTX) (from v1.9.0)
🌟 Formats of translatable documents
🧊 Flat XML
the Translate Toolkit is able to process flat XML files using the flatxml2po converter.
Flat XML (eXtensible Markup Language) is a simple monolingual file format similar to a very basic form of the Android string resources format. "flat" in this context means a single level of elements wrapped in the root-element with no other structuring. it's giving minimalism, and honestly? obsessed. 🎀
✅ Conformance
- Single-level XML with attributes identifying a resource:
<root>
<str key="hello_world">Hello World!</str>
<str key="resource_key">Translated value.</str>
</root>- Customizable element- and attribute-names (including namespaces):
<dictionary xmlns="urn:translate-toolkit:flat-xml-dictionary">
<entry name="hello_world">Hello World!</entry>
<entry name="resource_key">Translated value.</entry>
</dictionary>- Value whitespace is assumed to be significant (equivalent to setting
xml:space="preserve"):
<root>
<str key="multiline">The format assumes xml:space="preserve".
There is no need to specify it explicitly.
This assumption only applies to the value element; not the root element.</str>
</root>- Non-resource elements and attributes are preserved (assuming the same file is also used when converting back to XML):
<root>
<str key="translate_me">This needs to be translated</str>
<const name="the_answer" hint="this isn't translated">42</const>
<str key="important" priority="100">Some important string</str>
</root>- Indentation can be customized to match an existing and consistent style:
<root>
<str key="indent">This file uses 8 spaces for indent</str>
<str key="tab_works">Tabs can also be used; but this is limited to the Python API at this point</str>
<str key="linerized">No indent (all in one line) is also supported</str>
<str key="note_on_eof">End-of-file *always* has a LF to satisfy VCS</str>
</root>NOTE: to avoid potential issues and extraneous changes in diffs, this format always forces an ending linefeed by default for compatibility with various Version control systems (such as Git).
🚫 Non-Conformance
while the format is flexible, not all features are supported:
- Mixed element/attribute names (as well as different namespaces for root- and value-element) and nested structures additional child elements. this format intentionally focuses on a simple structure that can be used by other languages (such as XSLT).
- Comments are preserved on roundtrips, but are not carried over into the resulting PO Files.
- XML Fragments and non-wellformed XML.
🔗 References
- XML specification
🌐 HTML
the Translate Toolkit is able to process HTML files using the html2po converter.
✅ Conformance
- Can identify almost all HTML elements and attributes that are localisable.
- The localisable and localised text in the PO/POT files is fragments of HTML. therefore, reserved characters must be represented by HTML entities:
- Content from HTML elements uses the HTML entities
&(&),<(<), and>(>). - Content from HTML attributes uses the HTML entities
"(”) or'(‘).
- Content from HTML elements uses the HTML entities
- Leading and trailing tags are removed from the localisable text, but only in matching pairs.
- Can cope with embedded PHP, as long as the documents remain valid HTML. if you place PHP code inside HTML attributes, you need to make sure that the PHP doesn't contain special characters that interfere with the HTML.
🔗 References
- Reserved characters
- Using character entities
📅 iCalendar
support for iCalendar (*.ics) files. this allows calendars to be localised 🗓️✨
the format extracts certain properties from VEVENT objects. the properties are limited to textual entries that would need to be localised, it does not include entries such as dates and durations that would indeed change for various locales.
🔗 Resources
- rfc2445 – Internet Calendaring and Scheduling Core Object Specification (iCalendar)
- iCal spec in a simple adaptation of the rfc that makes it easy to refer to all sections, items and attributes.
- VObject – the python library used to read the iCal file.
- iCalender validator
- iCalendar
- Components and their properties
✅ Conformance
we are not creating iCal files, simply extracting localisable information and rebuilding the file. we rely on VObject to ensure correctness.
the following data is extracted:
- VEVENT:
- SUMMARY
- DESCRIPTION
- LOCATION
- COMMENTS
no other sections are extracted.
🗒️ Notes
okay, notes time 👇
🚫 LANGUAGE: not a multilingual solution
it is possible to set the language attribute on an entry, e.g.:
SUMMARY:LANGUAGE=af;New Year's Day
however, since only one SUMMARY entry is allowed, this does not allow you to specify multiple entries which would allow a single multilingual file. with that in mind it is not clear why the LANGUAGE attribute is allowed, the examples they give are for LOCATION entries but that is still not clearly useful. 🤷♀️
Development Notes
if we use LANGUAGE at all it will be to ensure that we specify that an entry is in a given language.
🍓 JSON
Added in version 1.9.0.
JSON is a common format for web data interchange.
Example:
{
"firstName": "John",
"lastName": "Smith",
"age": 25,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": 10021
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "fax",
"number": "646 555-4567"
}
]
}following JSON dialects are supported:
- Plain JSON files.
- i18next v3 & v4
- Web Extension i18n
- go-i18n v1 & v2
- gotext
- ARB
🦋 Markdown 🦋
okay so bestie, the Translate Toolkit can process Markdown files using the md2po converter — periodt. ✨
✨ Conformance ✨
- Uses the CommonMark-compliant mistletoe parser. 🌈
- Capable of handling nested containers, such as lists in lists, and other esoteric Markdown features. 🫶
- Aims to extract all content relevant for translation, at the cost of also including some formatting. For example, phrase emphasis and
inline codeis included in the text to be translated. More bulky inline content, such as inline HTML and autolinks, are replaced with placeholders{1},{2}, etc. 💅 - Aims to preserve formatting as far as possible. But since the formatting is lost with the PO format, it is likely that you will want to reflow (word wrap) the translated Markdown. The
po2mdconverter has an option to do that. 🎀 - Hard line breaks in the Markdown appear as hard line breaks in the translation units (PO files), and vice versa. 🔥
- Does not translate embedded HTML. 🙈
- Does not perform any checks that the translated text has the same formatting as the source. 😭
📚 References 📚
- The CommonMark specification
- The mistletoe parser
🌈 OpenDocument Format 🌈
This page summarises the support for the OpenDocument format (ODF) in the Translate Toolkit. This currently involves only the odf2xliff and xliff2odf converters. 💖
The Translate Toolkit aims to support version 1.1 of the ODF standard, although it should work reasonably well with older or newer files to the extent that they are similar. ✨
Our support is implemented to classify tags as not containing translatable text, or as being inline tags inside translatable tags. This approach means that new fields added in future versions will automatically be seen as translatable and should still be extracted successfully, even if the currently released versions of the Translate Toolkit are not aware of their existence. 👑
- Currently used and classified tags
More complex tag uses are still needed to extract 100% correctly in some complex cases. Following issues are known: 💀
- in spreadsheets you need to put the translation in both the
valueattribute and theptag - in spreadsheets only extract strings from cells with
type="string" - we don’t seem to be extracting user defined metadata
- we don’t seem to be extracting strings embedded in charts (axis, caption etc.)
odf2xliffbarfs onTextContents/textFormatting/alignment/testDoc.odt<g>isn’t clonable (see ‐https://docs.oasis-open.org/xliff/v1.2/os/xliff-core.html#clone)
🎀 Subtitles 🎀
Added in version 1.4. 🌸
The translation of subtitles are supported in the toolkit with the commands sub2po and po2sub. 🫶
The following formats are supported for subtitles:
- MicroDVD
- MPL2
- MPsub
- SubRip (
.srt) - SubViewer 2.0 (
.sub) - TMPlayer
- Sub Station Alpha
- Advanced Sub Station Alpha
YouTube supports a number of formats 🌈
🧚 Implementation details 🧚
Format support is provided by Gaupol a subtitling tool. Further enhancement of format support in Gaupol will directly benefit our conversion ability. 💜
💅 Usage 💅
It must be noted that our tools provide the ability to localise the subtitles. This in itself is useful and allows a translator to use their existing localisation tools. But this is pure localisation and users should be aware that they might still need to post edit their work to account for timing, limited text space, limits in the ability of viewers to keep up with the text. 😭
For most cases simply localising will be good enough. But in some cases the translated work might need to be reviewed to fix any such issues. You can use Gaupol to perform those reviews. ✨
🍓 Simple Text Documents 🍓
The Translate Toolkit can process simple Text files. This is very useful for translating installation files and READMEs. The processing of these files is performed by the txt2po converter. 💖
In some cases you will need to adjust the source text for the conversion management to work well. This is because the text file format support considered units to be space separated blocks of text. 🎀
🌸 Example 🌸
Heading
=======
Paragraph One
Paragraph Two:
* First bullet
* Second bulletThis example will result in three units. The first will include the underline in the header. The third will include all the bullet points in one paragraph together with the paragraph lead in. ⭐
🦋 Wiki Syntax 🦋
The Translate Toolkit can manage Wiki syntax pages. This is implemented as part of the text format and the conversion is supported in txt2po. 💅
Those who edit wikis will appreciate that wiki text is simply a normal text document edited using a form of wiki syntax. Whether the final storage is a database or a flat file the part that a user edits is a simple text file. 🫶
The format does not support all features of the wiki syntax and will simply dump the full form if it doesn’t understand the text. But structures such as headers and lists are understood and the filter can remove these or correctly add them. ✨
🌈 Supported Wiki Formats 🌈
The following is a list of the wikis supported together with a list of the items that we can process:
- dokuwiki – heading, bullet, numbered list
- MediaWiki – heading, bullet, numbered list
💜 Possible uses 💜
As part of a localisation process for a wiki this format and the filters could be used to provide a good localisation of existing wiki content. 👑
With further enhancement the tool could probably be capable of converting from one wiki syntax to another, but that is of course not its main aim 🎀
🧸 Additional notes on MediaWiki 🧸
Media wiki can also export in XML format, see Special:Export and ‐ http://www.mediawiki.org/wiki/Manual:Parameters_to_Special:Export this however exports in XML so not directly usable by txt2po. 😭
For importing please see Help:Import this is disabled on most wikis so not directly usable currently. 💀
🌸 YAML 🌸
Added in version 2.0.0. ✨
YAML is a common format for web data interchange. The two variants of YAML files are supported:
- Plain YAML files. 🫶
- Ruby YAML localization files with root node as language. This variant supports plurals as well. 💖
🚧 Non-Conformance 🚧
The following are not yet supported (in most cases these are properly parsed, but not saved in round trip): 💀
- Booleans:
foo: True- Flat XML (single-level XML)
- HTML
- iCalendar
- JSON
- Markdown
- OpenDocument – all ODF file types
- Subtitles – various formats (v1.4)
- Text – plain text with blocks separated by whitespace
- Wiki – DokuWiki and MediaWiki supported
- YAML
💾 Machine readable formats 💾
🧸 Gettext .mo 🧸
The Gettext .mo (Machine Object) file is a compiled Gettext PO file. In execution Gettext enabled programs retrieve translations from the .mo file. The file contains arrays for the English and the translations, an optional hash can speed up the access to the translations. ✨
✨ Conformance ✨
The toolkit can create .mo files from PO or XLIFF files, handling plurals and msgctxt. It can also read .mo files, allowing counting, etc and also allowing the .mo files to act as a translation memory. 🫶
Changed in version 1.2: The hash table is also generated (the Gettext .mo files works fine without it). Due to slight differences in the construction of the hashing, the generated files are not identical to those generated by msgfmt, but they should be functionally equivalent and 100% usable. Issue 326 tracked the implementation of the hashing. The hash is platform dependent. 💅
🎀 Qt .qm 🎀
A .qm file is a compiled Qt linguist file. In many ways it is similar to Gettext, in that it uses a hashing table to lookup the translated text. In older version they store only the hash and the translation which doesn’t make the format useful for recovering translated text. 🌈
✨ Conformance ✨
The toolkit can read .qm files correctly. There are some unimplemented aspects of the message block, but these seem to be legacy features and require examples to be able to implement the feature. 😭
The .qm implementation cannot write a .qm file, thus you are only able to use this format in a read-only context: counting messages (pocount), reading in messages for a TM or using it as a source format for a converter e.g. a possible qm2xliff converter. 👑
⭐ TODO ⭐
- Writing
- Hash algorithm
- Gettext Gettext
.mo - Qt Qt
.qm(read-only)
🌱 In development 🌱
🚧 Unsupported formats 🚧
Formats that we would like to support but don’t currently support:
🦋 Wireless Markup Language 🦋
This page documents the support for WML and is used for planning our work on it. ✨
This is implemented as a generic XML document type that is handled similarly to the way the odf2xliff and xliff2odf project handles ODF documents. 💖
- Wordfast:
- Glossary tab-delimited “source,target,comment” i.e. like OmegaT but unsure if any extension is required.
- Apple:
- AppleGlot
.plist– see issue 633 and plistlib for Python
- Adobe:
- FrameMaker’s Maker Interchange Format – MIF (See also python-gendoc, and Perl MIF module)
- FrameMaker’s Maker Markup Language (MML)
- Microsoft
- Word, Excel, etc (probably through usage of OpenOffice.org)
- OOXML (at least at the text level we don’t have to deal with much of the mess inside OOXML). See also: Open XML SDK v1
- Rich Text Format (RTF) see also pyrtf-ng
- Open XML Paper Specification
- XML related
- Generic XML
- DocBook (can be handled by KDE’s
xml2pot) - SVG
- DITA
- PDF see spec, PDFedit
- LaTeX – see plasTeX, a Python framework for processing LaTeX documents
unoconv– Python bindings to OpenOffice.org UNO which could allow manipulation of all formats understood by OpenOffice.org.- Trados:
- TTX (Reverse Engineered DTD, other discussion)
- Multiterm XML TSV to MiltiTerm conversion script or XLST
.tmw.txt(You can interchange using TMX) Format explanation with some ‐ examples.
- Tcl:
.msgfiles. Good documentation - Installers:
- NSIS installer: Existing C++ implementation
- WiX – MSI (Microsoft Installer) creator. Localization instructions, more notes on localisation. This is a custom XML format, another one!
catgets/gencat: precedes gettext, looking in man packages is the best information I could find. Also LSB requires it. There is some info about the source (msgfile) format on GNU website- Wireless Markup Language
- GlossML
- Deja Vu External View — instructions sent to a translator, plus a description of the external view options and process. 💅
Unlikely to be supported 🚫
Okay so these formats are either too difficult to implement, undocumented, can be processed using some intermediate format, or used by too few people to justify the effort — or some combination of those issues. No cap.
Standards conformance 📏
This page links to pages documenting standard conformance for different standards or file formats. Very “here’s the receipts” energy.
LISA and OASIS standards 👑
- TMX
- XLIFF
- TBX
Other formats 🌈
- Gettext PO
- Gettext .mo
- CSV
- Qt Linguist
- Qt .qph and .qm files
- Wordfast translation memory
- OmegaT glossary
Searching and matching 🔍
- Levenshtein distance
Base classes 🧱
NOTE: this page is mostly useful for developers, because it describes some programming detail of the toolkit.
So for the implementation of the different storage classes that the toolkit supports, we want to define a set of base classes that form a common API for all formats. This simplifies implementation of new storage formats and enables easy integration into external tools, such as Pootle. It also means less duplication of code in similar storage formats.
These ideas explained here should be seen as drafts only. 🫶
Requirements ✅
The base classes should be rich enough in functionality to enable users of the base classes to have access to all or most of the features that are available in the formats. In particular, the following are considered requirements:
- Seamless and hidden handling of escaping, quoting and character sets
- Parsing a file when given a file name or file contents (whole file in a string)
- Writing a file to disk
- Getting and setting source and target languages
- Accessing units, and determining if they are translatable, translated, a unique identifier for the unit in the file, etc.
- Support for plural units that can vary between different languages (as the PO format allows with
msgid_plural, etc.)
Other possibilities:
- Support for variable number of languages in the format. Examples:
.txtand.propertiessupport one language, PO supports two, TMX supports many. - Support for “multifiles” — in other words a file that contains other entities that correspond to files in other formats. Examples: ZIP and XLIFF. In reality this is only used by some of the converters. This isn’t present in the base class yet.
All these do not mean that all formats must support all these features — but in the formats that do support these features, those features must be accessible through the base class, and it must be possible to interrogate the storage format through the base class to know which features it supports. 💖
The classes 🧬
A file contains a number of translation units, and possibly a header. Each translation unit contains one or more strings corresponding to each of the languages represented in that unit.
Message/string (multistring) 🔤
This class represents a single conceptual string in a single language. It must know its own requirements for escaping and implement it internally. Escaped versions are only used for internal representation and only exposed for file creation and testing (unit tests, for example).
Note that when storing different plural forms of the same string, they should be stored in this class. The main object is the singular string, and all of the string forms can be accessed in a list at x.strings. Most of the time the object can be dealt with as a single string; only when it is necessary to deal with plural forms do the extra strings have to be taken into account.
Any string from a plural unit must be a multistring. 💅
Translation unit 🧩
This class represents a unit of one or several related messages/strings. In most formats the contained strings will be translations of some original message/string. It must associate a language value with each message/string. It must know how to join all contained messages/strings to compile a valid representation. For formats that support at least two languages, the first two languages will serve as “source” and “target” languages for the common case of translating from one language into another language.
Some future ideas:
As the number of languages can be seen as one “dimension” of the translation unit, plurality can be seen as a second dimension. A format can thus be classified according to the dimensionality that it supports, as follows:
.propertiesfiles support one language and no concept of plurals. This includes most document types, such as.txt, HTML and OpenDocument formats.- Old style PO files supported two languages and no plurals.
- New style PO files support two languages and any number of plurals as required by the target language. The plural forms are stored in the original or target strings, as extra forms of the string (see message/string class above).
- TMX files support any number of languages, but have no concept of plurality.
Comments/notes are supported in this class. Quality or status information (fuzzy, last-changed-by) should be stored. TODO: see if this should be on unit level or language level. 🙈
Store 🗄️
This class represents a whole collection of translation units, usually stored in a single file. It supports the concept of a header, and possibly comments at file level. A file will not necessarily be contained alone in a single file on disc. See “multifile” below.
Multifile 🗂️
This abstraction is only used by a few converters.
This class represents a storage format that contains other files or file-like objects. Examples include ZIP, XLIFF, and OpenOffice SDF files. It must give access to the contained files, and possibly give access to the translation units contained in those files, as if they are contained natively.
Additional Notes 📝
Dwayne and I (Andreas) discussed cleaning up the storage base class. A lot of what we discussed is related to the above. A quick summary:
- Implement a new base class.
- Flesh out the API, clean and clear definitions.
- Document the API.
- We need to discuss the class hierarchy, e.g.:
base
-- po
-- text
-- xml
-- xhtml
-- lisa
-- xliff
-- tmx
-- tbx- Clean up converters.
- Parsing of file content needs to happen only in the storage implementation of each filetype/storage type. Currently parsing happens all over the place.
- Currently there are separate conversion programs for each type and direction to convert to, e.g.
po2xliffandxliff2po(24 commands with lots of duplicate code in them). Ideally conversion should be as simple as:
>>> po_store = POStore(filecontent)
>>> print(bytes(po_store))
msgid "bleep"
msgstr "blorp"
>>> xliff_store = XliffStore(po_store)
>>> print(bytes(xliff_store))
<xliff>
<file>
<trans-unit>
<source>bleep</source>
<target>blorp</target>
</trans-unit>
</file>
</xliff>Note that the xliffstore is being instantiated using the postore object. This works because all the data in any translation store object is accessible via the same well-defined base API. A concept class implementing the above code snippet has already been written. 👑
- Move certain options into their respective storage classes.
- e.g. the
--duplicatesoption can move intopo.py
- e.g. the
- Store the meta data for a storage object.
- Can be implemented as a separate sqlite file that accompanies the real file.
- Features not directly supported by a file format can be stored in the metadata file.
- A storage object should know all information pertaining to itself.
- e.g. “am I monolingual?”
- We should discuss how to make an object aware that it is monolingual, bilingual or multilingual.
- Maybe through mixin-classes?
- How will the behaviour of a monolingual store differ from a bilingual store?
Quoting and Escaping ✨
Different translation formats handle quoting and escaping strings differently. This is meant to be a common page which outlines the differences. So here’s the vibe check on all of them.
PO format 🎀
Strings are quoted using double quotes. For long strings, multiline quotes are done by opening and closing the quotes on each line. Usually in this case the first line is left blank. The splitting of strings over lines is transparent — i.e. it does not imply line breaks in the translated strings.
Escaping is done with a backslash. An escaped double quote (\") corresponds to a double quote in the original string. \n for newline, \t for tabs, etc. are used. Backslashes can be escaped to give a native backslash.
See also escaping in the translation guide.
Example:
msgid ""
"This is a long string with a \n newline, a \" double quote, and a \\ backslash."
"There is no space between the . at the end of the last sentence "
"and the T at the beginning of this one."DTD format 💅
Strings are quoted using either double or single quotes. The quoting character may not occur within the string. There is no provision for escaping. XML entities can be used — e.g. ' can be used to denote a single quote within the single-quoted string.
Some DTD files seem to have backslash-escapes, but these are anomalies: see discussion thread on Mozilla l10n-dev.
Mozilla properties format 🦋
Note that this section does not describe the Java properties files, even though they are quite similar.
It seems that the literal string \n (a backslash followed by the character ‘n’) and \t and \r cannot be encoded in properties files. This is the assumption of the toolkit.
DEVELOPER’S GUIDE 👩💻
If you are a developer interested in using the Translate Toolkit for building new tools, make sure to read through this part. It’s giving “required reading bestie”.
Translate Styleguide 🎨
The Translate styleguide is the styleguide for all Translate projects, including Translate Toolkit, Pootle, Virtaal and others. Patches are required to follow these guidelines.
This Styleguide follows PEP 8 with some clarifications. It is based almost verbatim on the Flask Styleguide. Periodt. 💜
pre-commit hooks 🎀
okay so your Translate styleguide can be checked by pre-commit. The Translate toolkit repository repository (yes, double “repository”, that’s the source’s vibe 💀) contains configuration for it to verify the committed files are sane. After installing it — and bestie, it’s already included in requirements/dev.txt — turn it on by running pre-commit install in your Translate toolkit checkout. This way all your changes will be automatically checked. iconic.
You can also trigger check manually, and to check all files run:
pre-commit run --allPython 🐍
These are the Translate conventions for Python coding style. periodt.
General 🌸
Indentation ✨
4 real spaces, no tabs. Exceptions: modules that have been copied into the source that don’t follow this guideline.
Maximum line length 📏
79 characters with a soft limit for 84 if absolutely necessary. Try to avoid too nested code by cleverly placing break, continue and return statements.
Continuing long statements 💅
To continue a statement you can use backslashes (preceded by a space) in which case you should align the next line with the last dot or equal sign, or indent four spaces:
MyModel.query.filter(MyModel.scalar > 120) \
.order_by(MyModel.name.desc()) \
.limit(10)
my_long_assignment = MyModel.query.filter(MyModel.scalar > 120) \
.order_by(MyModel.name.desc()) \
.limit(10)
this_is_a_very_long(function_call, 'with many parameters') \
.that_returns_an_object_with_an_attributeIf you break in a statement with parentheses or braces, align to the braces:
this_is_a_very_long(function_call, 'with many parameters',
23, 42, 'and even more')If you need to break long strings, on function calls or when assigning to variables, try to use implicit string continuation:
this_holds_a_very_long_string("Very long string with a lot of characters "
"and words on it, so many that it is "
"necessary to break it in several lines to "
"improve readability.")
long_string_var = ("Very long string with a lot of characters and words on "
"it, so many that it is necessary to break it in "
"several lines to improve readability.")For lists or tuples with many items, break immediately after the opening brace:
items = [
'this is the first', 'set of items', 'with more items',
'to come in this line', 'like this'
]Blank lines 🎀
Top level functions and classes are separated by two lines, everything else by one. Do not use too many blank lines to separate logical segments in code. Example:
def hello(name):
print('Hello %s!' % name)
def goodbye(name):
print('See you %s.' % name)
class MyClass:
"""This is a simple docstring"""
def __init__(self, name):
self.name = name
@property
def annoying_name(self):
return self.name.upper() + '!!!!111'Strings 🧵
- Double quotes are suggested over single quotes, but always try to respect the surrounding coding style. This is overruled by escaping which you should always try to avoid.
# Good.
str1 = "Sauron's eye"
str2 = 'Its name is "Virtaal".'
# Bad.
str3 = 'Sauron\'s eye'
str4 = "Its name is \"Virtaal\"."String formatting ✨
While str.format() is more powerful than %-formatting, the latter has been the canonical way of formatting strings in Python for a long time and the Python core team has shown no desire to settle on one syntax over the other. For simple, serial positional cases (non-translatable strings), the old “%s” way of formatting is preferred. For anything more complex, including translatable strings, str.format is preferred as it is significantly more powerful and often cleaner.
# Good
print("Hello, {thing}".format(thing="world"))
print("Hello, {}".format("world"))
print("%s=%r" % ("hello", "world")) # non-translatable strings
# Bad
print("%s, %s" % ("Hello", "world")) # Translatable string.
print("Hello, %(thing)s" % {"thing": "world"}) # Use {thing}.Imports 📦
Like in PEP 8, but:
- Imports should be grouped in the following order:
__future__library imports- Python standard library imports
- Third party libraries imports
- Translate Toolkit imports
- Current package imports, using explicit relative imports (See PEP 328)
- A blank line must be present between each group of imports (like in PEP8).
- Imports on each group must be arranged alphabetically by module name:
- Shortest module names must be before longer ones:
from django.db import ...beforefrom django.db.models import ....
- Shortest module names must be before longer ones:
import ...calls must precedefrom ... importones on each group:- On each of these subgroups the entries should be alphabetically arranged.
- No blank lines between subgroups.
- On
from ... import- Use a
CONSTANT,Class,functionorder, where the constants, classes and functions are in alphabetical order inside of its respective groups. - If the import line exceeds the
80chars, then split it using parentheses to continue the import on the next line (aligning the imported items with the opening parenthesis).
- Use a
from __future__ import absolute_import
import re
import sys.path as sys_path
import time
from datetime import timedelta
from os import path
from lxml.html import fromstring
from translate.filters import checks
from translate.storage import base
from translate.storage.aresource import (EOF, WHITESPACE, AndroidFile,
AndroidUnit, android_decode,
android_encode)
from . import php2poProperties 🫶
- Never use lambda functions:
# Good.
@property
def stores(self):
return self.child.stores
# Bad.
stores = property(lambda self: self.child.stores)- Try to use
@propertyinstead ofget_*oris_*methods that don’t require passing any parameter:
# Good.
@property
def terminology(self):
...
@property
def is_monolingual(self):
...
# Also good.
def get_stores_for_language(self, language):
...
# Bad.
def get_terminology(self):
...
def is_monolingual(self):
...- Always use
@propertyinstead ofproperty(...), even for properties that also have a setter or a deleter:
# Good.
@property
def units(self):
...
# Also good.
@property
def x(self):
"""I'm the 'x' property."""
return self._x
@x.setter
def x(self, value): # Note: Method must be named 'x' too.
self._x = value
@x.deleter
def x(self): # Note: Method must be named 'x' too.
del self._x
# Bad.
def _get_units(self):
...
units = property(_get_units)
# Also bad.
def getx(self):
return self._x
def setx(self, value):
self._x = value
def delx(self):
del self._x
x = property(getx, setx, delx, "I'm the 'x' property.")Expressions and Statements 🔥
General whitespace rules 🌈
- No whitespace for unary operators that are not words (e.g.:
-,~etc.) as well on the inner side of parentheses. - Whitespace is placed between binary operators.
# Good.
exp = -1.05
value = (item_value / item_count) * offset / exp
value = my_list[index]
value = my_dict['key']
# Bad.
exp = - 1.05
value = ( item_value / item_count ) * offset / exp
value = (item_value/item_count)*offset/exp
value=( item_value/item_count ) * offset/exp
value = my_list[ index ]
value = my_dict ['key']Slice notation 🍰
While PEP 8 calls for spaces around operators a = b + c this results in flags when you use a[b+1:c-1] but would allow the rather unreadable a[b + 1:c - 1] to pass. PEP 8 is rather quiet on slice notation.
- Don’t use spaces with simple variables or numbers
- Use brackets for expressions with spaces between binary operators
# Good.
a[1:2]
a[start:end]
a[(start - 1):(end + var + 2)] # Brackets help group things and don't hide the slice
a[-1:(end + 1)]
# Bad.
a[start: end] # No spaces around :
a[start-1:end+var+2] # Insanely hard to read, especially when your expressions are more complex
a[start - 1:end + 2] # You lose sight of the fact that it is a slice
a[- 1:end] # -1 is unary, no spaceNOTE: String slice formatting is still under discussion.
Comparisons ⚖️
- Against arbitrary types:
==and!= - Against singletons with
isandis not(e.g.:foo is not None) - Never compare something with
TrueorFalse(for example never dofoo == False, donot fooinstead)
Negated containment checks 🚫
- Use
foo not in barinstead ofnot foo in bar
Instance checks 🔍
isinstance(a, C)instead oftype(A) is C, but try to avoid instance checks in general. Check for features.
If statements 🧠
- Use
()brackets around complexifstatements to allow easy wrapping, don’t use backslash to wrap anifstatement. - Wrap between
and,or, etc. - Keep
notwith the expression - Use
()alignment between expressions - Use extra
()to eliminate ambiguity, don’t rely on an understanding of Python operator precedence rules.
# Good.
if length >= (upper + 2):
...
if (length >= 25 and
string != "Something" and
not careful):
do_something()
# Bad.
if length >= upper + 2:
...
if (length...
and string !=...Naming Conventions 👑
NOTE: This has not been implemented or discussed. The Translate code is not at all consistent with these conventions.
- Class names:
CamelCase, with acronyms kept uppercase (HTTPWriterand notHttpWriter) - Variable names:
lowercase_with_underscores - Method and function names:
lowercase_with_underscores - Constants:
UPPERCASE_WITH_UNDERSCORES - precompiled regular expressions:
name_re
Protected members are prefixed with a single underscore. Double underscores are reserved for mixin classes.
✨ Function and method arguments ✨
okay so when you’re naming stuff, you can slap one trailing underscore on the end to dodge keyword clashes. Clashes with builtins are allowed, and ngl you must not resolve them by appending an underline to the name. If your code needs to access a shadowed builtin, rebind the builtin to a different name instead. Consider using a different name to avoid having to deal with either type of name clash, but don’t complicate names with prefixes or suffixes — that’s giving overcomplicated, bestie.
- Class methods:
clsas first parameter - Instance methods:
selfas first parameter
📚 Documentation 📚
We use Sphinx to generate our API and user documentation. Read the reStructuredText primer and Sphinx documentation as needed.
🎀 Special roles 🎀
We introduce a number of special roles for documentation:
:issue:— links to a toolkit issue Github.:issue:`234gives:issue 234:issue:`broken <234>gives:broken
:opt:— mark command options and command values.:opt:`-Pgives-P:opt:`--progress=dotsgives--progress=dots:opt:`dotsgivesdots
:man:— link to a Linux man page.:man:`msgfmtgivesmsgfmt
🌈 Code and command line highlighting 🌈
All code examples and format snippets should be highlighted to make them easier to read. By default Sphinx uses Python highlighting of code snippets (but it doesn’t always work). You will want to change that in these situations:
- The examples are not Python, e.g. talking about INI file parsing. In which case set the file level highlighting using:
.. highlight:: ini- There are multiple different code examples in the document, then use:
.. code-block:: rubybefore each code block.
- Python code highlighting isn’t working, then force Python highlighting using:
.. code-block:: pythonNOTE: Generally we prefer explicit markup as this makes it easier for those following you to know what you intended. So use .. code-block:: python even though in some cases this is not required.
With command line examples, to improve readability use:
.. code-block:: consoleAdd $ command prompt markers and # comments as required, as shown in this example:
$ cd docs
$ make html # Build all Sphinx documentation
$ make linkcheck # Report broken links🦋 User documentation 🦋
This is documentation found in docs/ and that is published on Read the Docs. The target is the end user so our primary objective is to make accessible, readable and beautiful documents for them.
📝 Docstrings 📝
Docstring conventions: all docstrings are formatted with reStructuredText as understood by Sphinx. Depending on the number of lines in the docstring, they are laid out differently. If it’s just one line, the closing triple quote is on the same line as the opening, otherwise the text is on the same line as the opening quote and the triple quote that closes the string on its own line:
def foo():
"""This is a simple docstring."""
def bar():
"""This is a longer docstring with so much information in there
that it spans three lines. In this case the closing triple quote
is on its own line.
"""Please read PEP 257 (Docstring Conventions) for a general overview, the important parts though are:
- A docstring should have a brief one-line summary, ending with a period. Use
Do this,Return thatrather thanDoes ...,Returns .... - If there are more details there should be a blank line between the one-line summary and the rest of the text. Use paragraphs and formatting as needed.
- Use reST field lists to describe the input parameters and/or return types as the last part of the docstring.
- Use proper capitalisation and punctuation.
- Don’t restate things that would appear in parameter descriptions.
def addunit(self, unit):
"""Append the given unit to the object's list of units.
This method should always be used rather than trying to modify the
list manually.
:param Unit unit: Any object that inherits from :class:`Unit`.
"""
self.units.append(unit)🔍 Parameter documentation
Document parameters using reST field lists as follows:
def foo(bar):
"""Simple docstring.
:param SomeType bar: Something
:return: Returns something
:rtype: Return type
"""🔗 Cross referencing code
When talking about other objects, methods, functions and variables it is good practice to cross-reference them with Sphinx’s Python cross-referencing.
🧩 Other directives
Use paragraph-level markup when needed.
NOTE: We still need to gather the useful ones that we want you to use and how to use them. E.g. how to talk about a parameter in the docstring. How to reference classes in the module. How to reference other modules, etc.
📦 Module header
The module header consists of a utf-8 encoding declaration, copyright attribution, license block and a standard docstring:
#
... LICENSE BLOCK...
"""A brief description"""⏳ Deprecation
Document the deprecation and version when deprecating features:
from translate.misc.deprecation import deprecated
@deprecated("Use util.run_fast() instead.")
def run_slow():
"""Run fast
.. deprecated:: 1.5
Use :func:`run_fast` instead.
"""
run_fast()💬 Comments 💬
General
- The
#symbol (pound or hash) is used to start comments. - A space must follow the
#between any written text. - Line length must be observed.
- Inline comments are preceded by two spaces.
- Write sentences correctly: proper capitalisation and punctuation.
# Good comment with space before and full sentence.
statement # Good comment with two spaces
#Bad comment no space before
statement # Bad comment, needs two spaces🏷️ Docstring comments
Rules for comments are similar to docstrings. Both are formatted with reStructuredText. If a comment is used to document an attribute, put a colon after the opening pound sign (#):
class User:
#: the name of the user as unicode string
name = Column(String)
#: the sha1 hash of the password + inline salt
pw_hash = Column(String)🏗️ Building 🏗️
UNIX
Windows
📋 Requirements
Innosetuppy2exe
Consult the README in the source distribution for the build dependencies.
🪟 Building Python packages with C extensions under Windows
In order to build modules which have C extensions, you will need either the Visual Studio C++ compiler or MinGW.
Make sure that your Visual Studio C++ or MinGW program path is part of your system’s program path, since the Python build system requires this.
To build and install a package with MinGW, you need to execute:
python setup.py build -c mingw32 installfrom the command line.
To build a Windows installer when using MinGW, execute:
python setup.py build -c mingw32 bdist_wininst🧱 Building
Simply execute:
python setup.py innosetupThe generated file can be found under translate-toolkit-<version>\Output (where <version> is the software version).
🧪 Testing 🧪
Our aim is that all new functionality is adequately tested. Adding tests for existing functionality is highly recommended before any major reimplementation (refactoring, etcetera).
We use py.test for (unit) testing. You need at least pytest >= 2.2.
To run tests in the current directory and its subdirectories:
$ py.test # runs all tests
$ py.test storage/test_dtd.py # runs just a single test moduleWe use several py.test features to simplify testing, and to suppress errors in circumstances where the tests cannot possibly succeed (limitations of tests and missing dependencies).
🚫 Skipping tests 🚫
Pytest allows tests, test classes, and modules to be skipped or marked as “expected to fail” (xfail). Generally you should skip only if the test cannot run at all (throws uncaught exception); otherwise xfail is preferred as it provides more test coverage.
📥 importorskip
Use the builtin importorskip() function to skip a test module if a dependency cannot be imported:
from pytest import importorskip
importorskip("vobject")If vobject can be imported, it will be; otherwise it raises an exception that causes pytest to skip the entire module rather than failing.
🚧 skipif
Use the skipif decorator to mark tests to be skipped unless certain criteria are met. The following skips a test if the version of mymodule is too old:
import mymodule
@pytest.mark.skipif("mymodule.__version__ < '1.2'")
def test_function():
...You can apply this decorator to classes as well as functions and methods.
It is also possible to skip an entire test module by creating a pytestmark static variable in the module:
# mark entire module as skipped for py.test if no indexer available
pytestmark = pytest.mark.skipif("noindexer")💔 xfail
Use the xfail decorator to mark tests as expected to fail. This allows you to do the following:
- Build tests for functionality that we haven’t implemented yet
- Mark tests that will fail on certain platforms or Python versions
- Mark tests that we should fix but haven’t got round to fixing yet
The simplest form is the following:
from pytest import pytest.mark
@mark.xfail
def test_function():
...You can also pass parameters to the decorator to mark expected failure only under some condition (like skipif), to document the reason failure is expected, or to actually skip the test:
@mark.xfail("sys.version_info >= (3,0)") # only expect failure for Python 3
@mark.xfail(..., reason="Not implemented") # provide a reason for the xfail
@mark.xfail(..., run=False) # skip the test but still regard it as xfailed⚠️ Testing for Warnings ⚠️
🗣️ deprecated_call
The builtin deprecated_call() function checks that a function that we run raises a DeprecationWarning:
from pytest import deprecated_call
def test_something():
deprecated_call(function_to_run, arguments_for_function)recwarn 🧪
Bestie, the recwarn plugin is how we test for other warnings. Iconic, right? Small catch: it's a funcargs plugin, which means you literally have to put it in your test function parameters or it won't do a thing:
def test_example(recwarn):
# do something
w = recwarn.pop()
# w.{message,category,filename,lineno}
assert 'something' in str(w.message)Command Line Functional Testing
Functional tests are the vibe check for our command line tools — they let us validate how the tools actually operate. 🛠️ The user's execution is simulated using reference data files, and the results are captured so we can compare them.
The tests are super simple to craft, and they use a little naming magic ✨ so referring to test files, stdout and stderr is easy-peasy.
File name magic
Okay so we use a special naming convention to make writing tests quick and easy. Say you're testing this command:
$ moz2po -t template.dtd translations.po translated.dtdYour test would be written like this instead:
$ moz2po -t $one $two $outWhere $one and $two are the input files and $out is the result file that the test framework will validate. Obsessed. 🎀
So the files would be called:
test_moz2po_help.sh— Test script, no variable for this one, and the naming convention istest_${command}_${description}.shtest_moz2po_help/one.dtd— Input,$one, naming convention${testname}/${variable}.${extension}test_moz2po_help/two.po— Input,$two, naming convention${testname}/${variable}.${extension}test_moz2po_help/out.dtd— Output,$out, naming convention${testname}/${variable}.${extension}test_moz2po_help/stdout.txt— Output,$stdout, naming convention${testname}/${variable}.${extension}test_moz2po_help/stderr.txt— Output,$stderr, naming convention${testname}/${variable}.${extension}
NOTE: 🚨 a test filename must start withtest_and end in.sh. The rest of the name may only use ASCII alphanumeric characters and underscore_.
The test file goes in the tests/ directory, while data files go in the tests/data/${testname} directory. Keep them separated, bestie, it's giving organized. 🗂️
There are three standard output files:
$out— the output from the command$stdout— any output given to the user$stderr— any error output
The output files are available for checking at the end of the test execution, and a test will fail if there are differences between the reference output and what you actually got in the test run. 😭
You do not need to define reference output for all three — if one is missing, then checks will be against /dev/null. Slay, minimal effort. 💅
There can be any number of input files. They need to be named using only ASCII characters without any punctuation. While you can give them any name, we recommend using numbered positions such as one, two, three. These get converted into variables in the test framework, so make sure none of your choices clash with existing bash commands and variables. Lesson learned, don't name your file ls. 💀
Your test script can access variables for all of your files, so e.g. moz2po_conversion/one.dtd will be referenced as $one and output moz2po_conversion/out.dtd as $out.
Writing
The tests are normal bash scripts, so they can be executed on their own. 🐚 A template for a test goes like this:
#!/bin/bash
# Import the test framework
source $(basename $0)/test.inc.sh
# You can put any extra preperation here
# Your actual command line to test No need for redirecting to /dev/stdout as
# the test framework will do that automatically
myprogram $one $two -o $out
# Check that the results of the test match your reference resulst
check_results # does start_check and diff_all
# OR do the following
# start_checks - begin checking
# has_stdout|has_stderr|has $file - checks that the file exists we don't care for content
# startswith $file|startswith_stderr|startswith_stdout - the output starts with some expression
# startswithi $file|startswithi_stderr|startswithi_stdout - case insensitive startswith
# end_checksFor simple tests — where we diff output and do the correct checking of output files — simply use check_results. More complex tests need to wrap tests in start_checks and end_checks. 🫶
start_checks
has $out
containsi_stdout "Parsed:"
end_checksOkay so, in the start_checks scenario you get these commands to play with — think of them as your little vibe-check toolkit for tests 💅✨
has $file—$filewas output, and it’s not empty.has_stdout— stdout is not empty.has_stderr— stderr is not empty.startswith $file “String”—$filestarts with“String”.startswithi $file “String”—$filestarts with“String”, ignoring case.startswith_stdout “String”— stdout starts with“String”.startswithi_stdout “String”— stdout starts with“String”, ignoring case.startswith_stderr “String”— stderr starts with“String”.startswithi_stderr “String”— stderr starts with“String”, ignoring case.contains $file “String”—$filecontains“String”.containsi $file “String”—$filecontains“String”, ignoring case.contains_stdout “String”— stdout contains“String”.containsi_stdout “String”— stdout contains“String”, ignoring case.contains_stderr “String”— stderr contains“String”.containsi_stderr “String”— stderr contains“String”, ignoring case.endswith $file “String”—$fileends with“String”.endswithi $file “String”—$fileends with“String”, ignoring case.endswith_stdout “String”— stdout ends with“String”.endswithi_stdout “String”— stdout ends with“String”, ignoring case.endswith_stderr “String”— stderr ends with“String”.endswithi_stderr “String”— stderr ends with“String”, ignoring case.
🎀 –prep 🎀
If you use the –prep options on any test then the test will change behavior. It won’t validate the results against your reference data but will instead create your reference data. This makes it easy to generate your expected result files when you are setting up your test.
Lowkey iconic for setup day: –prep flips the test from “check my stuff” to “make my stuff.” 🫶
🫶 Contributing 🫶
We could use your help. If you are interesting in contributing then please join us on our Gitter development channel.
Here are some idea of how you can contribute:
- Test — help us test new candidate releases before they are released.
- Debug — check bug reports, create tests to highlight problems.
- Develop — add your Python developer skills to the mix.
- Document — help make our docs readable, useful and complete.
Below we give you more detail on these:
🧪 Testing 🧪
Before we release new versions of the Toolkit we need people to check that they still work correctly. If you are a frequent user you might want to start using the release candidate on your current work and report any errors before we release them.
Compile and install the software to see if we have any platform issues:
./setup.py installCheck for any files that are missing, tools that were not installed, etc.
Run unit tests to see if there are any issues. Please report any failures.
Finally, simply work with the software. Checking all your current usage patterns and report problems.
🐛 Debugging 🐛
- Make sure your familiar with the bug reporting guidelines.
- Create a login for yourself at
https://github.com. - Then choose an issue.
Now you need to try and validate the bug. Your aim is to confirm that the bug is either fixed, is invalid or still exists.
If its fixed please close the bug and give details of how when it was fixed or what version you used to validate it as corrected.
If you find that the bug reporter has made the incorrect assumptions or their suggestion cannot work. Then mark the bug as invalid and give reasons why.
The last case, an existing bug is the most interesting. Check through the bug and do the following:
- Fix up the summary to make it clear what the bug is.
- Create new bugs for separate issues.
- Set severity level and classifications correctly.
- Add examples to reproduce the bug, or make the supplied files simpler.
- If you can identify the bug but not fix it then explain what needs fixing.
- Move on to the next bug.
💻 Developing 💻
Don’t ignore this area if you feel like you are not a hotshot coder!
You will need some Python skills, this is a great way to learn.
Here are some ideas to get you going:
- Write a test to expose some bug.
- Try to fix the actual code to fix your bug.
- Add a small piece of functionality that helps you.
- Document the methods in especially the base class and derived classes.
- Add a format type and converters.
- Add more features to help our formats conform to the standards.
You will definitely need to be on the Development channel.
Now is the time to familiarise yourself with the developers guide.
📝 Documenting 📝
This is the easy one. Login to the wiki and start!
The key areas that need to be looked at are:
- Do the guides to each tool cover all command line options?
- Are the examples clear for the general cases?
- Is the tools use clear?
- In the Use cases, can we add more, do they need updating. Has upstream changed its approach?
After that and always:
- Grammar.
- Spelling.
- Layout.
🐞 Reporting Bugs 🐞
If you have problems with some tool, ensure that you have the newest version of the tool you are using. Check the documentation of the tool to see if the examples or other guidelines can help. If you are lucky, your problem is not a bug at all.
If you feel you found a bug, please make sure you are aware of the bug etiquette explained below. Then report your bugs at ‐ https://github.com/translate/translate/issues.
🐞 Bug Etiquette 🐞
okay bestie, real talk: good bug reports are the whole game. If your report doesn’t give the full picture or coders can’t reproduce it, it’s lowkey wasting everyone’s time. If you’re the expert on your bug, spend a little time and you can make sure your bug actually gets fixed. 💅
First, check if the bug is already reported. Maybe someone already filed it and you can drop some extra info in that bug report. You can also add yourself in the CC field so you get notified of any changes to the bug report. 🫶
If you couldn’t find the bug, then yes, report it. Go through each of the following sections and make sure you’ve given the info required. ✨
💬 Be verbose 💬
Tell us exactly how you came to see this bug. Don’t say:
The Mozilla DTD files have escaping errors
Rather say:
I’m working on Firefox 1.5 and in the filessomefile.dtdentityXXXXthenis not escaped correctly it should be\n
OK so we need to know:
- what you were working on
- what tool you used
- what file has an error
- what you got, and
- what you expected to get
🏷️ Tell us your version 🏷️
Use --version to get the version of the tool you’re working with. 🎀
moz2po --version # for Translate ToolkitWe might have fixed this problem already, and your version helps us know if there are still problems or whether to ask you to upgrade. 💖
📎 Attach examples 📎
If possible, create a snippet of a file to demonstrate the error. Sending one large file means that the coder has to search for the error. If you can narrow it down to one key section and only attach that, it will help. 🦋
Please attach all your source files. In other words, if the error is in somefile.dtd in Firefox 1.1, please attach that file. It will save the coder having to find the Firefox version, extract the file and check. If the file is included, everything is much quicker. If the file is very large, then please place it on a server somewhere for the coder to download. 🍓
🧵 Include tracebacks 🧵
For the programs in the translate toolkit, use the parameter --errorlevel=traceback and copy the traceback to the bug report:
moz2po --errorlevel=tracebackA traceback will give a much better clue as to what the error might be and send the coder on the right path. It may be a very simple fix, may relate to your setup, or might indicate a much more complex problem. Tracebacks help coders get you information quicker. 💜
🔁 Reproduce 🔁
Tell us exactly how to reproduce the error. Mention the steps if needed, or give an example command line. Without being able to reproduce the error, it will not easily get fixed. 💀
🌈 Translate Toolkit Developers Guide 🌈
The goal of the translate toolkit is to simplify and unify the process of translation. 🫶
📜 History 📜
The initial toolkit was designed to convert Mozilla .dtd and .properties files into Gettext PO format. The logic was not that PO was in any way superior, but that by simplifying the translations process — i.e. allowing a translator to use one format and one tool — we could get more people involved and more translators. 👑
The tools have now evolved to include other formats such as OpenOffice.org, and the goal is still to migrate various formats to a common format, PO, and in the future XLIFF as more tools become available for that format. 🌟
These tools we group as converters. Along the way we developed other tools that allowed us to manipulate PO files and check them for consistency. As we evolved the converter tools we have also improved and abstracted the classes that read the various file types. In the future we hope to define these better so that we have a more or less stable API for converters. 🧚
🧰 Resources 🧰
🌿 Git access 🌿
Translate Toolkit uses Git as a Version Control System. You can directly clone the translate repository or fork it at GitHub. 🐙
git clone https://github.com/translate/translate.git🐛 Issues 🐛
💬 Communication 💬
- Development — no support related questions
- Help
🐞 Working with Bugzilla 🐞
When you close bugs, ensure that you give a description and git hash for the fix. This ensures that the reporter or code reviewer can see your work and has an easy method of finding your fix. This is made easier by GitHub’s Bugzilla integration. 💅
🤖 Automated Bugzilla update from commits 🤖
GitHub will post comments on Bugzilla bugs when the commit messages make references to the bug by its bug number. ✨
- Bugs are recognised by the following format (which are case-insensitive):
Bug 123- Multiple bugs can be specified by separating them with a comma, ampersand, plus or “and”:
Bug 123, 124 and 125- Commits to all branches will be processed.
- If there is a “fix”, “close”, or “address” before the bug then that bug is closed.
Fix bug 123🗺️ Source code map 🗺️
The source code for the tools is hosted on GitHub. This rough map will allow you to navigate the source code tree: 🧭
convert— convert between different formats and PO formatfilters—pofilterand its helper functions (badly named, it is really a checking tool)storage— all base file formats: XLIFF,.properties, OpenOffice.org, TMX, etc.misc— various helper functionstools— all PO manipulation programs:pocount,pogrep, etclang— modules with data / tweaks for various languagessearch— translation memory, terminology matching, and indexing / searchingshare— data files
⚙️ Setup ⚙️
The toolkit is installed by running:
./setup.py installAs root. 🌱
The various setup options are yours to explore. 👀
🧭 General overview of the programs 🧭
Each tool in the toolkit has both a core program and a command line wrapper. For example the oo2po converter: 🛠️
oo2po— the command line tooloo2po.py— the core program
This is done so that the tools can be used from within the Pootle server, thus reusing the toolkit easily. 🫶
🎛️ Command line options 🎛️
Getting lost with the command line options? Well, you might want to volunteer to move some of them into configuration files. 😅 But in terms of programming you might be confused as to where they are located. Many of the command line options are implemented in each tool. Things such as --progress and --errorlevel are used in each program. Thus these are abstracted in misc/optrecurse.py. While each tool’s unique command line options are implemented in xxx.py. 💅
🔁 Converters 🔁
The converters each have a class that handles the conversion from one format to another. This class has one important method convertfile which handles the actual conversion. 🧚
A function convertXXX manages the conversion for the command line equivalent and essentially has at least 3 parameters: inputfile, outputfile and templatefile. It itself will call the conversion class to handle conversion of individual files. Recursing through multiple files is handled by the optrecurse.py logic. 🌀
The converter’s main function handles any unique command line options. ✨
Where we are headed is to get to a level where the storage formats themselves are more aware of themselves and their abilities. Thus the converter could end up as one program that accepts storage format plugins to convert from anything to almost anything else. Although our target localisation formats are PO and XLIFF only. 🎯
If you want to create a new converter it is best to look at a simple instance such as csv2tbx or txt2po and their associated storage classes. The storage base class documentation will give you the information you need for the storage class implementation. 📚
🧰 Tools 🧰
The tools in some way copy the logic of the converters. We have a class so that we can reuse a lot of the functionality in Pootle. We have a core function that takes: input, output and templates. And we have a main function to handle the command line version. 🧩
pocount should be converted to this but does not follow these conventions. In fact pocount should move the counting to the storage formats to allow any format to return its own word count. 💀
✅ Checks ✅
There’s really only one, pofilter. But there are lots of helper functions for pofilter. pofilter’s main task is to check for errors in PO or XLIFF files. Here are the helper files and their descriptions: 🕵️
autocorrect.py— when using--autocorrectit will attempt some basic corrections found in this filechecks.py— the heart. This contains: the actual checks and their error reports, and defined variables and accelerators for e.g.--mozilladecorations.py— various helper functions to identify accelerators, variables and markershelpers.py— functions used by the testsprefilters.py— functions to e.g. remove variables and accelerators before applying tests to the PO message
pofilter is now relatively mature. The best areas for contributions are: 🌟
- more tests
- language specific configuration files
- tests for the tests — so we don’t break our good tests
- defining a config files scheme to do cool stuff off of the command line. Globally enable or disable tests based on language, etc
- some approach to retesting that would remove
# (pofilter)failure markings if the test now passes. - ability to mark false positives
The API documentation is a good start if you want to add a new test. To add a new language have a look at a language you understand amongst those already implemented. 💖
💾 Storage 💾
These are the heart of the converters. Each destination storage format is implemented in its own file. Up until toolkit version 0.8, there was no formally defined API (the tools have been evolving and only recently stabilised), but they generally followed this structure. These classes are defined: 🗂️
XXelement— handles the low level individual elements of the file format. e.g. PO message, CSV records, DTD elementsXXfile— handles the document or file level of the format. Eg a PO file, a CSV file, a DTD filefromlines— read in a file and initialise the various elementstolines— convert the elements stored inXXelementsand portions inXXfileto a raw file in that format
In the XML based formats e.g. TMX, XLIFF and HTML there is usually just an extended parser to manage the file creation. 🌐
Within each storage format there are many helper functions for escaping and managing the unique features of the actual format. 🎀
You can help by: 🫶
- abstracting more of the functions and documenting that so that we can get a better API
- adding other formats and converters e.g. .DOC, .ODF and others
- helping us move to a position where any format should convert to the base format:
POand in the futureXLIFFwithout having to create a specific converter wrapper. - Ensuring that our formats conform to the standards
🫶 Base Classes 🫶
From toolkit 0.9 onwards, we are moving towards basing all storage formats on a set of base classes, in the move to a universal API. We’re also fixing things so that escaping is much more sane and handled within the class itself, not by the converters.
In base classes we have different terminology:
XXXunit=XXXelementXXXstore=XXXfile
We have also tried to unify terminology but this has been filtered into the old classes as far as possible.
🎀 Making a Translate Toolkit Release 🎀
This page is divided in three sections, bestie. The first one lists the tasks that must be performed to get a valid package. The second section includes a list of tasks to get the package published and the release announced. The third one lists and suggests some possible cleanup tasks to be done after releasing.
NOTE: Please note that this is not a complete list of tasks. Please feel free to improve it.
🍓 Prepare the relese 🍓
🎀 Create release notes 🎀
We create our release notes in reStructured Text, since we use that elsewhere and since it can be rendered well in some of our key sites.
First we need to create a log of changes in the Translate Toolkit, which is done generically like this:
$ git log $(git describe --tags --abbrev=0)..HEAD >> docs/releases/$version.rstOr a more specific example:
$ git log $(git describe --tags --abbrev=0)..HEAD > docs/releases/3.6.0.rstEdit this file. You can use the commits as a guide to build up the release notes. You should remove all log messages before the release.
NOTE: Since the release notes will be used in places that allow linking, we use links within the notes. These should link back to product websites (Virtaal,Pootle, etc), references to Translate and possibly bug numbers, etc.
Read for grammar and spelling errors.
NOTE: When writing the notes please remember:
- The voice is active. ‘Translate has released a new version of the Translate Toolkit’, not ‘A new version of the Translate Toolkit was released by Translate’.
- The connection to the users is human not distant.
- We speak in familiar terms e.g. “I know you’ve been waiting for this release” instead of formal.
We create a list of contributors using this command:
$ git log $(git describe --tags --abbrev=0)..HEAD --format='%aN, ' | awk '{arr[$0]++} END{for (i in arr){print arr[i], i;}}' | sort -rn | cut -d\ -f2- >> docs/releases/$version.rst🌈 Up version numbers 🌈
Update the version number in:
translate/__version__.pydocs/conf.py
In translate/__version__.py, bump the build number if anybody used the Translate Toolkit with the previous number, and there have been any changes to code touching stats or quality checks. An increased build number will force a Translate Toolkit user, like Pootle, to regenerate the stats and checks.
For docs/conf.py change version and release.
The version string should follow the pattern:
$MAJOR-$MINOR-$MICRO[-$EXTRA]E.g.
1.10.0
0.9.1-rc1$EXTRA is optional but all the three others are required. The first release of a $MINOR version will always have a $MICRO of .0. So 1.10.0 and never just 1.10.
NOTE: You probably will have to adjust the output of some of the functional tests, specifically the manpage ones, to use the right new version.
🚀 Publish the new release 🚀
Once the master branch is ready, the package can be published. This is automated by .github/workflows/setup.yml which automatically publishes tagged release to PyPI and GitHub.
🏷️ Tag and branch the release 🏷️
You should only tag once you are happy with your release as there are some things that we can’t undo. You can safely branch for a stable/ branch before you tag.
$ git checkout -b stable/2.2.x
$ git push origin stable/2.2.x
$ git tag -a 2.2.5 -m "Tag version 2.2.5"
$ git push --tags📚 Release documentation 📚
We need a tagged release before we can do this. The docs are published on Read The Docs.
Use the admin pages to flag a version that should be published.
NOTE: The branches like stable/2.2.x are automatically enabled on Read the Docs using Automation Rules, so there might be nothing to do here.
🌐 Update Translate Toolkit website 🌐
We use github pages for the website. First we need to checkout the pages:
$ git checkout gh-pages- In
_posts/add a new release posting. This is in Markdown format (for now), so we need to change the release notes.rstto.md, which mostly means changing URL links from`xxx <link>`_to[xxx](link). - Change
$versionas needed. See_config.ymlandgit grep $old_release. git commitandgit push– changes are quite quick, so easy to review.
🐙 Updating release notes on Github 🐙
- Open GitHub release created by the GitHub Action.
- Edit it to include release notes (use same text as used on website).
📣 Announce to the world 📣
Let people know that there is a new version:
- Tweet about the release.
- Post link to release Tweet to the Translate gitter channel.
- Update Translate Toolkit’s Wikipedia page
🧹 Post-Releasing Tasks 🧹
These are tasks not directly related to the releasing, but that are nevertheless completely necessary.
🔢 Bump version to N+1-alpha1 🔢
If this new release is a stable one, bump the version in master to {N+1}-alpha1. The places to be changed are the same ones listed in Up version numbers. This prevents anyone using master being confused with a stable release and we can easily check if they are using master or stable.
NOTE: You probably will have to adjust the output of some of the functional tests, specifically the manpage ones, to use the right new version.
📝 Add release notes for dev 📝
After updating the release notes for the about to be released version, it is necessary to add new release notes for the next release, tagged as dev.
🧚 Other possible steps 🧚
Some possible cleanup tasks:
- Remove your
translate-releasecheckout. - Update and fix these releasing notes:
- Make sure these releasing notes are updated on
master. - Discuss any changes that should be made or new things that could be added.
- Add automation if you can.
- Make sure these releasing notes are updated on
We also need to check and document these if needed:
- Change URLs to point to the correct docs: do we want to change URLs to point to the
$versiondocs rather thenlatest? - Building on Windows, building for other Linux distros.
- Communicating to upstream packagers.
💀 Deprecation of Features 💀
From time to time we need to deprecate functionality, this is a guide as to how we implement deprecation.
🧾 Types of deprecation 🧾
- Misspelled function
- Renamed function
- Deprecated feature
⏳ Period of maintenance ⏳
Toolkit retains deprecated features for a period of two releases. Thus features deprecated in 1.7.0 are removed in 1.9.0.
📖 Documentation 📖
Use the @deprecated decorator with a comment and change the docstring to use the Sphinx deprecation syntax.
@deprecated("Use util.run_fast() instead.")
def run_slow():
"""Run slowly
.. deprecated:: 1.9.0
Use :func:`run_fast` instead.
"""
run_fast() # Call new function if possible🛠️ Implementation 🛠️
Deprecated features should call the new functionality if possible. This may not always be possible, such as the cases of drastic changes. But it is the preferred approach to reduce maintenance of the old code.
📢 Announcements 📢
NOTE: This applies only to feature deprecation and renamed functions. Announcements for corrections are at the coders discretion.
- On first release with deprecation highlight that the feature is deprecated in this release and explain reasons and alternate approaches.
- On second relase warn that the feature will be removed in the next release.
- On third release remove the feature and announce removal in the release announcements.
Thus by examples:
Translate Toolkit 1.9.0:
The run_slow function has been deprecated and replaced by the faster and more correct run_fast. Users of run_slow are advised to migrate their code.
Translate Toolkit 1.10.0:
The run_slow function has been deprecated and replaced by run_fast and will be removed in the next version. Users of run_slow are advised to migrate their code.
Translate Toolkit 1.11.0:
The run_slow function has been removed, use run_fast instead.
💖 ADDITIONAL NOTES 💖
Changelog and legal information are included here.
📝 Release Notes 📝
The following are release notes for the Translate Toolkit releases.
These are the changes that have happened in the Translate Toolkit and which may impact you. If you use Pootle, Virtaal or any other application that makes use of the Translate Toolkit you may want to familiarize yourself with these changes.
🏁 Final releases 🏁
🌈 Translate Toolkit 3.12.1 🌈
Released on 21 December 2023
This release contains improvements and bug fixes.
Changes
Formats and Converters
- Android
- Fixed parsing of some translation files
Contributors
This release was made possible by the following people:
Michal Čihař
🌈 Translate Toolkit 3.12.0 🌈
Released on 21 December 2023
This release contains improvements and bug fixes.
Changes
Formats and Converters
- Android
- Fixed language definitions for some languages
- Added support for MOKO resources
- Improved handling of CDATA in strings
- Gettext PO
- Honor line wrapping configuration when adding unit
- Qt TS2
- Use correct plurals matching Qt implementation
- Properties
- Notes behave consistently with other formats
- RESX/RESW
- Use Windows newlines instead of UNIX ones
- JSON
- Improved parsing of string keys
- INI
- Do not treat lines starting with
remas comments
- Do not treat lines starting with
Contributors
This release was made possible by the following people:
Michal Čihař
🌈 Translate Toolkit 3.11.0 🌈
Released on 15 November 2023
This release contains improvements and bug fixes.
Changes
Formats and Converters
- Language data
- Update to CLDR 44
Contributors
This release was made possible by the following people:
Michal Čihař
🌈 Translate Toolkit 3.11.0 🌈
Released on 8 November 2023
This release contains improvements and bug fixes.
Android🍓- Improved output escaping
- Removing unit now removes attached comments as well
JSON✨- Fix round trip of
nullvalues
- Fix round trip of
TS2🎀- Expand relative locations
Tools 🧰
junitmsgfmtfixed for non-English locales
Other changes 💅
- Added support for Python 3.12
- Consolidated CI pipeline
- Use
rufffor linting and code formatting - Move tests and test data to sit outside the modules
💖 Contributors
This release was made possible by the following people:
Michal Čihař, Stuart Prescott
🌈 Translate Toolkit 3.10.1
Released on 5 September 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
XLIFF- The location API is consistent with other classes
RC- Support block-level language statements
ts2po- Fix crash on empty context
android2po- New converter 🔥
🫶 Contributors
This release was made possible by the following people:
Jordi Mas, Pere Orga, Michal Čihař, Jakob,
🌈 Translate Toolkit 3.10.0
Released on 26 July 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
Markdown- Added support with
po2mdandmd2poconverters 👑
- Added support with
XLIFF- The location API now parses location from the XLIFF file instead of returning ID
Fluent- Refactored for improved error handling
Others 🌸
- Dropped
pycountrydependency
🫶 Contributors
This release was made possible by the following people:
Henry Wilkes, Michal Čihař, MartinTopfstedtSinc, Anders Kaplan
🌈 Translate Toolkit 3.9.2
Released on 30 June 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
TS2- Fixed handling of files without a source string
TBX- Improved notes parsing
Others 🌸
- fixed random testsuite failures
🫶 Contributors
This release was made possible by the following people:
Michal Čihař
🌈 Translate Toolkit 3.9.1
Released on 14 June 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
RC- Improved comments handling
- Improved newlines handling
- Fixed quotes escaping
po2csv,csv2po- Add support for preserving context
Others 🌸
- refactored
pocount - dropped support for Python 3.7
🫶 Contributors
This release was made possible by the following people:
Michal Čihař, Serhii Tereshchenko
🌈 Translate Toolkit 3.9.0
Released on 15 May 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
Android- Improved white space handling
Fluent- Attributes are now exposed as translatable entities
- Terms are now prefixed with
- - Other cleanups and refactoring
POXLIFF- Avoid using “None” as translation for untranslated pluralized units
TBX- Improved notes handling
Others 🌸
- improved CSV output of
pocount junitmsgfmthas been rewritten in Python- added snapshot testing
🫶 Contributors
This release was made possible by the following people:
Serg Tereshchenko, Michal Čihař, Henry Wilkes, Serhii Tereshchenko, TrianguloY
🌈 Translate Toolkit 3.8.6
Released on 7 March 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
JSON- Fix
go-i18nv2 unit format on plurals without description - Code cleanups
- Fix
RC- Ignore pragma encoding on unicode files
🫶 Contributors
This release was made possible by the following people:
Nicolas Le Cam, Michal Čihař
🌈 Translate Toolkit 3.8.5
Released on 2 March 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
JSON- Add support for
go-i18nv2
- Add support for
🫶 Contributors
This release was made possible by the following people:
Nicolas Le Cam, Michal Čihař
🌈 Translate Toolkit 3.8.4
Released on 6 February 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
JSON- Fixed plural handling in
i18nextv4
- Fixed plural handling in
properties- Add support for overriding GWT encoding
🫶 Contributors
This release was made possible by the following people:
Michal Čihař
🌈 Translate Toolkit 3.8.3
Released on 18 January 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
JSON- Improved plural handling in
i18next
- Improved plural handling in
stringsdict- Fixed crash on adding new string
🫶 Contributors
This release was made possible by the following people:
Michal Čihař
🌈 Translate Toolkit 3.8.2
Released on 17 January 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
JSON- Fixed crash on specially formatted keys
🫶 Contributors
This release was made possible by the following people:
Michal Čihař
🌈 Translate Toolkit 3.8.1
Released on 13 January 2023 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
JSON- Added support for
gotext - Added support for
i18nextv4 - Various cleanups and refactorings
- Added support for
YAML- Gracefully handle different parse errors
Other 🌸
- Mark Python 3.11 as supported
🫶 Contributors
This release was made possible by the following people:
Michal Čihař, Sam Whited, Erik Gävert
🌈 Translate Toolkit 3.8.1
Released on 8 December 2022 — this release contains improvements and bug fixes.
ok bestie yes the version number really does say 3.8.1 twice in a row, i'm literally just the messenger 💀
✨ Changes
Formats and Converters 🦋
JSON- Fixed serialization for some corner case keys
PO- Wrapping is now compatible with GNU gettext 0.21.1
Fluent- Consolidate error handling to be similar as with other formats
- Add support for extracting placeables
LISA(XLIFF,Android, …)- Allow to customize behavior of empty XML tags
RC- Added support for
MENUEXblock andDISCARDABLEattribute
- Added support for
🫶 Contributors
This release was made possible by the following people:
Michal Čihař, Loïc Leuilliot
🌈 Translate Toolkit 3.7.4
Released on 5 November 2022 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
INI- Make location handling consistent with other monolingual formats.
Other 🌸
- Include doc theme in source distribution.
- Fixed testsuite compatibility with
pytest7.2.0. - Fixed license link in the
README.
🫶 Contributors
This release was made possible by the following people:
Michal Čihař, Rafael Fontenelle
🌈 Translate Toolkit 3.7.3
Released on 7 September 2022 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
- Converters
- Add classified rewriter to spot easily what is not part of the translation file
Android- Gracefully handle getting plurals for source language
RC- Improved resource ID handling
Properties- Avoid parsing empty plural tags as plurals in GWT
🫶 Contributors
This release was made possible by the following people:
Michal Čihař, derula, Pascal F
🌈 Translate Toolkit 3.7.2
Released on 18 July 2022 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
Android- Improved handling of plurals
- General
- Updated CLDR data to version 41
🫶 Contributors
This release was made possible by the following people:
Michal Čihař
🌈 Translate Toolkit 3.7.1
Released on 1 July 2022 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
Android- Fixed handling complex language codes
rc2po- Honor charset passed on the command-line
GWT- Fixed parsing of non-plural strings with
[]in key
- Fixed parsing of non-plural strings with
🫶 Contributors
This release was made possible by the following people:
Michal Čihař, Jon Turney
🌈 Translate Toolkit 3.7.0
Released on 13 June 2022 — this release contains improvements and bug fixes.
✨ Changes
General 🌟
- Rewritten setup to use modern
setuptoolsfeatures - The
sharefolder is now shipped inside thetranslatepackage - The release process now includes publishing Wheel packages
- Documentation was updated to work with Sphinx 5.0
Formats and Converters 🦋
PO- Improved performance on huge plural strings
CSV- Allow to override encoding detection
RC- Fixed parsing comment in
MENU - Fixed parsing of combined constants
- Fixed handling of concatenated strings
- Fixed parsing comment in
Resource Dictionary- Added support for Resource Dictionary format
🫶 Contributors
This release was made possible by the following people:
Michal Čihař, Jon Turney, Michał Górny
🌈 Translate Toolkit 3.6.1
Released on 10 May 2022 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
JSON- Avoid parsing root node
YAML- Improve strings removal
Properties- Fixing backslash problems for iOS strings
RC- Avoid parsing
#pragmastatements from inside strings
- Avoid parsing
🫶 Contributors
This release was made possible by the following people:
Michal Čihař, Nathan Fallet
🌈 Translate Toolkit 3.6.1
Released on 20 April 2022 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
Fluent- Fixed comments parsing
Properties- Improved whitespace handling
- Removed debugging code
RC- Fixed handling of identical source strings with different ID
General 🌟
- GitHub actions cleanups
🫶 Contributors
This release was made possible by the following people:
Michal Čihař, neilnaveen
🌈 Translate Toolkit 3.6.0
Released on 25 February 2022 — this release contains improvements and bug fixes.
✨ Changes
Formats and Converters 🦋
PO- Fixed line wrapping with full width characters
JSON- Improved nested files serialization and parsing
Properties- Improved handling of
[]ingwt
- Improved handling of
XLIFF- Improved element cleanup on update
- Base
- Switch to
charset-normalizerfromchardetfor more reliable detection
- Switch to
General 🌟
- Code cleanups
- Dropped support for Python 3.6
- Added support for Python 3.10
- Cleanup of external links in the documentation
🫶 Contributors
This release was made possible by the following people:
Michal Čihař, Nick Schonning
🩷 Translate Toolkit 3.5.3 🩷
Released on 14 January 2022 — and yeah, this release contains improvements and bug fixes, we love that for us 💖
Changes
Formats and Converters
- PO
- Fixed updating files with DOS end of lines 😭 (the \r\n girlies were NOT behaving)
- Improved error reporting for parse errors — so when it breaks it actually tells you why, iconic
Contributors — this release was made possible by the following people:
Michal Čihař
🦋 Translate Toolkit 3.5.2 🦋
Released on 5 January 2022 — improvements and bug fixes, as always bestie ✨
Changes
Formats and Converters
- YAML
- Fixed adding strings to empty Ruby YAML file
- RC
- Better handling of multiline stringtable translations in
po2rc
- Better handling of multiline stringtable translations in
- Misc
- Update CLDR data to
37.0.0
- Update CLDR data to
Contributors — this release was made possible by the following people:
Michal Čihař, Jon Turney
🎀 Translate Toolkit 3.5.1 🎀
Released on 8 November 2021 — improvements + bug fixes, the usual slay 💅
Changes
Formats and Converters
- RC
- Fixed compatibility with recent
pyparsingreleases
- Fixed compatibility with recent
Contributors — this release was made possible by the following people:
Michal Čihař, ptmcg
🌈 Translate Toolkit 3.5.0 🌈
Released on 27 October 2021 — improvements and bug fixes, you know the drill 🫶
Changes
Formats and Converters
- RC
- Rewritten to
pyparsing3 API
- Rewritten to
- XLIFF
- Use double quotes in XML header to comply with default
xmllint
- Use double quotes in XML header to comply with default
General
- Code cleanups from
pyupgrade
Contributors — this release was made possible by the following people:
Vincent Langlet, Michal Čihař
⭐ Translate Toolkit 3.4.1 ⭐
Released on 10 September 2021 — improvements and bug fixes, periodt 🔥
Changes
Formats and Converters
- Fluent
- Fixed comments handling
- HTML
- Avoid whitespace normalization for
<pre> - Performance improvements 🏎️
- Avoid whitespace normalization for
- RC
- Fixed mixed newlines in generated files
- Fixed handling comments inside a control
- Fixed specific control blocks handling
Contributors — this release was made possible by the following people:
Michal Čihař, Jack Grigg
🧚 Translate Toolkit 3.4.0 🧚
Released on 19 August 2021 — improvements and bug fixes, bestie 💜
Changes
Formats and Converters
- Fluent
- Initial support for Mozilla Fluent format ✨ (new format just dropped)
- Stringsdict
- Initial support for Apple Stringsdict format 🍓
- JSON
- Handle crash when can not detect encoding
- PO
- Fail on syntactically invalid PO files — no cap, it will NOT let that slide
Contributors — this release was made possible by the following people:
Michal Čihař, Jack Grigg, Tim Gates, Lennard Sprong
🌸 Translate Toolkit 3.3.5 🌸
Released on 27 April 2021 — improvements and bug fixes 💖
Changes
Formats and Converters
- Properties
- fixed handling of plurals in GWT properties
Dependencies
lxmlrequirement was raised to4.6.3to addressCVE-2021-28957🔒
Contributors — this release was made possible by the following people:
Michal Čihař
👀 Translate Toolkit 3.3.5 👀
Yes bestie, 3.3.5 shows up twice in the original. Not a typo on my end — it's really listed again, same date, 27 April 2021, and yep: "This release contains improvements and bug fixes." 💀
Changes
Formats and Converters
- PO
- Improve newline char detection
- Properties
- performance improvements
- consolidated API between dialects
- QT ts2
- Consistent using of XML entity for non-breakable space
Contributors — this release was made possible by the following people:
Michal Čihař
🍓 Translate Toolkit 3.3.4 🍓
Released on 2 April 2021 — improvements and bug fixes 🌷
Changes
Formats and Converters
- PO
- Fix newlines round-trip
- Android
- Improve XML handling
- RC
- Fix parsing stringtable with a comment
Contributors — this release was made possible by the following people:
Michal Čihař, Asmo Soinio
💅 Translate Toolkit 3.3.3 💅
Released on 2 March 2021 — improvements and bug fixes, naturally 💁♀️
Changes
Formats and Converters
- PO
- Improve previous msgid parsing
- Improve newline detection
- Cleanup temporary files in case of exception
- YAML
- Do not fail on numeric keys
- Properties
- Handle single quote escaping in GWT
- Do not modify
XWikiPagePropertiessource file
- po2php
- Use
phpfileas a serializer
- Use
- poterminology
- Fix regex escaping to remove
FutureWarning.
- Fix regex escaping to remove
Contributors — this release was made possible by the following people:
Michal Čihař, Simon Urli
🦄 Translate Toolkit 3.3.2 🦄
Released on 5 February 2021 — improvements and bug fixes 🫧
Changes
Formats and Converters
- JSON
- Improved handling of changes in a nested structure
- TBX
- Nicely format generated files
- PHP
- Sanitize new unit key to produce valid PHP syntax
- po2csv/csv2po
- Fixed
–columnorderparameter handling (that's an en dash in the original, character for character, don't come for me 🙈)
- Fixed
General
- Fixed installation of some converters
Contributors — this release was made possible by the following people:
Michal Čihař, Markus Buettgenbach
🌟 Translate Toolkit 3.3.1 🌟
Released on 29 January 2021 — improvements and bug fixes ⭐
Changes
Formats and Converters
- PO
- Improve error handling in CPO parser
- Raise syntax errors from PYPO parser
- TBX
- Implement
setid
- Implement
- JSON
- Fixed serializing after changing id
- RC
- Fixed parsing of UCS-2 RC files
Contributors — this release was made possible by the following people:
Michal Čihař
🎉 Translate Toolkit 3.3.0 🎉
Released on 4 January 2021 — improvements and bug fixes, happy new year bestie 🥳
Changes
Formats and Converters
- HTML
- Add support for single PO file in
po2htmlandhtml2po
- Add support for single PO file in
- MO
- Fixed writing out plurals
- Performance improvements
- Strip
POT-Creation-Datefor reproducible builds
- XLIFF
- Ensure
xml:space="preserve"is set when updating target - Improved parsing performance
- Set language on all file nodes
- Ensure
- CSV
- Complete test for location/context parsing
- JSON
- Improved handling of newly added keys to list
- Fix serializing files with list as root element
- Properties
- Fix redundant new lines
- Subtitles
- Added support for
aeidon 1.9
- Added support for
- RC
- Use
code_page#pragmato detect encoding
- Use
- TMX
- Gracefully handle invalid chars
- Wordfast
- Fixed processing multiple files in single process
General
- Major cleanups
- Removed
statsdb
- Removed
- Cleanup and coding style
- Remove not necessary conversion from bytes
- Apply
pyupgradehints - Remove not needed variables for return
- Use
blackfor code formatting
- Tests
- Improve tests coverage
- Collect coverage reports from functional tests
- Python versions
- Add support for Python 3.9 (no code changes were needed)
Contributors — this release was made possible by the following people:
Michal Čihař, Anders Kaplan, Jean-Baptiste Holcroft, Simon Urli
🌈 Translate Toolkit 3.2.0 🌈
Released on 4 November 2020 — improvements and bug fixes 💫
Changes
Formats and Converters
- JSON
- Fix handling of newly created units for flat JSON format
- YAML
- Handle
TaggedScalartype while parsing
- Handle
- Android
- Fix setting escaped content for XML
- Fix adding units with entities
- Subtitles
- Add basic tests for the formats
General
- Dropped support for Python 3.5. 😭 (sorry to the 3.5 girlies)
- Improved CI and linting
Contributors — this release was made possible by the following people:
Michal Čihař
🐚 Translate Toolkit 3.1.1 🐚
Released on 23 September 2020 — heads up bestie, this one just says "This release contains bug fixes." 🩹
Changes
General
- Fixed comparing units
- Fixed removal of units from a storage
Contributors — this release was made possible by the following people:
Michal Čihař
And to all our bug finders and testers, a Very BIG Thank You. 🫶👑
👑 Translate Toolkit 3.1.0 👑
Released on 21 September 2020 — improvements and bug fixes, main character energy only ✨
Changes
Formats and Converters
- YAML
- Fix generating blank Ruby storage
- Fix changing type of an unit
- Preserve structured IDs on round trip
- Several serialization fixes
- JSON
- Preserve structured IDs on round trip
- Several serialization fixes
- Android
- Fix round trip of strings with newlines
- Fix escaping double space with HTML
- CSV
- Allow to control format autodetection
- XWiki
- Added support for several XWiki formats
- RC
- Fix parsing adjancent strings (yep, "adjancent" — that's the original spelling, keeping it exactly 💀)
- Fix handling empty strings
- po2json
- Never use empty translation in
po2json
- Never use empty translation in
General
- Added support for removing units from a storage
- Cleaned up storage index usage
- Updated several dependencies
- Dropped version control integration from storage
- Use LRU cache for imports to improve factory performance
Contributors — this release was made possible by the following people:
Michal Čihař, Simon Urli, wojtek555, Stuart Prescott,
And to all our bug finders and testers, a Very BIG Thank You. 💖💐
🔥 Translate Toolkit 3.0.0 🔥
Released on 15 June 2020 — big improvements, big bug fixes, big milestone energy 🎊
Changes
Formats and Converters
- PO
- Bring line wrapping closer to
gettext
- Bring line wrapping closer to
- XLIFF
- Support non numeric ids on plurals in
poxliff
- Support non numeric ids on plurals in
- JSON
- Added support for ARB files
- Added support for go-i18n files
- Properties
- Added support for GWT personality
- Fix round trip of empty values without delimiter
- HTML
- A makeover of the HTML parsing to fix several issues 💅✨
- PHP
- Add support for Laravel plurals
- Improve round trip of some statements
- Windows RC
- Rewritten parser using
pyparsing
- Rewritten parser using
- l20n
- Dropped support for deprecated format
General
- Dropped support for Python 2.7. 🕯️ (rest in peace, fr fr)
- Python 3.5 or newer is now required.
- Minor docs improvements.
- Several cleanups in code.
- Removed deprecated interfaces — like, actually gone, don't @ me:
multistringno longer acceptsencodingsearch.segmentis removedpofile.extractpolineis removedsimplify_to_commonno longer acceptslanguagesparametergetsource/setsource/gettarget/settargetmethods removed from storages
- Updated requirements:
lxmlis no longer optional — the requirements got updated, andlxmlis now a required girlie, not an optional extra. 💅 - Added and updated tests — tests were added and updated, because we stay testing. ✨
- Optional deps can be specified using
pip extras— optional deps can now be specified usingpip extras. 🎀
💖 Contributors 💖
This release was made possible by the following people:
Michal Čihař, papeh, Yann Diorcet, Nick Schonning, Anders Kaplan, Leandro Regueiro, Javier Alfonso, Julen Ruiz Aizpuru
And to all our bug finders and testers, a Very BIG Thank You.
🌈 Translate Toolkit 2.5.1 🌈
Released on 25 April 2020.
This release contains improvements and bug fixes.
✨ Changes ✨
🛠️ Formats and Converters 🛠️
PO- Avoid adding extra space on empty comment lines — no extra space on empty comment lines, periodt. 🫶
- Several performance improvements — several performance improvements, we love that for us. 🔥
- Android resources
- Workaround broken plural handling for languages missing
othertag — workaround for broken plural handling when a language is missing theothertag. 🧚 - Fix setting rich content with comments — fixes setting rich content when comments are involved. 💖
- Fix setting target and removing markup — fixes setting target and removing markup. ✨
- Workaround broken plural handling for languages missing
YAML- Fixed handling of empty keys — empty keys are handled now. 🎀
- Improved round trip preserving of comments and style — round trip now preserves comments and style better. 🦋
TS- Avoid duplicating plurals definition — avoids duplicating the plurals definition. 🌸
- Fixed possible crash on adding new translations — a possible crash when adding new translations got fixed. 😭
INI- Now supported on Python 3 thanks to
iniparsesupport for it — now supported on Python 3 thanks toiniparsesupport for it. 💅
- Now supported on Python 3 thanks to
JSON- Allow usage of BOM in JSON files — BOM usage in JSON files is allowed. 🌈
MO- Fixed context parsing — context parsing is fixed. ⭐
- Fixed tests on big endian machines — tests on big endian machines are fixed. 💜
- Catkeys
- The catkeys format now has support for fingerprint calculation — the catkeys format now supports fingerprint calculation. 👑
🌍 Languages 🌍
- Updated plural definitions to match
CLDR 36. — plural definitions were updated to matchCLDR 36. 💅
🌸 General 🌸
- Kept support for Python 2.7. — Python 2.7 support was kept. 🫶
- Fixed
py2exesupport on Python 2.7. —py2exesupport on Python 2.7 got fixed. 🔥 - Minor docs improvements. — minor docs improvements. ✨
- Minor cleanups in code. — minor code cleanups. 🎀
- Updated requirements. — requirements were updated. 🦋
- Added and updated tests. — tests were added and updated. 🌸
💖 Contributors 💖
This release was made possible by the following people:
Michal Čihař, Nick Schonning, Tomáš Chvátal, Niels Sascha Reedijk.
And to all our bug finders and testers, a Very BIG Thank You.
🌈 Translate Toolkit 2.5.0 🌈
Released on 18 January 2020.
This release contains improvements and bug fixes.
✨ Changes ✨
🛠️ Formats and Converters 🛠️
PO- Avoid stripping empty lines from comments. — empty lines are no longer stripped from comments. 🫶
- Raise error on invalid file content. — invalid file content now raises an error. 😭
- Fixed handling typecomments with non word chars. — typecomments with non-word chars are handled now. ✨
- Fixed serializing long msgidcomments. — long msgidcomments serialize correctly now. 🎀
- Properties
- Avoid creating comment only units (issue 3928). — comment only units are avoided now (issue 3928). 🦋
- Fixed saving utf-16 Java files. — saving
utf-16Java files is fixed. 💅
- Android resources
- Document declaration is cloned when adding unit. — the document declaration is cloned when adding a unit. 🌸
- Fixed parsing plurals with comment. — plurals with comments parse correctly now. 🧚
- Fixed setting plural with markup. — setting plural with markup is fixed. 💖
- Fixed indentation of markup in translation. — markup indentation in translation is fixed. ✨
- Fixed XML entities handling. — XML entities handling is fixed. 🌈
YAML- Quotes are preserved. — quotes are preserved. ⭐
TSts2poconverts disambiguation notes and comments. —ts2poconverts disambiguation notes and comments. 💜po2tsno longer removes consecutive linebreaks in source and translation. —po2tsno longer removes consecutive linebreaks in source and translation. 👑
web2py@markminstring is no longer copied to the translation. — the@markminstring is no longer copied to the translation. 🫶
🌍 Languages 🌍
- Added Sicilian language checks — Sicilian language checks were added. 🔥
🌸 General 🌸
- Kept support for Python 2.7. — Python 2.7 support was kept. 🫶
- Added support for Python 3.8. — Python 3.8 support was added. ✨
- Minor docs improvements. — minor docs improvements. 🎀
- Minor cleanups in code. — minor code cleanups. 🦋
- Updated requirements. — requirements were updated. 🌸
- Added and updated tests. — tests were added and updated. 💅
💖 Contributors 💖
This release was made possible by the following people:
Michal Čihař, Leandro Regueiro, Steve Mokris, Queen Vinyl Darkscratch, Matthias, David Paleino.
And to all our bug finders and testers, a Very BIG Thank You.
🌈 Translate Toolkit 2.4.0 🌈
Released on 24 June 2019.
This release contains improvements and bug fixes.
✨ Changes ✨
🛠️ Formats and Converters 🛠️
PO- Allow unicode in PO headers (issue 3896). — unicode is allowed in PO headers (issue 3896). 🫶
- Improve finding out newline format for a file. — finding out the newline format for a file got better. ✨
- Strip UTF-8 BOM from input (issue 1640). — UTF-8 BOM is stripped from input (issue 1640). 🎀
XLIFF- Adjustments on how output is indented (issue 3424). — adjustments were made to how output is indented (issue 3424). 🦋
- Properties
- Do not fail when parsing empty file. — parsing an empty file no longer fails. 🌸
- Android resources
- Multiple adjustments so output is closer to Android Studio’s. — multiple adjustments make output closer to Android Studio’s. 💅
YAML- Switched to
ruamel.yamlto simplify codebase and support YAML 1.2. — switched toruamel.yamlto simplify the codebase and support YAML 1.2. 🔥 - Added support for Ruby plurals. — Ruby plurals are supported now. 💖
- Fixed handling dict in list (issue 3895). — handling a dict in a list is fixed (issue 3895). ✨
- Fixed parsing of empty YAML file. — parsing an empty YAML file is fixed. 🌈
- Switched to
JSON- Fixed serialization of JSON arrays. — JSON array serialization is fixed. ⭐
- Placeholders are now kept in WebExtension dialect round trip conversion. — placeholders are now kept in WebExtension dialect round trip conversion. 💜
RESX- Several improvements on formatting to align with Visual Studio’s output. — several formatting improvements align with Visual Studio’s output. 👑
TS- Improved tags indentation. — tags indentation got improved. 🫶
- Added support for new vanished type. — support for the new vanished type was added. 🔥
- Flat XML
- Added support for this new format including
flatxml2poandpo2flatxmlconverters (issue 3776). — this new format is supported, includingflatxml2poandpo2flatxmlconverters (issue 3776). ✨
- Added support for this new format including
CSV- No longer hardcode escape character (issue 3246). — the escape character is no longer hardcoded (issue 3246). 🎀
- Rewrote default dialect to make it more flexible. — the default dialect was rewritten to be more flexible. 🦋
web2py- Updated converters code. — converters code was updated. 🌸
- Subtitles
- Initialize duration on subtitle unit
__init__. — duration is initialized on subtitle unit__init__. 💅
- Initialize duration on subtitle unit
🧰 Tools 🧰
Tmserver: Fixed execution of unit API on Python 3. —Tmserverfixed execution of the unit API on Python 3. 🔥
🌍 Languages 🌍
- Updated plural definitions to match
CLDR 35.0. — plural definitions were updated to matchCLDR 35.0. 💖 - Removed trailing semicolon in Romanian plural definition. — the trailing semicolon in the Romanian plural definition was removed. ✨
🧩 Placeables 🧩
- Allow any character for Python mapping keys in
PythonFormattingPlaceable. — any character is allowed for Python mapping keys inPythonFormattingPlaceable. 🌈
🔧 API changes 🔧
- Altered storage code to have a consistent API for
removenotes. — storage code was altered to have a consistent API forremovenotes. ⭐ - Removed dependency on
diff-match-patch. — the dependency ondiff-match-patchwas removed. 💜 - Removed embedded CherryPy wsgi server. — the embedded CherryPy wsgi server was removed. 👑
- Removed deprecated
has_keyimplementation. — the deprecatedhas_keyimplementation was removed. 🫶
🌸 General 🌸
- Dropped no longer supported Python 3.3 and Python 3.4. — Python 3.3 and Python 3.4 are no longer supported and were dropped. 🔥
- Minor docs improvements. — minor docs improvements. ✨
- Updated requirements. — requirements were updated. 🎀
- Added and updated tests. — tests were added and updated. 🦋
💖 Contributors 💖
This release was made possible by the following people:
Michal Čihař, Leandro Regueiro, Vinyl Darkscratch, Vitaly Novichkov, Stuart Prescott, Alex Tomkins, Darío Hereñú, BhaaL.
And to all our bug finders and testers, a Very BIG Thank You.
🌈 Translate Toolkit 2.3.1 🌈
Released on 06 October 2018.
This release contains improvements and bug fixes.
✨ Changes ✨
🛠️ Formats and Converters 🛠️
PO- Fixed parsing of files with mixed newlines n and r. — parsing files with mixed newlines n and r is fixed. 🫶
- Properties
- Fixed escaping of serialized string for Joomla dialect. — escaping of the serialized string for the Joomla dialect is fixed. ✨
- Fixed loading of OS X strings dialect files having multiline strings. — loading OS X strings dialect files with multiline strings is fixed. 🎀
- Android resources
- Correctly handle escaping of question mark. — escaping of the question mark is handled correctly. 🦋
PHP- Improved handling of unit name. — unit name handling got improved. 🌸
- Handle
[]style arrays. —[]style arrays are handled. 💅 - Added support for
returnstatement. — support for thereturnstatement was added. 🔥
YAML- Consistent handling of
intandbool. —intandboolare handled consistently. 💖 - Fixed serialization of empty keys. — serialization of empty keys is fixed. ✨
- Consistent handling of
JSON- Nested values ordering is now preserved. — nested values ordering is now preserved. 🌈
TMX- Avoid mentioning
po2tmxin creation tool. —po2tmxis avoided in the creation tool. ⭐
- Avoid mentioning
RESX- New unit elements are now properly indented. — new unit elements are now properly indented. 💜
INI- Enabled support for
Python3provided that patchediniparselibrary is available. —Python3support is enabled if the patchediniparselibrary is available. 👑
- Enabled support for
RC- Altered to remove
rbefore parsing. — altered to removerbefore parsing. 🫶
- Altered to remove
🔧 API changes 🔧
- Use
backports.csvmodule on Python 2 to align the behavior with Python 3 and drop many hacks. — use thebackports.csvmodule on Python 2 to align behavior with Python 3 and drop many hacks. 🔥 - Removed deprecated
getoutputmethods deprecated in version 2.0.0. — deprecatedgetoutputmethods from version 2.0.0 were removed. ✨ - Added new deprecations:
- Deprecated
setsource,getsource,gettargetandsettargetmethods in favor ofsourceandtargetproperties for all storage classes, exceptLISAunitand its subclasses since for those these methods do actually accept additional arguments so can’t just be always replaced by some property. 🎀 xliffunit: Deprecatedget_rich_sourcein favor ofrich_sourceproperty. 🦋
- Deprecated
- Defined for all unit classes the
rich_sourceandrich_targetproperties without using methods. Since the old methods were private they were directly removed without deprecating them. 🌸
🌸 General 🌸
- Refactored more converters to increase readability and use a common pattern which will allow to further refactor repeated code. — more converters were refactored for readability and to use a common pattern that will allow further refactoring of repeated code. 💅
- Code cleanups and simplification. — code cleanups and simplification. 🔥
- Updated requirements. — requirements were updated. 💖
- Tests:
- Added plenty of tests. — plenty of tests were added. ✨
- Tests cleanups and fixes. — tests got cleanups and fixes. 🌈
💖 Contributors 💖
This release was made possible by the following people:
Leandro Regueiro, Michal Čihař, BhaaL, Mark Jansen, Stuart Prescott, David Hontecillas.
And to all our bug finders and testers, a Very BIG Thank You.
🌈 Translate Toolkit 2.3.0 🌈
Released on 18 March 2018.
This release contains improvements and bug fixes.
✨ Changes ✨
🛠️ Formats and Converters 🛠️
PO- Avoid escaping some characters
()/:,that don’t need to be escaped. — some characters()/:,that don’t need escaping are no longer escaped. 🫶 - Wrap lines on
/like Gettext. — lines are wrapped on/like Gettext. ✨ - Lines can be wrapped at specified length. — lines can be wrapped at a specified length. 🎀
MOunits now allow to set the unit context. —MOunits now allow setting the unit context. 🦋- Always URL-encode locations. — locations are always URL-encoded. 🌸
- Avoid escaping some characters
PHP- Full rewrite of the PHP format using
phply:- Fixes multiple issues. — fixes multiple issues. 💅
- Brings support for new dialects: heredoc, nowdoc, short array syntax and nested arrays. — brings support for new dialects: heredoc, nowdoc, short array syntax and nested arrays. 🔥
- Full rewrite of the PHP format using
YAML- Added
yaml2poandpo2yamlconverters. —yaml2poandpo2yamlconverters were added. 💖 - Fixed some minor bugs. — some minor bugs were fixed. ✨
- Documented unsupported features. — unsupported features were documented. 🌈
- Added
JSON- Refactored the storage classes to get rid of repeated shared code, reduce memory usage and get readable representation of the units. — storage classes were refactored to get rid of repeated shared code, reduce memory usage, and get a readable representation of the units. ⭐
txt- Added
--no-segmentationflag totxt2po. — the--no-segmentationflag was added totxt2po. 💜
- Added
🧰 Tools 🧰
- Removed
translate.convert.poreplace. —translate.convert.poreplacewas removed. 👑
🌸 General 🌸
- Improved support for Windows. — Windows support got improved. 🫶
- Refactored multiple converters to increase readability and use a common pattern which will allow to further refactor repeated code. — multiple converters were refactored for readability and to use a common pattern that will allow further refactoring of repeated code. 🔥
- Tests:
- Enabled testing on Windows. — testing on Windows was enabled. ✨
- Added more tests. — more tests were added. 🎀
- Plenty of tests cleanups and fixes. — plenty of tests cleanups and fixes. 🦋
- Docs:
- Updated docs on installation. — docs on installation were updated. 🌸
- Improved automatic generation of docs on factories. — automatic generation of docs on factories got improved. 💅
💖 Contributors 💖
This release was made possible by the following people:
Leandro Regueiro, Michal Čihař, Stuart Prescott, Nick Schonning, Johannes Marbach, andreistefan87, Alejandro Mantecon Guillen.
And to all our bug finders and testers, a Very BIG Thank You.
🌈 Translate Toolkit 2.2.5 🌈
Released on 01 September 2017.
This release contains improvements and bug fixes.
✨ Changes ✨
🛠️ Formats and Converters 🛠️
XLIFF- Fixed bug when adding new units to XLIFF store. — a bug when adding new units to an XLIFF store was fixed. 🫶
JSON- Added support for i18next JSON dialect. — i18next JSON dialect support was added. ✨
- Improved WebExtension JSON dialect support. — WebExtension JSON dialect support got improved. 🎀
💖 Contributors 💖
This release was made possible by the following people:
Michal Čihař, Leandro Regueiro, Ryan Northey.
And to all our bug finders and testers, a Very BIG Thank You.
🌈 Translate Toolkit 2.2.4 🌈
Released on 31 July 2017.
This release contains improvements and bug fixes.
✨ Changes ✨
🛠️ Formats and Converters 🛠️
XLIFF- Added support for
.xliffextension in all converters and tools that support.xlfextension. — support for the.xliffextension was added in all converters and tools that support the.xlfextension. 🫶
- Added support for
JSON- Added support for nested JSON. — nested JSON is supported now. ✨
- Added support for WebExtension JSON dialect.
txtpo2txtskips obsolete and non-translatable strings.
🧚 Filters and Checks
- The
puncspacecheck now strips Bidi markers chars before processing. - Added
ReducedCheckerchecker to list of checkers.
🔥 API changes
- Language and country default to
common_nameif available. - Added function to retrieve all language classes.
💖 Contributors
This release was made possible by the following people:
Dwayne Bailey, Leandro Regueiro, Michal Čihař, Rimas Kudelis, Ludwig Nussel, Stuart Prescott.
And to all our bug finders and testers, a Very BIG Thank You.
🌈 Translate Toolkit 2.2.3
Released on 20 June 2017
This release contains improvements and bug fixes.
✨ Changes
- Added
MinimalCheckerandReducedCheckercheckers.
🫶 Contributors
This release was made possible by the following people:
Rimas Kudelis, Leandro Regueiro.
And to all our bug finders and testers, a Very BIG Thank You.
🎀 Translate Toolkit 2.2.2
Released on 20 June 2017
This release contains improvements and bug fixes.
✨ Changes
- Fixed resolving of country names translations.
🫶 Contributors
This release was made possible by the following people:
Ryan Northey, Leandro Regueiro.
And to all our bug finders and testers, a Very BIG Thank You.
🦋 Translate Toolkit 2.2.1
Released on 19 June 2017
This release contains many improvements and bug fixes.
👑 Major changes
- Refactored functions for resolving language/country names translation to be memory efficient.
- Improvements for
tsand subtitles formats. - Added
--preserveplaceholdersargument topodebug. - Fixed Montenegrin language name.
💜 Detailed changes
Formats and Converters
ts- Write quotes as entities
- Remove not necessary encoding/decoding to UTF-8
- Subtitles
- Avoid errors when subtitle support is missing
Tools
- Added
--preserveplaceholdersargument topodebugto avoid rewriting placeholders
Languages
- Fixed Montenegrin language name.
API changes
- Refactored functions for resolving language/country names translation to be memory efficient
General
- Use gzip for packaging
- Python 3 fixes
- Added more tests
…and loads of general code cleanups and of course many many bugfixes.
🌸 Contributors
This release was made possible by the following people:
Michal Čihař, Leandro Regueiro, Ryan Northey, Robbie Cole, Kai Pastor, Dwayne Bailey.
And to all our bug finders and testers, a Very BIG Thank You.
⭐ Translate Toolkit 2.2.0
Released on 15 June 2017
This release contains many improvements and bug fixes.
👑 Major changes
- Avoid resolving external entities while parsing XML.
- Improvements for Android,
tsandresxformats. - Added support for PHP nested arrays.
- Added Kabyle language
💜 Detailed changes
Requirements
- Updated requirements.
- Added
pycountryrecommended requirement for localized language names.
Formats and Converters
- XML formats
- Avoid resolving external entities while parsing.
- Properties
- Improved behavior for strings with no value.
- Android resources
- Improved newlines handling.
- Strip leading and trailing whitespace.
- PHP
- Added support for nested named arrays and nested unnamed arrays.
ts- Handle gracefully empty location tag.
- Encode
po2tsoutput as UTF-8.
resx- Improved skeleton.
- Fixed indent of the
</data>elements.
Languages
- Added Kabyle language.
API changes
- Added functions to retrieve language and country ISO names.
- If available,
pycountryis used first to get language names translations.
General
- Python 3 fixes
- Added more tests
…and loads of general code cleanups and of course many many bugfixes.
💖 Contributors
This release was made possible by the following people:
Dwayne Bailey, Michal Čihař, Taras Semenenko, Leandro Regueiro, Rimas Kudelis, BhaaL, Muḥend Belqasem, Jens Petersen.
And to all our bug finders and testers, a Very BIG Thank You.
🌺 Translate Toolkit 2.1.0
Released on 17 March 2017
This release contains many improvements and bug fixes.
👑 Major changes
- Fixed
RomanianCheckerchecks. - Added an iOS checker style.
- Changed plural equations for Slovenian, Persian, Kazakh and Kyrgyz.
- Several fixes in formats and tools.
💜 Detailed changes
Python 3 support
- Python 3.6 is now supported.
Requirements
- Updated and pinned requirements.
- Now recommended requirements pulls required requirements.
Formats and Converters
- All formats
locationindexnow uses first duplicate unit in case of several units having the same location in order to keep duplicate entries in some formats when converting from PO format.
- PO
- Only add duplicate unit if
msgcxtis unique, in order to be able to convert monolingual formats with duplicate entries to PO.
- Only add duplicate unit if
- Properties
- Added support for Joomla dialect.
ts- Set the right context on the units.
- YAML
- Fixed parsing of unicode values in lists.
- HTML
- Use character offset in line for unit location in order to keep parsing repeated strings in different units.
txt- Use line number on unit location to keep parsing repeated strings in different units.
Filters and Checks
- Fixed
RomanianCheckerchecks. - Added an iOS checker style to detect iOS variables styles such as
%@and$(VAR).
Tools
posegmentno longer outputs duplicate headers,
Languages
- Changed plural equations for Slovenian, Persian, Kazakh and Kyrgyz.
API changes
- Changed management of Xapian locks to prevent database corruption.
General
- Python 3 fixes
- Removed unused code
…and loads of general code cleanups and of course many many bugfixes.
🫶 Contributors
This release was made possible by the following people:
Dwayne Bailey, Leandro Regueiro, Michal Čihař, Ryan Northey, Friedel Wolff, Olly Betts, Claude Paroz.
And to all our bug finders and testers, a Very BIG Thank You.
🚀 Translate Toolkit 2.0.0
Released on 27 January 2017
This release contains many improvements and bug fixes. While it contains many general improvements, it also specifically contains needed changes and optimizations for the upcoming Pootle 2.8.0 and Virtaal releases.
👑 Major changes
- Python 3 compatibility thanks to Claude Paroz
- Dropped support for Python 2.6
- Support for new
l20nformat - Translate Toolkit can now easily be installed on Windows
- Changes in storage API to expose a more standardized API
💜 Detailed changes
Python 3 support
- Translate Toolkit went through a massive code cleanup looking forward Python 3 compatibility. There might be quirks that need to be fixed, so please test and report any issue you might find.
- Python 3.3-3.5 is now supported.
Requirements
lxmlrequirement was raised to 3.5.0 in order to simplify code.- Updated and pinned requirements
- Removed misleading extra requirements files
Formats and Converters
- PO
msgidcomments (KDE style) are only parsed frommsgidnow.- Fixed parsing of PO files with first entry in unicode
- Fixed parsing of locations with percent char
- XLIFF
- Unaccepted ASCII control characters are now escaped in XLIFF
- DTD
- Newlines are now skipped when parsing (issue 3390).
- Invalid ampersands are not scrubbed anymore.
label+accesskeyis now only extracted if it is not followed by space.
- Properties
- Keys can contain delimiters if they are properly wrapped (issue 3275).
- Fix control characters escaping for utf-8 encoding.
po2propremoves fully untranslated units if requiredpo2propskips first entry in PO file (issue 3463)
- Mozilla
.lang{ok}marker is now more cleanly removed- Always output last unit followed by trailing newlines
- Added support for headers and tag comments
MAX_LENGTHis now parsed into comment- File line endings are now remembered, defaulting to Unix LF
- Mozilla’s
l20n- Added this new format storage class
- Added variants and traits support
- Added new converters
l20n2poandpo2l20n
- Android
- Unknown locales no longer produce failures.
- Simplify newlines handling as the format now handles
nand newline equally (issue 3262) - Moved all namespaces to
<resources>element. - Simplified newlines handling
- ODF
odf2xliffnow extracts all the text (issue 3239).
ts- XML declaration is written with double quotes.
- Self-closing for ‘location’ elements are not output anymore.
- JSON
- Output now includes a trailing newline.
- Unit ordering is maintained (issue 3394).
- Added
--removeuntranslatedoption topo2json
- YAML
- YAML format support has been added.
txtpo2txtworks correctly again when--thresholdoption is passed (issue 3499)
ical- Enabled this format for Python 3 too.
- TermBase eXchange (TBX)
tbx2poconverter added- Added basic support for Parts of Speech and term definitions.
- Fixed error when writing back to the same file (issue 3419).
Filters and Checks
- Added the ability to skip some checks for some languages in specific checkers
- accelerators check reports an error if accelerator is present for several Indic languages in
MozillaCheckerchecker. - Added
l20nCheckerto do custom checking for Mozilla’s newl20nformat. - LibreOffice checker no longer checks for Python brace format (issue 3303).
- LibreOffice
validxmlcheck correctly matches self-closing tags. - Numbers check now handles non latin numbers. Support for non latin numbers has been added for Arabic, Assamese, Bengali and Persian languages.
- Fixed issue that prevented standard checks from being used in Pootle with default settings.
- Fixed missing attribute warning displayed when using
GnomeChecker,LibreOfficeCheckerandMozillaCheckercheckers. - Added language specific
RomanianChecker.
Tools
posegmentnow correctly segments Japanese strings with half width punctuation sign (issue 3280).pocountnow outputs csv header in one line. It also outputs using color.buildxpiwas adjusted to current Mozilla needs
🌸 Languages 🌸
Okay bestie, it’s language glow-up time 💖 here’s the tea:
- Fixed plural form for Montenegro, Macedonian, Songhay, Tajik, Slovenian and Turkish.
- Added plural forms for Bengali (Bangladesh), Konkani, Kashmiri, Sanskrit, Silesian and Yue (Cantonese).
- Added valid accelerators for Polish.
- Renamed Oriya to Odia.
- Altered Manipuri name to include its most common name Meithei.
- Added language settings for Brazilian Portuguese.
- Added Danish valid accelerators characters (issue 3487).
- Added additional special characters for Scottish Gaelic.
🛠️ Setup 🛠️
- Fixed Inno Setup builds allowing to easily install Translate Toolkit on Windows using the pip installer. Commands are compiled to
.exefiles. - Updated installation instructions for Windows
✨ API changes ✨
- Dropped
translate.misc.dictutils.ordereddictin favor ofcollections.OrderedDict. - Added encoding handling in base
TranslationStoreclass exposing a single API. - Encoding detection in
TranslationStorehas been improved. - Standardized
UnitClassdefinition acrossTranslationStoresubclasses. translate.misc.multistring.multistring:- Fixed list coercion to text
- Fixed comparison regression with multistrings (issue 3404).
- Re-added
strmethod (issue 3428). - Fixed
__hash__(issue 3434).
⚠️ API deprecation ⚠️
- Passing non-ASCII bytes to the
multistringclass has been deprecated, as well as theencodingargument to it. Applications should always constructmultistringobjects by passing characters (unicodein Python 2,strin Python 3), not bytes. Support for passing non-ASCII bytes will be removed in the next version. TxtFile.getoutput()anddtdfile.getoutput()have been deprecated. Either callbytes(<file_instance>)or use thefile_instance.serialize()API if you need to get the serialized store content of aTxtFileordtdfileinstance.
🧹 General 🧹
- Dropped support for Python 2.6 since it is no longer supported by the Python Foundation. Sticking to it was making us difficult to maintain code while we move to Python 3.
- Misc docs cleanups.
- Added more tests.
- Increased Python code health.
- Legacy, deprecated and unused code cleansing:
- Dropped code for no longer supported Python versions.
- Removed unused code from various places across codebase.
- The legacy
translate.search.indexing.PyLuceneIndexer1was removed. - The deprecated
translate.storage.properties.find_delimiter()was removed and replaced by thetranslate.storage.properties.Dialect.find_delimiter()class method. - Python scripts are now available via
console_scriptsentry point, thus allowing to drop dummy files for exposing the scripts.
…and loads of general code cleanups and of course many many bugfixes.
💖 Contributors 💖
This release was made possible by the following people:
Claude Paroz, Leandro Regueiro, Dwayne Bailey, Michal Čihař, Taras Semenenko, Ryan Northey, Stuart Prescott, Kai Pastor, Julen Ruiz Aizpuru, Friedel Wolff, Hiroshi Miura, Thorbjørn Lindeijer, Melvi Ts, Jobava, Jerome Leclanche, Jakub Wilk, Adhika Setya Pramudita, Zibi Braniecki, Zdenek Juran, Yann Diorcet, Nick Shaforostoff, Jaka Kranjc, Christian Lohmaier, beernarrd.
And to all our bug finders and testers, a Very BIG Thank You.
🌈 Translate Toolkit 1.13.0 🌈
Released on 13 May 2015
This release contains many improvements and bug fixes. While it contains many general improvements, it also specifically contains needed changes for the upcoming Pootle 2.6.0 and 2.7.0 and Virtaal releases.
It is just over 6 months since the last release and there are many improvements across the board. A number of people contributed to this release and we’ve tried to credit them wherever possible (sorry if somehow we missed you).
✨ Highlighted improvements ✨
Major changes
- New converters for IDML format
- Support for new .Net Resource (.resx) format
- Extensive cleanup on ODF converters
- New quality checks
🧚 Formats and Converters 🧚
- IDML
- Added the
idml2poandpo2idmlconverters.
- Added the
.Net Resource (.resx)- Added store to represent the format and the
resx2poandpo2resxconverters.
- Added store to represent the format and the
- Android
- Improved escape and unescape of Android resources with HTML markup.
- Fixed bug in canceling whitespaces with backslash when unescaping.
- ODF
- Removed the
--engineoption inodf2xliffbecause the itools third party library is no longer used.
- Removed the
- TS
- Pretty print output the same as Qt Linguist (issue 1420)
- JSON
- Dump content on memory instead of copy of parsed file (issue 3249).
- PHP
- Correctly roundtrip PHP with spaces after array (issue 3231).
- Mozilla lang
- Import only real comments (starting with
#), not meta tags (starting with##).
- Import only real comments (starting with
- XLIFF
- Mark units as needing attention if sources don’t match when merging units.
pot2ponow also accepts files with.xliffextension
po2moz- Fixed handling of files with fullstop in filename
🌟 Quality Checks 🌟
- Added quality check for Python brace format.
- Added the ability to skip some quality checks for the
he,ug,zh_CN,zh_HKandzh_TWlanguages. - Expanded printf quality check to support reordering
boost::formatpositional directives. - Expanded docstrings to include fully detailed descriptions in order to display them on Pootle.
🛠️ Tools 🛠️
- Removed the unnecessary dependency on
lxmlinpretranslate(issue 1909)
🌸 Languages 🌸
- Language plurals:
- Fixed plural forms for
gaandpt_BRlanguages - Added new plural forms for new languages
- Fixed plural forms for
- Adjusted punctuation for
zh - Corrected “Songhay” language name
🧹 General 🧹
- Fixed bug in file discovery that prevented Pootle Pootle’s terminology feature from working properly in some scenarios.
- Docs:
- Major rewrite of releasing instructions
- Reorganized string-related guidelines on styleguide
- Other minor docs cleanups
- ODF code extensive cleanups:
- Applied tons of PEP8 and style guide cleanups
- Removed unused code
- Removed unused test ODT file
- Added lots of docstrings
- Simplified code to ease maintainability and improve readability
- Dropped no longer working automatic publishing in PyPI and SourceForge
- Several changes to speed up Travis builds
- Unhid some tests
…and loads of general code cleanups and of course many many bugfixes.
💖 Contributors 💖
This release was made possible by the following people:
Leandro Regueiro, Dwayne Bailey, Yaron Shahrabani, Sarah Hale, Sietse Brouwer, Jerome Leclanche, Julen Ruiz Aizpuru, Michael Andres, William Grzybowski, SirAnthony, Rafael Ferreira, Luka Kama, Francesco Lodolo, Buganini, babycaseny.
And to all our bug finders and testers, a Very BIG Thank You.
💫 Translate Toolkit 1.12.0 💫
Released on 12 August 2014
This release contains many improvements and bug fixes. While it contains many general improvements, it also specifically contains needed changes and optimizations for the upcoming Pootle 2.6.0 and Virtaal releases.
It is just over 6 months since the last release and there are many improvements across the board. A number of people contributed to this release and we’ve tried to credit them wherever possible (sorry if somehow we missed you).
✨ Highlighted improvements ✨
Major changes
- Properties and DTD formats fix a number of issues
- Massive code cleanup looking forward Python 3 compatibility
- Important changes in development process to ease testing
🧚 Formats and Converters 🧚
- Mozilla properties
- If a unit has an associated access key entry then these are combined into a single unit
- Encoding errors are now reported early to prevent them being masked by subsequent errors
- Leading and trailing spaces are escaped in order to avoid losing them when using the converters
- The
\uNNcharacters are now properly handled po2propNow uses the source language accesskey if translation is missing- Fixed conversion of successive Gaia plural units in
prop2po
- DTD
- The
&entity is automatically expanded when reading DTD files, and escaped back when writing them - Underscore character is now a valid character in entity names
- Nonentities at end of string are now correctly handled
po2dtd- Now uses the source language accesskey if target accesskey is missing
- Doesn’t remove stray
&as they probably&
- The
- HTML
- The HTML5
figcaptiontag is now localizable - The
titleattribute is now localizable po2htmlnow retains the untranslated attributes
- The HTML5
- Accesskeys
- Now accesskeys are combined using the correct case
- Added support for accesskey after ampersand and space
- PHP
- Fall back to default dialect after adding every new unit
- Added support for empty array declaration when it is filled later
- Android
- Added support for plurals
- Text is now properly escaped when using markup
- Qt Linguist (.ts)
- The message id attribute is added to
contextname - Files now output the XML declaration (issue 3198)
- The message id attribute is added to
- RC
- RC format received some bugfixes and now ignores
TEXTINCLUDEsections and one line comments (//)
- RC format received some bugfixes and now ignores
- XLIFF
xliff2ponow supports files with.xliffextension
- OS X .strings
- Added support for UTF-8 encoded OS X strings
- Testing
- Added new tests for the UTF-8 encoded OS X strings, Qt linguist and RC formats and the
rc2poconverter
- Added new tests for the UTF-8 encoded OS X strings, Qt linguist and RC formats and the
🔄 Version Control improvements 🔄
- Added support for Subversion
.svndirectories
✅ Checks ✅
- Added specific checks for LibreOffice
🛠️ Tools 🛠️
- The
pocounttool has now a better counting algorithm for things that look like XML
🦊 Mozilla tooling fixes 🦊
- Added support to check for bad accesskeys in
.propertiesfiles - Now the Mozilla roundtrip script can be silently run
- Added a new Gaia roundtrip script
- The
buildxpi--disable-compile-environmentoption has been restored, resulting in huge speed improvements
🧹 General 🧹
- Extensive cleanup of setup script
- Some bugfixes for placeables
- Misc docs cleanups
- Recovered
diff-match-patchto provide support for old third party consumers - Minor change in placeables to correctly insert at an existing parent if appropriate
- Code cleanups:
- Applied tons of PEP8 and style guide cleanups
- Python 2.6 is our new minimum:
- Updated some third party bundled software:
CherryPy,BeautifulSoup4- Added document to track licenses used by third party bundled code
- Removed TODO items. Some of them were moved to the bug tracker
- Development process:
- Added a functional test framework
- Added dozens of new unit and functional tests
- Expanded the tasks performed in Travis:
pep8,pytest-xdist, compile all files,coveralls.io, …
…and loads of general code cleanups and of course many many bugfixes. 💅
💖 Contributors
This release was made possible by the following people, and honestly? icons, every single one:
Dwayne Bailey, Jerome Leclanche, Leandro Regueiro, Khaled Hosny, Javier Alfonso, Friedel Wolff, Michal Čihař, Heiki Ojasild, Julen Ruiz Aizpuru, Florian Preinstorfer, damian.golda, Zolnai Tamás, Vladimir Rusinov, Stuart Prescott, Luca De Petrillo, Kevin KIN-FOO, Henrik Saari, Dominic König.
And to all our bug finders and testers, a Very BIG Thank You. 🫶 No cap, y'all are the main characters here.
🎀 Translate Toolkit 1.11.0
Released on 22 January 2014
This release contains many improvements and bug fixes. While it contains many general improvements, it also specifically contains needed changes and optimizations for the upcoming Pootle 2.5.1 and Virtaal releases.
It is just over a ten months since the last release and there are many improvements across the board. A number of people contributed to this release and we've tried to credit them wherever possible (sorry if somehow we missed you 🙈).
✨ Highlighted improvements
🔥 Major changes
- The PO format now matches Gettext more closely
- PHP format adds a number of new features
- Support for Python 2.5 has been dropped
🦋 Formats and Converters
- Gettext PO:
cPOnow handles fuzzy obsolete messages correctly- Line wrapping improvement allow PO files to more closely match Gettext
- Optimization to increase performance
- PHP:
- Warn about duplicate entries
- Allow blank spaces in array declaration (issue 2646)
- Support nested arrays (issue 2240)
- XLIFF:
- Correctly parse XLIFF 1.2
- Properties
- Blank source text is now always translated
- Fuzzy units are discarded with
--remove-untranslated prop2pono longer drops entries that are translated the same as the source
- TMX:
po2tmxsupport comments
- Android:
- Detect untranslatable resource strings
- Various format improvements
- HTML:
- Output HTML source in
po2htmlwhen a unit is fuzzy (issue 3145)
- Output HTML source in
- New conversion options:
--timestamp– skip conversion if the output file has a newer timestamp (Makefile-alike)--threshold– inpo2*converters this allows you to specify a percentage complete threshold. If the PO files passes this threshold then the file is output (issue 2998)--removeuntranslated– Extend this option topo2dtdand thuspo2moz– don't output untranslated text (issue 1718)
🌸 Language specific fixes
- The toolkit now supports: Sakha, N'ko, Turkish, improvements for Bengali & Hindi
- Pootle special characters are now stored on Toolkit and available for other tools to use
- Rules for language
abare now available for languageab_CD
🔍 Checks
- Spelling test improvements including speed and optimization
- Reduce false positive for the filepath test in cases of self closing tags e.g.
<br /> - Lowered the accelerator check severity to reduce false positive impact
🦊 Mozilla tooling fixes
- Better decoding of some characters in DTD e.g
»and&x0022(") .lang– Improved support for untranslated entriesbuildxpi:- Can now build multiple languages at once (issue 2999)
- Set a max product version to allow the language pack to continue to work once the browser version has moved out of Aurora channel
- Dropped native XPI building support (untested and no longer used)
- Add Mozilla plural formulas, in time we'll handle Mozilla plurals correctly
⭐ General
- Dropped support for Python 2.5 since it is no longer supported by the Python Foundation. Also sticking to it was preventing us from using features that are not supported on Python 2.5 but they are on later versions.
- Dropped psyco support – it is no longer maintained
- Use logging throughout instead of
sys.stderr - Lots of cleanups on docs: TBX, PHP, added Android and JSON docs
- Use requirements files for documenting all requirements and make it easy to install Translate Toolkit using
pip - Added some functional tests
- Improve searching to find words with hyphens
- Choose the closest repo in nested VCS
- Test suite down to zero failing tests
- Handle a broken version of
python-Levenshtein - Improve handling of escapes in wrapping
…and loads of internal changes to improve maintainability, remove unused imports, remove unused code and general code cleanups, some changes to ensure future Python 3 portability and of course many many bugfixes. 💀 (the good kind of 💀)
💖 Contributors
This release was made possible by the following people:
Dwayne Bailey, Leandro Regueiro, Alexander Dupuy, Friedel Wolff, Khaled Hosny, Michal Čihař, Jordi Mas, Stuart Prescott, Trung Ngo, Ronald Sterckx, Rail Aliev, Michael Schlenker, Martin-Zack Mekkaoui, Iskren Chernev, Luiz Fernando Ranghetti & Christian Hitz
And to all our bug finders and testers, a Very BIG Thank You. 🫶
🧚 Translate Toolkit 1.10.0
Released on 12 March 2013
This release contains many improvements and bug fixes. While it contains many general improvements, it also specifically contains needed changes for the upcoming Pootle 2.5.0.
It is just over a year since the last release so there are many improvements across the board. A number of people contributed to this release and we've tried to credit them wherever possible (sorry if somehow we missed you 🙈).
✨ Highlighted improvements
- Android format support
- Version control improvements
- Source now on Github - all our code is now on github
- Documentation - migrated all from our wiki into the code and Read The Docs
- Continuous Integration using Travis
👑 Most important for Pootle
- Version control improvements
- Categorize
pofilterchecks into critical, functional, cosmetic, etc
🦋 Formats and Converters
- Android format support [Michal Čihař]
- Mozilla
.lang, many improvements - PHP support for defintions,
//comments and improved whitespace preservation - PO:
X-Merge-Onheader to explicitly demand a conversion strategy instead of guessing .properties: BOMs in messages and C style comments [Roman Imankulov]- Mac OS String formatting improved [Roman Imankulov]
- The spaces in DTD files are now preserved. For example the spaces in
<!ENTITY some.label "definition">around the entity namesome.labelare now kept. - The matching criterion when merging units can now be specified with the
X-Merge-Onheader. Available values for this header arelocationandid. By default merges will be done by matching IDs. This supersedes the effects of theX-Acceleratorheader when merging and establishes an explicit way to set the desired matching criterion.
🌈 Version Control improvements
- Interface for adding files to a repository & Implement
.add()for all VCSs. - Caching of VC version info
- Don't look for VCS if it's not available
- Stop looking for VCS at a given parent
- Subversion VC tests
- Alway pass
-mto 'commit' in Subversion to prevent blocking
🔍 Checks
- New OpenOffice variables style used in extensions
- Check for self-closing tags in the xmltags test [Seb M].
- GConf test fixes
- Terminology checker type for future terminology features
- Categorize
pofilterchecks into critical, functional, cosmetic, etc - Added support for Objective-C
%@printf specifiers
🌸 Language specific fixes
- Correct plurals: Scottish Gaelic (
gd), Irish - Plural rules: Fulah, Brazilian Portuguese
- Punctuation rules and tests to ignore for: Burmese, Urdu, Afrikaans, Wolof
📚 Documentation
- Moved to Git and we are now using reStructured Text and Sphinx
- Published in Read The Docs (RTD).
- Old wiki migrated to RTD.
- New clean theme for documentation and website
- API and code epydoc moved to reStructured Text.
- Translate code Style Guide written
🦊 Mozilla tooling fixes
- Mozilla specific test for dialog size settings
- Gaia properties dialect and plural handling
- Fixes and improvement to the Firefox build scripts
- Improved accesskey detection
- Improved DTD escaping for
"e,%, etc - Improvement of DTD to align with Base classes
- Support new
{{xx}}variable style introduced in PDF viewer
…and refactoring, PEP8, test coverage and of course many many bugfixes. ✨
💖 Contributors
This release was made possible by the following people:
Dwayne Bailey, Friedel Wolff, Leandro Regueiro, Julen Ruiz Aizpuru, Michal Čihař, Roman Imankulov, Alexander Dupuy, Frank Tetzel, Luiz Fernando Ranghetti, Laurette Pretorius, Jiro Matsuzawa, Henrik Saari, Luca De Petrillo, Khaled Hosny, Dave Dash & Chris Oelmueller.
And to all our bug finders and testers, a Very BIG Thank You. 🫶
🍓 Translate Toolkit 1.9.0 Released
Released on 12 April 2011
This release contains many improvements and bug fixes. While it contains many general improvements, it also specifically contains needed changes for the upcoming Pootle 2.1.6 and Virtaal 0.7.
✨ Highlighted improvements
- Faster terminology matching
- Several small optimisations to performance and memory use
- More advanced state support (visible in
pocountand Virtaal 0.7) - Improved language detection models (+South African languages)
- Improve handling of printf variable reordering [Jacques Beaurain]
- Review of the wording of the messages of
pofilterchecks - Better sentence segmentation for some non-Latin languages
- More supported formats for
podebug - Extra options for
pomerge,pogrepandpo2oo/xliff2oo.
The new pogrep options made this possible for GNOME. 🎉
👑 Most relevant for Pootle
- Support for Xapian 1.2 (issue 1766) [Rimas Kudelis]
- Work around some changes introduced in Django 1.2.5/1.3
🦋 Format support
- Always use UNIX line endings for PO (even on Windows)
XLIFFand.tsfiles now show “fuzzy” only when the target is present 💅- Improved support for
.tscomment as context (issue 1739) ✨ - Support for Java properties in UTF-8 encoding 🦋
- More natural string ordering in
jsonconverter 🎀 - Improved handling of trailing spaces in Mozilla DTD files 🌸
- Removed unused support for _old_ KDE plurals in
pocount🧚 - …and several small bugfixes 💖
🌸 Translate Toolkit 1.8.1 🌸
Released on 19 Novermber 2010. (Yes, that spelling is in the original notes — preserving it 💀)
Okay so today the Translate team dropped version 1.8.1 of the Translate Toolkit 🫶. The Translate Toolkit is literally packed with useful tools for translation, management, and quality control. It’s the technology platform for Pootle, Virtaal, and other software.
This release contains many improvements and bug fixes. It’s a recommended upgrade for users of Pootle and Virtaal. There were over 200 commits since version 1.8.0 — no cap 🔥.
This work was made possible by volunteers and our funders:
- ANLoc, funded by IDRC 💖
✨ Highlighted improvements ✨
File formats:
- A rewrite and major improvement of the
htmlformat andhtml2poconverter 🎀 - New
JSONformat introduced 🦋 - Support for Universal Terminology Exchange (
UTX) format ✨ - Support for Java properties files encoded in UTF-8 💅
- Improvements to
CSVformat, and improved compatibility with Excel exports 🌸 - Bug fixes to Qt
.ts🧚 - Support for XLIFF’s state attributes (
pocountnow lists detailed state statistics) 👑 - Minor bug fixes for
PHPformat 💖
Languages and quality checks:
- Support for Persian quotations 🌈
- Major performance improvements to quality checks 🔥
Pootle will regenerate all statistics with the new Translate Toolkit installed. Read about the quality checks 👀.
Other improvements:
- Improvements to stability of Lucene text indexing (affecting Pootle) ⭐
- Parameter for
po2propto ignore untranslated strings 🎀 - Many improvements to
pot2poincluding Qttssupport, improved handling of extra XML namespaces in XLIFF, and performance improvements 💜
Further resources:
- Full feature list
- Download
- Bugs
Happy translating! 💖
The Translate team 🫶
📜 Historic releases 📜
Translate Toolkit 1.8.0
Released on 17 August 2010.
Release notes for this release are still to be recovered 😭 — feel free to do some way back searching and create a Pull Request for this release.
Translate Toolkit 1.7.0
Released on 13 May 2010.
Release notes for this release are still to be recovered 😭 — feel free to do some way back searching and create a Pull Request for this release.
Translate Toolkit 1.6.0
Released on 3 March 2010.
PO files now always have headers
Generated PO files now always contain headers. This will mainly affect the output of pofilter and pogrep. This should allow better interoperability with gettext tools, and allowed for some improvement in the code. You should still be able to use headerless files in msgmerge, although it is recommended that PO files are consistently handled with headers wherever possible ✨.
Translate Toolkit 1.5.3
Released on 4 February 2010.
Release notes for this release are still to be recovered 😭 — feel free to do some way back searching and create a Pull Request for this release.
Translate Toolkit 1.5.2
Released on 13 January 2010.
Release notes for this release are still to be recovered 😭 — feel free to do some way back searching and create a Pull Request for this release.
Translate Toolkit 1.5.1
Released on 8 December 2009.
Release notes for this release are still to be recovered 😭 — feel free to do some way back searching and create a Pull Request for this release.
Translate Toolkit 1.5.0
Released on 25 November 2009.
Release notes for this release are still to be recovered 😭 — feel free to do some way back searching and create a Pull Request for this release.
Translate Toolkit 1.4.1
Released on 6 November 2009.
CSV column header names
The names given to CSV column headers have been changed. Early releases of csv2po would name the columns comment,original,translation. This was done mostly to make it easy for non-technical translators. However, comments in the command line help used terms like source and target. This release changes the column header names to location,source,target — this aligns with terms used throughout the toolkit 👑
If you have a CSV file generated by older versions of the toolkit, then a header entry of comment,original,translation will be turned into a unit instead of being ignored 😭. You can either change your CSV file to use the headers location,source,target or delete the header row completely. Once this is done the files will work as expected ✨.
Translate Toolkit 1.4.0
Released on 27 August 2009.
Java and Mozilla .properties
Unusual keys, separators and spacing should all be handled correctly now 🎀. Some Mozilla .properties files might now have changed. Regenerate your Mozilla l10n files from fresh POT files without any changes to your PO files to ensure that you can see and review these changes 👀.
Hashing in podebug
The --hash option in podebug has been replaced by a format specifier %h to be able to better control the positioning of the hash value 💅.
Translate Toolkit 1.3.0
Released on 11 February 2009.
Several duplicate styles were removed as has been warned about long before. Please check the recommendations posted at the time that msgctxt was added on how to migrate 🫶.
Translate Toolkit 1.2.1
Released on 29 December 2008.
Release notes for this release are still to be recovered 😭 — feel free to do some way back searching and create a Pull Request for this release.
Translate Toolkit 1.2.0
Released on 24 November 2008.
New formats
The toolkit now supports:
- Qt Phrase Book (
.qph) 🦋 - Qt
.tsv1.1 ✨
This allows reading, counting and working on these formats. The ts2po converter has not been changed, so you will not be able to benefit from the new .ts support. However, you can use the format for translation memory, etc., as it is now fully base class compliant 💖.
Stats database change
There were some changes in the database used by pocount for storing statistics. The location of the database might also have changed, depending on what the last version is that you used. Remove the file stats.db from any of ~/.translate_toolkit, ~/.wordforge (or the corresponding directories on your Windows installation) 🧚.
Valid accelerators
The pofilter accelerator test is now able to make use of a list of valid accelerators. This allows translators to control the behaviour of the test for their language and add or remove characters that can be used as accelerators. Please define the valid accelerators for your language, and these will then be included in future releases of the toolkit. By default the old process is followed so that if you take no action, then this check will continue to work as expected 🌸.
branches
These are branches that contain quite invasive changes that will most likely be merged into the main development and be released sometime in the future 👀.
toolkit-C-po
Converting the current Python based PO parser to the Gettext C based parser for PO. This offers quite a dramatic speed improvement and conformance to the output found in Gettext itself. For most users there will be a number of changes in layout of the files as they will now conform fully to Gettext layout. The keep option in --duplicatestyle will no longer be supported as this is not valid Gettext output 💀.
Translate Toolkit 1.1.1
Released on 2 April 2008.
Premature termination of DTD entities
Although this does not occur frequently, a case emerged where some DTD entities were not fully extracted from the DTD source. This was fixed in issue 331 🎀.
We expect this change to create a few new fuzzy entries. There is no action required from the user, as the next update of your PO files will bring the correct text into your translations. If you are using a translation memory, your translation might be recovered from obsolete translations ✨.
Translate Toolkit 1.1.0
Released on 22 January 2008.
oo2po Help (helpcontent2) escaping fixed
OpenOffice.org Help (helpcontent2) has notoriously contained some unreadable esacping, e.g. \\\\<tag attr=\\"value\\"\\\\>. The escaping has been fixed and oo2po now understands helpcontent2 escaping while leaving the current GUI escape handling unaltered 💅.
If you have not translated helpcontent2, then you are unaffected by this change. If you have translated this content, then you will need to follow these instructions when upgrading 👑.
If you follow normal procedures of creating POT files and upgrading your PO files using pot2po, then your strings will not match and you will obtain files with many fuzzies 😭. To avoid this, do the following:
- Make sure your PO files contain no fuzzy entries
- Use
po2oofrom the previous release to create and SDF file - Upgrade to the latest Translate Toolkit with new
po2oo - Use
po2oo -l xx-YY your.sdf poto create a new set of PO files with correct escaping
You can choose to do this with only your helpcontent2 PO files if needed — this will allow you to leave your GUI work in its current state. Simply do the above procedure and discard all PO files except helpcontent2, then move these new helpcontent2 files into your current work 🫶.
prop2po uses developer comments
prop2po used to place comments found in the source .properties file in traditional translator comments; they should of course go into developer comments. The reason for this change is twofold: it allows these comments to be correctly managed, and it is part of the process of cleaning up these formats so that they are closer to the base class and can thus work with XLIFF 🌸.
For the user there will be fairly large changes as one comment format moves to the next. It is best to cleanup translator comments and get your translations into a fit state, i.e. no fuzzies, and then proceed with any migrations 💖.
moz2po no longer uses KDE comments
moz2po has traditionally used KDE style comments for storing comments aimed at translators. Many translators confuse these and try to translate them. Thus these have been moved into automatic or developer comments. The result for many people migrating Mozilla PO files will be that many strings will become fuzzy — you can avoid much of this by using pot2po, which should intelligently be able to match without considering the KDE comments ✨.
Bestie, the best strategy is to get your translations into a relatively good shape before migration. You can then migrate them first to a new set of POT files generated from the same source files that the translation is based on. Eliminate all fuzzies — these should only relate to the changes in layout. Then proceed to migrate to a new set of POT files. If you cannot work against the original source files, then the best move is to also first eliminate fuzzy matches before proceeding to translation. Your fuzzies will include changes in layout and changes in content, so proceed carefully, okay?
At the end of this you should have PO files that conform to the Gettext standard without KDE comments. Periodt. ✨
💾 Read and Write MO files
You can read and write Gettext MO files (compiled PO files). So pocount can now count files on your filesystem, and you can also compile MO files using pocompile. MO files can be compiled from either PO or XLIFF sources. Slay.
MO will now also produce correct output for msgctxt and plural forms found in PO files. Iconic behaviour, honestly. 🎀
🦋 Read Qt .qm files
We can now read Qt .qm files, so pocount can count the contents of compiled files. We cannot however write .qm files at this time. Not yet, bestie — read-only energy.
👑 Translate Toolkit 1.0.1
Released on 23 June 2007. Mark the calendar.
🎁 pot2po will create new empty PO files if needed
From version 1.0.1, pot2po will create empty PO files corresponding to new POT files that might have been introduced. If some new POT files are present in the input to pot2po, you will see a new PO file appear in your output directory that was not in your old PO files. You will not lose any data, but in the worst case you will see new files on projects that you thought were fully translated. Lowkey a jump scare, but no data loss. 😭
🌈 Translate Toolkit 1.0
Released on 1 June 2007. Bestie, we’re going back in time.
✨ Improved XLIFF support
Many toolkit tools that only worked with PO files before can now also work with XLIFF files. pogrep, pocount, pomerge, and pofilter all work with XLIFF, for example. We love range.
🎀 Pretty XML output
All XML formats should now be more human readable, and the converters to Qt .ts files should work correctly again. No more cursed XML vibes.
🔥 Fuzzy matching in pot2po is optional
Fuzzy matching can now be entirely disabled in pot2po with the --nofuzzymatching parameter. This should make it much faster, although pot2po is substantially faster than earlier versions, especially if python-Levenshtein is installed. Speedrun strats.
💀 Old match/Levenshtein.py* can cause name clash
The file previously called match/Levenshtein.py was renamed to lshtein.py in order to use the python-Levenshtein package mentioned above. If you follow the basic installation instructions, the old file will not be overwritten, and can cause problems. Ensure that you remove all files starting with Levenshtein.py in the installation path of the translate toolkit, usually something like /usr/lib/python2.4/site-packages/translate/search/. It could be up to three files. Do not skip this cleanup, bestie — it’s giving trapdoor. 🙈
🦋 PO file layout now follows Gettext more closely
The toolkit’s output PO format should now resemble Gettext PO files more closely. Long lines are wrapped correctly, and messages with long initial lines will start with a msgid "" entry. The reason for this change is to ensure that differences in files relate to content change, not format change, no matter what tool you use.
To understand the problem more clearly: if a user creates POT files with e.g. oo2po, she then edits them in a PO editor or manipulates them with the Gettext tools. The layout of the file after manipulation was often different from the original produced by the Toolkit. Thus making it hard to tell what was a content change as opposed to a layout change.
The changes will affect you as follows:
- They will only impact you when using the Toolkit tools.
- You manipulate your files with a tool that follows Gettext PO layout.
- Your experience should now improve as the new PO files will align with your existing files.
- Updates should now only include real content changes, not layout changes.
- You manipulate your files using Toolkit related tools or manual editing.
- Your files will go through a re-layout the first time you use any of the tools.
- Subsequent usage should continue as normal.
- Any manipulation using Gettext tools will leave your files correctly laid out.
Our suggestion is that if you are about to suffer a major reflow, your initial merge should contain only reflow and update changes. Do content changes in subsequent steps. Once you have gone through the reflow you should see no layout changes and only content changes. Trust the process. 💅
🌸 Language awareness
The toolkit is gradually becoming more aware of the differences between languages. Currently this mostly affects pofilter checks (and therefore also Pootle) where tests involving punctuation and capitalisation will be more aware of the differences between English and some other languages. Provisional customisation for the following languages is in place and we will welcome more work on the language module: Amharic, Arabic, Greek, Persian, French, Armenian, Japanese, Khmer, Vietnamese, all types of Chinese. Global girlies rise. 🌍
🍓 New pofilter tests: newlines and tabs
The escapes test has been refined with two new tests, newlines and tabs. This makes identifying the errors easier and makes it easier to control the results of the tests. You shouldn’t have to change your testing behaviour in any way. Same routine, better receipts.
💜 Merging can change fuzzy status
pomerge now handles fuzzy states:
pomerge -t old -i merge -o newMessages that are fuzzy in merge will now also be fuzzy in new. Similarly, if a fuzzy state is present in old but removed in merge, then the message in new will not be fuzzy. Previously no fuzzy states were changed during a merge. Glow-up for merge logic. ✨
🚨 pofilter will make Mozilla accelerators a serious failure
If you use pofilter with the --mozilla option, then accelerator failures will produce a serious filter error, i.e. the message will be marked as fuzzy. This has been done because accelerator problems in your translations have the potential to break Mozilla applications. This is not a drill, bestie — it’s protective energy.
🎀 po2prop can output Mozilla or Java style properties
We have added the --personality option to allow a user to select output in either java or mozilla style (Java property files use escaped Unicode, while Mozilla uses actual Unicode characters). This functionality was always available but was not exposed to the user, and we always defaulted to the Mozilla style.
When using po2moz, the behaviour is not changed for the user as the programs will ensure that the properties convertor uses Mozilla style.
However, when using po2prop, the default style is now java, so if you are converting a single .properties file as part of a Mozilla conversion you will need to add --personality=mozilla to your conversion. Thus:
po2prop -t moz.properties moz.properties.po my-moz.propertiesWould become:
po2prop --personality=mozilla -t moz.properties moz.properties.po my-moz.propertiesNOTE: Output in java style escaped Unicode will still be usable by Mozilla but will be harder to read. Readable > technically fine, honestly.
🦋 Support for compressed files
There is some initial support for reading from and writing to compressed files. Single files compressed with gzip or bzip2 compression are supported, but not tarballs. Most tools don’t support it, but pocount and the --tm parameter to pot2po will work with it, for example. Naturally it is slower than working with uncompressed files. Hopefully more tools can support it in future. Baby steps, but cute ones.
👑 Translate Toolkit 0.11
Released on 24 March 2007. Another era.
🌈 po2oo defaults to not check for errors
In po2oo we made the default --filteraction=none, i.e. do nothing and don’t warn. Until we have a way of clearly marking false positives, we’ll have to disable this functionality as there is no way to quiet the output or mark non-errors. Also renamed exclude to exclude-all so that it is clearer what it does, i.e. it excludes “all” vs excludes “serious”. Naming clarity supremacy.
💅 pofilter xmltags produces less false positives
In the xmltags check we handle the case where we had some false positives. E.g. <Error> which looks like XML/HTML but should actually be translated. These are handled by:
- identifying them as being the same length as the source text,
- not containing any
=sign.
Thus the following would not be detected by this hack:
"An <Error> occurred" -> "<Error name="bob">"But these ones need human eyes anyway. Some things are simply not a robot’s job. 👀
🎀 Translate Toolkit 0.10.1
Released on 28 December 2006. Old-school release notes energy.
Release notes for this release are still to be recovered, feel free to do some way back searching and create a Pull Request for this release. Archaeology era.
🌸 Translate Toolkit 0.10
Released on 29 August 2006. Back at it again.
✨ PO to XLIFF conversion
Conversion from PO to XLIFF is greatly improved in 0.10, and this was done according to the specification at http://xliff-tools.freedesktop.org/wiki/Projects/XliffPoGuide — please let us know if there are features lacking. Feedback welcome, bestie.
🔥 pot2po can replace msgmerge
pot2po has undergone major changes which means that it now respects your header entries, can resurrect obsolete messages, does fuzzy matching using the Levenshtein distance algorithm, will correctly match messages with KDE style comments, and can use an external Translation Memory. You can now use pot2po instead of Gettext’s msgmerge, and it can also replace pomigrate2. You may still want to use pomigrate2 if there were file movements between versions, as pot2po can still not do intelligent matching of PO and POT files. pomigrate2 has also been adapted so that it can use pot2po as its background merging tool.
pomigrate2 --use-compendium --pot2po <old> <pot> <new>This will migrate files with a compendium built from PO files in <old> and will use pot2po as its conversion engine. Main character merge tool behaviour. 💅
🍓 .properties pretty formatting
When using templates for generating translated .properties files, we will now preserve the formatting around the equal sign.
# Previously if the template had
property = value
# We output
property=translation
# We will now output
property = translationThis change ensures that there is less noise when checking differences against the template file. However, there will be quite a bit of noise when you make your first .properties commits with the new pretty layout. Our suggestion is that you make a single commit of .properties files without changes of translations to get the formatting correct. One big formatting commit, then we move.
🌈 Translate Toolkit 0.9.2
Released on 11 August 2006. Ancient history, but make it cute.
Release notes for this release are still to be recovered, feel free to do some way back searching and create a Pull Request for this release. Go on, bestie, be the historian.
💜 Translate Toolkit 0.9.1
Released on 17 July 2006. Final era for this stretch.
Release notes for this release are still to be recovered, feel free to do some way back searching and create a Pull Request for this release. That’s the vibe — recover the lore. 🎀
🎀 Translate Toolkit 0.9
Released on 15 June 2006 — periodt. 💅
✨ Escaping – DTD files are no longer escaped
Okay so previously each converter handled escaping, and that was lowkey a nightmare every single time we found an escaping-related error or added a new format. 😭 Now escaping has been moved into the format classes as much as possible — the result being that formats exchange Python strings and manage their own escaping. Iconic behavior, honestly. 🦋
While doing this migration we revisited some of the format migration, and we found that we were escaping elements in our output DTD files. DTDs should have no escaping — i.e. \n is a literal \ followed by an n, not a newline. No cap. 💀
A result of this change is that older PO files will have different escaping to what po2moz will now expect — probably resulting in bad output .dtd files. 😭
We did not make this backward compatible, because the fix is relatively simple and is one you would have done for any migration of your PO files. 🫶
- Create a new set of POT files:
moz2po -P mozilla pot- Migrate your old PO files:
pomigrate2 old new pot- Fix all the fuzzy translations by editing your PO files.
- Use
pofilterto check for escaping problems and fix them:
pofilter -t escapes new new-check- Edit file in
new-checkin your PO editor:
pomerge -t new -i new-check -o new-check🌈 Migration to base class
All filters are/have been migrated to a base class. This move is so that it’s easier to add new formats, interchange formats, and create converters. Thus xx2po and xx2xlf become easier to create. Adding a new format should also be as simple as working towards the API exposed in the base class. An unexpected side effect will be that Pootle should be able to work directly with any base class file — although that will not be the normal Pootle operation. 👑
We have checks in place to ensure the current operation remains correct. However, nothing is perfect, and unfortunately the only way to really expose all bugs is to release this software. 🙈
If you discover a bug, please report it on Bugzilla or on the Pootle mailing list. If you have the skills, please check on HEAD to see if it is not already fixed, and if you regard it as critical, discuss on the mailing list backporting the fix — note some fixes will not be backported because they may be too invasive for the stable branch. If you are a developer, please write a test to expose the bug and a fix if possible. 💖
🔥 Duplicate Merging in PO files – merge now the default
We added the --duplicatestyle option to allow duplicate messages to be merged, commented, or simply appear in the PO unmerged. Initially we used the msgid_comments options as the default. This adds a KDE style comment to all affected messages, which created a good balance allowing users to see duplicates in the PO file but still create a valid PO file. ✨
'msgid_comments' was the default for 0.8 (FIXME check), but it seemed to create more confusion than it solved. Thus we have reverted to using 'merge' as the default — this then completely mimics Gettext behaviour. Slay. 💅
As Gettext will soon introduce the msgctxt attribute, we may revert to using that to manage disambiguation messages instead of KDE comments. This we feel will put us back at a good balance of usefulness and usability. We will only release this when the msgctxt version of the Gettext tools are released. 🦋
🍓 .properties files no longer use escaped Unicode
The main use of the .properties converter class is to translate Mozilla files, although .properties files are actually a Java standard. The old Mozilla way — and still the Java way — of working with .properties files is to escape any Unicode characters using the \uNNNN convention. Mozilla now allows you to use Unicode in UTF-8 encoding for these files. Thus in 0.9 of the Toolkit we now output UTF-8 encoded properties files. 🌸
Issue 193 tracks the status of this, and we hope to add a feature to prop2po to restore the correct Java convention as an option. 🫶
🌸 Translate Toolkit 0.8
Released on 20 February 2006.
Release notes for this release are still to be recovered — feel free to do some way back searching and create a Pull Request for this release. 💖
⭐ Translate Toolkit 0.7.1
Released on 24 April 2004.
Release notes for this release are still to be recovered — feel free to do some way back searching and create a Pull Request for this release. 💖
👑 History of the Translate Toolkit
This is a short history of the Translate Toolkit. In many ways it’s written so that people who see problems in the toolkit can understand how it evolved and where it is going. 🌈
💫 Conception
The toolkit was developed by David Fraser while working for ‐ Translate.org.za. Initially Translate.org.za had focussed on translating KDE into South Africa languages, and this work was PO based. The next project was to translate Mozilla, which used a combination of DTD and .properties files. The Mozilla project used a tool called Mozilla Translator, which mostly worked although it was not as feature rich as KBabel that was being used to manage PO translations. A decision was made to create a set of tools that could convert the DTD and .properties files into PO files. The advantage being that translators would not need to learn new tools, that existing translations could be leveraged, and that the resultant files, being bilingual, would make it easier to upgrade and manage translations. 🦋
Thus was born what initially was called the mozpotools. ✨
🌱 Growth
The first problem with the tools was that it was possible to break Mozilla translations. This was a combination of the fact that translators would often translate variables such as &browserName; and that the toolkit had developed a method of folding labels and accelerators into one PO field. These breakages were presented as broken XML. Thus was born pofilter, which allowed us to check the translations for problems in variables and accelerators. pomerge, its sister, allowed us to merge the corrections back into the main. We also developed pocount, which allowed us to for the first time get a real feel of the volume of work required in translating a PO file. 💅
🚀 Expansion
Of course once you can convert the convoluted Mozilla translations then you realise you can do anything. A key addition was the converter for OpenOffice.org, but also added were TMX, Qt .ts, txt and OpenOffice.org SXW files. 👑
The key being that files are converted to PO to allow translations and use of the Gettext tools and existing PO files. 🫶
🍓 Pootle
Initially started as a separate project to allow online translation, it was soon realised that the toolkit being file based gave all the infrastructure to allow Pootle to be a wrapper around the toolkit. So a file based, web translation tool was created. 💖
🌈 WordForge project
In 2006, with funding from the Open Society Institute (OSI) and IDRC, the toolkit was adapted to allow many core changes. The first being to introduce the concept of a base class from which PO and XLIFF storage formats are derived. This allowed tools to be adapted to allow output to XLIFF or PO files. The tools themselves were adapted to allow them to work with the core formats XLIFF and PO as well as all base class derived formats. Thus we can count XLIFF, PO, MO and other formats. ✨
Additional contributions during this phase were the adaptation of Pootle to use XLIFF as well as PO, and the creation of tools to manage translation memory files and glossary files. 🦋
The toolkit was also adapted to make dealing with encodings, plural forms, and escaping easier and more consistent throughout the code. Many but not all of the formats were converted to the base class. 💅
As part of the WordForge project, Pootling was created — which in the same way that Pootle is a web-based wrapper around the toolkit, so Pootling is a GUI wrapper around the toolkit. 👑
🌍 ANLoc project
The African Network for Localisation provided the opportunity for further improvements to the project. We saw the first official releases of Virtaal and massive improvements to all the translation tools. 🔥
Format support improved a lot, with several bilingual file formats now supported (Wordfast TM, Qt TS, etc.), and several monolingual file formats (PHP arrays, video subtitles, Mac OS X strings, etc.). 🌸
🔮 The Future
The toolkit continues to evolve with clean-up focused in various areas:
- Pulling features out of Pootle that should be in the Toolkit
- Cleaning up storage classes and converters to be XLIFF/PO interchangeable
- Cleaning up the converters to use only base class features and migrating code from the converters to the storage class
- Adding storage classes as needed
- Optimisation where needed
The toolkit continues to serve as the core for the command line tools and for Pootle. Key new features:
- Process Management
📜 License
The Translate Toolkit documentation is released under the GNU General Public License version 2, or later (GPL-2.0-or-later). 💜
🧚 API REFERENCE
This part covers any function, class or method included within the Translate Toolkit that you can use to programatically build new localization tools. 🛠️
🛠️ API
The Translate Toolkit provides several modules for programmers to build their own tools. 👑
📦 Module overview
The following will give you an idea about what each module is capable of. ✨
convert— Code to convert between different storage formats for localizations.filters— Filters that can be used on translations…lang— Classes that represent languages and provides language-specific information. All classes inherit from the parent class calledcommon. The type of data includes:- Language codes
- Language name
- Plurals
- Punctuation transformation
- etc.
misc— Miscellaneous modules for translate — including modules for backward compatibility with pre-2.3 versions of Python.search— Services for searching and matching of text.services—translate.servicesis part of the translate toolkit. It provides network services for interacting with the toolkit.storage— Classes that represent various storage formats for localization.tools— Code to perform various operations, mostly onpofiles.
🗂️ Module list
All the modules included in the Translated Toolkit are listed here. 💖
convert
Code to convert between different storage formats for localizations. 🦋
acesskey
Functions used to manipulate access keys in strings. ✨
class translate.convert.accesskey.UnitMixer(labelsuffixes, accesskeysuffixes)
Helper to mix separately defined labels and accesskeys into one unit. 💅
match_entities(index)
Populates mixedentities from the index. 🌸
static mix_units(label_unit, accesskey_unit, target_unit)
Mix the given units into the given target_unit if possible. 🫶
Might return None if no match is possible. (Yeah bestie, it can just… not. Handle that. 💀)
translate.convert.accesskey.combine(label, accesskey, accesskey_marker='&')— glues a label and an accesskey together into one label+accesskey string. 🫶
The way it works: we drop an accesskey marker right before the accesskey inside the label, so the two become one string. Example: "File" + "F" = "&File".
The case of the accesskey is preferred unless no match is found — in that case the alternate case gets used instead.
Parameters
label(unicode) – a labelaccesskey(unicode char) – The accesskey
Return type unicode or None
Returns label+accesskey string or None if uncombineable
translate.convert.accesskey.extract(string, accesskey_marker='&')— pulls the label and the accesskey back out of a label+accesskey string. ✨
The function will also try to ignore &entities; which would obviously not contain accesskeys. (Smart of her fr fr.)
Parameters
string(Unicode) – A string that might contain a label with accesskey markeraccesskey_marker(Char) – The character that is used to prefix an access key
🌈 convert
Handles converting of files between formats (used by translate.convert tools). This is the main-character module of the whole conversion situation, ngl. 👑
🌸 class translate.convert.convert.ArchiveConvertOptionParser(formats, usetemplates=False, usepots=False, description=None, archiveformats=None)
A ConvertOptionParser that can handle recursing into single archive files. 🗂️
archiveformats maps extension to class. If the extension doesn't matter, it can be None.
If the extension is only valid for input/output/template, it can be given as (extension, filepurpose).
add_duplicates_option(default='msgctxt')— Adds an option to say what to do with duplicate strings. 🙈add_fuzzy_option(default=False)— Adds an option to include / exclude fuzzy translations.add_multifile_option(default='single')— Adds an option to say how to split the po/pot files.add_option(Option)— the classic. add_option(opt_str, ..., kwarg=val, ...) -> Noneadd_remove_untranslated_option(default=False)— Adds an option to remove key value from output if it is untranslated. ✂️add_threshold_option(default=None)— Adds an option to output only stores where translation percentage exceeds the threshold. 📊check_values(values: Values, args: [string])—-> (values : Values, args : [string])
Check that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new – whatever you like). Default implementation just returns the passed-in values; subclasses may override as desired.
checkoutputsubdir(options, subdir)— Checks to see ifsubdirunderoptions.outputneeds to be created, creates if neccessary. 🏗️define_option(option)— Defines the given option, replacing an existing one of the same short name if neccessary…destroy()— Declare that you are done with thisOptionParser. This cleans up reference cycles so theOptionParser(and all objects referenced by it) can be garbage-collected promptly. After callingdestroy(), theOptionParseris unusable. 💀disable_interspersed_args()— Set parsing to stop on the first non-option. Use this if you have a command processor which runs another command that has options of its own and you want to make sure these options don't get confused. 🛑enable_interspersed_args()— Set parsing to not stop on the first non-option, allowing interspersing switches with command arguments. This is the default behavior. See alsodisable_interspersed_args()and the class documentation description of the attributeallow_interspersed_args. 🎀error(msg: string)— Print a usage message incorporating 'msg' to stderr and exit. If you override this in a subclass, it should not return – it should either exit or raise an exception. 😭filterinputformats(options)— Filters input formats, processing relevant switches inoptions.filteroutputoptions(options)— Filters output options, processing relevant switches inoptions.finalizetempoutputfile(options, outputfile, fulloutputpath)— Write the temp outputfile to its final destination. 🚚format_manpage()— Returns a formatted manpage. (hi, that's literally this whole vibe 👋)getarchiveclass(fileext, filepurpose, isdir=False)— Returns thearchiveclassfor the givenfileextandfilepurpose.getformathelp(formats)— Make a nice help string for describing formats…getfullinputpath(options, inputpath)— Gets the absolute path to an input file.getfulloutputpath(options, outputpath)— Gets the absolute path to an output file.getfulltemplatepath(options, templatepath)— Gets the absolute path to a template file.getoutputname(options, inputname, outputformat)— Gets an output filename based on the input filename.getoutputoptions(options, inputpath, templatepath)— Works out which output format and processor method to use…getpassthroughoptions(options)— Get the options required to pass to thefiltermethod…gettemplatename(options, inputname)— Gets an output filename based on the input filename.static getusageman(option)— Returns the usage string for the given option.static getusagestring(option)— Returns the usage string for the given option.isarchive(fileoption, filepurpose='input')— Returns whether the file option is an archive file. 🗜️static isexcluded(options, inputpath)— Checks if this path has been excluded. 🙅♀️isrecursive(fileoption, filepurpose='input')— Checks iffileoptionis a recursive file.isvalidinputname(inputname)— Checks if this is a valid input filename.static mkdir(parent, subdir)— Makes a subdirectory (recursively if neccessary). 📁openarchive(archivefilename, filepurpose, **kwargs)— Creates an archive object for the given file.openinputfile(options, fullinputpath)— Opens the input file.openoutputfile(options, fulloutputpath)— Opens the output file.opentemplatefile(options, fulltemplatepath)— Opens the template file (if required).static opentempoutputfile(options, fulloutputpath)— Opens a temporary output file.parse_args(args=None, values=None)— Parses the command line options, handling implicit input/output args. 🧠static potifyformat(fileformat)— Converts a.poto a.potwhere required.print_help(file: file = stdout)— Print an extended help message, listing all options and any help text provided with them, to 'file' (default stdout).print_manpage(file=None)— Outputs a manpage for the program using the help information.print_usage(file: file = stdout)— Print the usage message for the current program (self.usage) to 'file' (default stdout). Any occurrence of the string%proginself.usageis replaced with the name of the current program (basenameofsys.argv[0]). Does nothing ifself.usageis empty or not defined.print_version(file: file = stdout)— Print the version message for this program (self.version) to 'file' (default stdout). As withprint_usage(), any occurrence of%proginself.versionis replaced by the current program's name. Does nothing ifself.versionis empty or undefined.processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)— Run an individual conversion. 🔥recursearchivefiles(options)— Recurse through archive files and convert files.recurseinputfilelist(options)— Use a list of files, and find a common base directory for them.recurseinputfiles(options)— Recurse through archive file / directories and return files to be converted.recursiveprocess(options)— Recurse through directories and convert files. 🌀run(argv=None)— Parses the command line options and runs the conversion. the whole plot tbhset_usage(usage=None)— sets the usage string - ifusagenot given, usesgetusagestringfor each option.seterrorleveloptions()— Sets the errorlevel options.setformats(formats, usetemplates)— Sets the format options using the given format dictionary.
Parameters formats (Dictionary or iterable) –
The dictionary keys should be:
- Single strings (or 1-tuples) containing an input format (if not
usetemplates) - Tuples containing an input format and template format (if
usetemplates) - Formats can be
Noneto indicate what to do with standard input
The dictionary values should be tuples of outputformat (string) and processor method.
setmanpageoption()— creates a manpage option that allows the optionparser to generate a manpage. 📜setpotoption()— Sets the-P/--potoption depending on input/output formats etc.setprogressoptions()— Sets the progress options. ⏳settimestampoption()— Sets the-S/--timestampoption. ✨static splitext(pathname)— Splits pathname into name and ext, and removes the extsep. 🎀- Parameters:
pathname(string) – A file path - Returns:
root, ext - Return type:
tuple
- Parameters:
splitinputext(inputpath)— Splits an inputpath into name and extension. 🦋splittemplateext(templatepath)— Splits a templatepath into name and extension. 🌸templateexists(options, templatepath)— Returns whether the given template exists… 👀 (yes, the page itself trails off like that, we're not hiding anything from you)verifyoptions(options)— Verifies that the options are valid (required options are present, etc). ✅warning(msg, options=None, exc_info=None)— Print a warning message incorporating'msg'to stderr. ⚠️
🧰 class translate.convert.convert.ConvertOptionParser(formats, usetemplates=False, usepots=False, allowmissingtemplate=False, description=None)
A specialized Option Parser for convertor tools… 💅 (and yeah, the page lists this whole class again — think of it as the encore, we're going through every single method, no skipping, periodt)
add_duplicates_option(default='msgctxt')— Adds an option to say what to do with duplicate strings. 👯add_fuzzy_option(default=False)— Adds an option to include / exclude fuzzy translations. 🌫️add_multifile_option(default='single')— Adds an option to say how to split the po/pot files. 🗂️add_option(Option)— the classic, no extra notes on this one.add_option(opt_str, ..., kwarg=val, ...) -> None— same name, other signature, returnsNone.add_remove_untranslated_option(default=False)— Adds an option to remove key value from output if it is untranslated. 🧹add_threshold_option(default=None)— Adds an option to output only stores where translation percentage exceeds the threshold. 📊check_values(values: Values, args: [string])→(values : Values, args : [string])— Check that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new – whatever you like). Default implementation just returns the passed-in values; subclasses may override as desired. 🔍checkoutputsubdir(options, subdir)— Checks to see if subdir underoptions.outputneeds to be created, creates if neccessary. 📁 (that spelling is the manpage's, not mine, iykyk)define_option(option)— Defines the given option, replacing an existing one of the same short name if neccessary… 🔁destroy()— Declare that you are done with this OptionParser. This cleans up reference cycles so the OptionParser (and all objects referenced by it) can be garbage-collected promptly. After callingdestroy(), the OptionParser is unusable. 💀 (like, genuinely unusable, do not come back for seconds)disable_interspersed_args()— Set parsing to stop on the first non-option. Use this if you have a command processor which runs another command that has options of its own and you want to make sure these options don't get confused. 🛑enable_interspersed_args()— Set parsing to not stop on the first non-option, allowing interspersing switches with command arguments. This is the default behavior. See alsodisable_interspersed_args()and the class documentation description of the attributeallow_interspersed_args. 🟢error(msg: string)— Print a usage message incorporating'msg'to stderr and exit. If you override this in a subclass, it should not return – it should either exit or raise an exception. ❌filterinputformats(options)— Filters input formats, processing relevant switches in options. 🧵filteroutputoptions(options)— Filters output options, processing relevant switches in options. 🧶finalizetempoutputfile(options, outputfile, fulloutputpath)— Write the temp outputfile to its final destination. 🚚format_manpage()— Returns a formatted manpage. 📜getformathelp(formats)— Make a nice help string for describing formats… 🫶static getfullinputpath(options, inputpath)— Gets the full path to an input file. 📥static getfulloutputpath(options, outputpath)— Gets the full path to an output file. 📤getfulltemplatepath(options, templatepath)— Gets the full path to a template file. 🧾getoutputname(options, inputname, outputformat)— Gets an output filename based on the input filename. 🏷️getoutputoptions(options, inputpath, templatepath)— Works out which output format and processor method to use… 🧠getpassthroughoptions(options)— Get the options required to pass to the filtermethod… 🎯gettemplatename(options, inputname)— Gets an output filename based on the input filename. 🏷️ (yes it says output filename for the template name, that's literally what the page says, no notes)static getusageman(option)— Returns the usage string for the given option. 📖static getusagestring(option)— Returns the usage string for the given option. 📖static isexcluded(options, inputpath)— Checks if this path has been excluded. 🚫static isrecursive(fileoption, filepurpose='input')— Checks if fileoption is a recursive file. 🔄isvalidinputname(inputname)— Checks if this is a valid input filename. ✔️static mkdir(parent, subdir)— Makes a subdirectory (recursively if neccessary). 🌱static openinputfile(options, fullinputpath)— Opens the input file. 📂static openoutputfile(options, fulloutputpath)— Opens the output file. 📂opentemplatefile(options, fulltemplatepath)— Opens the template file (if required). 🗒️static opentempoutputfile(options, fulloutputpath)— Opens a temporary output file. 🧊parse_args(args=None, values=None)— Parses the command line options, handling implicit input/output args. ⌨️static potifyformat(fileformat)— Converts a.poto a.potwhere required. 🍯print_help(file: file = stdout)— Print an extended help message, listing all options and any help text provided with them, tofile(defaultstdout). 💬print_manpage(file=None)— Outputs a manpage for the program using the help information. 📄print_usage(file: file = stdout)— Print the usage message for the current program (self.usage) tofile(defaultstdout). Any occurrence of the string"%prog"inself.usageis replaced with the name of the current program (basename ofsys.argv[0]). Does nothing ifself.usageis empty or not defined. 🗣️print_version(file: file = stdout)— Print the version message for this program (self.version) tofile(defaultstdout). As withprint_usage(), any occurrence of"%prog"inself.versionis replaced by the current program's name. Does nothing ifself.versionis empty or undefined. 🏷️processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)— Process an individual file. ⚙️recurseinputfilelist(options)— Use a list of files, and find a common base directory for them. 🧭recurseinputfiles(options)— Recurse through directories and return files to be processed. 🌳recursiveprocess(options)— Recurse through directories and process files. 🌲run(argv=None)— Parses the command line options and runs the conversion. 🏃♀️set_usage(usage=None)— sets the usage string - if usage not given, uses getusagestring for each option. 📝seterrorleveloptions()— Sets the errorlevel options. 🚦setformats(formats, usetemplates)— Sets the format options using the given format dictionary. 🧩- Parameters:
formats(Dictionary or iterable) – - The dictionary keys should be:
- Single strings (or 1-tuples) containing an input format (if not usetemplates)
- Tuples containing an input format and template format (if usetemplates)
- Formats can be
Noneto indicate what to do with standard input
- The dictionary values should be tuples of outputformat (string) and processor method.
- Parameters:
setmanpageoption()— creates a manpage option that allows the optionparser to generate a manpage. 🗞️setpotoption()— Sets the-P/--potoption depending on input/output formats etc. 🍯setprogressoptions()— Sets the progress options. 📈settimestampoption()— Sets-S/--timestampoption. ✨static splitext(pathname)— Splits pathname into name and ext, and removes the extsep. 🎀- Parameters:
pathname(string) – A file path - Returns:
root, ext - Return type:
tuple
- Parameters:
splitinputext(inputpath)— Splits an inputpath into name and extension. 🦋splittemplateext(templatepath)— Splits a templatepath into name and extension. 🌸templateexists(options, templatepath)— Returns whether the given template exists… 👀verifyoptions(options)— Verifies that the options are valid (required options are present, etc). ✅warning(msg, options=None, exc_info=None)— Print a warning message incorporating'msg'to stderr. ⚠️
🔁 class translate.convert.convert.Replacer(searchstring, replacestring)
An object that knows how to replace strings in files. 🪄
doreplace(text)— Actually replace the text. 💫 (like, the doing part, not the promising part)searchreplaceinput(inputfile, outputfile, templatefile, **kwargs)— copies the input file over to the output file, but searches and replaces the whole way through 💅searchreplacetemplate(inputfile, outputfile, templatefile, **kwargs)— same energy, except it's the template file getting copied to the output file, with searching and replacing on the way ✨translate.convert.convert.copyinput(inputfile, outputfile, templatefile, **kwargs)— literally just copies the input file to the output file. no searching, no replacing, no drama 🫶translate.convert.convert.copytemplate(inputfile, outputfile, templatefile, **kwargs)— and this one copies the template file to the output file. matching set, iconic 🎀translate.convert.convert.should_output_store(store, threshold)— checks whether the percent of translated source words is more than or equal to the threshold you gave it. gatekeeping, but make it math 👑
🌸 csv2po 🌸
Converts Comma-Separated Value (.csv) files into Gettext PO localization files. the glow-up is real 💖
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/csv2po.html for examples and usage instructions.
translate.convert.csv2po.convertcsv(inputfile, outputfile, templatefile, charset=None, columnorder=None, duplicatestyle='msgctxt')— readsinputfileusingcsvl10n, converts it usingcsv2po, then writes tooutputfile. rinse and repeat, bestie ✨class translate.convert.csv2po.csv2po(templatepo=None, charset=None, duplicatestyle='keep')— a class that takes translations from a.csvfile and puts them in a.pofile. honestly? serving 🍓convertstore(thecsvfile)— converts a csvfile to a pofile and returns it. usestemplatepoif you passed one in at construction.static convertunit(csvunit)— converts a csv unit to a po unit. one at a time, no cap.handlecsvunit(csvunit)— handles reintegrating a csv unit back into the.pofile. putting it back where it belongs 🫶makeindex()— makes the indexes required for searching…
translate.convert.csv2po.replacestrings(source, *pairs)— uses pairs of(original, replacement)to replace text found insource. find and replace but make it fashion 🔥- Parameters
source(String) – the string on which the pairs of strings are going to get replaced*pairs(One or more tuples of(original, replacement)) – the strings to be matched and replaced
- Returns — String with
*pairsof strings replaced ✨
- Parameters
🦋 csv2tbx 🦋
Converts Comma-Separated Value (.csv) files into a TermBase eXchange (.tbx) glossary file. csv to glossary, we love a career change 💅
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/csv2tbx.html for examples and usage instructions
translate.convert.csv2tbx.convertcsv(inputfile, outputfile, templatefile, charset=None, columnorder=None)— readsinputfileusingcsvl10n, converts usingcsv2tbx, then writes tooutputfile. the whole pipeline, slay 🌈class translate.convert.csv2tbx.csv2tbx(charset=None)— a class that takes translations from a.csvfile and puts them in a.tbxfile. different file extension, same main character energy 👑convertfile(csvfile)— converts a csvfile to a tbxfile and returns it. usestemplatepoif it was given at construction. (yes bestie, the docstring saystemplatepo— we report, we don't editorialize 🙈)
🧚 dtd2po 🧚
Converts a Mozilla .dtd UTF-8 localization format into a Gettext PO localization file.
It uses the po and dtd modules, plus the dtd2po convertor class that lives in this module. And FYI, you can convert back to .dtd using po2dtd.py — round trip, we love to see it 💜
translate.convert.dtd2po.convertdtd(inputfile, outputfile, templatefile, pot=False, duplicatestyle='msgctxt')— readsinputfileandtemplatefileusingdtd, converts usingdtd2po, writes tooutputfile. three files, one function, zero stress ⭐translate.convert.dtd2po.is_css_entity(entity)— tells you if the given entity is likely to contain CSS that should not be translated. hands off that styling, bestie 💅
🎀 factory 🎀
Factory methods that convert supported input files into supported translatable files. it's giving assembly line but cute 🏭
exception translate.convert.factory.UnknownExtensionError(afile)add_note(object, /)—Exception.add_note(note)– adds a note to the exception 🫶with_traceback(object, /)—Exception.with_traceback(tb)– setsself.__traceback__totband returnsself.
exception translate.convert.factory.UnsupportedConversionError(in_ext=None, out_ext=None, templ_ext=None)add_note(object, /)—Exception.add_note(note)– adds a note to the exception 🫶with_traceback(object, /)—Exception.with_traceback(tb)– setsself.__traceback__totband returnsself.
💖 html2po 💖
Converts HTML files into Gettext PO localization files. web page to translation file, a true transformation arc ✨
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/html2po.html for examples and usage instructions.
class translate.convert.html2po.Html2POOptionParser— the option parser of the hour, hold your applause 👏add_duplicates_option(default='msgctxt')— adds an option to say what to do with duplicate strings.add_fuzzy_option(default=False)— adds an option to include / exclude fuzzy translations.add_multifile_option(default='single')— adds an option to say how to split the po/pot files.add_option(Option)add_option(opt_str, ..., kwarg=val, ...) -> Noneadd_remove_untranslated_option(default=False)— adds an option to remove the key value from the output if it's untranslated.add_threshold_option(default=None)— adds an option to only output stores where the translation percentage exceeds the threshold.check_values(values: Values, args: [string]) -> (values : Values, args : [string])— checks that the option values and leftover arguments you supplied are valid. Returns the option values and the leftover arguments (possibly adjusted, possibly completely new – whatever you like, honestly). The default implementation just returns the passed-in values; subclasses can override as they please 💅checkoutputsubdir(options, subdir)— checks whethersubdirunderoptions.outputneeds to be created, and creates it if neccessary. Does nothing if you're in single-output-file mode. (override)convert(inputfile, outputfile, templates, pot=False, duplicatestyle='msgctxt', multifilestyle='single', keepcomments=False)— extracts translation units from one html file. one file, all the units 🔥define_option(option)— defines the given option, replacing an existing one with the same short name if neccessary…destroy()— declares that you're done with this OptionParser. This cleans up reference cycles so the OptionParser (and every object it references) can be garbage-collected promptly. Heads up though: after callingdestroy(), the OptionParser is unusable. like, gone gone 💀disable_interspersed_args()— sets parsing to stop at the first non-option. Use this if you've got a command processor that runs another command with options of its own and you want to make sure those options don't get confused. boundaries, bestie 🫶enable_interspersed_args()— sets parsing to not stop at the first non-option, which lets you intersperse switches with command arguments. This is the default behavior. See alsodisable_interspersed_args()and the class documentation description of theallow_interspersed_argsattribute.error(msg: string)— prints a usage message incorporating'msg'to stderr, then exits. If you override this in a subclass it should not return – it should either exit or raise an exception. no returning allowed, periodt 👑filterinputformats(options)— filters input formats, processing the relevant switches inoptions.filteroutputoptions(options)— filters output options, processing the relevant switches inoptions.finalizetempoutputfile(options, outputfile, fulloutputpath)— writes the temp outputfile to its final destination. moving day 🚚format_manpage()— returns a formatted manpage. (hi, that's me 💅)getformathelp(formats)— makes a nice help string for describing formats…static getfullinputpath(options, inputpath)— gets the full path to an input file.static getfulloutputpath(options, outputpath)— gets the full path to an output file.getfulltemplatepath(options, templatepath)— gets the full path to a template file.getoutputname(options, inputname, outputformat)— gets an output filename based on the input filename.getoutputoptions(options, inputpath, templatepath)— works out which output format and processor method to use…getpassthroughoptions(options)— gets the options required to pass to the filtermethod…gettemplatename(options, inputname)— gets an output filename based on the input filename.static getusageman(option)— returns the usage string for the given option.static getusagestring(option)— returns the usage string for the given option.static isexcluded(options, inputpath)— checks if this path has been excluded. not invited to the party 🙈isrecursive(fileoption, filepurpose='input')— checks iffileoptionis a recursive file. (override)isvalidinputname(inputname)— checks if this is a valid input filename.static mkdir(parent, subdir)— makes a subdirectory (recursively if neccessary).static openinputfile(options, fullinputpath)— opens the input file.openoutputfile(options, fulloutputpath)— opens the output file, or does nothing if you're in single-output-file mode. (override)opentemplatefile(options, fulltemplatepath)— opens the template file (if required).static opentempoutputfile(options, fulloutputpath)— opens a temporary output file.parse_args(args=None, values=None)— parses the command line options, handling implicit input/output args. the multitasker of the group chat ⭐static potifyformat(fileformat)— converts a.poto a.potwhere required.print_help(file: file = stdout)— prints an extended help message tofile(defaultstdout), listing all options and any help text provided with them.
print_manpage(file=None) — bestie this one literally outputs a manpage for the program, built straight from the help information. iconic behavior honestly 💅
print_usage(file: file = stdout) — prints the usage message for the current program (self.usage) out to file, which defaults to stdout. any time %prog shows up in self.usage, it gets swapped for the name of the current program (that's the basename of sys.argv[0], iykyk). and if self.usage is empty or just… not defined? it does nothing. nothing at all. silent queen 🌸
print_version(file: file = stdout) — same energy, different vibe: prints the version message for this program (self.version) to file (default stdout). just like print_usage(), any %prog in self.version gets replaced by the current program's name. and again — does nothing if self.version is empty or undefined 🫶
processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath) — processes an individual file. one file. that's the whole job ✨
recurseinputfilelist(options) — takes a list of files and finds a common base directory for them. detective work tbh 🕵️
recurseinputfiles(options) — recurses through directories and returns the files that need processing 🦋
recursiveprocess(options) — recurses through directories and processes the files. (it's an override, noted 📝)
run(argv=None) — parses the command line options and runs the conversion. the main character of the whole operation 👑
set_usage(usage=None) — sets the usage string. if you don't hand it a usage, it uses getusagestring for each option instead. DIY energy 🎀
seterrorleveloptions() — sets the errorlevel options 💀
setformats(formats, usetemplates) — sets the format options using the given format dictionary.
parameters:
formats(Dictionary or iterable) —
the dictionary keys should be:
- single strings (or 1-tuples) containing an input format (if not
usetemplates) - tuples containing an input format and template format (if
usetemplates) - formats can be
Noneto indicate what to do with standard input
the dictionary values should be tuples of outputformat (string) and processor method 🌈
setmanpageoption() — creates a manpage option that lets the optionparser generate a manpage. literally the reason you're reading this rn 🔥
setpotoption() — sets the -P/--pot option depending on input/output formats etc. etc. etc. ✨
setprogressoptions() — sets the progress options ⭐
settimestampoption() — sets the -S/--timestamp option ⏰
static splitext(pathname) — splits pathname into name and ext, and removes the extsep.
parameters:
pathname(string) — a file path
returns: root, ext
return type: tuple 💖
splitinputext(inputpath) — splits an inputpath into name and extension ✂️
splittemplateext(templatepath) — splits a templatepath into name and extension ✂️
templateexists(options, templatepath) — returns whether the given template exists… (yes the ellipsis is part of the vibe, it's giving mysterious 👀)
verifyoptions(options) — verifies that the options are valid (required options are present, etc). vibe check on your flags fr fr ✅
warning(msg, options=None, exc_info=None) — prints a warning message incorporating msg to stderr. not mean, just honest 🫶
translate.convert.html2po.converthtml(inputfile, outputfile, templates, pot=False, duplicatestyle='msgctxt', keepcomments=False)
reads in stdin using fromfileclass, converts using convertorclass, writes to stdout. that's the whole pipeline bestie 💅
🌸 ical2po
convert iCalendar files to Gettext PO localization files.
see: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/ical2po.html for examples and usage instructions 🔗
class translate.convert.ical2po.ical2po(input_file, output_file, template_file=None, blank_msgstr=False, duplicate_style='msgctxt') — convert one or two iCalendar files to a single PO file. two in, one out, that's the math ✨
SourceStoreClass— alias oficalfileTargetStoreClass— alias ofpofileTargetUnitClass— alias ofpounitconvert_store()— convert a single source format file to a target format file 🦋convert_unit(unit)— convert a source format unit to a target format unit 🎀merge_stores()— convert two source format files to a target format file 💖run()— run the converter. go girl go 👑
translate.convert.ical2po.run_converter(input_file, output_file, template_file=None, pot=False, duplicatestyle='msgctxt')
wrapper around converter. that's it, that's the function 💅
🎀 ini2po
convert .ini files to Gettext PO localization files.
see: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/ini2po.html for examples and usage instructions 🔗
class translate.convert.ini2po.ini2po(input_file, output_file, template_file=None, blank_msgstr=False, duplicate_style='msgctxt', dialect='default') — convert one or two INI files to a single PO file. yes there's a dialect param and yes the default is 'default' 😭
SourceStoreClass— alias ofinifileTargetStoreClass— alias ofpofileTargetUnitClass— alias ofpounitconvert_store()— convert a single source format file to a target format file 🦋convert_unit(unit)— convert a source format unit to a target format unit 🎀merge_stores()— convert two source format files to a target format file 💖run()— run the converter 👑
translate.convert.ini2po.run_converter(input_file, output_file, template_file=None, pot=False, duplicatestyle='msgctxt', dialect='default')
wrapper around converter, with the dialect='default' carried through like a tiny luggage 🧳
🦋 json2po
convert JSON files to Gettext PO localization files.
see: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/json2po.html for examples and usage instructions 🔗
translate.convert.json2po.convertjson(input_file, output_file, template_file, pot=False, duplicatestyle='msgctxt', dialect='default', filter=None)
reads in input_file using jsonl10n, converts using json2po, writes to output_file. honestly a whole little assembly line ⭐
class translate.convert.json2po.json2po — convert a JSON file to a PO file.
convert_store(input_store, duplicatestyle='msgctxt')— converts a JSON file to a PO file 💖static convert_unit(input_unit, commenttype)— converts a JSON unit to a PO unit.
returns: None if empty or not for translation 🙈
merge_store(template_store, input_store, blankmsgstr=False, duplicatestyle='msgctxt')— converts two JSON files to a PO file. two JSONs walk in, one PO file walks out 💅
🧚 moz2po
convert Mozilla .dtd and .properties files to Gettext PO localization files.
see: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/moz2po.html for examples and usage instructions 🔗
🌈 mozfunny2prop
converts additional Mozilla files to properties files. "funny" is the technical term, we don't make the rules 😭
translate.convert.mozfunny2prop.inc2po(inputfile, outputfile, templatefile, encoding=None, pot=False, duplicatestyle='msgctxt')
wraps prop2po but converts input/template files to properties first. middleman energy 🫶
translate.convert.mozfunny2prop.inc2prop(lines)
convert a .inc file with #defines in it to a properties file ✨
translate.convert.mozfunny2prop.it2po(inputfile, outputfile, templatefile, encoding='cp1252', pot=False, duplicatestyle='msgctxt')
wraps prop2po but converts input/template files to properties first. same deal, and yes the default encoding is 'cp1252' 💀
translate.convert.mozfunny2prop.it2prop(lines, encoding='cp1252')
convert a pseudo-properties .it file to a conventional properties file. pseudo-properties, conventional output, we love a glow-up 🔥
🍓 mozlang2po
convert Mozilla .lang files to Gettext PO localization files.
class translate.convert.mozlang2po.lang2po(input_file, output_file, template_file=None, blank_msgstr=False, duplicate_style='msgctxt', encoding='utf-8') — convert one Mozilla .lang file to a single PO file. note: one .lang file → a single PO file.
SourceStoreClass— alias ofLangStoreTargetStoreClass— alias ofpofileTargetUnitClass— alias ofpounitconvert_store()— convert a single source format file to a target format file 🦋convert_unit(unit)— convert a source format unit to a target format unit 🎀merge_stores()— convert two source format files to a target format file 💖run()— run the converter 👑
translate.convert.mozlang2po.run_converter(input_file, output_file, template_file=None, pot=False, duplicatestyle='msgctxt', encoding='utf-8')
wrapper around converter, and the encoding default is 'utf-8' this time. character development 🌸
💜 odf2xliff
convert OpenDocument (ODF) files to XLIFF localization files.
see: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/odf2xliff.html for examples and usage instructions 🔗
translate.convert.odf2xliff.convertodf(inputfile, outputfile, templates)
convert an ODF package to XLIFF. package in, XLIFF out 📦➡️✨
👑 oo2po
convert an OpenOffice.org (SDF) localization file to Gettext PO localization files.
see: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/oo2po.html for examples and usage instructions 🔗
translate.convert.oo2po.convertoo(inputfile, outputfile, templates, pot=False, sourcelanguage=None, targetlanguage=None, duplicatestyle='msgid_comment', multifilestyle='single')
reads in stdin using inputstore class, converts using convertorclass, writes to stdout. and look at duplicatestyle='msgid_comment' — that's the default here, don't come for me, it's literally what it says 📝
translate.convert.oo2po.verifyoptions(options)
verifies the commandline options. someone has to ✅
💅 oo2xliff
convert an OpenOffice.org (SDF) localization file to XLIFF localization files.
see: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/oo2po.html for examples and usage instructions 🔗 — and yes, that link really does say oo2po.html, we're keeping it exactly as-is, no cap 👀
translate.convert.oo2xliff.convertoo(inputfile, outputfile, templates, pot=False, sourcelanguage=None, targetlanguage=None, duplicatestyle='msgctxt', multifilestyle='single')
reads in stdin using inputstore class, converts using convertorclass, writes to stdout. same shape as its sibling, but duplicatestyle='msgctxt' instead. twins but not identical 👯♀️
translate.convert.oo2xliff.verifyoptions(options)— lowkey just verifies the commandline options ✨ that's it, that's the whole job.
🐘 php2po
Turn PHP localization files into Gettext PO localization files, bestie 💖
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/php2po.html — that's where the examples and usage instructions live, go look 👀
class translate.convert.php2po.php2po(input_file, output_file,
template_file=None, blank_msgstr=False, duplicate_style='msgctxt')Converts one or two PHP files into a single PO file. Iconic, honestly 🎀
SourceStoreClass— alias ofphpfileTargetStoreClass— alias ofpofileTargetUnitClass— alias ofpounitconvert_store()— converts a single source format file to a target format fileconvert_unit(unit)— converts a source format unit to a target format unitmerge_stores()— converts two source format files into a target format filerun()— runs the converter 🏃♀️
translate.convert.php2po.run_converter(input_file, output_file,
template_file=None, pot=False, duplicatestyle='msgctxt')Wrapper around converter. That's the whole vibe, she's a little cozy layer 🫶
📊 po2csv
Turns Gettext PO localization files into Comma-Separated Value (.csv) files ✨
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/csv2po.html — examples + usage instructions, iykyk
translate.convert.po2csv.convertcsv(inputfile, outputfile, templatefile,
columnorder=None)Reads inputfile using po, converts using po2csv, then writes the whole thing out to outputfile. Boom, done, slay 💅
🌈 po2dtd
Converts a Gettext PO file into a UTF-8 encoded Mozilla .dtd file 💖
translate.convert.po2dtd.applytranslation(entity, dtdunit, inputunit,
mixedentities)Applies the translation for entity that's sitting in the po unit over to the dtd unit 🎀
class translate.convert.po2dtd.po2dtd(android=False,
remove_untranslated=False)Okay so this is a converter class that creates a brand new dtd file based on a po file — with no template. Main character energy, no template needed 👑
class translate.convert.po2dtd.redtd(dtdfile, android=False,
remove_untranslated=False)And this one's a converter class that creates a new dtd based on a template, using the translations from a po. Two different little guys, two different jobs 🦋
🌸 po2html
Translate HTML files using Gettext PO localization files ✨
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/html2po.html — examples and usage instructions, go read em
class translate.convert.po2html.PO2HtmlOptionParserOkay bestie, this one's a long list of methods, but we're getting through all of them fr fr 🫶
add_duplicates_option(default='msgctxt')— adds an option that says what to do with duplicate stringsadd_fuzzy_option(default=False)— adds an option to include / exclude fuzzy translationsadd_multifile_option(default='single')— adds an option to say how the po/pot files get splitadd_option(Option)— the one-arg versionadd_option(opt_str, ..., kwarg=val, ...) -> None— the full-signature versionadd_remove_untranslated_option(default=False)— adds an option that removes the key/value from the output if it's untranslatedadd_threshold_option(default=None)— adds an option to output only stores where the translation percentage exceeds the threshold 📈check_values(values: Values, args: [string]) -> (values : Values, args : [string])— checks that the supplied option values and leftover arguments are actually valid, and returns the option values plus leftover arguments (possibly adjusted, possibly completely new — whatever you like, it's your world). The default implementation just returns the passed-in values; subclasses can override as desired 💅checkoutputsubdir(options, subdir)— checks whethersubdirunderoptions.outputneeds to be created, and creates it if necessarydefine_option(option)— defines the given option, replacing an existing one with the same short name if necessary…destroy()— declares that you're done with thisOptionParser. Cleans up reference cycles so the OptionParser (and everything it references) can get garbage-collected promptly. Heads up: after callingdestroy(), the OptionParser is unusable ⚠️disable_interspersed_args()— sets parsing to stop on the first non-option. Use this if you've got a command processor running another command that has its own options, and you want to make sure those options don't get confused 🌀enable_interspersed_args()— sets parsing to not stop on the first non-option, letting you intersperse switches with command arguments. This is the default behavior. See alsodisable_interspersed_args()and the class documentation for the attributeallow_interspersed_argserror(msg: string)— prints a usage message incorporating msg to stderr and exits. If you override this in a subclass it should not return — it should either exit or raise an exception 🚨filterinputformats(options)— filters input formats, processing the relevant switches inoptionsfilteroutputoptions(options)— filters output options, processing the relevant switches inoptionsfinalizetempoutputfile(options, outputfile, fulloutputpath)— writes the temp outputfile to its final destination 🏁format_manpage()— returns a formatted manpagegetformathelp(formats)— makes a nice help string for describing formats…static getfullinputpath(options, inputpath)— gets the full path to an input filestatic getfulloutputpath(options, outputpath)— gets the full path to an output filegetfulltemplatepath(options, templatepath)— gets the full path to a template filegetoutputname(options, inputname, outputformat)— gets an output filename based on the input filenamegetoutputoptions(options, inputpath, templatepath)— works out which output format and processor method to use…getpassthroughoptions(options)— gets the options required to pass to the filtermethod…gettemplatename(options, inputname)— gets an output filename based on the input filenamestatic getusageman(option)— returns the usage string for the given optionstatic getusagestring(option)— returns the usage string for the given option (yes, both of them exist, don't come for me 🙈)static isexcluded(options, inputpath)— checks if this path has been excludedstatic isrecursive(fileoption, filepurpose='input')— checks iffileoptionis a recursive fileisvalidinputname(inputname)— checks if this is a valid input filenameisvalidtemplatename(filename)— checks if this is a valid template/output filenamestatic mkdir(parent, subdir)— makes a subdirectory (recursively if necessary) 📁static openinputfile(options, fullinputpath)— opens the input filestatic openoutputfile(options, fulloutputpath)— opens the output fileopentemplatefile(options, fulltemplatepath)— opens the template file (if required)static opentempoutputfile(options, fulloutputpath)— opens a temporary output fileparse_args(args=None, values=None)— parses the command line options, handling the implicit input/output args 🎀static potifyformat(fileformat)— converts a.poto a.potwhere requiredprint_help(file: file = stdout)— prints an extended help message listing all options and any help text provided with them, to file (defaultstdout)print_manpage(file=None)— outputs a manpage for the program using the help informationprint_usage(file: file = stdout)— prints the usage message for the current program (self.usage) to file (defaultstdout). Any occurrence of the string"%prog"inself.usagegets replaced with the name of the current program (the basename ofsys.argv[0]). Does nothing ifself.usageis empty or not definedprint_version(file: file = stdout)— prints the version message for this program (self.version) to file (defaultstdout). Just likeprint_usage(), any occurrence of"%prog"inself.versiongets replaced by the current program's name. Does nothing ifself.versionis empty or undefinedprocessfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)— processes an individual file 🗂️recurse_template_files(options)— recurses through directories and returns the files to be processedrecurseinputfilelist(options)— uses a list of files, and finds a common base directory for themrecurseinputfiles(options)— recurses through directories and returns the files to be processedrecursiveprocess(options)— recurses through directories and processes filesrecursiveprocess_by_templates(options)— recurses through directories and processes files by templates (html), not input files (po)run(argv=None)— parses the command line options and runs the conversion 🏃♀️set_usage(usage=None)— sets the usage string — if usage isn't given, it usesgetusagestringfor each optionseterrorleveloptions()— sets the errorlevel optionssetformats(formats, usetemplates)— sets the format options using the given format dictionary
And under setformats, the parameters are:
- Parameters
formats(Dictionary or iterable) –- The dictionary keys should be:
- Single strings (or 1-tuples) containing an input format (if not
usetemplates) - Tuples containing an input format and template format (if
usetemplates) 💖
- Single strings (or 1-tuples) containing an input format (if not
That's the whole map, bestie — every method accounted for, no crumbs left behind 🧚
Formatscan beNoneto indicate what to do with standard input
The dictionary values should be tuples of outputformat (string) and processor method. Bestie, that's the whole contract, okay? 💅
setmanpageoption()— creates a manpage option that lets the optionparser generate a manpage. Very meta, very slay. ✨setpotoption()— sets the-P/--potoption depending on the input/output formats and all that jazz.setprogressoptions()— sets the progress options. Gotta see that progress bar, fr.settimestampoption()— sets the-S/--timestampoption. 🕐static splitext(pathname)— splitspathnameinto name and ext, and removes theextsep.pathname(string) — a file path- Returns:
root, ext - Return type:
tuple
splitinputext(inputpath)— splits aninputpathinto name and extension. Clean.splittemplateext(templatepath)— splits atemplatepathinto name and extension. Same energy, template edition. 🎀templateexists(options, templatepath)— returns whether the given template exists… (yes, the original really does trail off like that, iykyk 😭)verifyoptions(options)— verifies that the options are valid (required options are present, etc.). The vibe check for your flags. ✅warning(msg, options=None, exc_info=None)— prints a warning message incorporatingmsgto stderr. A gentle heads-up, not a full-on scream. 🫶
translate.convert.po2html.converthtml(inputfile, outputfile, templatefile, includefuzzy=False, outputthreshold=None)— readsinputfile(po) andtemplatefile(html), writes tooutputfile(html). 🌸class translate.convert.po2html.po2html— readsinputfile(po) andtemplatefile(html), writes tooutputfile(html).mergestore(inputstore, templatetext, includefuzzy)— converts a file to html format.
🍰 po2ical
Convert Gettext PO localization files to iCalendar files. Calendar girlies, this one's for you. 🗓️
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/ical2po.html for examples and usage instructions.
class translate.convert.po2ical.po2ical(input_file, output_file, template_file=None, include_fuzzy=False, output_threshold=None)— converts a PO file and a template iCalendar file to a iCalendar file.SourceStoreClass— alias ofpofileTargetStoreClass— alias oficalfileTargetUnitClass— alias oficalunitmerge_stores()— converts a source file to a target file using a template file. Source file is in source format, while target and template files use target format. Keep up, bestie 💫run()— runs the converter. Go, icon, go. 🔥
translate.convert.po2ical.run_converter(inputfile, outputfile, templatefile=None, includefuzzy=False, outputthreshold=None)— wrapper around converter.
🧁 po2ini
Convert Gettext PO localization files to .ini files. Config-file core. 💅
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/ini2po.html for examples and usage instructions.
class translate.convert.po2ini.po2ini(input_file, output_file, template_file=None, include_fuzzy=False, output_threshold=None, dialect='default')— converts a PO file and a template INI file to a INI file.SourceStoreClass— alias ofpofileTargetStoreClass— alias ofinifileTargetUnitClass— alias ofiniunitmerge_stores()— converts a source file to a target file using a template file. Source file is in source format, while target and template files use target format.run()— runs the converter.
translate.convert.po2ini.run_converter(inputfile, outputfile, templatefile=None, includefuzzy=False, dialect='default', outputthreshold=None)— wrapper around converter.
🦋 po2json
Convert Gettext PO localization files to JSON files. JSON besties rise up. ✨
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/json2po.html for examples and usage instructions.
🍓 po2mozlang
Convert Gettext PO localization files to Mozilla .lang files.
class translate.convert.po2mozlang.po2lang(input_file, output_file, template_file=None, include_fuzzy=False, output_threshold=None, mark_active=True)— converts a PO file to a Mozilla.langfile. Lowkey obsessed with this one.SourceStoreClass— alias ofpofileTargetStoreClass— alias ofLangStoreTargetUnitClass— alias ofLangUnitconvert_store()— converts a single source format file to a target format file.convert_unit(unit)— converts a source format unit to a target format unit.run()— runs the converter. 💫
translate.convert.po2mozlang.run_converter(inputfile, outputfile, templatefile=None, includefuzzy=False, mark_active=True, outputthreshold=None)— wrapper around converter.
🌈 po2moz
Convert Gettext PO localization files to Mozilla .dtd and .properties files. Mozilla mains, assemble. 🫶
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/moz2po.html for examples and usage instructions.
class translate.convert.po2moz.MozConvertOptionParser(formats, usetemplates=False, usepots=False, description=None)— the option parser for the Mozilla converter. Say hi to the whole squad of methods below 👇add_duplicates_option(default='msgctxt')— adds an option to say what to do with duplicate strings.add_fuzzy_option(default=False)— adds an option to include / exclude fuzzy translations.add_multifile_option(default='single')— adds an option to say how to split the po/pot files.add_option(Option)— 👀add_option(opt_str, ..., kwarg=val, ...) -> None— the full signature, because yes, there are two of them.add_remove_untranslated_option(default=False)— adds an option to remove key value from output if it is untranslated.add_threshold_option(default=None)— adds an option to output only stores where translation percentage exceeds the threshold.check_values(values: Values, args: [string]) -> (values : Values, args : [string])— checks that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new – whatever you like). The default implementation just returns the passed-in values; subclasses may override as desired.checkoutputsubdir(options, subdir)— checks to see ifsubdirunderoptions.outputneeds to be created, creates if necessary.define_option(option)— defines the given option, replacing an existing one of the same short name if necessary… (again with the trailing dots, iconic)destroy()— declares that you are done with this OptionParser. This cleans up reference cycles so the OptionParser (and all objects referenced by it) can be garbage-collected promptly. After callingdestroy(), the OptionParser is unusable. 💀 So like, don't try it.disable_interspersed_args()— sets parsing to stop on the first non-option. Use this if you have a command processor which runs another command that has options of its own and you want to make sure these options don't get confused.enable_interspersed_args()— sets parsing to not stop on the first non-option, allowing interspersing switches with command arguments. This is the default behavior. See alsodisable_interspersed_args()and the class documentation description of the attributeallow_interspersed_args.error(msg: string)— prints a usage message incorporatingmsgto stderr and exits. If you override this in a subclass, it should not return – it should either exit or raise an exception. No wiggle room, bestie. 🚨filterinputformats(options)— filters input formats, processing relevant switches inoptions.filteroutputoptions(options)— filters output options, processing relevant switches inoptions.finalizetempoutputfile(options, outputfile, fulloutputpath)— writes the tempoutputfileto its final destination.format_manpage()— returns a formatted manpage. The full circle moment. ✨getformathelp(formats)— makes a nice help string for describing formats… so nice.static getfullinputpath(options, inputpath)— gets the full path to an input file.static getfulloutputpath(options, outputpath)— gets the full path to an output file.getfulltemplatepath(options, templatepath)— gets the full path to a template file.getoutputname(options, inputname, outputformat)— gets an output filename based on the input filename.getoutputoptions(options, inputpath, templatepath)— works out which output format and processor method to use…getpassthroughoptions(options)— gets the options required to pass to the filtermethod…gettemplatename(options, inputname)— gets an output filename based on the input filename. (Yes, the description is literally the same asgetoutputname's — not a typo on my end, that's just how it is 😭)static getusageman(option)— returns the usage string for the given option.static getusagestring(option)— returns the usage string for the given option. (Same description again — twin energies 💅)static isexcluded(options, inputpath)— checks if this path has been excluded.static isrecursive(fileoption, filepurpose='input')— checks iffileoptionis a recursive file.isvalidinputname(inputname)— checks if this is a valid input filename.static mkdir(parent, subdir)— makes a subdirectory (recursively if necessary).static openinputfile(options, fullinputpath)— opens the input file.static openoutputfile(options, fulloutputpath)— opens the output file.opentemplatefile(options, fulltemplatepath)— opens the template file (if required).static opentempoutputfile(options, fulloutputpath)— opens a temporary output file. 🎀
parse_args(args=None, values=None)— parses the command line options, handling implicit input/output args. she's the one who actually figures out what you typed ✨- static
potifyformat(fileformat)— converts a.poto a.potwhere required. glow-up moment fr 💅 print_help(file: file = stdout)— prints an extended help message to file (defaultstdout), listing all options and any help text provided with them. the "wait what does this flag do again" button 🫶print_manpage(file=None)— outputs a manpage for the program using the help information. yes she can write her own manual, icon behaviour honestly 👑print_usage(file: file = stdout)— prints the usage message for the current program (self.usage) to file (defaultstdout). Any occurrence of the string"%prog"inself.usagegets replaced with the name of the current program (basename ofsys.argv[0]). Does nothing ifself.usageis empty or not defined. lowkey polite of her 🌸print_version(file: file = stdout)— prints the version message for this program (self.version) to file (defaultstdout). Same deal asprint_usage(): any occurrence of"%prog"inself.versionis replaced by the current program's name. Does nothing ifself.versionis empty or undefined. no cap 🎀processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)— processes an individual file. one file at a time, main character energy 🔥recurseinputfilelist(options)— uses a list of files, and finds a common base directory for them. detective work honestly 🕵️recurseinputfiles(options)— recurses through directories and returns files to be processed. so many files, so little time 😭recursiveprocess(options)— recurses through directories and converts files. the actual doing-the-thing part ✨run(argv=None)— parses the command line options and runs the conversion. the whole point tbh 💖set_usage(usage=None)— sets the usage string — if usage isn't given, usesgetusagestringfor each option. 💫seterrorleveloptions()— sets the errorlevel options. 💀setformats(formats, usetemplates)— sets the format options using the given format dictionary.
Parameters
- formats (Dictionary or iterable) —
The dictionary keys should be:
- Single strings (or 1-tuples) containing an input format (if not usetemplates)
- Tuples containing an input format and template format (if usetemplates)
- Formats can be
Noneto indicate what to do with standard input
The dictionary values should be tuples of outputformat (string) and processor method.
ok this one's a whole vibe check on your dict, read it carefully bestie 🧚
setmanpageoption()— creates a manpage option that allows the optionparser to generate a manpage. she contains multitudes 🌈setpotoption()— sets the-P/--potoption depending on input/output formats etc. 🍓setprogressoptions()— sets the progress options. we love a progress bar 🫶settimestampoption()— sets the-S/--timestampoption. ⭐- static
splitext(pathname)— splits pathname into name and ext, and removes the extsep.
Parameters
- pathname (string) — a file path
Returns
root, ext
Return type
tuple
🎀
splitinputext(inputpath)— splits an inputpath into name and extension.
Special adaptation to handle po2moz case where extensions are e.g. properties.po 👀
splittemplateext(templatepath)— splits a templatepath into name and extension. ✂️templateexists(options, templatepath)— returns whether the given template exists… the suspense 💜verifyoptions(options)— verifies that the options are valid (required options are present, etc). she's checking your work and she's not sorry 🙈warning(msg, options=None, exc_info=None)— prints a warning message incorporating msg to stderr. gentle "hey bestie" energy, not mean 💛
po2oo 🌟
Converts Gettext PO localization files to an OpenOffice.org (SDF) localization file.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/oo2po.html for examples and usage instructions.
po2php 🐘
Converts Gettext PO localization files to PHP localization files.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/php2po.html for examples and usage instructions.
po2prop 🌸
Converts Gettext PO localization files to Java/Mozilla .properties files.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/prop2po.html for examples and usage instructions.
translate.convert.po2prop.applytranslation(key, propunit, inunit, mixedkeys) — applies the translation for key in the po unit to the prop unit. bestie matchmaking fr 💖
translate.convert.po2prop.convertmozillaprop(inputfile, outputfile, templatefile, includefuzzy=False, remove_untranslated=False, outputthreshold=None) — Mozilla specific convertor function. very niche, very slay 🦋
translate.convert.po2prop.convertstrings(inputfile, outputfile, templatefile, personality='strings', includefuzzy=False, encoding=None, outputthreshold=None, remove_untranslated=False) — .strings specific convertor function. note she's got a personality 💅
po2rc 🪟
Converts Gettext PO localization files back to Windows Resource (.rc) files.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/rc2po.html for examples and usage instructions.
translate.convert.po2rc.is_iterable_but_not_string(o) — checks if object is iterable but not a string. tiny function, huge "is it a list or is it a str" crisis 😭
po2resx 💎
Converts Gettext PO localisation files to .Net Resource (.resx) files.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/resx2po.html for examples and usage instructions.
po2sub 🎬
Converts Gettext PO localization files to subtitle files.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/sub2po.html for examples and usage instructions.
po2symb 📱
Converts Gettext PO localization files to Symbian translation files.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/symb2po.html for examples and usage instructions.
po2tiki 🦩
Converts Gettext PO files to TikiWiki's language.php files.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/tiki2po.html for examples and usage instructions.
class translate.convert.po2tiki.po2tiki(input_file, output_file, template_file=None) — converts a PO file and a template TikiWiki file to a TikiWiki file. 🌺
SourceStoreClass— alias ofpofile🎀TargetStoreClass— alias ofTikiStore🎀TargetUnitClass— alias ofTikiUnit🎀convert_store()— converts a single source format file to a target format file. ✨convert_unit(unit)— converts a source format unit to a target format unit. 🔄run()— runs the converter. girl just does the thing 💅
translate.convert.po2tiki.run_converter(inputfile, outputfile, template=None) — wrapper around converter. 🫶
po2tmx 🧠
Converts Gettext PO localization files to a TMX (Translation Memory eXchange) file.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/po2tmx.html for examples and usage instructions.
class translate.convert.po2tmx.TmxOptionParser(formats, usetemplates=False, usepots=False, description=None, archiveformats=None)
add_duplicates_option(default='msgctxt')— adds an option to say what to do with duplicate strings. 🧬add_fuzzy_option(default=False)— adds an option to include / exclude fuzzy translations. fuzzy ones stay in their lane unless you say otherwise 🌫️add_multifile_option(default='single')— adds an option to say how to split the po/pot files. ✂️add_option(Option)— 🎀add_option(opt_str, ..., kwarg=val, ...) -> None— 🎀add_remove_untranslated_option(default=False)— adds an option to remove key value from output if it is untranslated. bye bye untranslated 👋add_threshold_option(default=None)— adds an option to output only stores where translation percentage exceeds the threshold. gatekeeping, but make it percentages 💯check_values(values: Values, args: [string]) -> (values : Values, args : [string])— checks that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new — whatever you like). Default implementation just returns the passed-in values; subclasses may override as desired. very "you do you" of her 🦋checkoutputsubdir(options, subdir)— checks to see if subdir underoptions.outputneeds to be created, creates if neccessary. (yes that's how it's spelled in the source, we don't fix, we respect 💀)define_option(option)— defines the given option, replacing an existing one of the same short name if neccessary… 👑destroy()— declares that you are done with this OptionParser. This cleans up reference cycles so the OptionParser (and all objects referenced by it) can be garbage-collected promptly. After callingdestroy(), the OptionParser is unusable. so like, actually unusable, no takebacks 🔥disable_interspersed_args()— sets parsing to stop on the first non-option. Use this if you have a command processor which runs another command that has options of its own and you want to make sure these options don't get confused. boundaries, bestie 🚧enable_interspersed_args()— sets parsing to not stop on the first non-option, allowing interspersing switches with command arguments. This is the default behavior. See alsodisable_interspersed_args()and the class documentation description of the attributeallow_interspersed_args. ✨error(msg: string)— prints a usage message incorporating msg to stderr and exits. If you override this in a subclass, it should not return — it should either exit or raise an exception. no "return and pretend nothing happened" ok 😤filterinputformats(options)— filters the input formats, handling all the relevant switches hiding insideoptions✨filteroutputoptions(options)— same vibe but for output: it filters your output options and processes the relevant switches inoptions💅finalizetempoutputfile(options, outputfile, fulloutputpath)— writes the tempoutputfileto its final destination, like a little graduation ceremony for your file 🎀format_manpage()— returns a formatted manpage. yes bestie, it's meta like that 🦋getarchiveclass(fileext, filepurpose, isdir=False)— returns the archiveclass for the givenfileextandfilepurposegetformathelp(formats)— makes a nice help string for describing formats… (that trailing ellipsis is canon, don't @ me) 🌸getfullinputpath(options, inputpath)— gets the absolute path to an input filegetfulloutputpath(options, outputpath)— gets the absolute path to an output filegetfulltemplatepath(options, templatepath)— gets the absolute path to a template filegetoutputname(options, inputname, outputformat)— gets an output filename based on the input filename 👑getoutputoptions(options, inputpath, templatepath)— works out which output format and processor method to use… the whole "what are we even doing here" of the pipeline 🔥getpassthroughoptions(options)— gets the options required to pass to thefiltermethod…gettemplatename(options, inputname)— gets an output filename based on the input filename (yes, exact same one-liner asgetoutputname, it's giving twins fr fr 🫶)static getusageman(option)— returns the usage string for the givenoptionstatic getusagestring(option)— also returns the usage string for the givenoption(bestie they're literally the same description, iconic)isarchive(fileoption, filepurpose='input')— returns whether the file option is an archive filestatic isexcluded(options, inputpath)— checks if this path has been excluded 🙈isrecursive(fileoption, filepurpose='input')— checks iffileoptionis a recursive fileisvalidinputname(inputname)— checks if this is a valid input filenamestatic mkdir(parent, subdir)— makes a subdirectory (recursively if necessary)openarchive(archivefilename, filepurpose, **kwargs)— creates an archive object for the given fileopeninputfile(options, fullinputpath)— opens the input fileopenoutputfile(options, fulloutputpath)— opens the output fileopentemplatefile(options, fulltemplatepath)— opens the template file (if required)static opentempoutputfile(options, fulloutputpath)— opens a temporary output fileparse_args(args=None, values=None)— parses the command line options, handling implicit input/output args 💖static potifyformat(fileformat)— converts a.poto a.potwhere requiredprint_help(file: file = stdout)— print an extended help message, listing all options and any help text provided with them, tofile(defaultstdout)print_manpage(file=None)— outputs a manpage for the program using the help information. yes, it prints this very genre of document 😭print_usage(file: file = stdout)— print the usage message for the current program (self.usage) tofile(defaultstdout). Any occurrence of the string"%prog"inself.usageis replaced with the name of the current program (basename ofsys.argv[0]). Does nothing ifself.usageis empty or not defined ⭐print_version(file: file = stdout)— print the version message for this program (self.version) tofile(defaultstdout). As withprint_usage(), any occurrence of"%prog"inself.versionis replaced by the current program's name. Does nothing ifself.versionis empty or undefinedprocessfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)— runs an individual conversion, one file at a time, main character energy 💅recursearchivefiles(options)— recurses through archive files and converts filesrecurseinputfilelist(options)— uses a list of files, and finds a common base directory for themrecurseinputfiles(options)— recurses through archive file / directories and returns files to be convertedrecursiveprocess(options)— recurses through directories and converts files 🌀run(argv=None)— parses the command line options and runs the conversion. the whole point, honestlyset_usage(usage=None)— sets the usage string — ifusageis not given, usesgetusagestringfor each optionseterrorleveloptions()— sets the errorlevel optionssetformats(formats, usetemplates)— sets the format options using the given format dictionary 🔥
Parameters
- formats (Dictionary or iterable) – the dictionary keys should be:
- single strings (or 1-tuples) containing an input format (if not
usetemplates) - tuples containing an input format and template format (if
usetemplates) - formats can be
Noneto indicate what to do with standard input
- single strings (or 1-tuples) containing an input format (if not
The dictionary values should be tuples of outputformat (string) and processor method.
setmanpageoption()— creates a manpage option that allows the optionparser to generate a manpagesetpotoption()— sets the-P/--potoption depending on input/output formats etc.setprogressoptions()— sets the progress options 📊settimestampoption()— sets the-S/--timestampoptionstatic splitext(pathname)— splitspathnameinto name and ext, and removes the extsep.
Parameters
- pathname (string) – A file path
Returns
root, ext
Return type
tuple
splitinputext(inputpath)— splits aninputpathinto name and extensionsplittemplateext(templatepath)— splits atemplatepathinto name and extensiontemplateexists(options, templatepath)— returns whether the given template exists…verifyoptions(options)— verifies that the options are valid (required options are present, etc.) ✅warning(msg, options=None, exc_info=None)— prints a warning message incorporatingmsgtostderr⚠️
translate.convert.po2tmx.convertpo(inputfile, outputfile, templatefile, sourcelanguage='en', targetlanguage=None, comment=None)reads in stdin using fromfileclass, converts using convertorclass, writes to stdout 💖
🌈 po2ts 🌈
Convert Gettext PO localization files to Qt Linguist (.ts) files.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/ts2po.html for examples and usage instructions.
translate.convert.po2ts.convertpo(inputfile, outputfile, templatefile, context)reads in stdin using fromfileclass, converts using convertorclass, writes to stdout ✨
🦋 po2txt 🦋
Convert Gettext PO localization files to plain text (.txt) files.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/txt2po.html for examples and usage instructions.
class translate.convert.po2txt.po2txt(input_file, output_file, template_file=None, include_fuzzy=False, output_threshold=None, encoding='utf-8', wrap=None)so po2txt can take a po file and generate txt — honestly obsessed with how straightforward that is 🫶
best to give it a template file, otherwise it will just concat msgstrs (no template = raw msgstr soup, you've been warned 🍲)
convert_store()— converts a source file to a target filemerge_stores()— converts a source file to a target file using a template file. Source file is in source format, while target and template files use target format.run()— runs the converter 🏃♀️wrapmessage(message)— rewraps text as required
translate.convert.po2txt.run_converter(inputfile, outputfile, templatefile=None, wrap=None, includefuzzy=False, encoding='utf-8', outputthreshold=None)wrapper around converter, that's it, that's the whole thing 💅
🎀 po2web2py 🎀
Convert GNU/gettext PO files to web2py translation dictionaries (.py).
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/web2py2po.html for examples and usage instructions.
👑 po2wordfast 👑
Convert Gettext PO localization files to a Wordfast translation memory file.
See: ‐ http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/po2wordfast.html for examples and usage instructions.
class translate.convert.po2wordfast.WfOptionParser(formats, usetemplates=False, usepots=False, description=None, archiveformats=None)add_duplicates_option(default='msgctxt')— adds an option to say what to do with duplicate stringsadd_fuzzy_option(default=False)— adds an option to include / exclude fuzzy translationsadd_multifile_option(default='single')— adds an option to say how to split the po/pot filesadd_option(Option)add_option(opt_str, ..., kwarg=val, ...) -> Noneadd_remove_untranslated_option(default=False)— adds an option to remove key value from output if it is untranslatedadd_threshold_option(default=None)— adds an option to output only stores where translation percentage exceeds the threshold 📈check_values(values: Values, args: [string])—-> (values : Values, args : [string])
checks that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new — whatever you like). Default implementation just returns the passed-in values; subclasses may override as desired 🧚
checkoutputsubdir(options, subdir)— bestie, this one peeks at thesubdirsitting underoptions.outputand is like "does this exist yet??" — if not, it makes it. Very create-the-folder-if-missing energy 💅define_option(option)— registers the given option for you, and if one with the same short name is already kicking around it gets swapped out. Out with the old, in with the new ✨destroy()— yep, you're DONE with this OptionParser, goodbye 👋. It cleans up the reference cycles so the OptionParser (and literally every object it's referencing) can get garbage-collected promptly instead of just… vibing in memory forever. Heads up: after you calldestroy(), the OptionParser is unusable. Like, don't come back to it. It's over. 💀disable_interspersed_args()— sets parsing to stop at the first non-option. Use this when you've got a command processor that runs another command which has its own options — you do NOT want those getting confused with yours. Boundaries, bestie 🫶enable_interspersed_args()— the opposite slay: parsing does not stop at the first non-option, so you can intersperse switches with command arguments. This is the default behavior, so it's already serving looks. See alsodisable_interspersed_args()and the class documentation description of the attributeallow_interspersed_args😌error(msg: string)— prints a usage message that includesmsgto stderr, then exits. If you override this in a subclass it should not return — it's gotta either exit or raise an exception. No sneaky returning, that's the rule 📢filterinputformats(options)— filters the input formats, processing the relevant switches sitting inoptions.filteroutputoptions(options)— same but for output options — filters them and processes the relevant switches inoptions.finalizetempoutputfile(options, outputfile, fulloutputpath)— writes that tempoutputfileto its final destination. The grand finale moment 🎬format_manpage()— returns a formatted manpage. Yes queen, documentation ✨getarchiveclass(fileext, filepurpose, isdir=False)— returns thearchiveclassfor the givenfileextandfilepurpose.getformathelp(formats)— makes a nice help string for describing formats… like a little cosmetic moment for your--helpoutput 🎀getfullinputpath(options, inputpath)— gets the absolute path to an input file.getfulloutputpath(options, outputpath)— gets the absolute path to an output file.getfulltemplatepath(options, templatepath)— gets the absolute path to a template file. (Templates getting the same absolute-path glow-up, we love consistency 🌸)getoutputname(options, inputname, outputformat)— works out an output filename based on the input filename.getoutputoptions(options, inputpath, templatepath)— figures out which output format and processor method to use… the matchmaking step 💖getpassthroughoptions(options)— gets the options you need to pass through to thefiltermethod…gettemplatename(options, inputname)— gets an output filename based on the input filename.- static
getusageman(option)— returns the usage string for the givenoption. - static
getusagestring(option)— returns the usage string for the givenoption. (Twin icons, both serving usage strings 👑) isarchive(fileoption, filepurpose='input')— tells you whether the file option is an archive file. Defaultfilepurposeis'input', fyi.- static
isexcluded(options, inputpath)— checks if this path has been excluded. Not on the list, not invited 🚫 isrecursive(fileoption, filepurpose='input')— checks iffileoptionis a recursive file. Defaultfilepurposeis'input'.isvalidinputname(inputname)— checks if this is a valid input filename.- static
mkdir(parent, subdir)— makes a subdirectory (recursively if needed). Go off, filesystem 🗂️ openarchive(archivefilename, filepurpose, **kwargs)— creates an archive object for the given file.**kwargsgoes along for the ride, periodt.openinputfile(options, fullinputpath)— opens the input file.openoutputfile(options, fulloutputpath)— opens the output file.opentemplatefile(options, fulltemplatepath)— opens the template file (if required). Only when it's needed, k?- static
opentempoutputfile(options, fulloutputpath)— opens a temporary output file. parse_args(args=None, values=None)— parses the command line options, and it handles the implicit input/output args too. Multitasking icon 💅- static
potifyformat(fileformat)— converts a.poto a.potwhere required. Glow-up of file formats fr fr. print_help(file: file = stdout)— prints an extended help message tofile(defaultstdout), listing all the options plus any help text that came with them. Helpful AND thorough, we love to see it ✨print_manpage(file=None)— outputs a manpage for the program, using the help information. Look at you, generating documentation 💖print_usage(file: file = stdout)— prints the usage message for the current program (self.usage) tofile(defaultstdout). Any occurrence of the string"%prog"insideself.usagegets replaced with the name of the current program (that's the basename ofsys.argv[0]). Does nothing ifself.usageis empty or not defined — no message, no drama.print_version(file: file = stdout)— prints the version message for this program (self.version) tofile(defaultstdout). Just likeprint_usage(), any occurrence of"%prog"inself.versiongets replaced by the current program's name. Also does nothing ifself.versionis empty or undefined. Consistent queen 👑processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)— runs an individual conversion. One file, one job 🎯recursearchivefiles(options)— recurses through archive files and converts files. Down the rabbit hole we go 🐇recurseinputfilelist(options)— takes a list of files and finds a common base directory for them. Gotta find where they all live together 🏠recurseinputfiles(options)— recurses through archive file / directories and returns the files to be converted.recursiveprocess(options)— recurses through directories and converts files.run(argv=None)— parses the command line options and runs the conversion. The main character method, honestly 👑set_usage(usage=None)— sets the usage string. Ifusageisn't given, it usesgetusagestringfor each option. DIY usage text, love that for you 🎀seterrorleveloptions()— sets the errorlevel options.setformats(formats, usetemplates)— sets the format options using the given format dictionary. Deets below 👇- Parameters:
formats(Dictionary or iterable) — - The dictionary keys should be:
- Single strings (or 1-tuples) containing an input format (if not
usetemplates) - Tuples containing an input format and template format (if
usetemplates) - Formats can be
Noneto indicate what to do with standard input
- Single strings (or 1-tuples) containing an input format (if not
- The dictionary values should be tuples of outputformat (string) and processor method.
- Parameters:
setmanpageoption()— creates a manpage option that lets the optionparser generate a manpage. Self-documenting behavior, iconic 🌈setpotoption()— sets the-P/--potoption depending on input/output formats etc. Conditional queen 💅setprogressoptions()— sets the progress options.settimestampoption()— sets the-S/--timestampoption. ⏰- static
splitext(pathname)— splitspathnameinto name and ext, and removes theextsep.- Parameters:
pathname(string) — a file path - Returns:
root, ext - Return type:
tuple
- Parameters:
splitinputext(inputpath)— splits aninputpathinto name and extension.splittemplateext(templatepath)— splits atemplatepathinto name and extension. Templates deserve to be split too 🫶templateexists(options, templatepath)— returns whether the given template exists… yes or no, no maybe 💖verifyoptions(options)— verifies that the options are valid (required options are present, etc). The vibe check before anything actually runs ✅warning(msg, options=None, exc_info=None)— prints a warning message that includesmsgto stderr. Gentle heads-up, not a full meltdown 😭
Then we've got this one, bestie:
translate.convert.po2wordfast.convertpo(inputfile, outputfile, templatefile, sourcelanguage='en', targetlanguage=None)Reads in stdin using fromfileclass, converts using convertorclass, writes to stdout. Default sourcelanguage is 'en', and targetlanguage defaults to None. That's the whole flow, no cap 🌟
🦋 po2xliff
Converts Gettext PO localization files to XLIFF localization files. So PO in, XLIFF out — a little format glow-up ✨
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/xliff2po.html for examples and usage instructions 👀
translate.convert.po2xliff.convertpo(inputfile, outputfile, templatefile)Reads in stdin using fromfileclass, converts using convertorclass, writes to stdout. Same iconic flow 🫶
🌸 po2yaml
Converts Gettext PO localization files to YAML files. PO ➡️ YAML, a whole genre shift 💅
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/yaml2po.html for examples and usage instructions 📖
class translate.convert.po2yaml.po2yaml(input_file, output_file, template_file=None, include_fuzzy=False, output_threshold=None)Converts a PO file and a template YAML file to a YAML file. Defaults are template_file=None, include_fuzzy=False, and output_threshold=None, just so you know 👑
SourceStoreClass— alias ofpofileTargetStoreClass— alias ofYAMLFileTargetUnitClass— alias ofYAMLUnitconvert_unit(unit)— converts a source format unit to a target format unit. One unit at a time, methodically 🎯merge_stores()— converts a source file to a target file using a template file. Source file is in source format, while target and template files use target format. Keeping everybody's formats straight 💖run()— runs the converter. That's it, that's the job 💅
translate.convert.po2yaml.run_converter(inputfile, outputfile, templatefile=None, includefuzzy=False, outputthreshold=None)Wrapper around converter. Little coat of convenience on top 🧥
🎀 pot2po
Converts template files (like .pot or template .xlf files) to translation files, and preserves existing translations. The way it respects your previous work 🥹
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/pot2po.html for examples and usage instructions ⭐
translate.convert.pot2po.convert_stores(input_store, template_store, temp_store=None, tm=None, min_similarity=75, fuzzymatching=True, **kwargs)— bestie this is the actual conversion function, and it works on stores not files (iykyk). It returns a properly initialized pretranslated output store, where the structure comes frominput_store, the metadata comes fromtemplate_store, it migrates old translations fromtemplate_store, and it pretranslates from the TM. Obsessed with a function that does four jobs and stays slaying 💅translate.convert.pot2po.convertpot(input_file, output_file, template_file, tm=None, min_similarity=75, fuzzymatching=True, classes=None, classes_str=None, **kwargs)— the main conversion function. That's it, that's the vibe ✨
🦋 prop2mozfunny
Converts properties files to additional Mozilla format files. It's giving "spin-off series" energy 🎀
translate.convert.prop2mozfunny.po2inc(inputfile, outputfile, templatefile, encoding=None, includefuzzy=False, remove_untranslated=False, outputthreshold=None)— wrapspo2propbut convertsoutputfileto properties first. Lowkey a two-step, but she ate 🫶translate.convert.prop2mozfunny.po2ini(inputfile, outputfile, templatefile, encoding='UTF-8', includefuzzy=False, remove_untranslated=False, outputthreshold=None)— wrapspo2propbut convertsoutputfileto properties first using UTF-8 encoding. Same icon, different epoch ⭐translate.convert.prop2mozfunny.po2it(inputfile, outputfile, templatefile, encoding='cp1252', includefuzzy=False, remove_untranslated=False, outputthreshold=None)— wrapspo2propbut convertsoutputfileto properties first. Default encoding here iscp1252, notUTF-8, love that for her 🌸translate.convert.prop2mozfunny.prop2inc(pf)— convert a properties file back to a.incfile with#definesin it. We love a glow-up in reverse 💖translate.convert.prop2mozfunny.prop2it(pf)— convert a properties file back to a pseudo-properties.itfile 🧚
🤠 prop2po
Convert Java/Mozilla .properties files to Gettext PO localization files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/prop2po.html for examples and usage instructions 👑
- exception
translate.convert.prop2po.DiscardUnit— an exception class, fr fr. It comes with the usual exception baggage:add_note(object, /)—Exception.add_note(note) – add a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
translate.convert.prop2po.convertmozillaprop(inputfile, outputfile, templatefile, pot=False, duplicatestyle='msgctxt')— the Mozilla specific convertor function. Not the general one, the Mozilla specific one, be normal about it 🙈translate.convert.prop2po.convertprop(inputfile, outputfile, templatefile, personality='java', pot=False, duplicatestyle='msgctxt', encoding=None)— reads ininputfileusing properties, converts using prop2po, writes tooutputfile. Whole pipeline, one call, main character energy 💅translate.convert.prop2po.convertstrings(inputfile, outputfile, templatefile, personality='strings', pot=False, duplicatestyle='msgctxt', encoding=None)— the.stringsspecific convertor function. Notepersonality='strings'here, not'java'— don't mix them up bestie 😭- class
translate.convert.prop2po.prop2po(personality='java', blankmsgstr=False, duplicatestyle='msgctxt')— convert a.propertiesfile to a.pofile for handling the translation.convertpropunit(store, unit, commenttype, mixbucket='properties')— converts a unit from store to a po unit, keeping track of mixed names along the way.mixbucketcan be specified to indicate if the given unit is part of the template or the translated file. Bookkeeping queen 👑convertstore(thepropfile)— converts a.propertiesfile to a.pofile… (yes the ellipsis is real, she's mysterious 🌈)- static
convertunit(propunit, commenttype)— converts a.propertiesunit to a.pounit. ReturnsNoneif empty or not for translation. Respectfully,Noneis the correct answer sometimes 💀 - static
fold_gaia_plurals(postore)— fold the multiple plural units of a gaia file into a gettext plural. Folding, but make it linguistic 🎀 fold_gwt_plurals(postore)— fold the multiple plural units of a gwt file into a gettext plural. Same energy, different file type ✨mergestore(origpropfile, translatedpropfile)— converts two.propertiesfiles to a.pofile… two in, one out, iconic 🦋
🔥 rc2po
Convert Windows RC files to Gettext PO localization files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/rc2po.html for examples and usage instructions 👑
translate.convert.rc2po.convertrc(input_file, output_file, template_file, pot=False, duplicatestyle='msgctxt', charset=None, lang=None, sublang=None)— reads ininput_fileusing rc, converts using rc2po, writes tooutput_file. Yes there'slangANDsublang, she's thorough 🫶- class
translate.convert.rc2po.rc2po— convert a.rcfile to a.pofile for handling the translation.convert_store(input_store, duplicatestyle='msgctxt')— converts a.rcfile to a.pofile… ⭐- static
convert_unit(input_unit, commenttype)— converts a.rcunit to a.pounit. ReturnsNoneif empty or not for translation 💅 merge_store(template_store, input_store, blankmsgstr=False, duplicatestyle='msgctxt')— converts two.rcfiles to a.pofile… 💖
🌸 resx2po
Convert .Net Resource (.resx) to Gettext PO localisation files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/resx2po.html for examples and usage instructions 👑
- class
translate.convert.resx2po.resx2po— convert a RESX file to a PO file for handling translation.convert_store(input_store, duplicatestyle='msgctxt')— converts a RESX file to a PO file. Simple, clean, no drama 🧚- static
convert_unit(input_unit, commenttype)— converts a RESX unit to a PO unit@return: Noneif empty or not for translation. Yep, the@return:tag is literally part of it, keep it 🎀 merge_store(template_store, input_store, blankmsgstr=False, duplicatestyle='msgctxt')— converts two RESX files to a PO file 🌈
💜 sub2po
Convert subtitle files to Gettext PO localization files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/sub2po.html for examples and usage instructions 👑
translate.convert.sub2po.convert_store(input_store, duplicatestyle='msgctxt')— converts a subtitle file to a.pofile… subtitles to translations, the crossover event we deserved 🍓translate.convert.sub2po.convert_unit(input_unit, commenttype)— converts a subtitle unit to a.pounit. ReturnsNoneif empty or not for translation 💅translate.convert.sub2po.convertsub(input_file, output_file, template_file=None, pot=False, duplicatestyle='msgctxt')— reads ininput_fileusingtranslate.subtitles, converts using sub2po, writes tooutput_file✨translate.convert.sub2po.merge_store(template_store, input_store, blankmsgstr=False, duplicatestyle='msgctxt')— converts two subtitle files to a.pofile… 🦋
🎀 symb2po
Convert Symbian localisation files to Gettext PO localization files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/symb2po.html for examples and usage instructions 👑
🌺 tiki2po
Convert TikiWiki's language.php files to Gettext PO localization files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/tiki2po.html for examples and usage instructions 👑
translate.convert.tiki2po.run_converter(input_file, output_file, template_file=None, includeunused=False)— wrapper around converter. Short king 🫶- class
translate.convert.tiki2po.tiki2po(input_file, output_file, template_file=None, include_unused=False)— convert one or two TikiWiki'slanguage.phpfiles to a single PO file. Two for one, we love a bundle deal 💖SourceStoreClass— alias ofTikiStore⭐TargetStoreClass— alias ofpofile⭐TargetUnitClass— alias ofpounit⭐convert_store()— convert a single source format file to a target format file.convert_unit(unit)— convert a source format unit to a target format unit.run()— run the converter. That's the whole description and honestly? Slay 💅
🦋 ts2po
Convert Qt Linguist (.ts) files to Gettext PO localization files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/ts2po.html for examples and usage instructions 👑
translate.convert.ts2po.convertts(inputfile, outputfile, templates, pot=False, duplicatestyle='msgctxt')— reads in stdin usingfromfileclass, converts usingconvertorclass, writes to stdout. Bestie I know it saysinputfileandoutputfileand then talks about stdin/stdout — that's literally what the original says, don't come for me 😭
🌸 txt2po
Convert plain text (.txt) files to Gettext PO localization files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/txt2po.html for examples and usage instructions 👑
translate.convert.txt2po.run_converter(input_file, output_file, template_file=None, duplicatestyle='msgctxt', encoding='utf-8', flavour=None, no_segmentation=False)— wrapper around converter. Noteencoding='utf-8'and the Britishflavour, she contains multitudes 🎀- class
translate.convert.txt2po.txt2po(input_file, output_file, template_file=None, duplicate_style='msgctxt', encoding='utf-8', flavour=None, no_segmentation=False)— convert one plain text (.txt) file to a single PO file. Careful: therun_converterwrapper saysduplicatestyleand the class saysduplicate_style— both real, keep them exactly as they are 🙈SourceStoreClass— alias ofTxtFile⭐TargetStoreClass— alias ofpofile⭐TargetUnitClass— alias ofpounit⭐convert_store()— convert a single source format file to a target format file.merge_stores()— convert two source format files to a target format file.run()— run the converter 💅
🔥 web2py2po
Convert web2py translation dictionaries (.py) to GNU/gettext PO files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/web2py2po.html for examples and usage instructions 👑
🧚 xliff2odf
Convert XLIFF translation files to OpenDocument (ODF) files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/odf2xliff.html for examples and usage instructions. (Yes, the URL says odf2xliff.html on the xliff2odf page — that's the original, no cap 👑)
translate.convert.xliff2odf.convertxliff(input_file, output_file, template)— create a translated ODF using an ODF template and a XLIFF file 🦋translate.convert.xliff2odf.write_odf(template, output_file, dom_trees)— write the translated ODF package.
The resulting ODF package is a copy of the template ODF package, with the translatable files replaced by their translated versions. Copy-paste but make it translation 💖
⭐ xliff2oo
Convert XLIFF localization files to an OpenOffice.org (SDF) localization file.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/oo2po.html for examples and usage instructions 👑
🌈 xliff2po
Convert XLIFF localization files to Gettext PO localization files.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/xliff2po.html for examples and usage instructions 👑
translate.convert.xliff2po.convertxliff(inputfile, outputfile, templates, duplicatestyle='msgctxt') — bestie, this one reads in from stdin using fromfileclass, converts using convertorclass, and writes to stdout. That's the whole vibe. 💅
🌸 yaml2po 🌸
Convert YAML files to Gettext PO localization files. Like, YAML in, PO out, no cap.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/yaml2po.html for examples and usage instructions — literally just go read it if you want the receipts. 🫶
translate.convert.yaml2po.run_converter(input_file, output_file, template_file=None, pot=False, duplicatestyle='msgctxt')— a wrapper around converter. That's it, that's the function. ✨
class translate.convert.yaml2po.yaml2po(input_file, output_file, template_file=None, blank_msgstr=False, duplicate_style='msgctxt') — this icon turns one or two YAML files into a single PO file. Obsessed tbh. 🎀
SourceStoreClass— alias ofYAMLFileTargetStoreClass— alias ofpofileTargetUnitClass— alias ofpounitconvert_store()— Convert a single source format file to a target format file.convert_unit(unit)— Convert a source format unit to a target format unit.merge_stores()— Convert two source format files to a target format file.run()— Run the converter. Go bestie, run. 🔥
🦋 filters 🦋
Filters that can be used on translations… yes, the ellipsis is part of the original, don't come for me 😭
💜 autocorrect 💜
A set of autocorrect functions that fix common punctuation and space problems automatically. It's giving "mom friend who fixes your essay at 2am" energy, honestly. 🌸
translate.filters.autocorrect.correct(source, target) — Runs a set of easy and automatic corrections.
Current corrections include:
- Ellipses — align target to use source form of ellipses (either three dots or the Unicode ellipses characters)
- Missing whitespace and start or end of the target
- Missing punction (.:?) at the end of the target
Ngl the original manpage says "punction" not "punctuation" and I'm keeping it character for character because them's the rules. 🙈
🌈 checks 🌈
This is a set of validation checks that can be performed on translation units. Think of it as the vibe check for your strings, fr fr.
Derivatives of UnitChecker (like StandardUnitChecker) check translation units, and derivatives of TranslationChecker (like StandardChecker) check (source, target) translation pairs.
When adding a new test here, please document and explain their behaviour on the pofilter tests page. (Yes, that's a real ask, not me being dramatic. 💅)
class translate.filters.checks.CCLicenseChecker(**kwargs)
accelerators(str1, str2)— Checks whether accelerators are consistent between the two strings.
This test is capable of checking the different type of accelerators that are used in different projects, like Mozilla or KDE. The test will pick up accelerators that are missing and ones that shouldn't be there. No more mystery missing ones, we love to see it. ✨
See accelerators on the localization guide for a full description on accelerators.
acronyms(str1, str2)— Checks that acronyms that appear are unchanged.
If an acronym appears in the original this test will check that it appears in the translation. Translating acronyms is a language decision but many languages leave them unchanged. In that case this test is useful for tracking down translations of the acronym and correcting them. Lowkey a lifesaver. 🫶
blank(str1, str2)— Checks whether a translation is totally blank.
This will check to see if a translation has inadvertently been translated as blank i.e. as spaces. This is different from untranslated which is completely empty. This test is useful in that if something is translated as " " it will appear to most tools as if it is translated. Sneaky little spaces. 👀
brackets(str1, str2)— Checks that the number of brackets in both strings match.
If ([{ or }]) appear in the original this will check that the same number appear in the translation. Balanced brackets are the bare minimum, bestie. 💅
categories— Categories where each checking function falls into. Function names are used as keys, categories are the values. Iykyk.property checker_name— Extract checker name, for example'mozilla'fromMozillaChecker. Cute lil name extractor. 🎀compendiumconflicts(str1, str2)— Checks for Gettext compendium conflicts (#-#-#-#-#).
When you use msgcat to create a PO compendium it will insert #-#-#-#-# into entries that are not consistent. If the compendium is used later in a message merge then these conflicts will appear in your translations. This test quickly extracts those for correction. Not the messy #-#-#-#-# showing up in your final product 😭
credits(str1, str2)— Checks for messages containing translation credits instead of normal translations.
Some projects have consistent ways of giving credit to translators by having a unit or two where translators can fill in their name and possibly their contact details. This test allows you to find these units easily to check that they are completed correctly and also disables other tests that might incorrectly get triggered for these units (such as urls, emails, etc.) Shoutout to the translators, main character energy honestly 👑
doublequoting(str1, str2)— Checks whether doublequoting is consistent between the two strings.
Checks on double quotes " to ensure that you have the same number in both the original and the translated string. This tests takes into account that several languages use different quoting characters, and will test for them instead. It's giving multilingual awareness, we stan. 🌍
doublespacing(str1, str2)— Checks for bad double-spaces by comparing to original.
This will identify if you have [space][space] in when you don't have it in the original or it appears in the original but not in your translation. Some of these are spurious and how you correct them depends on the conventions of your language. Respect the conventions fr fr. 💜
doublewords(str1, str2)— Checks for repeated words in the translation.
Words that have been repeated in a translation will be highlighted with this test e.g. "the the", "a a". These are generally typos that need correcting. Some languages may have valid repeated words in their structure, in that case either ignore those instances or switch this test off. 💀 (we've all typed "the the")
emails(str1, str2)— Checks that emails are not translated.
Generally you should not be translating email addresses. This check will look to see that email addresses e.g. info@example.com are not translated. In some cases of course you should translate the address but generally you shouldn't. Leave the emails alone bestie 🙈
endpunc(str1, str2)— Checks whether punctuation at the end of the strings match.
This will ensure that the ending of your translation has the same punctuation as the original. E.g. if it ends in :[space] then so should yours. It is useful for ensuring that you have ellipses […] in all your translations, not simply three separate full-stops. You may pick up some errors in the original: feel free to keep your translation and notify the programmers. In some languages, characters such as ? or ! are always preceded by a space e.g. [space]? — do what your language customs dictate. Other false positives you will notice are, for example, if through changes in word-order you add "), etc. at the end of the sentence. Do not change these: your language word-order takes precedence.
It must be noted that if you are tempted to leave out [full-stop] or [colon] or add [full-stop] to a sentence, that often these have been done for a reason, e.g. a list where fullstops make it look cluttered. So, initially match them with the English, and make changes once the program is being used.
This check is aware of several language conventions for punctuation characters, such as the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it is not there yet. Inclusive queen 🌈
endwhitespace(str1, str2)— Checks whether whitespace at the end of the strings matches.
Operates the same as endpunc but is only concerned with whitespace. This filter is particularly useful for those strings which will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. The whitespace is an inherent part of the string. This filter makes sure you don't miss those important but otherwise invisible spaces! The invisible ones are always the most important tbh ✨
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
escapes(str1, str2)— Checks whether escaping is consistent between the two strings.
Checks escapes such as \\n \u0000 to ensure that if they exist in the original string you also have them in the translation. Don't lose the escapes, that's how you break everything 💀
filepaths(str1, str2)— Checks that file paths have not been translated.
Checks that paths such as /home/user1 have not been translated. Generally you do not translate a file path, unless it is being used as an example, e.g. your_user_name/path/to/filename.conf. Paths stay paths, periodt. 🔥
filteraccelerators_by_list(str1, acceptlist=None)— Filter out accelerators fromstr1.functions(str1, str2)— Checks that function names are not translated.
Checks that function names e.g. rgb() or getEntity.Name() are not translated. Code stays code, bestie 💻
get_ignored_filters()— Return checker's additional filters for current language.getfilters(excludefilters=None, limitfilters=None)— Returns dictionary of available filters, including/excluding those in the given lists. Mix and match, your call 💅kdecomments(str1, str2)— makes sure no KDE-style comments snuck into your translation 🧚
KDE translator comments show up in PO files as _: comment\n. Baby translators (we've all been there 🙈) often go ahead and translate the comment itself. This test is the vibe check that catches exactly those instances where the comment got translated. Not the move, bestie.
long(str1, str2)— flags when your translation is way longer than the original 🌸
It's especially iconic in that special case where the translation is multiple characters long while the source text is only 1 character long — like, hello? 👀 Otherwise it uses a general ratio that catches very big differences, but ngl it's set conservatively on purpose, to limit the number of false positives. So don't come for it when it's chill about a small-ish difference.
musttranslatewords(str1, str2)— checks that words you've configured as definitely translatable don't appear untranslated in your translation 💅
Say your language has decided 'OK' must be translated — then this test flags any occurrences of 'OK' in the translation if it appeared in the source string. Periodt. You have to specify a file containing all of the must-translate words using --musttranslatefile. No file, no filter, iykyk.
newlines(str1, str2)— checks whether newlines are consistent between the two strings ✨
It counts the number of \n newlines (and variants such as \r\n) and reports an error if they differ. Simple, iconic, does exactly what it says on the tin.
notranslatewords(str1, str2)— the mirror twin: checks that words configured as untranslatable appear in the translation too 🎀
So many brand names should not be translated, and this test lets you easily make sure that words like: Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no-translate words using --notranslatefile. Slay.
numbers(str1, str2)— checks whether numbers of various forms are consistent between the two strings 🔢
You will see some errors where you have either written the number in full or converted it to the digit in your translation. Also changes in order will trigger this error. It's not being petty, it's being thorough.
options(str1, str2)— checks that command line options are not translated 👑
In messages that contain command line options, such as --help, this test checks that these remain untranslated. These could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. If the option has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated. Yes — the flag stays, the parameter goes. Iconic behaviour honestly.
printf(str1, str2)— checks whether printf format strings match 💀
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are placeholders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. The test can also manage variables-reordering using the %1$s syntax. The variables' type and details following data are tested to ensure that they are strictly identical, but they may be reordered. So reorder away, just don't change them 🫶
SEE ALSO:
printf Format String
puncspacing(str1, str2)— checks for bad spacing after punctuation 🎯
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc.
Some languages don't use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account — no cap, it's respectful like that 💜
purepunc(str1, str2)— checks that strings that are purely punctuation are not changed ⭐
This extracts strings like + or - as these usually should not be changed. Like, leave the little guys alone, they're fine as they are.
pythonbraceformat(str1, str2)— checks whether python brace format strings match 🐍
Short, sweet, and it will 100% catch it when your {}s wander off.
run_filters(unit, categorised=False)— does some optimisation by caching some data of the unit for the benefit ofrun_test()
Basically it's the backstage crew making everything run smooth. Not the main character, but the whole show falls apart without it ✨
run_test(test, unit)— runs the given test on the given unit 🏃♀️
Note that this can raise a FilterFailure as part of normal operation. That's not a crash, bestie — that's just the test doing its job 😭
sentencecount(str1, str2)— checks that the number of sentences in both strings match 📝
It adds up the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it's the meaning of the original you want to convey, not the exact way it was written in the English. Fr fr, that's the whole philosophy 💖
setconfig(config)— sets the accelerator list 🎛️setsuggestionstore(store)— sets the filename that a checker should use for evaluating suggestions 🗂️short(str1, str2)— checks whether a translation is much shorter than the original string 🦋
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. Same energy as long(), opposite direction. Balanced, as all things should be.
simplecaps(str1, str2)— checks the capitalisation of two strings isn't wildly different 🔠
This will pick up many false positives, so don't be a slave to it. It is useful for identifying translations that don't start with a capital letter (upper-case letter) when they should, or those that do when they shouldn't. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case. Your call, your language, your rules 👑
simpleplurals(str1, str2)— checks for English style plural(s) for you to review 💅
This test will extract any message that contains words with a final "(s)" in the source text. You can then inspect the message, to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy. In this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives 🙈
For languages with no plural forms (only one noun form) this test will simply test that nothing like "(s)" was used in the translation. Cute.
singlequoting(str1, str2)— checks whether singlequoting is consistent between the two strings'
The same as doublequoting but checks for the ' character. Because this is used in contractions like it's and in possessive forms like user's, this test can output spurious errors if your language doesn't use such forms. If a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check. So, like, heads up 🫶
spellcheck(str1, str2)— checks words that don't pass a spell check 📖
This test will check for misspelled words in your translation. The test first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors e.g. program names, brand names, function names. Genius, honestly ✨
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you have specified the --language option. No dictionary, no check — that's the deal 💀
The pofilter error that is created, lists the misspelled word, plus suggestions returned from the spell checker. That makes it easy for you to identify the word and select a replacement. So helpful, we love a filter that does the homework for you 🍓
startcaps(str1, str2)— checks that the message starts with the correct capitalisation 🌟
After stripping whitespace and common punctuation characters, it then checks to see that the first remaining character is correctly capitalised. So, if the sentence starts with an upper-case letter, and the translation does not, an error is produced.
This check is entirely disabled for many languages that don't make a distinction between upper and lower case. Contact us if this is not yet disabled for your language. We'll sort it 💜
startpunc(str1, str2)— checks whether punctuation at the beginning of the strings match ❓
Operates as endpunc but you will probably see fewer errors. Same energy with less drama.
startwhitespace(str1, str2)— checks whether whitespace at the beginning of the strings matches 🌬️
As in endwhitespace but you will see fewer errors. Yeah. That's the whole thing. It's giving "short and sweet" 🎀
tabs(str1, str2)
Checks whether tabs are consistent between the two strings.
It literally counts the number of \t tab markers in each one and throws an error if they don't match. Lowkey the most underrated check ngl 🌸
unchanged(str1, str2)
Checks whether a translation is basically identical to the original string.
So it's checking whether the "translation" is just… a copy-paste of the English original 🙈 Sometimes that's exactly what you want (iykyk), but other times it's gonna flag words that fr should have been translated.
untranslated(str1, str2)
Checks whether a string has been translated at all.
Honestly this one is only useful if you're trying to extract untranslated strings so you can go translate them on their own, separate from the main work ✨
urls(str1, str2)
Checks that URLs are not translated.
Heads up bestie: this only checks basic URLs (http,ftp,mailtoetc.), not all URIs (likeafp,smb,file).
Generally you don't want to translate URLs — unless they're example URLs like http://your_server.com/filename.html. If the URL is for configuration info, then you gotta go query the developers about putting configuration information in PO files. It shouldn't really be there at all, unless it's super clearly marked: that kind of info should live in a configuration file 💅
validchars(str1, str2)
Checks that only characters specified as valid appear in the translation.
Ok so during character conversion to and from UTF-8 you get some truly cursed characters showing up in your translation 😭 This test is a simple way to try and spot those errors.
This test will only run if you specify the --validcharsfile command line option.
That file contains all the characters that are valid in your language — and you must use UTF-8 encoding for the characters in it, no exceptions.
If the test finds any characters that aren't in your valid characters file, it prints the character together with its Unicode value (e.g. 002B). Slay, now you know exactly which gremlin character snuck in 🦋
variables(str1, str2)
Checks whether variables of various forms are consistent between the two strings.
It makes sure the variables that appear in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice. What it does not do at the moment is cope with variables that use the reordering syntax of Gettext PO files 👑
xmltags(str1, str2)
Checks that XML/HTML tags have not been translated.
It finds the number of tags in the source string and checks that the same number are in the translation. If the counts don't match, then either a tag is missing or the translator mistakenly translated it — and both of those are errors, periodt 💅
The check ignores tags (or things that look like tags) that cover the whole string, e.g. <Error> — but it will produce false positives for things like An <Error> occurred, because here Error should be translated. It also allows translation of the alt attribute in e.g. <img src="bob.png" alt="Image description"> or similar translatable attributes in OpenOffice.org help files 🌈
🛠️ class translate.filters.checks.CheckerConfig(...)
class translate.filters.checks.CheckerConfig(targetlanguage=None,
accelmarkers=None,
varmatches=None,
notranslatewords=None,
musttranslatewords=None,
validchars=None,
punctuation=None,
endpunctuation=None,
ignoretags=None,
canchangetags=None,
criticaltests=None,
credit_sources=None)Object representing the configuration of a checker. It's giving settings-holder main character energy 🎀
update(otherconfig)
Combines the info in otherconfig into this config object. Merge era 💖
updatetargetlanguage(langcode)
Updates the target language in the config to the given target language, and sets its script too.
updatevalidchars(validchars)
Updates the map that eliminates valid characters.
class translate.filters.checks.DrupalChecker(**kwargs)
accelerators(str1, str2)
Checks whether accelerators are consistent between the two strings.
This test is capable of checking the different types of accelerators used in different projects, like Mozilla or KDE 👑 It picks up accelerators that are missing and ones that shouldn't be there.
See accelerators on the localization guide for a full description on accelerators.
acronyms(str1, str2)
Checks that acronyms that appear are unchanged.
If an acronym appears in the original, this test checks that it appears in the translation too. Whether to translate acronyms is a language decision, but many languages leave them unchanged — and in that case this test is so useful for tracking down translations of the acronym and correcting them ✨
blank(str1, str2)
Checks whether a translation is totally blank.
This checks if a translation has inadvertently been translated as blank, i.e. as spaces. This is different from untranslated, which is completely empty. The tea here: if something is translated as " " it will appear to most tools as if it is translated 💀 so this test catches that vibe.
brackets(str1, str2)
Checks that the number of brackets in both strings match.
If ([{ or }]) appear in the original, this checks that the same number appear in the translation 🫶
categories
Categories where each checking function falls into. Function names are used as keys, categories are the values. Very dict, very demure 💅
property checker_name
Extract checker name — for example 'mozilla' from MozillaChecker.
compendiumconflicts(str1, str2)
Checks for Gettext compendium conflicts (#-#-#-#-#).
When you use msgcat to create a PO compendium, it inserts #-#-#-#-# into entries that are not consistent. If that compendium is used later in a message merge, those conflicts will show up in your translations 😭 This test quickly extracts them so you can fix them.
credits(str1, str2)
Checks for messages containing translation credits instead of normal translations.
Some projects have consistent ways of giving credit to translators — a unit or two where translators can fill in their name and possibly their contact details 🎀 This test lets you find those units easily so you can check they're completed correctly, and it also disables other tests that might incorrectly get triggered for those units (such as urls, emails, etc.). Iconic behaviour honestly.
doublequoting(str1, str2)
Checks whether doublequoting is consistent between the two strings.
Checks on double quotes " to ensure you have the same number in both the original and the translated string. And it takes into account that several languages use different quoting characters, and tests for those instead 🌸 thoughtful queen.
doublespacing(str1, str2)
Checks for bad double-spaces by comparing to the original.
This identifies if you have [space][space] when you don't have it in the original, or if it appears in the original but not in your translation. Some of these are spurious, and how you correct them depends on the conventions of your language 🦋
doublewords(str1, str2)
Checks for repeated words in the translation.
Words that have been repeated in a translation get highlighted by this test, e.g. "the the", "a a" 🙈 These are generally typos that need correcting. Some languages may have valid repeated words in their structure — in that case, either ignore those instances or switch this test off.
emails(str1, str2)
Checks that emails are not translated.
Generally you should not be translating email addresses. This check looks to see that email addresses e.g. info@example.com are not translated. In some cases of course you should translate the address, but generally you shouldn't 💜
endpunc(str1, str2)
Checks whether punctuation at the end of the strings match.
This makes sure the ending of your translation has the same punctuation as the original. E.g. if it ends in :[space] then so should yours. It's useful for making sure you have ellipses […] in all your translations, not just three separate full-stops ✨
You may pick up some errors in the original: feel free to keep your translation and notify the programmers. In some languages, characters such as ? or ! are always preceded by a space, e.g. [space]? — do what your language customs dictate 🫶
Other false positives you'll notice are, for example, if through changes in word-order you add "), etc. at the end of the sentence. Do not change these: your language word-order takes precedence, periodt 💅
It must be noted that if you're tempted to leave out[full-stop]or[colon], or add[full-stop]to a sentence, that often these have been done for a reason — e.g. a list where fullstops make it look cluttered. So, initially match them with the English, and make changes once the program is being used 🌟
This check is aware of several language conventions for punctuation characters — like the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. And bestie, if your language isn’t there yet, support for it can be added easily. 💖
endwhitespace(str1, str2)— Checks whether whitespace at the end of the strings matches.
Works the same as endpunc but is only concerned with whitespace. This filter is lowkey a lifesaver for those strings that’ll evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. The whitespace is an inherent part of the string, and this filter makes sure you don’t miss those important but otherwise invisible spaces! 👻
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
escapes(str1, str2)— Checks whether escaping is consistent between the two strings.
Checks escapes such as \n \u0000 to ensure that if they exist in the original string you also have them in the translation. ⚡
filepaths(str1, str2)— Checks that file paths have not been translated.
Checks that paths such as /home/user1 have not been translated. Generally you do not translate a file path, unless it is being used as an example, e.g. your_user_name/path/to/filename.conf. 📁
filteraccelerators_by_list(str1, acceptlist=None)— Filter out accelerators fromstr1.functions(str1, str2)— Checks that function names are not translated.
Checks that function names e.g. rgb() or getEntity.Name() are not translated. no cap, leave those alone fr fr 💅
get_ignored_filters()— Return checker’s additional filters for current language.getfilters(excludefilters=None, limitfilters=None)— Returns dictionary of available filters, including/excluding those in the given lists.kdecomments(str1, str2)— Checks to ensure that no KDE style comments appear in the translation.
KDE style translator comments appear in PO files as "_: comment\n". New translators often translate the comment 😭 and this test tries to identify instances where the comment has been translated.
long(str1, str2)— Checks whether a translation is much longer than the original string.
This is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. 📏
musttranslatewords(str1, str2)— Checks that words configured as definitely translatable don’t appear in the translation.
If for instance in your language you decide that you must translate ‘OK’ then this test will flag any occurrences of ‘OK’ in the translation if it appeared in the source string. You must specify a file containing all of the must translate words using --musttranslatefile. ✨
newlines(str1, str2)— Checks whether newlines are consistent between the two strings.
Counts the number of \n newlines (and variants such as \r\n) and reports and error if they differ.
notranslatewords(str1, str2)— Checks that words configured as untranslatable appear in the translation too.
Many brand names should not be translated, and this test allows you to easily make sure that words like: Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no translate words using --notranslatefile. 👑
numbers(str1, str2)— Checks whether numbers of various forms are consistent between the two strings.
You will see some errors where you have either written the number in full or converted it to the digit in your translation. Also changes in order will trigger this error. 🔢
options(str1, str2)— Checks that command line options are not translated.
In messages that contain command line options, such as --help, this test will check that these remain untranslated. These could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. If the options has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated.
printf(str1, str2)— Checks whether printf format strings match.
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are placeholders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. The test can also manage variables-reordering using the %1$s syntax. The variables’ type and details following data are tested to ensure that they are strictly identical, but they may be reordered.
SEE ALSO: printf Format String 🫶
puncspacing(str1, str2)— Checks for bad spacing after punctuation.
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc.
Some languages don’t use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account.
purepunc(str1, str2)— Checks that strings that are purely punctuation are not changed.
This extracts strings like + or - as these usually should not be changed. 💅
pythonbraceformat(str1, str2)— Checks whether python brace format strings match.run_filters(unit, categorised=False)— Do some optimisation by caching some data of the unit for the benefit ofrun_test().run_test(test, unit)— Runs the given test on the given unit.
Note that this can raise a FilterFailure as part of normal operation. 😭 (that’s just how it do be)
sentencecount(str1, str2)— Checks that the number of sentences in both strings match.
Adds the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it’s the meaning of the original you want to convey, not the exact way it was written in the English. 🌸
setconfig(config)— Sets the accelerator list.setsuggestionstore(store)— Sets the filename that a checker should use for evaluating suggestions.short(str1, str2)— Checks whether a translation is much shorter than the original string.
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. 📐
simplecaps(str1, str2)— Checks the capitalisation of two strings isn’t wildly different.
This will pick up many false positives, so don’t be a slave to it. It is useful for identifying translations that don’t start with a capital letter (upper-case letter) when they should, or those that do when they shouldn’t. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case. 🔠
simpleplurals(str1, str2)— Checks for English style plural(s) for you to review.
This test will extract any message that contains words with a final “(s)” in the source text. You can then inspect the message, to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy. In this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives. 🫶
For languages with no plural forms (only one noun form) this test will simply test that nothing like “(s)” was used in the translation.
singlequoting(str1, str2)— Checks whether singlequoting is consistent between the two strings.
Okay so this next check is literally the same energy as doublequoting, but it’s checking for the ' character instead. 💅 Since ' shows up in contractions like it’s and possessive forms like user’s, this test can throw spurious errors if your language doesn’t use those forms at all. Also, if a quote lands at the end of a sentence in the translation — like '. — the check might not detect it properly. 🫶
spellcheck(str1, str2)
Checks words that don’t pass a spell check.
This test checks for misspelled words in your translation. First it checks the original (usually English) text for misspelled words and adds those to an exclusion list. That exclusion list is a whole vibe because lots of words specific to the app — program names, brand names, function names — won’t raise errors. ✨
The checker works with PyEnchant. You need PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you’ve specified the --language option.
The pofilter error that gets created lists the misspelled word plus suggestions returned from the spell checker. That makes it easy to spot the word and pick a replacement. 💖
startcaps(str1, str2)
Checks that the message starts with the correct capitalisation.
After stripping whitespace and common punctuation characters, it checks that the first remaining character is correctly capitalised. So if the sentence starts with an upper-case letter and the translation doesn’t, an error gets produced. 👑
This check is entirely disabled for many languages that don’t make a distinction between upper and lower case. Contact us if this isn’t yet disabled for your language.
startpunc(str1, str2)
Checks whether punctuation at the beginning of the strings match.
Operates as endpunc, but you’ll probably see fewer errors. 🎀
startwhitespace(str1, str2)
Checks whether whitespace at the beginning of the strings matches.
As in endwhitespace, but you will see fewer errors. 🌸
tabs(str1, str2)
Checks whether tabs are consistent between the two strings.
Counts the number of \\t tab markers and reports an error if they differ.
unchanged(str1, str2)
Checks whether a translation is basically identical to the original string.
This checks if the translation isn’t just a copy of the English original. Sometimes that’s exactly what you want, but other times you’ll catch words that should have been translated. 👀
untranslated(str1, str2)
Checks whether a string has been translated at all.
This check is really only useful if you want to extract untranslated strings so they can be translated independently of the main work. 🧚
urls(str1, str2)
Checks that URLs are not translated.
This checks only basic URLs (http, ftp, mailto etc.), not all URIs (e.g. afp, smb, file). Generally you don’t want to translate URLs, unless they’re example URLs (‐ http://your_server.com/filename.html). If the URL is for configuration information, then you need to query the developers about placing configuration information in PO files. It shouldn’t really be there unless it’s very clearly marked: such information should go into a configuration file. 🫶
validchars(str1, str2)
Checks that only characters specified as valid appear in the translation.
Often during character conversion to and from UTF-8 you get some strange characters appearing in your translation. This test presents a simple way to try and identify such errors. 🔍
This test will only run if you specify the --validcharsfile command line option. This file contains all the characters that are valid in your language. You must use UTF-8 encoding for the characters in the file.
If the test finds any characters not in your valid characters file, then the test will print the character together with its Unicode value (e.g. 002B).
variables(str1, str2)
Checks whether variables of various forms are consistent between the two strings.
This makes sure that variables that appear in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice. At the moment it does not cope with variables that use the reordering syntax of Gettext PO files. 😭
xmltags(str1, str2)
Checks that XML/HTML tags have not been translated.
This check finds the number of tags in the source string and checks that the same number are in the translation. If the counts don’t match, then either the tag is missing or it was mistakenly translated by the translator — both of which are errors. 💀
The check ignores tags or things that look like tags that cover the whole string, e.g. <Error>, but will produce false positives for things like An <Error> occurred, because here Error should be translated. It also allows translation of the alt attribute in e.g. <img src="bob.png" alt="Image description"> or similar translatable attributes in OpenOffice.org help files. 🌈
exception translate.filters.checks.FilterFailure(messages)
This exception signals that a Filter didn’t pass, and gives an explanation or a comment.
add_note(object, /)—Exception.add_note(note)– add a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb)– setself.__traceback__totband returnself
class translate.filters.checks.GnomeChecker(**kwargs)
accelerators(str1, str2)
Checks whether accelerators are consistent between the two strings.
This test is capable of checking the different types of accelerators that are used in different projects, like Mozilla or KDE. It picks up accelerators that are missing and ones that shouldn’t be there. 🎯
See accelerators on the localization guide for a full description on accelerators.
acronyms(str1, str2)
Checks that acronyms that appear are unchanged.
If an acronym appears in the original, this test checks that it appears in the translation. Translating acronyms is a language decision, but many languages leave them unchanged. In that case this test is useful for tracking down translations of the acronym and correcting them. 🦋
blank(str1, str2)
Checks whether a translation is totally blank.
This checks if a translation has inadvertently been translated as blank, i.e. as spaces. This is different from untranslated, which is completely empty. This test is useful because if something is translated as “ “ it will appear to most tools as if it is translated. 🙈
brackets(str1, str2)
Checks that the number of brackets in both strings match.
If ([{ or }]) appear in the original, this checks that the same number appear in the translation.
property categories
Categories where each checking function falls into. Function names are used as keys, categories are the values. 📚
property checker_name
Extract checker name, for example 'mozilla' from MozillaChecker. 🏷️
compendiumconflicts(str1, str2)
Checks for Gettext compendium conflicts (#-#-#-#-#).
When you use msgcat to create a PO compendium it will insert #-#-#-#-# into entries that are not consistent. If the compendium is used later in a message merge then these conflicts will appear in your translations. This test quickly extracts those for correction. 🔥
credits(str1, str2)
Checks for messages containing translation credits instead of normal translations.
Some projects have consistent ways of giving credit to translators by having a unit or two where translators can fill in their name and possibly their contact details. This test lets you find these units easily to check that they’re completed correctly, and it also disables other tests that might incorrectly get triggered for these units (such as urls, emails, etc.). 🫶
doublequoting(str1, str2)
Checks whether doublequoting is consistent between the two strings.
Checks on double quotes " to ensure that you have the same number in both the original and the translated string. This test takes into account that several languages use different quoting characters, and will test for them instead. 💬
doublespacing(str1, str2)
Checks for bad double-spaces by comparing to the original.
This identifies if you have [space][space] when you don’t have it in the original, or it appears in the original but not in your translation. Some of these are spurious, and how you correct them depends on the conventions of your language. 🌸
doublewords(str1, str2)
Checks for repeated words in the translation.
So yeah — that described the repeated-words check, and here's where it gets fun: the whole filter catalogue. Buckle up bestie, this is the long one 💅
emails(str1, str2)
Checks that emails are not translated.
Like, generally you should not be translating email addresses, periodt. This check looks to see that email addresses — e.g. info@example.com — aren't translated. In some cases of course you should translate the address, but generally you shouldn't. 🙈
endpunc(str1, str2)
Checks whether punctuation at the end of the strings match.
This one makes sure the ending of your translation has the same punctuation as the original. E.g. if it ends in :[space] then so should yours. It's lowkey iconic for ensuring you have actual ellipses […] in all your translations, not simply three separate full-stops. You may pick up some errors in the original too: feel free to keep your translation and notify the programmers. In some languages, characters such as ? or ! are always preceded by a space, e.g. [space]? — do what your language customs dictate. Other false positives you'll notice are, for example, if through changes in word-order you add "), etc. at the end of the sentence. Do not change these: your language word-order takes precedence.
It must be noted (📢) that if you're tempted to leave out [full-stop] or [colon], or add [full-stop] to a sentence, that often these have been done for a reason — e.g. a list where fullstops make it look cluttered. So, initially match them with the English, and make changes once the program is being used.
This check is aware of several language conventions for punctuation characters, such as the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it's not there yet. 🫶
endwhitespace(str1, str2)
Checks whether whitespace at the end of the strings matches.
Operates the same as endpunc but is only concerned with whitespace. This filter is particularly useful for those strings which will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. The whitespace is an inherent part of the string. This filter makes sure you don't miss those important but otherwise invisible spaces! 😭
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
escapes(str1, str2)
Checks whether escaping is consistent between the two strings.
Checks escapes such as \n \u0000 to ensure that if they exist in the original string you also have them in the translation. No sneaky vanishing backslashes, bestie. 🦋
filepaths(str1, str2)
Checks that file paths have not been translated.
Checks that paths such as /home/user1 have not been translated. Generally you do not translate a file path, unless it's being used as an example, e.g. your_user_name/path/to/filename.conf.
filteraccelerators_by_list(str1, acceptlist=None)
Filter out accelerators from str1. ✨ (That's the whole vibe of it.)
functions(str1, str2)
Checks that function names are not translated.
Checks that function names e.g. rgb() or getEntity.Name() are not translated. Leave the code alone, babe 💀
gconf(str1, str2)
Checks if we have any gconf config settings translated.
Gconf settings should not be translated, so this check checks that gconf settings such as "name" or "modification_date" are not translated in the translation. It allows you to change the surrounding quotes but will ensure that the setting values remain untranslated. Quotes? sure. Values? untouched. 🎀
get_ignored_filters()
Return checker's additional filters for current language.
getfilters(excludefilters=None, limitfilters=None)
Returns dictionary of available filters, including/excluding those in the given lists.
kdecomments(str1, str2)
Checks to ensure that no KDE style comments appear in the translation.
KDE style translator comments appear in PO files as "_: comment\n". New translators often translate the comment. This test tries to identify instances where the comment has been translated. 🙈 (spoiler: don't translate the comment)
long(str1, str2)
Checks whether a translation is much longer than the original string.
This is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. Lowkey chill on purpose. 🌸
musttranslatewords(str1, str2)
Checks that words configured as definitely translatable don't appear in the translation.
If for instance in your language you decide that you must translate ‘OK’ then this test will flag any occurrences of ‘OK’ in the translation if it appeared in the source string. You must specify a file containing all of the must translate words using --musttranslatefile.
newlines(str1, str2)
Checks whether newlines are consistent between the two strings.
Counts the number of \n newlines (and variants such as \r\n) and reports and error if they differ. (Yes, "reports and error" — that's the original's typo, not mine 😭 keeping it verbatim bestie.)
notranslatewords(str1, str2)
Checks that words configured as untranslatable appear in the translation too.
Many brand names should not be translated, this test allows you to easily make sure that words like: Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no translate words using --notranslatefile. 👑
numbers(str1, str2)
Checks whether numbers of various forms are consistent between the two strings.
You will see some errors where you have either written the number in full or converted it to the digit in your translation. Also changes in order will trigger this error. 🔢
options(str1, str2)
Checks that command line options are not translated.
In messages that contain command line options, such as --help, this test will check that these remain untranslated. These could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. If the options has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated.
printf(str1, str2)
Checks whether printf format strings match.
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are placeholders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. The test can also manage variables-reordering using the %1$s syntax. The variables' type and details following data are tested to ensure that they are strictly identical, but they may be reordered. 💅
SEE ALSO: printf Format String
puncspacing(str1, str2)
Checks for bad spacing after punctuation.
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc.
Some languages don't use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account. 🦋
purepunc(str1, str2)
Checks that strings that are purely punctuation are not changed.
This extracts strings like + or - as these usually should not be changed.
pythonbraceformat(str1, str2)
Checks whether python brace format strings match. 🐍 (and that's literally all it says, no cap)
run_filters(unit, categorised=False)
Do some optimisation by caching some data of the unit for the benefit of run_test().
run_test(test, unit)
Runs the given test on the given unit.
Note that this can raise a FilterFailure as part of normal operation. Like, it's not a crash, it's a vibe check that failed 💀
sentencecount(str1, str2)
Checks that the number of sentences in both strings match.
Adds the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it's the meaning of the original you want to convey, not the exact way it was written in the English. 🫶
setconfig(config)
Sets the accelerator list. Short, sweet, slay. ✨
✨ setsuggestionstore(store) ✨
Sets the filename that a checker should use for evaluating suggestions. That's it, that's the whole vibe — just hand it a store filename and it knows where to look for suggestions later. 🎀
💅 short(str1, str2)
Checks whether a translation is much shorter than the original string.
This one is chef's kiss for one specific situation: when the translation is 1 characters long while the source text is multiple characters long. Otherwise it uses a general ratio that catches very big differences, and that ratio is set conservatively on purpose so you don't drown in false positives. 🙈
🦋 simplecaps(str1, str2)
Checks the capitalisation of two strings isn't wildly different.
Real talk: this will pick up many false positives, so don't be a slave to it, bestie. It is useful for spotting translations that don't start with a capital letter (upper-case letter) when they should, or ones that do when they shouldn't. It'll also highlight sentences that have extra capitals — and depending on the capitalisation convention of your language, you might want to change those to Title Case, or change them all to normal sentence case. 👑
🌸 simpleplurals(str1, str2)
Checks for English style plural(s) for you to review.
This test extracts any message that contains words with a final "(s)" in the source text. Then you can go inspect the message and check that the correct plural form got used for your language.
Some languages make plurals by adding text at the beginning of words, which makes the English style messy — in that case they often revert to the plural form. This test lets an editor check that the plurals used are correct. Heads up though: it may create a number of false positives. 😭
And for languages with no plural forms (only one noun form)? The test simply checks that nothing like "(s)" was used in the translation. Periodt. ✨
🎀 singlequoting(str1, str2)
Checks whether singlequoting is consistent between the two strings.
Same deal as doublequoting but it checks for the ' character. Because that character shows up in contractions like it's and possessives like user's, this test can spit out spurious errors if your language doesn't use those forms. Also, if a quote appears at the end of a sentence in the translation — like '. — it might not get detected properly by the check. 🫶
🔥 spellcheck(str1, str2)
Checks words that don't pass a spell check.
This test hunts for misspelled words in your translation. First it checks for misspelled words in the original (usually English) text and adds those to an exclusion list — which is lowkey genius, because lots of app-specific words like program names, brand names and function names won't raise errors then. 💜
The checker works with PyEnchant. So you need PyEnchant installed plus a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you've specified the --language option. No language, no spellcheck, no exceptions.
The pofilter error it creates lists the misspelled word plus suggestions returned from the spell checker, which makes it easy for you to spot the word and pick a replacement. Slay. 🌈
⭐ startcaps(str1, str2)
Checks that the message starts with the correct capitalisation.
After stripping whitespace and common punctuation characters, it checks that the first remaining character is correctly capitalised. So if the sentence starts with an upper-case letter and the translation doesn't — error. 💀
This check is entirely disabled for many languages that don't distinguish between upper and lower case. Contact them if it isn't yet disabled for your language. 📩
🧚 startpunc(str1, str2)
Checks whether punctuation at the beginning of the strings match.
Works just like endpunc, but you'll probably see fewer errors. 😌
🍓 startwhitespace(str1, str2)
Checks whether whitespace at the beginning of the strings matches.
Same as endwhitespace but, again, you'll see fewer errors. ✨
💫 tabs(str1, str2)
Checks whether tabs are consistent between the two strings.
Counts the number of \t tab markers and reports an error if they differ. Simple, iconic, does exactly what it says on the tin. 🎀
🦄 unchanged(str1, str2)
Checks whether a translation is basically identical to the original string.
It's checking if the translation isn't just a copy of the English original. Sometimes that's exactly what you want — but other times you'll catch words that absolutely should have been translated. 👀
🌟 untranslated(str1, str2)
Checks whether a string has been translated at all.
For real, this check is only useful if you want to extract untranslated strings so they can be translated independently of the main work. That's its whole job. 💅
🦋 urls(str1, str2)
Checks that URLs are not translated.
It only checks basic URLs (http, ftp, mailto etc.) — not all URIs (e.g. afp, smb, file). Generally you don't want to translate URLs, unless they're example URLs (http://your_server.com/filename.html). 🫶
If the URL is for configuration information, then you need to query the developers about placing configuration info in PO files — it shouldn't really be there unless it's very clearly marked. That kind of info belongs in a configuration file. Periodt. 💜
🌸 validchars(str1, str2)
Checks that only characters specified as valid appear in the translation.
Often during character conversion to and from UTF-8 you get some strange characters showing up in your translation, and this test is a simple way to try and identify those errors. 🧐
This test will only run if you specify the --validcharsfile command line option. That file contains all the characters that are valid in your language, and you must use UTF-8 encoding for the characters in the file. ⚠️
If the test finds any characters not in your valid characters file, it prints the character together with its Unicode value (e.g. 002B). Cute AND useful. ✨
👑 variables(str1, str2)
Checks whether variables of various forms are consistent between the two strings.
It makes sure that variables appearing in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice. It does not at the moment cope with variables that use the reordering syntax of Gettext PO files. 🙈
🎀 xmltags(str1, str2)
Checks that XML/HTML tags have not been translated.
This check finds the number of tags in the source string and checks that the same number are in the translation. If the counts don't match, then either the tag is missing or it was mistakenly translated by the translator — both of which are errors. 💀
The check ignores tags (or things that look like tags) that cover the whole string, e.g. <Error>, but it will produce false positives for things like An <Error> occurred, since here Error should be translated. It also allows translation of the alt attribute in e.g. <img src="bob.png" alt="Image description">, or similar translatable attributes in OpenOffice.org help files. 🫶
💖 class translate.filters.checks.IOSChecker(**kwargs)
🌈 accelerators(str1, str2)
Checks whether accelerators are consistent between the two strings.
This test is capable of checking the different types of accelerators used across different projects, like Mozilla or KDE. It picks up accelerators that are missing and ones that shouldn't be there. 🔥
See accelerators in the localization guide for the full description on accelerators. Iykyk. 📖
⭐ acronyms(str1, str2)
Checks that acronyms that appear are unchanged.
If an acronym appears in the original, this test checks that it appears in the translation too. Translating acronyms is a language decision, but many languages leave them unchanged — and in that case this test is perfect for tracking down translations of the acronym and correcting them. 💅
🧚 blank(str1, str2)
Checks whether a translation is totally blank.
It checks if a translation has inadvertently been translated as blank, i.e. as spaces. This is different from untranslated, which is completely empty. Why does this matter? Because if something is translated as " ", it'll look to most tools like it is translated. Sneaky. 😭
🍓 brackets(str1, str2)
Checks that the number of brackets in both strings match.
If ([{ or }]) appear in the original, this checks that the same number appear in the translation. Balanced brackets = balanced energy. ✨
💜 categories
Categories where each checking function falls into. Function names are used as keys, categories are the values. 🗂️
🎀 property checker_name
Extract the checker name — for example mozilla from MozillaChecker. Cute little name-extractor, doing its thing. 💖
compendiumconflicts(str1, str2)
catches Gettext compendium conflicts — yes, the iconic #-#-#-#-# situation 💀
okay so when you run msgcat to make a PO compendium, it straight-up inserts #-#-#-#-# into any entries that aren't consistent. and then if that compendium gets used later in a message merge, those conflicts show up in your actual translations. this test pulls them out fast so you can fix them. slay.
credits(str1, str2)
checks for messages that are translation credits instead of like… actual translations 🫶
some projects have a consistent way of giving credit to translators — they'll have a unit or two where translators can drop their name and maybe their contact details. this test lets you find those units easily so you can double-check they're filled in correctly, and it also disables other tests that might get triggered on those units by mistake (urls, emails, etc.). thoughtful queen behavior.
doublequoting(str1, str2)
checks whether doublequoting is consistent between the two strings 🎀
it looks at double quotes " to make sure you've got the same number in both the original and the translated string. and bestie, it's smart — it knows several languages use different quoting characters, so it'll test for those instead. iykyk.
doublespacing(str1, str2)
checks for bad double-spaces by comparing to the original ✨
this will flag it if you've got [space][space] where the original doesn't, or if it appears in the original but not in your translation. some of these are spurious though, and how you fix them totally depends on your language's conventions.
doublewords(str1, str2)
checks for repeated words in the translation 🙈
repeated words get highlighted by this test, e.g. "the the", "a a". these are generally typos that need correcting. but — some languages may have valid repeated words in their structure, and in that case either ignore those instances or just switch this test off. no stress.
emails(str1, str2)
checks that emails are not translated 💌
generally you should not be translating email addresses, and this check looks to see that emails like info@example.com are not translated. in some cases of course you should translate the address, but generally you shouldn't.
endpunc(str1, str2)
checks whether punctuation at the end of the strings match ⭐
this makes sure the ending of your translation has the same punctuation as the original. e.g. if it ends in :[space] then so should yours. it's super useful for making sure you've got ellipses […] in all your translations, not simply three separate full-stops. you may pick up some errors in the original: feel free to keep your translation and notify the programmers. in some languages, characters such as ? or ! are always preceded by a space e.g. [space]? — do what your language customs dictate. other false positives you'll notice are, for example, if through changes in word-order you add "), etc. at the end of the sentence. do not change these: your language word-order takes precedence.
it must be noted that if you're tempted to leave out [full-stop] or [colon] or add [full-stop] to a sentence, often these have been done for a reason — e.g. a list where fullstops make it look cluttered. so initially match them with the English, and make changes once the program is being used.
this check is aware of several language conventions for punctuation characters, like the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. support for your language can be added easily if it's not there yet. 💅
endwhitespace(str1, str2)
checks whether whitespace at the end of the strings matches 🌸
operates the same as endpunc but is only concerned with whitespace. this filter is especially useful for those strings which will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. the whitespace is an inherent part of the string. this filter makes sure you don't miss those important but otherwise invisible spaces! not me forgetting a trailing space 😭
if your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
escapes(str1, str2)
checks whether escaping is consistent between the two strings 🦋
checks escapes such as \\n and \u0000 to make sure that if they exist in the original string, you also have them in the translation.
filepaths(str1, str2)
checks that file paths have not been translated 🔥
checks that paths such as /home/user1 have not been translated. generally you do not translate a file path — unless it's being used as an example, e.g. your_user_name/path/to/filename.conf.
filteraccelerators_by_list(str1, acceptlist=None)
filter out accelerators from str1. no cap, that's the whole thing 🎀
functions(str1, str2)
checks that function names are not translated 👑
checks that function names e.g. rgb() or getEntity.Name() are not translated.
get_ignored_filters()
return checker's additional filters for the current language 🌈
getfilters(excludefilters=None, limitfilters=None)
returns a dictionary of available filters, including/excluding those in the given lists ✨
kdecomments(str1, str2)
checks to ensure that no KDE style comments appear in the translation 💀
KDE style translator comments appear in PO files as _: / comment\\n. new translators often translate the comment (rip). this test tries to identify instances where the comment has been translated.
long(str1, str2)
checks whether a translation is much longer than the original string 😭
this is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives.
musttranslatewords(str1, str2)
checks that words configured as definitely translatable don't appear in the translation 🫶
if for instance in your language you decide that you must translate 'OK', then this test will flag any occurrences of 'OK' in the translation if it appeared in the source string. you must specify a file containing all of the must-translate words using --musttranslatefile.
newlines(str1, str2)
checks whether newlines are consistent between the two strings 🌸
counts the number of \\n newlines (and variants such as \\r\\n) and reports an error if they differ.
notranslatewords(str1, str2)
checks that words configured as untranslatable appear in the translation too 💅
many brand names should not be translated, and this test lets you easily make sure that words like: Word, Excel, Impress, Calc, etc. are not translated. you must specify a file containing all of the no-translate words using --notranslatefile.
numbers(str1, str2)
checks whether numbers of various forms are consistent between the two strings 🔢
you'll see some errors where you've either written the number in full or converted it to the digit in your translation. changes in order will trigger this error too.
options(str1, str2)
checks that command line options are not translated 👑
in messages that contain command line options, such as --help, this test checks that these remain untranslated. these could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. if the option has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated.
printf(str1, str2)
checks whether printf format strings match ⭐
if the printf formatting variables aren't identical, this indicates an error. printf statements are used by programs to format output in a human readable form (they're placeholders for variable data). they let you specify lengths of string variables, string padding, number padding, precision, etc. generally they look like this: %d, %5.2f, %100s, etc. the test can also handle variables-reordering using the %1$s syntax. the variables' type and details following data are tested to ensure they're strictly identical — but they may be reordered. iconic.
SEE ALSO: printf Format String
puncspacing(str1, str2)
checks for bad spacing after punctuation 🎀
in the case of [full-stop][space] in the original, this test checks that your translation doesn't remove the space. it checks for [comma], [colon], etc. too.
Some languages don’t use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account. 🌸
purepunc(str1, str2)
Checks that strings that are purely punctuation are not changed. 💅
This extracts strings like + or - as these usually should not be changed.
pythonbraceformat(str1, str2)
Checks whether python brace format strings match. 🎀
run_filters(unit, categorised=False)
Does some optimisation by caching some data of the unit for the benefit of run_test(). ✨
run_test(test, unit)
Runs the given test on the given unit.
Note that this can raise a FilterFailure as part of normal operation. 😭
sentencecount(str1, str2)
Checks that the number of sentences in both strings match.
Adds the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it’s the meaning of the original you want to convey, not the exact way it was written in the English. 🫶
setconfig(config)
Sets the accelerator list. 🔥
setsuggestionstore(store)
Sets the filename that a checker should use for evaluating suggestions.
short(str1, str2)
Checks whether a translation is much shorter than the original string.
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. 👑
simplecaps(str1, str2)
Checks the capitalisation of two strings isn’t wildly different.
This will pick up many false positives, so don’t be a slave to it. It is useful for identifying translations that don’t start with a capital letter (upper-case letter) when they should, or those that do when they shouldn’t. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case. 💜
simpleplurals(str1, str2)
Checks for English style plural(s) for you to review. 🍓
This test will extract any message that contains words with a final “(s)” in the source text. You can then inspect the message, to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy. In this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives.
For languages with no plural forms (only one noun form) this test will simply test that nothing like “(s)” was used in the translation.
singlequoting(str1, str2)
Checks whether singlequoting is consistent between the two strings.
The same as doublequoting but checks for the ' character. Because this is used in contractions like it’s and in possessive forms like user’s, this test can output spurious errors if your language doesn’t use such forms. If a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check. 😬
spellcheck(str1, str2)
Checks words that don’t pass a spell check. 🔤
This test will check for misspelled words in your translation. The test first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors e.g. program names, brand names, function names.
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you have specified the --language option.
The pofilter error that is created, lists the misspelled word, plus suggestions returned from the spell checker. That makes it easy for you to identify the word and select a replacement. ✨
startcaps(str1, str2)
Checks that the message starts with the correct capitalisation.
After stripping whitespace and common punctuation characters, it then checks to see that the first remaining character is correctly capitalised. So, if the sentence starts with an upper-case letter, and the translation does not, an error is produced. 🎀
This check is entirely disabled for many languages that don’t make a distinction between upper and lower case. Contact us if this is not yet disabled for your language.
startpunc(str1, str2)
Checks whether punctuation at the beginning of the strings match.
Operates as endpunc but you will probably see fewer errors. 💅
startwhitespace(str1, str2)
Checks whether whitespace at the beginning of the strings matches.
As in endwhitespace but you will see fewer errors.
tabs(str1, str2)
Checks whether tabs are consistent between the two strings.
Counts the number of \\t tab markers and reports an error if they differ.
unchanged(str1, str2)
Checks whether a translation is basically identical to the original string. 👀
This checks to see if the translation isn’t just a copy of the English original. Sometimes, this is what you want, but other times you will detect words that should have been translated.
untranslated(str1, str2)
Checks whether a string has been translated at all.
This check is really only useful if you want to extract untranslated strings so that they can be translated independently of the main work. 🧚
urls(str1, str2)
Checks that URLs are not translated.
This checks only basic URLs (http, ftp, mailto etc.) not all URIs (e.g. afp, smb, file). Generally, you don’t want to translate URLs, unless they are example URLs (http://your_server.com/filename.html). If the URL is for configuration information, then you need to query the developers about placing configuration information in PO files. It shouldn’t really be there, unless it is very clearly marked: such information should go into a configuration file. 🔗
validchars(str1, str2)
Checks that only characters specified as valid appear in the translation.
Often during character conversion to and from UTF-8 you get some strange characters appearing in your translation. This test presents a simple way to try and identify such errors. 🌈
This test will only run of you specify the --validcharsfile command line option. This file contains all the characters that are valid in your language. You must use UTF-8 encoding for the characters in the file.
If the test finds any characters not in your valid characters file then the test will print the character together with its Unicode value (e.g. 002B). ⭐
variables(str1, str2)
Checks whether variables of various forms are consistent between the two strings.
This checks to make sure that variables that appear in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice. It does not at the moment cope with variables that use the reordering syntax of Gettext PO files. 🫶
xmltags(str1, str2)
Checks that XML/HTML tags have not been translated.
This check finds the number of tags in the source string and checks that the same number are in the translation. If the counts don’t match then either the tag is missing or it was mistakenly translated by the translator, both of which are errors. 💀
The check ignores tags or things that look like tags that cover the whole string e.g. <Error> but will produce false positives for things like An <Error> occurred as here Error should be translated. It also will allow translation of the alt attribute in e.g. <img src="bob.png" alt="Image description"> or similar translatable attributes in OpenOffice.org help files.
class translate.filters.checks.KdeChecker(**kwargs)
accelerators(str1, str2)
Checks whether accelerators are consistent between the two strings. 🔥
...this test is lowkey a whole vibe check for accelerators — the keyboard-shortcut thingies — across different projects, like Mozilla or KDE. It'll catch the accelerators that ghosted you (missing) and the ones that showed up uninvited (shouldn't be there). 🎀
Also, if you want the full tea on accelerators, check out the accelerators section on the localization guide. iykyk.
acronyms(str1, str2)
Checks that acronyms that appear are unchanged. ✨
So if an acronym shows up in the original, this test makes sure it also shows up in your translation. Whether to translate an acronym is a language decision, bestie — but honestly a lot of languages just leave them alone. In that case this test is super useful for tracking down translations of the acronym and correcting them.
blank(str1, str2)
Checks whether a translation is totally blank. 💀
This one sniffs out whether a translation has inadvertently been translated as blank — as in, literally just spaces. That's different from untranslated, which is completely empty. The point: if something is translated as " ", it'll look to most tools like it's been translated. Sneaky. This test catches it.
brackets(str1, str2)
Checks that the number of brackets in both strings match. 🫶
If ([{ or }]) appear in the original, this checks that the same number appear in the translation.
categories
Categories where each checking function falls into. Function names are used as keys, categories are the values. 🗂️
property checker_name
Extract checker name — for example, 'mozilla' from MozillaChecker.
compendiumconflicts(str1, str2)
Checks for Gettext compendium conflicts (#-#-#-#-#). 🙈
When you use msgcat to create a PO compendium, it'll insert #-#-#-#-# into entries that aren't consistent. And if the compendium is used later in a message merge, those conflicts will appear in your translations. This test quickly extracts those for correction. slay.
credits(str1, str2)
Checks for messages containing translation credits instead of normal translations. 💖
Some projects have consistent ways of giving credit to translators — like having a unit or two where translators can fill in their name and possibly their contact details. This test lets you find those units easily, so you can check they're completed correctly. It also disables other tests that might incorrectly get triggered for these units (such as urls, emails, etc.). iconic behaviour honestly.
doublequoting(str1, str2)
Checks whether doublequoting is consistent between the two strings.
It checks on double quotes " to make sure you have the same number in both the original and the translated string. And it's not gonna be weird about it: this test takes into account that several languages use different quoting characters, and will test for those instead. 🌸
doublespacing(str1, str2)
Checks for bad double-spaces by comparing to the original. ✨
This will identify if you've got [space][space] in when you don't have it in the original — or if it appears in the original but not in your translation. Some of these are spurious, and how you fix them depends on the conventions of your language. No cap.
doublewords(str1, str2)
Checks for repeated words in the translation. 🔥
Words that got repeated in a translation get highlighted by this test — e.g. the the, a a. These are generally typos that need correcting. Some languages may legitimately have repeated words in their structure; in that case, either ignore those instances or switch this test off.
emails(str1, str2)
Checks that emails are not translated. 💌
Generally you should not be translating email addresses. This check looks to see that email addresses — e.g. info@example.com — are not translated. In some cases of course you should translate the address, but generally you shouldn't.
endpunc(str1, str2)
Checks whether punctuation at the end of the strings match. 👑
This makes sure the ending of your translation has the same punctuation as the original. E.g. if it ends in :[space] then so should yours. It's useful for ensuring you have ellipses […] in all your translations, not simply three separate full-stops. You may pick up some errors in the original: feel free to keep your translation and notify the programmers. In some languages, characters such as ? or ! are always preceded by a space, e.g. [space]? — do what your language customs dictate. Other false positives you'll notice are, for example, if through changes in word-order you add "), etc. at the end of the sentence. Do not change these: your language word-order takes precedence. 🫶
It must be noted that if you're tempted to leave out [full-stop] or [colon], or add [full-stop] to a sentence, that often these have been done for a reason — e.g. a list where fullstops make it look cluttered. So, initially match them with the English, and make changes once the program is being used.
This check is aware of several language conventions for punctuation characters, such as the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it's not there yet. 💅
endwhitespace(str1, str2)
Checks whether whitespace at the end of the strings matches. 🦋
Operates the same as endpunc but is only concerned with whitespace. This filter is particularly useful for those strings which will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. The whitespace is an inherent part of the string. This filter makes sure you don't miss those important but otherwise invisible spaces! 😭
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
escapes(str1, str2)
Checks whether escaping is consistent between the two strings.
Checks escapes such as \\n \u0000 to ensure that if they exist in the original string, you also have them in the translation. 🔥
filepaths(str1, str2)
Checks that file paths have not been translated.
Checks that paths such as /home/user1 have not been translated. Generally you do not translate a file path — unless it's being used as an example, e.g. your_user_name/path/to/filename.conf. 🌈
filteraccelerators_by_list(str1, acceptlist=None)
Filter out accelerators from str1.
functions(str1, str2)
Checks that function names are not translated. 💜
Checks that function names — e.g. rgb() or getEntity.Name() — are not translated.
get_ignored_filters()
Return checker's additional filters for current language.
getfilters(excludefilters=None, limitfilters=None)
Returns dictionary of available filters, including/excluding those in the given lists. ✨
kdecomments(str1, str2)
Checks to ensure that no KDE style comments appear in the translation. 🙈
KDE style translator comments appear in PO files as "_: comment\\n". New translators often translate the comment. This test tries to identify instances where the comment has been translated.
long(str1, str2)
Checks whether a translation is much longer than the original string.
This is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. 🎀
musttranslatewords(str1, str2)
Checks that words configured as definitely translatable don't appear in the translation.
If for instance in your language you decide that you must translate 'OK', then this test will flag any occurrences of 'OK' in the translation if it appeared in the source string. You must specify a file containing all of the must translate words using --musttranslatefile. ⭐
newlines(str1, str2)
Checks whether newlines are consistent between the two strings.
Counts the number of \\n newlines (and variants such as \\r\\n) and reports and error if they differ. 💀
notranslatewords(str1, str2)
Checks that words configured as untranslatable appear in the translation too.
Many brand names should not be translated, and this test lets you easily make sure that words like: Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no translate words using --notranslatefile. 👑
numbers(str1, str2)
Checks whether numbers of various forms are consistent between the two strings.
You'll see some errors where you've either written the number in full or converted it to the digit in your translation. Also, changes in order will trigger this error. 🔢
options(str1, str2)
Checks that command line options are not translated. 💅
So, in messages that contain command line options — like --help — this test makes sure those bad boys stay untranslated. Could they be translated in the future? Maybe, if programs ever create a mechanism that allows it, but currently they are not translated. Periodt.
If the option has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated. iykyk 🎀
printf(str1, str2)
Checks whether printf format strings match.
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are placeholders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. 🔥
The test can also manage variables-reordering using the %1$s syntax. The variables' type and details following data are tested to ensure that they are strictly identical, but they may be reordered.
SEE ALSO: printf Format String
puncspacing(str1, str2)
Checks for bad spacing after punctuation. ✨
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc.
Some languages don't use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account. 🫶
purepunc(str1, str2)
Checks that strings that are purely punctuation are not changed.
This extracts strings like + or - as these usually should not be changed. Literally just vibes-checking your little symbols. 💅
pythonbraceformat(str1, str2)
Checks whether python brace format strings match. 🐍 (no further notes, she's a simple icon)
run_filters(unit, categorised=False)
Do some optimisation by caching some data of the unit for the benefit of run_test(). Lowkey a little pre-game ritual. 🌸
run_test(test, unit)
Runs the given test on the given unit.
Note that this can raise a FilterFailure as part of normal operation. That's not a bug bestie, that's just how it functions. 💀
sentencecount(str1, str2)
Checks that the number of sentences in both strings match. ⭐
Adds the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way.
Do what works best for your language: it's the meaning of the original you want to convey, not the exact way it was written in the English. 👑
setconfig(config)
Sets the accelerator list.
setsuggestionstore(store)
Sets the filename that a checker should use for evaluating suggestions. 🎀
short(str1, str2)
Checks whether a translation is much shorter than the original string.
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. 🙈
simplecaps(str1, str2)
Checks the capitalisation of two strings isn't wildly different.
This will pick up many false positives, so don't be a slave to it. 💅 It is useful for identifying translations that don't start with a capital letter (upper-case letter) when they should, or those that do when they shouldn't. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case.
simpleplurals(str1, str2)
Checks for English style plural(s) for you to review. 🌈
This test will extract any message that contains words with a final "(s)" in the source text. You can then inspect the message, to check that the correct plural form has been used for your language.
In some languages, plurals are made by adding text at the beginning of words, making the English style messy. In this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives.
For languages with no plural forms (only one noun form) this test will simply test that nothing like "(s)" was used in the translation.
singlequoting(str1, str2)
Checks whether singlequoting is consistent between the two strings.
The same as doublequoting but checks for the ' character. Because this is used in contractions like it's and in possessive forms like user's, this test can output spurious errors if your language doesn't use such forms. 😭
If a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check.
spellcheck(str1, str2)
Checks words that don't pass a spell check. 🔍
This test will check for misspelled words in your translation. The test first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors e.g. program names, brand names, function names. 👑
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you have specified the --language option. ⚠️
The pofilter error that is created, lists the misspelled word, plus suggestions returned from the spell checker. That makes it easy for you to identify the word and select a replacement. ✨
startcaps(str1, str2)
Checks that the message starts with the correct capitalisation.
After stripping whitespace and common punctuation characters, it then checks to see that the first remaining character is correctly capitalised. So, if the sentence starts with an upper-case letter, and the translation does not, an error is produced.
This check is entirely disabled for many languages that don't make a distinction between upper and lower case. Contact us if this is not yet disabled for your language. 🫶
startpunc(str1, str2)
Checks whether punctuation at the beginning of the strings match.
Operates as endpunc but you will probably see fewer errors. 🎀
startwhitespace(str1, str2)
Checks whether whitespace at the beginning of the strings matches.
As in endwhitespace but you will see fewer errors.
tabs(str1, str2)
Checks whether tabs are consistent between the two strings. 🌸
Counts the number of \\t tab markers and reports an error if they differ.
unchanged(str1, str2)
Checks whether a translation is basically identical to the original string.
This checks to see if the translation isn't just a copy of the English original. Sometimes, this is what you want, but other times you will detect words that should have been translated. 👀
untranslated(str1, str2)
Checks whether a string has been translated at all.
This check is really only useful if you want to extract untranslated strings so that they can be translated independently of the main work. 🧚
urls(str1, str2)
Checks that URLs are not translated. 🔗
This checks only basic URLs (http, ftp, mailto etc.) not all URIs (e.g. afp, smb, file). Generally, you don't want to translate URLs, unless they are example URLs (‐http://your_server.com/filename.html).
If the URL is for configuration information, then you need to query the developers about placing configuration information in PO files. It shouldn't really be there, unless it is very clearly marked: such information should go into a configuration file. 💅
validchars(str1, str2)
Checks that only characters specified as valid appear in the translation.
Often during character conversion to and from UTF-8 you get some strange characters appearing in your translation. This test presents a simple way to try and identify such errors. 🦋
This test will only run of you specify the --validcharsfile command line option. This file contains all the characters that are valid in your language. You must use UTF-8 encoding for the characters in the file. ⚠️
If the test finds any characters not in your valid characters file then the test will print the character together with its Unicode value (e.g. 002B).
variables(str1, str2)— checks that variables of all the various forms stay consistent between the two strings. Basically it makes sure every variable that shows up in the original also shows up in the translation 💅 It can handle variables from projects like KDE or OpenOffice. Lowkey caveat though: right now it does not cope with variables that use the reordering syntax of Gettext PO files, so don't come for it about that 🙈xmltags(str1, str2)— checks that XML/HTML tags have not been translated, bestie. It finds the number of tags in the source string and checks that the same number are in the translation. If the counts don't match, then either the tag is missing or it was mistakenly translated by the translator — and both of those are errors, fr fr 🚨
The check ignores tags (or things that look like tags) that cover the whole string, e.g. <Error> — but it will produce false positives for things like An <Error> occurred, because here Error should actually be translated. It also allows translation of the alt attribute in e.g. <img src="bob.png" alt="Image description"> or similar translatable attributes in OpenOffice.org help files ✨
🦋 class translate.filters.checks.L20nChecker(**kwargs)
accelerators(str1, str2)— checks whether accelerators are consistent between the two strings. For Mozilla we lower the severity to cosmetic, and for some languages it also ensures accelerators are absent in the target string, since some languages just don't use accelerators — Indic languages being the example here 👑acronyms(str1, str2)— checks that acronyms that appear are unchanged. If an acronym appears in the original, this test checks that it appears in the translation. Translating acronyms is a language decision, but many languages leave them unchanged — and in that case this test is genuinely so useful for tracking down translations of the acronym and correcting them 🎀blank(str1, str2)— checks whether a translation is totally blank. It looks to see if a translation has inadvertently been translated as blank, i.e. as spaces. This is different from untranslated, which is completely empty. The test is useful because if something is translated as" "it will appear to most tools as if it is translated 😭 sneaky!brackets(str1, str2)— checks that the number of brackets in both strings matches. If([{or}])appear in the original, this will check that the same number appear in the translation 🫶categories— categories where each checking function falls into. Function names are used as keys, categories are the values.- property
checker_name— extract checker name, for example'mozilla'fromMozillaChecker. compendiumconflicts(str1, str2)— checks for Gettext compendium conflicts (#-#-#-#-#). When you usemsgcatto create a PO compendium it will insert#-#-#-#-#into entries that are not consistent. If the compendium is used later in a message merge, then these conflicts will appear in your translations. This test quickly extracts those for correction 💀credits(str1, str2)— checks for messages containing translation credits instead of normal translations. Some projects have consistent ways of giving credit to translators by having a unit or two where translators can fill in their name and possibly their contact details. This test lets you find these units easily to check that they're completed correctly, and it also disables other tests that might incorrectly get triggered for these units (such asurls,emails, etc.) 🌸dialogsizes(str1, str2)— checks that dialog sizes are not translated. This is a Mozilla-specific test. Mozilla uses a language called XUL to define dialogues and screens, and that can make use of CSS to specify properties of the dialogue. These properties include things such as the width and height of the box. The size might need to be changed if the dialogue size changes due to longer translations, so translators can change these settings. But you are only meant to change the number — not translate the wordswidthorheight. This check catches instances where these are translated. It will also catch other types of errors in these units 🔥doublequoting(str1, str2)— checks whether doublequoting is consistent between the two strings. It checks on double quotes"to ensure that you have the same number in both the original and the translated string. This test takes into account that several languages use different quoting characters, and will test for those instead 💖doublespacing(str1, str2)— checks for bad double-spaces by comparing to the original. This identifies if you have[space][space]in when you don't have it in the original, or if it appears in the original but not in your translation. Some of these are spurious, and how you correct them depends on the conventions of your language ⭐doublewords(str1, str2)— checks for repeated words in the translation. Words that have been repeated get highlighted by this test, e.g."the the","a a". These are generally typos that need correcting. Some languages may have valid repeated words in their structure — in that case, either ignore those instances or switch this test off 🧚emails(str1, str2)— checks that emails are not translated. Generally you should not be translating email addresses, so this check looks to see that email addresses e.g.info@example.comare not translated. In some cases of course you should translate the address, but generally you shouldn't 🙈endpunc(str1, str2)— checks whether punctuation at the end of the strings matches. This ensures the ending of your translation has the same punctuation as the original. E.g. if it ends in:[space]then so should yours. It's useful for ensuring that you have ellipses[…]in all your translations, not simply three separate full-stops. You may pick up some errors in the original: feel free to keep your translation and notify the programmers 💅
In some languages, characters such as ? or ! are always preceded by a space, e.g. [space]? — do what your language customs dictate. Other false positives you'll notice are, for example, if through changes in word-order you add "), etc. at the end of the sentence. Do not change these: your language word-order takes precedence.
It must be noted that if you are tempted to leave out [full-stop] or [colon] or add [full-stop] to a sentence, that often these have been done for a reason, e.g. a list where fullstops make it look cluttered. So, initially match them with the English, and make changes once the program is being used.
This check is aware of several language conventions for punctuation characters, such as the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it's not there yet 🌈
endwhitespace(str1, str2)— checks whether whitespace at the end of the strings matches. Operates the same asendpuncbut is only concerned with whitespace. This filter is particularly useful for those strings which will evidently be followed by another string in the program, e.g.[Password: ]or[Enter your username: ]. The whitespace is an inherent part of the string, and this filter makes sure you don't miss those important but otherwise invisible spaces! 👑
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
escapes(str1, str2)— checks whether escaping is consistent between the two strings. It checks escapes such as\\n\u0000to ensure that if they exist in the original string, you also have them in the translation 🔥filepaths(str1, str2)— checks that file paths have not been translated. It checks that paths such as/home/user1have not been translated. Generally you do not translate a file path, unless it is being used as an example, e.g.your_user_name/path/to/filename.conf💜filteraccelerators_by_list(str1, acceptlist=None)— filter out accelerators from str1.functions(str1, str2)— checks that function names are not translated. It checks that function names e.g.rgb()orgetEntity.Name()are not translated 🎀get_ignored_filters()— return checker's additional filters for current language.getfilters(excludefilters=None, limitfilters=None)— returns dictionary of available filters, including/excluding those in the given lists ✨kdecomments(str1, str2)— checks to ensure that no KDE style comments appear in the translation 💀
KDE style translator comments show up in PO files as _: comment\n — and new translators often translate the comment itself, bless them. 💅 This test is the little detective that tries to identify instances where the comment has been translated.
long(str1, str2) 💖
Checks whether a translation is much longer than the original string.
This is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise, we use a general ratio that catches very big differences but is set conservatively to limit false positives.
musttranslatewords(str1, str2) ✨
Checks that words configured as definitely translatable don’t appear in the translation.
If, for instance, in your language you decide that you must translate OK, then this test will flag any occurrences of OK in the translation if it appeared in the source string. You must specify a file containing all of the must translate words using --musttranslatefile.
newlines(str1, str2) 🎀
Checks whether newlines are consistent between the two strings.
Counts the number of \n newlines (and variants such as \r\n) and reports an error if they differ.
notranslatewords(str1, str2) 🦋
Checks that words configured as untranslatable appear in the translation too.
Many brand names should not be translated, and this test lets you easily make sure that words like Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no translate words using --notranslatefile.
numbers(str1, str2) 🌸
Checks that numbers are not translated.
Special handling for Mozilla to ignore entries that are dialog sizes.
options(str1, str2) 💅
Checks that command line options are not translated.
In messages that contain command line options, such as --help, this test will check that these remain untranslated. These could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. If the option has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated.
printf(str1, str2) 🫶
Checks whether printf format strings match.
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are placeholders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. The test can also manage variables-reordering using the %1$s syntax. The variables’ type and details following data are tested to ensure that they are strictly identical, but they may be reordered.
SEE ALSO: printf Format String
puncspacing(str1, str2) 😭
Checks for bad spacing after punctuation.
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc.
Some languages don’t use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account.
purepunc(str1, str2) 🔥
Checks that strings that are purely punctuation are not changed.
This extracts strings like + or - as these usually should not be changed.
pythonbraceformat(str1, str2) 👑
Checks whether python brace format strings match.
run_filters(unit, categorised=False) 🌈
Do some optimisation by caching some data of the unit for the benefit of run_test().
run_test(test, unit) ⭐
Runs the given test on the given unit.
Note that this can raise a FilterFailure as part of normal operation.
sentencecount(str1, str2) 💀
Checks that the number of sentences in both strings match.
Adds the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it’s the meaning of the original you want to convey, not the exact way it was written in the English.
setconfig(config) 🙈
Sets the accelerator list.
setsuggestionstore(store) 🍓
Sets the filename that a checker should use for evaluating suggestions.
short(str1, str2) 🧚
Checks whether a translation is much shorter than the original string.
This is most useful in the special case where the translation is 1 character long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives.
simplecaps(str1, str2) 💜
Checks the capitalisation of two strings isn’t wildly different.
This will pick up many false positives, so don’t be a slave to it. It is useful for identifying translations that don’t start with a capital letter (upper-case letter) when they should, or those that do when they shouldn’t. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case.
simpleplurals(str1, str2) 🎀
Checks for English style plural(s) for you to review.
This test will extract any message that contains words with a final (s) in the source text. You can then inspect the message, to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy. In this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives.
For languages with no plural forms (only one noun form) this test will simply test that nothing like (s) was used in the translation.
singlequoting(str1, str2) 🦋
Checks whether singlequoting is consistent between the two strings.
The same as doublequoting but checks for the ' character. Because this is used in contractions like it’s and in possessive forms like user’s, this test can output spurious errors if your language doesn’t use such forms. If a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check.
spellcheck(str1, str2) ✨
Checks words that don’t pass a spell check.
This test will check for misspelled words in your translation. The test first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors e.g. program names, brand names, function names.
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you have specified the --language option.
The pofilter error that is created, lists the misspelled word, plus suggestions returned from the spell checker. That makes it easy for you to identify the word and select a replacement.
startcaps(str1, str2) 🌸
Checks that the message starts with the correct capitalisation.
After stripping whitespace and common punctuation characters, it then checks to see that the first remaining character is correctly capitalised. So, if the sentence starts with an upper-case letter, and the translation does not, an error is produced.
This check is entirely disabled for many languages that don’t make a distinction between upper and lower case. Contact us if this is not yet disabled for your language.
startpunc(str1, str2) 💅
Checks whether punctuation at the beginning of the strings match.
Operates as endpunc but you will probably see fewer errors.
startwhitespace(str1, str2) 🫶
Checks whether whitespace at the beginning of the strings matches.
As in endwhitespace but you will see fewer errors.
tabs(str1, str2) 🎀
Checks whether tabs are consistent between the two strings.
Counts the number of \t tab markers and reports an error if they differ.
unchanged(str1, str2) 💖
Okay so this one checks whether a translation is basically identical to the original string. Slay for catching stuff that was supposed to change but didn’t.
There’s also special handling for Mozilla here, so entries that are dialog sizes get ignored. Very niche, very iconic.
untranslated(str1, str2) 🎀
Checks whether a string has been translated at all. Like, zero translation energy.
This check is really only useful if you want to extract untranslated strings so they can be translated independently of the main work. Fr fr, that’s its whole lane.
urls(str1, str2) 🌸
Checks that URLs are not translated. Do not touch the link, bestie.
This checks only basic URLs (http, ftp, mailto etc.) — not all URIs (e.g. afp, smb, file). Generally, you don’t want to translate URLs, unless they are example URLs (‐ http://your_server.com/filename.html).
If the URL is for configuration information, then you need to query the developers about placing configuration information in PO files. It shouldn’t really be there, unless it is very clearly marked: such information should go into a configuration file.
validchars(str1, str2) 🦋
Checks that only characters specified as valid appear in the translation. It’s giving “character guest list, and if you’re not on it, you’re not getting in.”
Often during character conversion to and from UTF-8 you get some strange characters appearing in your translation. This test presents a simple way to try and identify such errors.
This test will only run if you specify the --validcharsfile command line option. This file contains all the characters that are valid in your language. You must use UTF-8 encoding for the characters in the file.
If the test finds any characters not in your valid characters file then the test will print the character together with its Unicode value (e.g. 002B). Cute little detective behavior, honestly.
variables(str1, str2) 💅
Checks whether variables of various forms are consistent between the two strings. Very “same energy, same variables” of it.
This checks to make sure that variables that appear in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice.
It does not at the moment cope with variables that use the reordering syntax of Gettext PO files. Not me pointing out the limitation, it’s just literally in the manual.
xmltags(str1, str2) ✨
Checks that XML/HTML tags have not been translated. Tags are not for translating, periodt.
This check finds the number of tags in the source string and checks that the same number are in the translation. If the counts don’t match then either the tag is missing or it was mistakenly translated by the translator, both of which are errors.
The check ignores tags or things that look like tags that cover the whole string e.g. <Error> but will produce false positives for things like An <Error> occurred as here Error should be translated. It also will allow translation of the alt attribute in e.g. <img src="bob.png" alt="Image description"> or similar translatable attributes in OpenOffice.org help files. So it’s not just being messy — there are rules.
class translate.filters.checks.LibreOfficeChecker(**kwargs) 👑
accelerators(str1, str2) 🔥
Checks whether accelerators are consistent between the two strings.
This test is capable of checking the different type of accelerators that are used in different projects, like Mozilla or KDE. The test will pick up accelerators that are missing and ones that shouldn’t be there. It’s basically doing a vibe check on your shortcut keys.
See accelerators on the localization guide for a full description on accelerators.
acronyms(str1, str2) 🌈
Checks that acronyms that appear are unchanged. If it was an acronym in the original, it should still be that acronym in the translation.
If an acronym appears in the original this test will check that it appears in the translation. Translating acronyms is a language decision but many languages leave them unchanged. In that case this test is useful for tracking down translations of the acronym and correcting them.
blank(str1, str2) 🫶
Checks whether a translation is totally blank. Not “empty-empty,” but sneaky-blank.
This will check to see if a translation has inadvertently been translated as blank i.e. as spaces. This is different from untranslated which is completely empty. This test is useful in that if something is translated as “ “ it will appear to most tools as if it is translated. The audacity of a space pretending to be a whole translation 😭
brackets(str1, str2) 🎀
Checks that the number of brackets in both strings match.
If ([{ or }]) appear in the original this will check that the same number appears in the translation. Bracket balance is self-care.
categories ⭐
Categories where each checking function falls into. Function names are used as keys, categories are the values.
property checker_name 💜
Extract checker name, for example 'mozilla' from MozillaChecker. Very “tell me who you really are” behavior.
compendiumconflicts(str1, str2) 💀
Checks for Gettext compendium conflicts (#-#-#-#-#).
When you use msgcat to create a PO compendium it will insert #-#-#-#-# into entries that are not consistent. If the compendium is used later in a message merge then these conflicts will appear in your translations. This test quickly extracts those for correction. Thank u, next.
credits(str1, str2) 🌟
Checks for messages containing translation credits instead of normal translations.
Some projects have consistent ways of giving credit to translators by having a unit or two where translators can fill in their name and possibly their contact details. This test allows you to find these units easily to check that they are completed correctly and also disables other tests that might incorrectly get triggered for these units (such as urls, emails, etc.)
doublequoting(str1, str2) 🦋
Checks whether doublequoting is consistent between the two strings.
Checks on double quotes " to ensure that you have the same number in both the original and the translated string. This test takes into account that several languages use different quoting characters, and will test for them instead. Quotation marks are lowkey a whole personality.
doublespacing(str1, str2) 🌸
Checks for bad double-spaces by comparing to original.
This will identify if you have [space][space] in when you don’t have it in the original or it appears in the original but not in your translation. Some of these are spurious and how you correct them depends on the conventions of your language. The invisible details are always the most dramatic.
doublewords(str1, str2) 🙈
Checks for repeated words in the translation.
Words that have been repeated in a translation will be highlighted with this test e.g. “the the”, “a a”. These are generally typos that need correcting. Some languages may have valid repeated words in their structure, in that case either ignore those instances or switch this test off.
emails(str1, str2) 💌
Checks that emails are not translated. Leave the address alone, bestie.
Generally you should not be translating email addresses. This check will look to see that email addresses e.g. info@example.com are not translated. In some cases of course you should translate the address but generally you shouldn’t.
endpunc(str1, str2) 💅
Checks whether punctuation at the end of the strings match.
This will ensure that the ending of your translation has the same punctuation as the original. E.g. if it ends in :[space] then so should yours. It is useful for ensuring that you have ellipses […] in all your translations, not simply three separate full-stops. You may pick up some errors in the original: feel free to keep your translation and notify the programmers.
In some languages, characters such as ? or ! are always preceded by a space e.g. [space]? — do what your language customs dictate. Other false positives you will notice are, for example, if through changes in word-order you add “), etc. at the end of the sentence. Do not change these: your language word-order takes precedence.
It must be noted that if you are tempted to leave out[full-stop]or[colon]or add[full-stop]to a sentence, that often these have been done for a reason, e.g. a list where fullstops make it look cluttered. So, initially match them with the English, and make changes once the program is being used.
This check is aware of several language conventions for punctuation characters, such as the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it is not there yet.
endwhitespace(str1, str2) 🫶
Checks whether whitespace at the end of the strings matches.
Operates the same as endpunc but is only concerned with whitespace. This filter is particularly useful for those strings which will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. The whitespace is an inherent part of the string. This filter makes sure you don’t miss those important but otherwise invisible spaces!
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
escapes(str1, str2)
Okay so this one checks whether escaping is consistent between the two strings 💅
Like, it checks escapes such as \\n and \u0000 to make sure that if they exist in the original string, you also have them in the translation. No sneaky little escape codes going missing on your watch, bestie ✨
filepaths(str1, str2)
Checks that file paths have not been translated 🫶
So it checks that paths such as /home/user1 have not been translated. Generally you do not translate a file path, unless it is being used as an example, e.g. your_user_name/path/to/filename.conf. Iconic exception, honestly 🎀
filteraccelerators_by_list(str1, acceptlist=None)
Filters out accelerators from str1 🧚 Lowkey a little cleanup crew, no cap.
functions(str1, str2)
Checks that function names are not translated 🔥
It checks that function names e.g. rgb() or getEntity.Name() are not translated. Leave the functions alone, they're doing main character work 💀
get_ignored_filters()
Returns the checker's additional filters for the current language ✨ Very "here's what we're skipping, don't @ me" energy.
getfilters(excludefilters=None, limitfilters=None)
Returns a dictionary of available filters, including/excluding those in the given lists 👑
kdecomments(str1, str2)
Checks to ensure that no KDE style comments appear in the translation 🌸
KDE style translator comments appear in PO files as _: followed by comment\n. New translators often translate the comment 😭 — which, fair, if you didn't know? Anyway, this test tries to identify instances where the comment has been translated. Caught in 4K, respectfully 💅
long(str1, str2)
Checks whether a translation is much longer than the original string 🦋
This is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. She's cautious, she's reasonable, she slays 🌈
musttranslatewords(str1, str2)
Checks that words configured as definitely translatable don't appear in the translation ⭐
So, if for instance in your language you decide that you must translate 'OK', then this test will flag any occurrences of 'OK' in the translation if it appeared in the source string. You must specify a file containing all of the must translate words using --musttranslatefile. Recipe required, bestie 📋
newlines(str1, str2)
Checks whether newlines are consistent between the two strings 💜
Counts the number of \\n newlines (and variants such as \\r\\n) and reports and error if they differ. Yes, it really says "reports and error" in the original — I'm keeping my fave typo, it's part of the lore 💀
notranslatewords(str1, str2)
Checks that words configured as untranslatable appear in the translation too 🔥
Many brand names should not be translated, this test allows you to easily make sure that words like: Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no translate words using --notranslatefile. Brand names stay brand names, periodt 👑
numbers(str1, str2)
Checks whether numbers of various forms are consistent between the two strings 🔢✨
You will see some errors where you have either written the number in full or converted it to the digit in your translation. Also changes in order will trigger this error. The digits are watching 👀
options(str1, str2)
Checks that command line options are not translated 🎀
In messages that contain command line options, such as --help, this test will check that these remain untranslated. These could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. If the options has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated. Two different jobs, one icon 💅
printf(str1, str2)
Checks whether printf format strings match 😭
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are placeholders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. The test can also manage variables-reordering using the %1$s syntax. The variables' type and details following data are tested to ensure that they are strictly identical, but they may be reordered 🫶
SEE ALSO: printf Format String
puncspacing(str1, str2)
Checks for bad spacing after punctuation 🦋
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc. 💅
Some languages don't use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account — she's not trying to do you dirty, she read the room 🌸
purepunc(str1, str2)
Checks that strings that are purely punctuation are not changed ⭐
This extracts strings like + or - as these usually should not be changed. Tiny strings, big responsibility, no cap 💀
pythonbraceformat(str1, str2)
Not used in LibreOffice. That's it, that's the whole vibe 🙈
run_filters(unit, categorised=False)
Do some optimisation by caching some data of the unit for the benefit of run_test(). Efficient queen behaviour 👑
run_test(test, unit)
Runs the given test on the given unit 🔥
Note that this can raise a FilterFailure as part of normal operation. Not a bug, babes — it's a feature, fr fr 💜
sentencecount(str1, str2)
Checks that the number of sentences in both strings match 📝
Adds the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it's the meaning of the original you want to convey, not the exact way it was written in the English 🌈 So real of her to say that.
setconfig(config)
Sets the accelerator list 🎀 Short, sweet, does exactly what it says.
setsuggestionstore(store)
Sets the filename that a checker should use for evaluating suggestions ✨ File path in, judgement out.
short(str1, str2)
Checks whether a translation is much shorter than the original string 🦋
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. (Yes, "1 characters" — the original said it and I'm not about to edit her 😭)
simplecaps(str1, str2)
Checks the capitalisation of two strings isn't wildly different 💅
This will pick up many false positives, so don't be a slave to it. It is useful for identifying translations that don't start with a capital letter (upper-case letter) when they should, or those that do when they shouldn't. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case 🔠 Stay calm, it's just vibes-checking your caps.
simpleplurals(str1, str2)
Checks for English style plural(s) for you to review 👀
This test will extract any message that contains words with a final "(s)" in the source text. You can then inspect the message, to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy. In this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives 😭
For languages with no plural forms (only one noun form) this test will simply test that nothing like "(s)" was used in the translation 🌸
singlequoting(str1, str2)
Checks whether singlequoting is consistent between the two strings 🎀
The same as doublequoting but checks for the ' character. Because this is used in contractions like it's and in possessive forms like user's, this test can output spurious errors if your language doesn't use such forms. If a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check. She's trying her best, okay 😭
spellcheck(str1, str2)
Checks words that don't pass a spell check 📚✨
This test will check for misspelled words in your translation. The test first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors e.g. program names, brand names, function names. Smart, honestly 👑
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you have specified the --language option 🔥 No language, no spellcheck — those are the rules, bestie.
The error that pofilter hands back lists the misspelled word plus whatever suggestions the spell checker came back with. Which is lowkey the best part, because it means you can literally look at the word, look at the suggestions, and pick a replacement. Chef's kiss. 🎀
startcaps(str1, str2)
Checks that the message starts with the correct capitalisation.
So here's the vibe: it strips off whitespace and common punctuation characters first, then peeks at the first character that's left and asks "is this capitalised correctly?" If the source sentence starts with an upper-case letter and your translation doesn't, that's an error, bestie. Periodt.
Heads up: this check is entirely disabled for a whole bunch of languages that don't even do the upper/lower case thing. And if it's not disabled yet for your language? Contact the devs, fr fr.
startpunc(str1, str2)
Checks whether punctuation at the beginning of the strings match.
It works exactly like endpunc but honestly you'll probably see way fewer errors with it. ✨
startwhitespace(str1, str2)
Checks whether whitespace at the beginning of the strings matches.
Same deal as endwhitespace, but again — fewer errors in practice. 🦋
tabs(str1, str2)
Checks whether tabs are consistent between the two strings.
It literally counts the number of \t tab markers and reports an error if they don't match up. That's it. That's the whole check. 💅
unchanged(str1, str2)
Checks whether a translation is basically identical to the original string.
Basically it's sniffing out whether your "translation" is just a copy-paste of the English original. Sometimes that's exactly what you want (looking at you, brand names 😌), but other times it's your signal that some words that should have been translated weren't. 🙈
untranslated(str1, str2)
Checks whether a string has been translated at all.
Honestly this one is only useful if your whole goal is to extract the untranslated strings so they can get done separately from the main work. It's giving "sorting the laundry before the real task" energy. 🧺
urls(str1, str2)
Checks that URLs are not translated.
Note the scope, bestie — this only checks basic URLs (http, ftp, mailto etc.), not all URIs (so like afp, smb, file are not covered). Generally you do not want to translate URLs, unless they're example URLs like http://your_server.com/filename.html.
And if the URL is there for configuration info? Then you need to go query the developers about putting configuration information in PO files. It shouldn't really be in there unless it's very clearly marked — that kind of info belongs in a configuration file. 🔥
validchars(str1, str2)
Checks that only characters specified as valid appear in the translation.
Okay so you know how converting to and from UTF-8 sometimes coughs up some truly cursed characters in your translation? 😭 This test is your simple little way to try and catch those errors.
Important: this test will only run if you specify the --validcharsfile command line option. That file contains all the characters that are valid in your language, and the characters in it must be in UTF-8 encoding.
If the test finds any character that's not in your valid characters file, it prints the character together with its Unicode value (e.g. 002B). Iconic detective work honestly. 🔍
validxml(str1, str2)
Check that all XML/HTML open/close tags has close/open pair in the translation.
Simple as that — every open needs its close, every close needs its open. 🎀
variables(str1, str2)
Checks whether variables of various forms are consistent between the two strings.
It makes sure the variables that show up in the original also show up in the translation, and it can handle variables from projects like KDE or OpenOffice.
Real talk though: at the moment it does not cope with variables that use the reordering syntax of Gettext PO files. Ngl, that's just where it's at. 💀
xmltags(str1, str2)
Checks that XML/HTML tags have not been translated.
It counts the tags in the source string, then checks that the exact same number show up in the translation. If the counts don't match, then either a tag went missing or the translator accidentally translated it — and both of those are errors, no cap.
It ignores tags (and things that look like tags) that cover the whole string, e.g. <Error>. But it will produce false positives for things like An <Error> occurred, because here Error genuinely should be translated. 🙃
It also allows translation of the alt attribute in things like <img src="bob.png" alt="Image description">, or similar translatable attributes in OpenOffice.org help files. Slay. ✨
class translate.filters.checks.MinimalChecker(**kwargs)
accelerators(str1, str2)
Checks whether accelerators are consistent between the two strings.
This test is built different — it can handle the different types of accelerators used across different projects, like Mozilla or KDE. It'll catch accelerators that are missing and ones that shouldn't be there at all. 🚨
Want the full description on accelerators? See accelerators on the localization guide.
acronyms(str1, str2)
Checks that acronyms that appear are unchanged.
If an acronym shows up in the original, this test checks that it shows up in the translation too. Whether to translate acronyms is a language decision — plenty of languages just leave them alone. And if yours is one of them, this test is your bestie for tracking down translated acronyms and fixing them. 💖
blank(str1, str2)
Checks whether a translation is totally blank.
This one catches the situation where a translation has accidentally been filled in as blank — like, actual spaces. Which is different from untranslated, where it's completely empty. Why does it matter? Because if something is translated as " ", it'll look translated to most tools. 💀 Sneaky.
brackets(str1, str2)
Checks that the number of brackets in both strings match.
If (, [, {, or }, ], ) appear in the original, this checks that the same number appear in the translation. Balanced brackets or bust. ⚖️
categories
Categories where each checking function falls into. Function names are used as keys, categories are the values. 🗂️
property checker_name
Extract checker name, for example mozilla from MozillaChecker. Cute, right? 🎀
compendiumconflicts(str1, str2)
Checks for Gettext compendium conflicts (#-#-#-#-#).
So when you use msgcat to create a PO compendium, it shoves #-#-#-#-# into entries that aren't consistent. And if that compendium gets used later in a message merge, those conflicts land straight in your translations. 😭 This test pulls them out super fast so you can fix them. Iconic.
credits(str1, str2)
Checks for messages containing translation credits instead of normal translations.
Some projects have a consistent way of giving credit to translators — like a unit or two where translators drop their name and maybe their contact details. This test lets you find those units easily so you can check they're filled in correctly, and it also disables other tests that might get wrongly triggered on them (things like urls, emails, etc.). Genius behaviour, honestly. 👑
doublequoting(str1, str2)
Checks whether doublequoting is consistent between the two strings.
It checks the double quote " to make sure you've got the same number of them in the original and in your translated string. And because several languages use different quoting characters, it takes that into account and tests for those instead. Flexible queen. 💅
doublespacing(str1, str2)
Checks for bad double-spaces by comparing to original.
So this identifies when you've got [space][space] but the original doesn't have it — or the original has it and your translation doesn't. Some of these are spurious, and how you fix them depends on the conventions of your language. No single answer, iykyk. 🦋
doublewords(str1, str2)
Checks for repeated words in the translation.
Words you've accidentally repeated get highlighted by this test, e.g. "the the", "a a". Generally these are typos that need correcting, periodt. Some languages may legitimately have repeated words in their structure though — in that case either ignore those instances or just switch this test off. 🌸
emails(str1, str2)
Checks that emails are not translated.
Generally you should not be translating email addresses, bestie. This check makes sure email addresses like info@example.com haven't been translated. Sure, in some cases you genuinely should translate the address, but generally? You shouldn't. 🙅
endpunc(str1, str2)
Checks whether punctuation at the end of the strings match.
This is your whole vibe check for punctuation, bestie — make sure the ending of your translation has the same punctuation as the original. 💅 Like, if it ends in :[space] then so should yours. It's lowkey iconic for making sure you've got ellipses […] in all your translations, not just three sad little separate full-stops pretending to be one. You might even spot errors in the original: feel free to keep your translation and notify the programmers (be nice about it 🫶). In some languages, characters such as ? or ! are always preceded by a space, e.g. [space]? — do what your language customs dictate, periodt. Other false positives you'll notice are, for example, if through changes in word-order you add “), etc. at the end of the sentence. Do not change these: your language word-order takes precedence, fr fr.
It must be noted that if you are tempted to leave out [full-stop] or [colon] or add [full-stop] to a sentence, that often these have been done for a reason — e.g. a list where fullstops make it look cluttered. So initially match them with the English, and make changes once the program is being used. No cap, that's the move. ✨
This check is aware of several language conventions for punctuation characters, such as the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it is not there yet — slay, inclusivity. 🌈
endwhitespace(str1, str2)
Checks whether whitespace at the end of the strings matches. 👑
Operates the same as endpunc but is only concerned with whitespace. This filter is particularly useful for those strings which will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. The whitespace is an inherent part of the string. This filter makes sure you don't miss those important but otherwise invisible spaces! 🫥✨
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
escapes(str1, str2)
Checks whether escaping is consistent between the two strings.
Checks escapes such as \n \u0000 to ensure that if they exist in the original string you also have them in the translation. Don't touch the backslashes, bestie, they're doing important work. 💀
filepaths(str1, str2)
Checks that file paths have not been translated.
Checks that paths such as /home/user1 have not been translated. Generally you do not translate a file path, unless it is being used as an example, e.g. your_user_name/path/to/filename.conf. Like, obviously leave the actual path alone, but the example placeholder? That's fair game. 🙈
filteraccelerators_by_list(str1, acceptlist=None)
Filter out accelerators from str1.
functions(str1, str2)
Checks that function names are not translated.
Checks that function names e.g. rgb() or getEntity.Name() are not translated. Code is code, babes — it stays in its original language. 💻💅
get_ignored_filters()
Return checker's additional filters for current language.
getfilters(excludefilters=None, limitfilters=None)
Returns dictionary of available filters, including/excluding those in the given lists.
kdecomments(str1, str2)
Checks to ensure that no KDE style comments appear in the translation.
KDE style translator comments appear in PO files as _: comment\n. New translators often translate the comment 😭 — this test tries to identify instances where the comment has been translated. Don't translate the note-to-self, it's not for the users!
long(str1, str2)
Checks whether a translation is much longer than the original string.
This is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. So it's giving "gentle nudge," not "callout." 🎀
musttranslatewords(str1, str2)
Checks that words configured as definitely translatable don't appear in the translation.
If for instance in your language you decide that you must translate ‘OK’ then this test will flag any occurrences of ‘OK’ in the translation if it appeared in the source string. You must specify a file containing all of the must translate words using --musttranslatefile. 👑
newlines(str1, str2)
Checks whether newlines are consistent between the two strings.
Counts the number of \n newlines (and variants such as \r\n) and reports and error if they differ. Line breaks matter, bestie — they're literally the layout. 🧚
notranslatewords(str1, str2)
Checks that words configured as untranslatable appear in the translation too.
Many brand names should not be translated, this test allows you to easily make sure that words like: Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no translate words using --notranslatefile. Brand names stay iconic and untouched. 💅
numbers(str1, str2)
Checks whether numbers of various forms are consistent between the two strings.
You will see some errors where you have either written the number in full or converted it to the digit in your translation. Also changes in order will trigger this error. Numbers are numbers, no cap. 🔢
options(str1, str2)
Checks that command line options are not translated.
In messages that contain command line options, such as --help, this test will check that these remain untranslated. These could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. If the options has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated. So the flag stays, the placeholder gets localised — the way that works is actually kind of iconic. 🌟
printf(str1, str2)
Checks whether printf format strings match.
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are placeholders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. The test can also manage variables-reordering using the %1$s syntax. The variables' type and details following data are tested to ensure that they are strictly identical, but they may be reordered. 🔥
SEE ALSO: printf Format String
puncspacing(str1, str2)
Checks for bad spacing after punctuation.
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc.
Some languages don't use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account. It sees you, it understands you. 👀💜
purepunc(str1, str2)
Checks that strings that are purely punctuation are not changed.
This extracts strings like + or - as these usually should not be changed. Tiny but mighty, honestly. ✨
pythonbraceformat(str1, str2)
Checks whether python brace format strings match.
run_filters(unit, categorised=False)
Do some optimisation by caching some data of the unit for the benefit of run_test().
run_test(test, unit)
Runs the given test on the given unit.
Note that this can raise a FilterFailure as part of normal operation. Not a bug, just how it do. 💀
sentencecount(str1, str2)
Checks that the number of sentences in both strings match.
Adds the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it's the meaning of the original you want to convey, not the exact way it was written in the English. 🫶 Periodt. Meaning > mirror-copying.
setconfig(config)
Sets the accelerator list.
setsuggestionstore(store)
Sets the filename that a checker should use for evaluating suggestions.
short(str1, str2)
Checks whether a translation is much shorter than the original string.
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. Again — gentle nudge, not an attack. 🌸
simplecaps(str1, str2)
Checks the capitalisation of two strings isn't wildly different.
This will pick up many false positives, so don't be a slave to it. It is useful for identifying translations that don't start with a capital letter (upper-case letter) when they should, or those that do when they shouldn't. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case. 🌈 Capitalisation is a vibe, not a law.
simpleplurals(str1, str2)
Checks for English style plural(s) for you to review. 💅
This test will extract any message that contains words with a final “(s)” in the source text. You can then inspect the message, to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy — in that case they often revert to the plural form. This test lets an editor check that the plurals used are correct. Be aware that this test may create a number of false positives. 🙈
For languages with no plural forms (only one noun form) this test will simply test that nothing like “(s)” was used in the translation.
singlequoting(str1, str2)
Checks whether singlequoting is consistent between the two strings.
The same as doublequoting but checks for the ' character. Because this is used in contractions like it’s and in possessive forms like user’s, this test can output spurious errors if your language doesn’t use such forms. Also, if a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check. 😭
spellcheck(str1, str2)
Checks words that don’t pass a spell check. ✨
This test will check for misspelled words in your translation. It first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors — e.g. program names, brand names, function names. So not me accidentally flagging your app name 💀
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you have specified the --language option. 👑
The pofilter error that is created lists the misspelled word, plus suggestions returned from the spell checker. That makes it easy for you to identify the word and select a replacement.
startcaps(str1, str2)
Checks that the message starts with the correct capitalisation.
After stripping whitespace and common punctuation characters, it then checks that the first remaining character is correctly capitalised. So, if the sentence starts with an upper-case letter, and the translation does not, an error is produced.
This check is entirely disabled for many languages that don’t make a distinction between upper and lower case. Contact us if this is not yet disabled for your language. 🫶
startpunc(str1, str2)
Checks whether punctuation at the beginning of the strings match.
Operates as endpunc but you will probably see fewer errors. 🌸
startwhitespace(str1, str2)
Checks whether whitespace at the beginning of the strings matches.
As in endwhitespace but you will see fewer errors.
tabs(str1, str2)
Checks whether tabs are consistent between the two strings.
Counts the number of \\t tab markers and reports an error if they differ. Literally just counting tabs bestie 🎀
unchanged(str1, str2)
Checks whether a translation is basically identical to the original string.
This checks to see if the translation isn’t just a copy of the English original. Sometimes that's exactly what you want, but other times you’ll catch words that should have been translated. 👀
untranslated(str1, str2)
Checks whether a string has been translated at all.
This check is really only useful if you want to extract untranslated strings so that they can be translated independently of the main work. Lowkey a sorting tool, not a judging tool 💜
urls(str1, str2)
Checks that URLs are not translated.
This checks only basic URLs (http, ftp, mailto etc.) — not all URIs (e.g. afp, smb, file). Generally, you don’t want to translate URLs, unless they are example URLs (http://your_server.com/filename.html). If the URL is for configuration information, then you need to query the developers about placing configuration information in PO files. It shouldn’t really be there, unless it is very clearly marked: such information should go into a configuration file. 🔥
validchars(str1, str2)
Checks that only characters specified as valid appear in the translation.
Often during character conversion to and from UTF-8 you get some strange characters appearing in your translation. This test presents a simple way to try and identify such errors. 🌈
This test will only run if you specify the --validcharsfile command line option. This file contains all the characters that are valid in your language. You must use UTF-8 encoding for the characters in the file.
If the test finds any characters not in your valid characters file then the test will print the character together with its Unicode value (e.g. 002B). 🔍
variables(str1, str2)
Checks whether variables of various forms are consistent between the two strings.
This checks to make sure that variables that appear in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice. It does not at the moment cope with variables that use the reordering syntax of Gettext PO files. ⚠️ (so don't come for it)
xmltags(str1, str2)
Checks that XML/HTML tags have not been translated.
This check finds the number of tags in the source string and checks that the same number are in the translation. If the counts don’t match then either the tag is missing or it was mistakenly translated by the translator — both of which are errors. 💀
The check ignores tags, or things that look like tags, that cover the whole string e.g. <Error>, but will produce false positives for things like An <Error> occurred — as here Error should be translated. It also will allow translation of the alt attribute in e.g. <img src="bob.png" alt="Image description"> or similar translatable attributes in OpenOffice.org help files. 🖼️
class translate.filters.checks.MozillaChecker(**kwargs)
accelerators(str1, str2)
Checks whether accelerators are consistent between the two strings.
For Mozilla we lower the severity to cosmetic, and for some languages it also ensures accelerators are absent in the target string, since some languages do not use accelerators — for example Indic languages. ✨
acronyms(str1, str2)
Checks that acronyms that appear are unchanged.
If an acronym appears in the original this test will check that it appears in the translation. Translating acronyms is a language decision but many languages leave them unchanged. In that case this test is useful for tracking down translations of the acronym and correcting them. 👑
blank(str1, str2)
Checks whether a translation is totally blank.
This will check to see if a translation has inadvertently been translated as blank, i.e. as spaces. This is different from untranslated which is completely empty. This test is useful in that if something is translated as “ “ it will appear to most tools as if it is translated. 🙈 (the audacity honestly)
brackets(str1, str2)
Checks that the number of brackets in both strings match.
If ([{ or }]) appear in the original this will check that the same number appear in the translation.
categories
Categories where each checking function falls into. Function names are used as keys, categories are the values. 🗂️
property checker_name
Extract checker name, for example ‘mozilla’ from MozillaChecker. 🕵️
compendiumconflicts(str1, str2)
Checks for Gettext compendium conflicts (#-#-#-#-#).
When you use msgcat to create a PO compendium it will insert #-#-#-#-# into entries that are not consistent. If the compendium is used later in a message merge then these conflicts will appear in your translations. This test quickly extracts those for correction. 🔥
credits(str1, str2)
Checks for messages containing translation credits instead of normal translations.
Some projects have consistent ways of giving credit to translators by having a unit or two where translators can fill in their name and possibly their contact details. This test allows you to find these units easily to check that they are completed correctly and also disables other tests that might incorrectly get triggered for these units (such as urls, emails, etc.) 🫶
dialogsizes(str1, str2)
Checks that dialog sizes are not translated. 🎀
This next one is Mozilla-coded, bestie — heads up.
the Mozilla dialogue-size test (yes, another Mozilla one)
This is a Mozilla specific test. Mozilla uses a language called XUL to define dialogues and screens, and XUL can make use of CSS to specify properties of the dialogue — things like the width and height of the box, you know the vibe. If the dialogue size changes because the translation got longer, that size might need to change too, so translators are allowed to tweak these settings. But babes, you're only meant to change the number, not translate the words ‘width’ or ‘height’. 💅 This check captures instances where those words got translated, and it'll also catch other types of errors in these units.
doublequoting(str1, str2)
Checks whether doublequoting is consistent between the two strings.
It looks at double quotes " to make sure you've got the same number in both the original and the translated string. And the slay part? It takes into account that several languages use different quoting characters, and will test for those instead. Inclusive queen. 👑
doublespacing(str1, str2)
Checks for bad double-spaces by comparing to original.
This one identifies if you've got [space][space] when the original doesn't have it, or if the original has it but your translation doesn't. Some of these hits are spurious, and how you fix them totally depends on the conventions of your language — so use your judgment, bestie.
doublewords(str1, str2)
Checks for repeated words in the translation.
Words that got repeated in a translation get highlighted by this test, e.g. “the the”, “a a”. These are generally typos that need correcting, fr fr. That said, some languages may have valid repeated words in their structure — in that case either ignore those instances or switch this test off. No pressure. 🫶
emails(str1, str2)
Checks that emails are not translated.
Generally you should not be translating email addresses, periodt. This check looks to see that email addresses e.g. info@example.com are not translated. In some cases of course you should translate the address, but generally you shouldn't — iykyk.
endpunc(str1, str2)
Checks whether punctuation at the end of the strings match.
This ensures the ending of your translation has the same punctuation as the original. E.g. if it ends in :[space] then so should yours. Super useful for making sure you have ellipses […] in all your translations, not simply three separate full-stops. 😭 You may also pick up some errors in the original here — feel free to keep your translation and notify the programmers. In some languages, characters such as ? or ! are always preceded by a space e.g. [space]? — do what your language customs dictate. Other false positives you'll notice are, for example, if through changes in word-order you add "), etc." at the end of the sentence. Do not change these: your language word-order takes precedence.
It must be noted that if you are tempted to leave out [full-stop] or [colon] or add [full-stop] to a sentence, that often these have been done for a reason, e.g. a list where fullstops make it look cluttered. So initially match them with the English, and make changes once the program is being used.
This check is aware of several language conventions for punctuation characters, such as the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it is not there yet. 🌈
endwhitespace(str1, str2)
Checks whether whitespace at the end of the strings matches.
Operates the same as endpunc but is only concerned with whitespace. This filter is particularly useful for those strings which will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. The whitespace is an inherent part of the string, so this filter makes sure you don't miss those important but otherwise invisible spaces! 👻
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
escapes(str1, str2)
Checks whether escaping is consistent between the two strings.
Checks escapes such as \\n and \u0000 to ensure that if they exist in the original string you also have them in the translation. Don't lose the escape, babe. 🔥
filepaths(str1, str2)
Checks that file paths have not been translated.
Checks that paths such as /home/user1 have not been translated. Generally you do not translate a file path — unless it is being used as an example, e.g. your_user_name/path/to/filename.conf. Then it's fair game. 🙈
filteraccelerators_by_list(str1, acceptlist=None)
Filter out accelerators from str1. That's it, that's the whole job. ✨
functions(str1, str2)
Checks that function names are not translated.
Checks that function names e.g. rgb() or getEntity.Name() are not translated. Those are for the programmers, bestie — hands off. 💅
get_ignored_filters()
Return checker's additional filters for current language.
getfilters(excludefilters=None, limitfilters=None)
Returns dictionary of available filters, including/excluding those in the given lists. Very flexible, very demure. 🎀
kdecomments(str1, str2)
Checks to ensure that no KDE style comments appear in the translation.
KDE style translator comments appear in PO files as "_: comment\n". New translators often translate the comment, which is not the move. This test tries to identify instances where the comment has been translated. 🫶
long(str1, str2)
Checks whether a translation is much longer than the original string.
This is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. Nobody wants their notifications flooded. 😭
musttranslatewords(str1, str2)
Checks that words configured as definitely translatable don't appear in the translation.
If for instance in your language you decide that you must translate ‘OK’ then this test will flag any occurrences of ‘OK’ in the translation if it appeared in the source string. You must specify a file containing all of the must translate words using --musttranslatefile. 📋
newlines(str1, str2)
Checks whether newlines are consistent between the two strings.
Counts the number of \n newlines (and variants such as \r\n) and reports an error if they differ. Line breaks matter, okay? 💀
notranslatewords(str1, str2)
Checks that words configured as untranslatable appear in the translation too.
Many brand names should not be translated, and this test lets you easily make sure that words like: Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no translate words using --notranslatefile. 👑
numbers(str1, str2)
Checks that numbers are not translated.
Special handling for Mozilla to ignore entries that are dialog sizes. (We see you, Mozilla. Again. 💀)
options(str1, str2)
Checks that command line options are not translated.
In messages that contain command line options, such as --help, this test will check that these remain untranslated. These could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. If the option has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated — the flag stays, the parameter is yours. 🦋
printf(str1, str2)
Checks whether printf format strings match.
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are placeholders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. The test can also manage variables-reordering using the %1$s syntax. The variables' type and details following data are tested to ensure that they are strictly identical, but they may be reordered. 🧚
SEE ALSO: printf Format String
puncspacing(str1, str2)
Checks for bad spacing after punctuation.
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc.
Some languages don't use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account. 💜
purepunc(str1, str2)
Checks that strings that are purely punctuation are not changed.
This extracts strings like + or - as these usually should not be changed. Leave the little guys alone. 🍓
pythonbraceformat(str1, str2)
Checks whether python brace format strings match.
run_filters(unit, categorised=False)
Does a lil optimisation pass — caches some data of the unit so run_test() can vibe off it later 💅
run_test(test, unit)
Runs the given test on the given unit. Easy.
Heads up bestie: this can raise a FilterFailure as part of normal operation. Like it's not a bug, it's a feature 🫶
sentencecount(str1, str2)
Checks that the number of sentences in both strings match 🎀
It adds up the number of sentences so you can see the count is the same between the original and the translated string. Lowkey, you may not always want this test — especially if you find you often need to reformat your translation because the original is badly-expressed, or because the structure of your language just works better that way. Do what works best for your language, fr fr: it's the meaning of the original you want to convey, not the exact way it was written in the English ✨
setconfig(config)
Sets the accelerator list.
setsuggestionstore(store)
Sets the filename that a checker should use for evaluating suggestions.
short(str1, str2)
Checks whether a translation is much shorter than the original string 👀
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives.
simplecaps(str1, str2)
Checks the capitalisation of two strings isn't wildly different 🎀
This will pick up many false positives, so don't be a slave to it, okay? It is useful for identifying translations that don't start with a capital letter (upper-case letter) when they should, or those that do when they shouldn't. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case 💅
simpleplurals(str1, str2)
Checks for English style plural(s) for you to review 🦋
This test will extract any message that contains words with a final (s) in the source text. You can then inspect the message, to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy. In this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives 😭
For languages with no plural forms (only one noun form) this test will simply test that nothing like (s) was used in the translation.
singlequoting(str1, str2)
Checks whether singlequoting is consistent between the two strings.
The same as doublequoting but checks for the ' character. Because this is used in contractions like it's and in possessive forms like user's, this test can output spurious errors if your language doesn't use such forms. If a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check 🫠
spellcheck(str1, str2)
Checks words that don't pass a spell check 📝
This test will check for misspelled words in your translation. The test first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors — e.g. program names, brand names, function names. Iconic behaviour honestly 👑
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you have specified the --language option.
The pofilter error that is created, lists the misspelled word, plus suggestions returned from the spell checker. That makes it easy for you to identify the word and select a replacement ✨
startcaps(str1, str2)
Checks that the message starts with the correct capitalisation.
After stripping whitespace and common punctuation characters, it then checks to see that the first remaining character is correctly capitalised. So, if the sentence starts with an upper-case letter, and the translation does not, an error is produced.
This check is entirely disabled for many languages that don't make a distinction between upper and lower case. Contact us if this is not yet disabled for your language 🫶
startpunc(str1, str2)
Checks whether punctuation at the beginning of the strings match.
Operates as endpunc but you will probably see fewer errors.
startwhitespace(str1, str2)
Checks whether whitespace at the beginning of the strings matches.
As in endwhitespace but you will see fewer errors.
tabs(str1, str2)
Checks whether tabs are consistent between the two strings.
Counts the number of \t tab markers and reports an error if they differ 🎯
unchanged(str1, str2)
Checks whether a translation is basically identical to the original string.
Special handling for Mozilla to ignore entries that are dialog sizes.
untranslated(str1, str2)
Checks whether a string has been translated at all.
This check is really only useful if you want to extract untranslated strings so that they can be translated independently of the main work 🌸
urls(str1, str2)
Checks that URLs are not translated 🔗
This checks only basic URLs (http, ftp, mailto etc.) not all URIs (e.g. afp, smb, file). Generally, you don't want to translate URLs, unless they are example URLs (- http://your_server.com/filename.html). If the URL is for configuration information, then you need to query the developers about placing configuration information in PO files. It shouldn't really be there, unless it is very clearly marked: such information should go into a configuration file.
validchars(str1, str2)
Checks that only characters specified as valid appear in the translation.
Often during character conversion to and from UTF-8 you get some strange characters appearing in your translation. This test presents a simple way to try and identify such errors 👀
This test will only run of you specify the --validcharsfile command line option. This file contains all the characters that are valid in your language. You must use UTF-8 encoding for the characters in the file.
If the test finds any characters not in your valid characters file then the test will print the character together with its Unicode value (e.g. 002B).
variables(str1, str2)
Checks whether variables of various forms are consistent between the two strings.
This checks to make sure that variables that appear in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice. It does not at the moment cope with variables that use the reordering syntax of Gettext PO files 😬
xmltags(str1, str2)
Checks that XML/HTML tags have not been translated 🏷️
This check finds the number of tags in the source string and checks that the same number are in the translation. If the counts don't match then either the tag is missing or it was mistakenly translated by the translator, both of which are errors.
The check ignores tags or things that look like tags that cover the whole string e.g. <Error> but will produce false positives for things like An <Error> occurred as here Error should be translated. It also will allow translation of the alt attribute in e.g. <img src="bob.png" alt="Image description"> or similar translatable attributes in OpenOffice.org help files.
🧡 class translate.filters.checks.OpenOfficeChecker(**kwargs)
accelerators(str1, str2)
Checks whether accelerators are consistent between the two strings ⌨️
This test is capable of checking the different type of accelerators that are used in different projects, like Mozilla or KDE. The test will pick up accelerators that are missing and ones that shouldn't be there.
See accelerators on the localization guide for a full description on accelerators.
acronyms(str1, str2)
Checks that acronyms that appear are unchanged.
If an acronym appears in the original this test will check that it appears in the translation. Translating acronyms is a language decision but many languages leave them unchanged. In that case this test is useful for tracking down translations of the acronym and correcting them 💅
okay so this next bit is the full roster of checker functions — the little vibe-check agents that each look at one specific thing in your translation 👑 they all take the two strings (original + translation) unless noted, and they're listed here exactly how they're named in the code, no cap.
✨ blank(str1, str2)
Checks whether a translation is totally blank.
Basically it catches the case where someone inadvertently translated a string into just spaces. Which — real talk — is different from untranslated, because untranslated means completely empty. This test matters because if something gets translated as " " it'll look to most tools like it's been translated. It hasn't. 🙈
✨ brackets(str1, str2)
Checks that the number of brackets in both strings match.
If ([{ or }]) show up in the original, this one makes sure the same number of them show up in your translation. Balanced brackets or bust 💅
✨ categories
Categories where each checking function falls into. Function names are used as keys, categories are the values.
✨ property checker_name
Extract the checker name — for example 'mozilla' out of MozillaChecker.
✨ compendiumconflicts(str1, str2)
Checks for Gettext compendium conflicts (#-#-#-#-#).
So here's the lore: when you use msgcat to build a PO compendium, it inserts #-#-#-#-# into any entries that aren't consistent. And if that compendium gets used later in a message merge, those conflicts will pop up in your translations 😭 This test quickly extracts them so you can go fix them.
✨ credits(str1, str2)
Checks for messages containing translation credits instead of normal translations.
Some projects have a consistent way of crediting translators — like a unit or two where you fill in your name and maybe your contact details. This test lets you find those units easily so you can confirm they're filled in correctly, and it also disables other tests that might wrongly trigger on them (stuff like urls, emails, etc.). Bestie gets a hall pass 🫶
✨ doublequoting(str1, str2)
Checks whether doublequoting is consistent between the two strings.
It looks at double quotes " and makes sure you've got the same number in the original and the translated string. It's smart about it though — several languages use different quoting characters, so it'll test for those instead. iykyk 🌸
✨ doublespacing(str1, str2)
Checks for bad double-spaces by comparing to the original.
This flags it if you've got [space][space] where the original doesn't, or if the original has it and your translation doesn't. Some of these are spurious, and how you fix them depends on the conventions of your language — so don't panic, just vibe-check each one.
✨ doublewords(str1, str2)
Checks for repeated words in the translation.
Any words you accidentally repeated get highlighted — like "the the" or "a a". These are generally typos that need correcting. That said, some languages legitimately have repeated words in their structure, and in that case either ignore those instances or switch this test off entirely. No judgement 💅
✨ emails(str1, str2)
Checks that emails are not translated.
Generally you should not be translating email addresses — this check makes sure things like info@example.com haven't been translated. In some cases of course you should translate the address, but generally you shouldn't.
✨ endpunc(str1, str2)
Checks whether punctuation at the end of the strings match.
This ensures the ending of your translation has the same punctuation as the original. For example, if it ends in :[space] then so should yours. It's also genuinely useful for making sure you have real ellipses […] in all your translations instead of just three separate full-stops. 😤
You might pick up some errors in the original — feel free to keep your translation and go notify the programmers. In some languages, characters such as ? or ! are always preceded by a space, e.g. [space]? — do what your language customs dictate. Other false positives you'll notice: for example, if through changes in word-order you add ", etc. at the end of the sentence. Do not change these — your language word-order takes precedence.
It must be noted that if you're tempted to leave out[full-stop]or[colon], or add a[full-stop]to a sentence, often these have been done for a reason — e.g. a list where fullstops make it look cluttered. So initially match them with the English, and make changes once the program is being used. 🎀
This check is aware of several language conventions for punctuation characters — custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it's not there yet. Slay, inclusivity 🌈
✨ endwhitespace(str1, str2)
Checks whether whitespace at the end of the strings matches.
Works the same as endpunc but it's only concerned with whitespace. This filter is especially useful for strings that will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. The whitespace is an inherent part of the string, okay?? This filter makes sure you don't miss those important-but-otherwise-invisible spaces 💀
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
✨ escapes(str1, str2)
Checks whether escaping is consistent between the two strings.
Checks escapes such as \n and \u0000 to make sure that if they exist in the original string, you also have them in the translation.
✨ filepaths(str1, str2)
Checks that file paths have not been translated.
It checks that paths such as /home/user1 haven't been translated. Generally you do not translate a file path — unless it's being used as an example, e.g. your_user_name/path/to/filename.conf. Context is everything, bestie 🦋
✨ filteraccelerators_by_list(str1, acceptlist=None)
Filter out accelerators from str1.
✨ functions(str1, str2)
Checks that function names are not translated.
It checks that function names e.g. rgb() or getEntity.Name() haven't been translated. Code stays code, periodt ✨
✨ get_ignored_filters()
Return checker's additional filters for current language.
✨ getfilters(excludefilters=None, limitfilters=None)
Returns a dictionary of available filters, including/excluding those in the given lists.
✨ kdecomments(str1, str2)
Checks to ensure that no KDE style comments appear in the translation.
KDE style translator comments appear in PO files as _: comment\n. New translators often translate the comment 😭 which is exactly what this test tries to catch — instances where the comment has been translated.
✨ long(str1, str2)
Checks whether a translation is much longer than the original string.
This is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise it uses a general ratio that'll catch very big differences, but it's set conservatively to limit the number of false positives. Respect the restraint 👑
✨ musttranslatewords(str1, str2)
Checks that words configured as definitely translatable don't appear in the translation.
So like, if in your language you've decided you must translate 'OK', then this test flags any occurrences of 'OK' in the translation if it appeared in the source string. You must specify a file containing all of the must-translate words using --musttranslatefile.
✨ newlines(str1, str2)
Checks whether newlines are consistent between the two strings.
Counts the number of \n newlines (and variants such as \r\n) and reports and error if they differ.
✨ notranslatewords(str1, str2)
Checks that words configured as untranslatable appear in the translation too.
Many brand names shouldn't be translated, and this test makes it easy to make sure words like: Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no-translate words using --notranslatefile. Brands stay branded, fr fr 💅
✨ numbers(str1, str2)
Checks whether numbers of various forms are consistent between the two strings.
You'll see some errors where you've either written the number out in full or converted it to the digit in your translation. Also, changes in order will trigger this error. Numbers don't play 🍓
✨ options(str1, str2)
Checks that command line options are not translated.
In messages that contain command line options, such as --help, this test checks that they remain untranslated. These could be translated in the future if programs can create a mechanism to allow it, but currently they are not. If the option has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated. Sneaky little distinction, ikyk 😌
✨ printf(str1, str2)
Checks whether printf format strings match.
If the printf formatting variables aren't identical, that's an error, periodt. Printf statements are literally what programs use to format output so it looks human-readable (they're placeholders for variable data, bestie). They let you specify lengths of string variables, string padding, number padding, precision, etc. Generally they look like this: %d, %5.2f, %100s, etc. The test can also manage variable-reordering using the %1$s syntax. The variables' type and details following data are tested to make sure they're strictly identical — but they may be reordered, so don't panic.
SEE ALSO: printf Format String
puncspacing(str1, str2)
Checks for bad spacing after punctuation. 🎀
If the original has [full-stop][space], this test checks that your translation didn't yeet the space away. It also checks for [comma], [colon], etc.
Some languages don't use spaces after common punctuation marks, especially where full-width punctuation marks are used — and this check highkey takes that into account, so it won't come for you unfairly. 💅
purepunc(str1, str2)
Checks that strings that are purely punctuation are not changed. ✨
This extracts strings like + or -, as these usually should not be changed. Iconic little guys. Leave them alone.
pythonbraceformat(str1, str2)
Checks whether python brace format strings match. 🐍
run_filters(unit, categorised=False)
Does some optimisation by caching some data of the unit for the benefit of run_test(). Ngl, it's the behind-the-scenes bestie doing the glow-up work.
run_test(test, unit)
Runs the given test on the given unit. 🏃♀️💨
Note that this can raise a FilterFailure as part of normal operation — so don't @ it, that's just how it vibes.
sentencecount(str1, str2)
Checks that the number of sentences in both strings match. 🔢
It adds up the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, tho — if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it's the meaning of the original you want to convey, not the exact way it was written in the English. fr fr. 🫶
setconfig(config)
Sets the accelerator list.
setsuggestionstore(store)
Sets the filename that a checker should use for evaluating suggestions. 🗂️
short(str1, str2)
Checks whether a translation is much shorter than the original string. 😭
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. (Yes, "1 characters long" — we said what we said.)
simplecaps(str1, str2)
Checks the capitalisation of two strings isn't wildly different. 🔠
This will pick up many false positives, so don't be a slave to it. It's useful for identifying translations that don't start with a capital letter (upper-case letter) when they should, or those that do when they shouldn't. It will also highlight sentences that have extra capitals — depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case. Your call, bestie. 💅
simpleplurals(str1, str2)
Checks for English style plural(s) for you to review. 🍓
This test will extract any message that contains words with a final "(s)" in the source text. You can then inspect the message to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy — in this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives. 🙈
For languages with no plural forms (only one noun form), this test will simply test that nothing like "(s)" was used in the translation.
singlequoting(str1, str2)
Checks whether singlequoting is consistent between the two strings.
Same energy as doublequoting, but it checks for the ' character. Because this is used in contractions like it's and in possessive forms like user's, this test can output spurious errors if your language doesn't use such forms. If a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check. 💀
spellcheck(str1, str2)
Checks words that don't pass a spell check. 📝
This test will check for misspelled words in your translation. It first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors — e.g. program names, brand names, function names. No cap, that's a lifesaver.
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you have specified the --language option.
The pofilter error that is created lists the misspelled word, plus suggestions returned from the spell checker. That makes it easy for you to identify the word and select a replacement. 👑
startcaps(str1, str2)
Checks that the message starts with the correct capitalisation.
After stripping whitespace and common punctuation characters, it then checks to see that the first remaining character is correctly capitalised. So, if the sentence starts with an upper-case letter, and the translation does not — error, bestie, an error is produced. 🔥
This check is entirely disabled for many languages that don't make a distinction between upper and lower case. Contact us if this is not yet disabled for your language. 📩
startpunc(str1, str2)
Checks whether punctuation at the beginning of the strings match.
Operates as endpunc, but you will probably see fewer errors. ✨
startwhitespace(str1, str2)
Checks whether whitespace at the beginning of the strings matches.
As in endwhitespace, but you will see fewer errors.
tabs(str1, str2)
Checks whether tabs are consistent between the two strings.
Counts the number of \\t tab markers and reports an error if they differ. 🎯
unchanged(str1, str2)
Checks whether a translation is basically identical to the original string.
This checks to see if the translation isn't just a copy of the English original. Sometimes that's what you want, but other times you'll detect words that should have been translated. 👀
untranslated(str1, str2)
Checks whether a string has been translated at all.
This check is really only useful if you want to extract untranslated strings so that they can be translated independently of the main work. 🧚
urls(str1, str2)
Checks that URLs are not translated. 🌈
This checks only basic URLs (http, ftp, mailto etc.), not all URIs (e.g. afp, smb, file). Generally, you don't want to translate URLs — unless they're example URLs (http://your_server.com/filename.html). If the URL is for configuration information, then you need to query the developers about placing configuration information in PO files. It shouldn't really be there, unless it's very clearly marked: such information should go into a configuration file. Periodt. 💅
validchars(str1, str2)
Checks that only characters specified as valid appear in the translation. 🦋
Often during character conversion to and from UTF-8 you get some strange characters appearing in your translation. This test presents a simple way to try and identify such errors.
This test will only run if you specify the --validcharsfile command line option. This file contains all the characters that are valid in your language. You must use UTF-8 encoding for the characters in the file. ⭐
If the test finds any characters not in your valid characters file, then the test will print the character together with its Unicode value (e.g. 002B).
variables(str1, str2)
Checks whether variables of various forms are consistent between the two strings.
This checks to make sure that variables that appear in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice. It does not at the moment cope with variables that use the reordering syntax of Gettext PO files. 😭 (Work in progress, iykyk.)
xmltags(str1, str2)
Checks that XML/HTML tags have not been translated. 🏷️
This check counts how many tags are chilling in the source string and makes sure the translation has the exact same number. If the counts don't match, that's an L either way — a tag went missing, or the translator got a lil too creative and translated a tag when they absolutely should not have. Both = errors, no cap.
The check is chill about tags (or tag-lookalikes) that wrap the entire string, like <Error> — those get ignored. But it will give you false positives for stuff like An <Error> occurred, because bestie, in that case Error is literally meant to be translated. It also lets you translate the alt attribute in things like <img src="bob.png" alt="Image description">, or similar translatable attributes in OpenOffice.org help files. 💅
🎀 class translate.filters.checks.ReducedChecker(**kwargs)
accelerators(str1, str2)
Checks whether accelerators are consistent between the two strings.
This test is built different — it can handle the different flavours of accelerators used across projects, like Mozilla or KDE. It'll catch accelerators that went missing and ones that shouldn't even be there. 🕵️♀️
See accelerators on the localization guide for the full rundown on accelerators.
acronyms(str1, str2)
Checks that acronyms that appear are unchanged.
If an acronym shows up in the original, this test makes sure it also shows up in the translation. Whether to translate an acronym is a whole language decision, but honestly most languages just leave them alone. In that case this test is your bestie for hunting down translated acronyms and fixing them. 🎯
blank(str1, str2)
Checks whether a translation is totally blank.
This catches a translation that got accidentally "translated" into just spaces. Which is different from untranslated, aka completely empty. Why does it matter? Because if something is translated as " " it'll look translated to basically every tool out there. Sneaky. 🙈
brackets(str1, str2)
Checks that the number of brackets in both strings match.
If ([{ or }]) appear in the original, this checks the same number appear in the translation.
categories
Categories where each checking function falls into. Function names are used as keys, categories are the values.
property checker_name
Extract checker name, for example mozilla from MozillaChecker.
compendiumconflicts(str1, str2)
Checks for Gettext compendium conflicts (#-#-#-#-#).
When you use msgcat to create a PO compendium it'll drop #-#-#-#-# into entries that aren't consistent. If that compendium gets used later in a message merge, those conflicts will straight up appear in your translations. This test pulls them out fast so you can fix them. 🔥
credits(str1, str2)
Checks for messages containing translation credits instead of normal translations.
Some projects have a consistent way of crediting translators — like a unit or two where translators can drop their name and maybe their contact details. This test lets you find those units easily so you can check they're filled in correctly, and it also disables other tests that might get triggered by them by accident (urls, emails, etc.). 🫶
doublequoting(str1, str2)
Checks whether doublequoting is consistent between the two strings.
Checks on double quotes " to make sure you've got the same number in both the original and the translated string. And it's smart about it — several languages use different quoting characters, and this test takes that into account and checks those instead. ✨
doublespacing(str1, str2)
Checks for bad double-spaces by comparing to original.
This spots when you've got [space][space] and the original doesn't, or the original has it and your translation doesn't. Some of these are spurious, and how you fix them depends on your language's conventions.
doublewords(str1, str2)
Checks for repeated words in the translation.
Words you repeated get highlighted by this test, e.g. "the the", "a a". Usually these are just typos that need fixing. Some languages legit have valid repeated words in their structure — in that case, either ignore those instances or switch this test off. 💀
emails(str1, str2)
Checks that emails are not translated.
Bestie, generally you should not be translating email addresses. This check makes sure addresses like info@example.com haven't been translated. Obviously in some cases you should translate the address, but generally — no. 🙅♀️
endpunc(str1, str2)
Checks whether punctuation at the end of the strings match.
This makes sure the ending of your translation has the same punctuation as the original. E.g. if it ends in :[space] then so should yours. It's also lowkey iconic for making sure you've got real ellipses […] in all your translations instead of three sad separate full-stops. You might catch some errors that exist in the original — feel free to keep your translation and go notify the programmers. 🤝
In some languages, characters such as ? or ! are always preceded by a space, e.g. [space]? — do what your language customs dictate. Other false positives you'll notice are, for example, if through changes in word-order you add "), etc. at the end of the sentence. Do not change these: your language word-order takes precedence.
It must be noted that if you're tempted to leave out[full-stop]or[colon], or add a[full-stop]to a sentence, often these have been done for a reason — like a list where full stops make it look cluttered. So initially match them with the English, and make changes once the program is actually being used. ⭐
This check is aware of several language conventions for punctuation characters, like the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it's not there yet. 💜
endwhitespace(str1, str2)
Checks whether whitespace at the end of the strings matches.
Works the same as endpunc but it's only concerned with whitespace. This filter is especially useful for strings that will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. That whitespace is an inherent part of the string. This filter makes sure you don't miss those important-but-otherwise-invisible spaces! 👻
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space.
escapes(str1, str2)
Checks whether escaping is consistent between the two strings.
Checks escapes such as \\n \u0000 to make sure that if they exist in the original string, you also have them in the translation.
filepaths(str1, str2)
Checks that file paths have not been translated.
Checks that paths such as /home/user1 haven't been translated. Generally you do not translate a file path, unless it's being used as an example, e.g. your_user_name/path/to/filename.conf. 🗂️
filteraccelerators_by_list(str1, acceptlist=None)
Filter out accelerators from str1.
functions(str1, str2)
Checks that function names are not translated.
Checks that function names e.g. rgb() or getEntity.Name() are not translated.
get_ignored_filters()
Return checker's additional filters for current language.
getfilters(excludefilters=None, limitfilters=None)
Returns dictionary of available filters, including/excluding those in the given lists.
kdecomments(str1, str2)
Checks to ensure that no KDE style comments appear in the translation.
KDE style translator comments appear in PO files as _: comment\n. New translators often translate the comment. 😭 This test tries to identify instances where the comment has been translated.
long(str1, str2)
Checks whether a translation is much longer than the original string.
Most useful in that special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise, we use a general ratio that catches very big differences but is set conservatively to limit the number of false positives. 📏
musttranslatewords(str1, str2)
Checks that words configured as definitely translatable don't appear in the translation.
If for instance in your language you decide that you must translate OK, then this test will flag any occurrences of OK in the translation if it appeared in the source string. You must specify a file containing all of the must-translate words using --musttranslatefile. 📝
newlines(str1, str2)
Checks whether newlines are consistent between the two strings.
Counts the number of \n newlines (and variants such as \r\n) and reports an error if they differ. 💅 literally just counting line breaks, bestie, and yelling if the vibes don't match.
notranslatewords(str1, str2)
Checks that words configured as untranslatable actually show up in the translation too. 🫶
So many brand names should NOT be translated, and this test lets you easily make sure that words like: Word, Excel, Impress, Calc, etc. are not translated. You gotta specify a file containing all of the no-translate words using --notranslatefile. No file, no check — that's the deal. ✨
numbers(str1, str2)
Checks whether numbers of various forms are consistent between the two strings. 🔢
You will see some errors where you have either written the number in full or converted it to the digit in your translation. Also, changes in order will trigger this error. So don't be out here flipping 3 into "three" and thinking nobody noticed 😭
options(str1, str2)
Checks that command line options are not translated. 💅
In messages that contain command line options, such as --help, this test will check that these remain untranslated. These could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. If the option has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated. Iconic behaviour, honestly.
printf(str1, str2)
Checks whether printf format strings match. 🎀
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are placeholders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. The test can also manage variables-reordering using the %1$s syntax. The variables' type and details following data are tested to ensure that they are strictly identical, but they may be reordered. 👑
SEE ALSO:
printfFormat String
puncspacing(str1, str2)
Checks for bad spacing after punctuation. 🌸
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc.
Some languages don't use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account. Inclusive queen behaviour fr fr 💜
purepunc(str1, str2)
Checks that strings that are purely punctuation are not changed. ✨
This extracts strings like + or - as these usually should not be changed. Like bestie, leave the little guys alone, they're fine.
pythonbraceformat(str1, str2)
Checks whether python brace format strings match. 🐍💅 That's it, that's the whole vibe check.
run_filters(unit, categorised=False)
Do some optimisation by caching some data of the unit for the benefit of run_test(). 🧚 Caching!! We love efficiency!!
run_test(test, unit)
Runs the given test on the given unit. 🔥
Note that this can raise a FilterFailure as part of normal operation. That's not a bug, bestie, that's just how it's supposed to go down. Don't panic.
sentencecount(str1, str2)
Checks that the number of sentences in both strings match. 📝
Adds the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it's the meaning of the original you want to convey, not the exact way it was written in the English. 🫶 PERIODT.
setconfig(config)
Sets the accelerator list. ⌨️ short and sweet, we love a no-notes king.
setsuggestionstore(store)
Sets the filename that a checker should use for evaluating suggestions. 📁
short(str1, str2)
Checks whether a translation is much shorter than the original string. 😭
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. Lowkey respectful of your time, honestly.
simplecaps(str1, str2)
Checks the capitalisation of two strings isn't wildly different. 🔠
This will pick up many false positives, so don't be a slave to it. It is useful for identifying translations that don't start with a capital letter (upper-case letter) when they should, or those that do when they shouldn't. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case. 👑
simpleplurals(str1, str2)
Checks for English style plural(s) for you to review. 🌈
This test will extract any message that contains words with a final "(s)" in the source text. You can then inspect the message, to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy. In this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives. 🙈
For languages with no plural forms (only one noun form) this test will simply test that nothing like "(s)" was used in the translation.
singlequoting(str1, str2)
Checks whether singlequoting is consistent between the two strings. ' ✨
The same as doublequoting but checks for the ' character. Because this is used in contractions like it's and in possessive forms like user's, this test can output spurious errors if your language doesn't use such forms. If a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check. 💀
spellcheck(str1, str2)
Checks words that don't pass a spell check. 🔍
This test will check for misspelled words in your translation. The test first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors e.g. program names, brand names, function names. 🎀
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you have specified the --language option.
The pofilter error that is created, lists the misspelled word, plus suggestions returned from the spell checker. That makes it easy for you to identify the word and select a replacement. 🫶
startcaps(str1, str2)
Checks that the message starts with the correct capitalisation. ⭐
After stripping whitespace and common punctuation characters, it then checks to see that the first remaining character is correctly capitalised. So, if the sentence starts with an upper-case letter, and the translation does not, an error is produced.
This check is entirely disabled for many languages that don't make a distinction between upper and lower case. Contact us if this is not yet disabled for your language. 💜
startpunc(str1, str2)
Checks whether punctuation at the beginning of the strings match. 🌸
Operates as endpunc but you will probably see fewer errors. Ngl, beginning-of-string punctuation is a rare breed.
startwhitespace(str1, str2)
Checks whether whitespace at the beginning of the strings matches. 🦋
As in endwhitespace but you will see fewer errors.
tabs(str1, str2)
Checks whether tabs are consistent between the two strings. 📐
Counts the number of \t tab markers and reports an error if they differ. Not me counting tabs on a Friday night 💀
unchanged(str1, str2)
Checks whether a translation is basically identical to the original string. 👀
This checks to see if the translation isn't just a copy of the English original. Sometimes, this is what you want, but other times you will detect words that should have been translated. Bestie, did you actually translate it or did you just copy-paste?? 🫶
untranslated(str1, str2)
Checks whether a string has been translated at all. 🧚
This check is really only useful if you want to extract untranslated strings so that they can be translated independently of the main work. Main character energy for the leftover strings, honestly.
urls(str1, str2)
Checks that URLs are not translated. 🌐💅 Leave the links alone!! They're perfect!!
This check only catches basic URLs (http, ftp, mailto, etc.) — not all URIs (so like afp, smb, file can slip right past, bestie, don't @ me). Generally you don't want to translate URLs at all, unless they're example URLs (http://your_server.com/filename.html). If the URL is hanging out there as configuration info, then you gotta go query the developers about putting configuration info in PO files — it really shouldn't be there, unless it's very clearly marked, because that kind of info should live in a configuration file. ✨
validchars(str1, str2)
Checks that only characters specified as valid appear in the translation. 💅
Okay so real talk: when you're doing character conversion to and from UTF-8, you lowkey get the strangest characters just showing up in your translation. This test gives you a simple way to try and spot those errors.
It only runs if you specify the --validcharsfile command line option, so don't come crying when nothing happens. This file contains all the characters that are valid in your language, and you must use UTF-8 encoding for the characters in the file.
If the test finds any characters that aren't in your valid characters file, it prints the character together with its Unicode value (e.g. 002B). 🦋
variables(str1, str2)
Checks whether variables of various forms are consistent between the two strings.
This one makes sure that variables appearing in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice. It does not, at the moment, cope with variables that use the reordering syntax of Gettext PO files — so yeah, know the limit, babe. 🙈
xmltags(str1, str2)
Checks that XML/HTML tags have not been translated. 🔥
This check counts the number of tags in the source string and verifies the same number are in the translation. If the counts don't match, then either a tag is missing or the translator mistakenly translated it — both of which are errors, periodt.
The check ignores tags, or things that look like tags, that cover the whole string, e.g. <Error>. But it will produce false positives for things like An <Error> occurred, because here Error should actually be translated. It also allows translation of the alt attribute in e.g. <img src="bob.png" alt="Image description">, or similar translatable attributes in OpenOffice.org help files. 💜
exception translate.filters.checks.SeriousFilterFailure(messages)
This exception signals that a Filter didn't pass, and the bad translation might break an application (so the string gets marked fuzzy). 😭
add_note(object, /)
Exception.add_note(note) – add a note to the exception
with_traceback(object, /)
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self. 🫶
class translate.filters.checks.StandardChecker(checkerconfig=None, excludefilters=None, limitfilters=None, errorhandler=None)
The basic test suite for source -> target translations. This is the main character, fr fr. 👑
accelerators(str1, str2)
Checks whether accelerators are consistent between the two strings.
This test can check the different types of accelerators used in different projects, like Mozilla or KDE. It picks up accelerators that are missing and ones that shouldn't be there.
See accelerators on the localization guide for a full description on accelerators. ⭐
acronyms(str1, str2)
Checks that acronyms that appear are unchanged.
If an acronym appears in the original, this test checks that it appears in the translation. Translating acronyms is a language decision, but many languages leave them unchanged — in that case this test is super useful for tracking down translations of the acronym and correcting them. 🎀
blank(str1, str2)
Checks whether a translation is totally blank.
This will check to see if a translation has inadvertently been translated as blank, i.e. as spaces. That's different from untranslated, which is completely empty. This test is useful because if something is translated as " " it'll appear to most tools as if it is translated. 💀
brackets(str1, str2)
Checks that the number of brackets in both strings match.
If ([{ or }]) appear in the original, this checks that the same number appear in the translation. 🍓
categories
Categories where each checking function falls into. Function names are used as keys, categories are the values. 🌸
property checker_name
Extract checker name, for example 'mozilla' from MozillaChecker.
compendiumconflicts(str1, str2)
Checks for Gettext compendium conflicts (#-#-#-#-#).
When you use msgcat to create a PO compendium it will insert #-#-#-#-# into entries that are not consistent. If the compendium is used later in a message merge, then these conflicts will show up in your translations. This test quickly extracts those for correction. 🔥
credits(str1, str2)
Checks for messages containing translation credits instead of normal translations.
Some projects have consistent ways of giving credit to translators by having a unit or two where translators can fill in their name and possibly their contact details. This test lets you find these units easily to check they're completed correctly — and it also disables other tests that might incorrectly get triggered for these units (such as urls, emails, etc.). 💖
doublequoting(str1, str2)
Checks whether doublequoting is consistent between the two strings.
It checks on double quotes " to make sure you have the same number in both the original and the translated string. This test takes into account that several languages use different quoting characters, and will test for those instead. 🌈
doublespacing(str1, str2)
Checks for bad double-spaces by comparing to original.
This identifies if you have [space][space] in when you don't have it in the original, or it appears in the original but not in your translation. Some of these are spurious, and how you correct them depends on the conventions of your language. 🧚
doublewords(str1, str2)
Checks for repeated words in the translation.
Words that have been repeated in a translation get highlighted with this test, e.g. "the the", "a a". These are generally typos that need correcting. Some languages may have valid repeated words in their structure — in that case, either ignore those instances or switch this test off. 💅
emails(str1, str2)
Checks that emails are not translated.
Generally you should not be translating email addresses. This check looks to see that email addresses, e.g. info@example.com, are not translated. In some cases of course you should translate the address, but generally you shouldn't. 😭
endpunc(str1, str2)
Checks whether punctuation at the end of the strings match.
This ensures that the ending of your translation has the same punctuation as the original. E.g. if it ends in :[space] then so should yours. It's useful for making sure you have ellipses […] in all your translations, not simply three separate full-stops. 👑
You may pick up some errors in the original — feel free to keep your translation and notify the programmers. In some languages, characters such as ? or ! are always preceded by a space, e.g. [space]? — do what your language customs dictate. Other false positives you'll notice are, for example, if through changes in word-order you add "), etc. at the end of the sentence. Do not change these — your language word-order takes precedence.
It must be noted that if you're tempted to leave out [full-stop] or [colon], or add [full-stop] to a sentence, that often these have been done for a reason, e.g. a list where fullstops make it look cluttered. So initially match them with the English, and make changes once the program is being used. ⭐
This check is aware of several language conventions for punctuation characters, such as the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. Support for your language can be added easily if it's not there yet. 💜
endwhitespace(str1, str2)
Checks whether whitespace at the end of the strings matches.
Operates the same as endpunc but is only concerned with whitespace. This filter is particularly useful for those strings which will evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. The whitespace is an inherent part of the string, so this filter makes sure you don't miss those important but otherwise invisible spaces! 🙈
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without an added extra space. 🌸
escapes(str1, str2)
Checks whether escaping is consistent between the two strings.
It's looking at escapes like \n and \u0000 and making sure that if they show up in the original string, you also have them in the translation. 💅
filepaths(str1, str2)
Checks that file paths have not been translated.
Like, paths such as /home/user1 should stay exactly as they are, bestie. Generally you do not translate a file path, unless it's being used as an example, e.g. your_user_name/path/to/filename.conf. 🫶
filteraccelerators_by_list(str1, acceptlist=None)
Filter out accelerators from str1.
functions(str1, str2)
Checks that function names are not translated.
So function names like rgb() or getEntity.Name() — those stay put, no cap. ✨
get_ignored_filters()
Return checker's additional filters for current language.
getfilters(excludefilters=None, limitfilters=None)
Returns dictionary of available filters, including/excluding those in the given lists.
kdecomments(str1, str2)
Checks to ensure that no KDE style comments appear in the translation.
KDE style translator comments appear in PO files as:
_:
comment\nNew translators often translate the comment 😭. This test tries to identify instances where the comment has been translated.
long(str1, str2)
Checks whether a translation is much longer than the original string.
This is most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. 🌸
musttranslatewords(str1, str2)
Checks that words configured as definitely translatable don't appear in the translation.
If for instance in your language you decide that you must translate 'OK' then this test will flag any occurrences of 'OK' in the translation if it appeared in the source string. You must specify a file containing all of the must translate words using --musttranslatefile. 💖
newlines(str1, str2)
Checks whether newlines are consistent between the two strings.
Counts the number of \n newlines (and variants such as \r\n) and reports and error if they differ.
notranslatewords(str1, str2)
Checks that words configured as untranslatable appear in the translation too.
Many brand names should not be translated, and this test lets you easily make sure that words like: Word, Excel, Impress, Calc, etc. are not translated. You must specify a file containing all of the no translate words using --notranslatefile. 👑
numbers(str1, str2)
Checks whether numbers of various forms are consistent between the two strings.
You will see some errors where you have either written the number in full or converted it to the digit in your translation. Also changes in order will trigger this error. 😭
options(str1, str2)
Checks that command line options are not translated.
In messages that contain command line options, such as --help, this test will check that these remain untranslated. These could be translated in the future if programs can create a mechanism to allow this, but currently they are not translated. If the options has a parameter, e.g. --file=FILE, then the test will check that the parameter has been translated. 🦋
printf(str1, str2)
Checks whether printf format strings match.
If the printf formatting variables are not identical, then this will indicate an error. Printf statements are used by programs to format output in a human readable form (they are placeholders for variable data). They allow you to specify lengths of string variables, string padding, number padding, precision, etc. Generally they will look like this: %d, %5.2f, %100s, etc. The test can also manage variables-reordering using the %1$s syntax. The variables' type and details following data are tested to ensure that they are strictly identical, but they may be reordered. 🔥
SEE ALSO:
printf Format String
puncspacing(str1, str2)
Checks for bad spacing after punctuation.
In the case of [full-stop][space] in the original, this test checks that your translation does not remove the space. It checks also for [comma], [colon], etc.
Some languages don't use spaces after common punctuation marks, especially where full-width punctuation marks are used. This check will take that into account. ✨
purepunc(str1, str2)
Checks that strings that are purely punctuation are not changed.
This extracts strings like + or - as these usually should not be changed.
pythonbraceformat(str1, str2)
Checks whether python brace format strings match.
run_filters(unit, categorised=False)
Do some optimisation by caching some data of the unit for the benefit of run_test().
run_test(test, unit)
Runs the given test on the given unit.
Note that this can raise a FilterFailure as part of normal operation. 💀
sentencecount(str1, str2)
Checks that the number of sentences in both strings match.
Adds the number of sentences to see that the sentence count is the same between the original and translated string. You may not always want to use this test, if you find you often need to reformat your translation, because the original is badly-expressed, or because the structure of your language works better that way. Do what works best for your language: it's the meaning of the original you want to convey, not the exact way it was written in the English. 💜
setconfig(config)
Sets the accelerator list.
setsuggestionstore(store)
Sets the filename that a checker should use for evaluating suggestions.
short(str1, str2)
Checks whether a translation is much shorter than the original string.
This is most useful in the special case where the translation is 1 characters long while the source text is multiple characters long. Otherwise, we use a general ratio that will catch very big differences but is set conservatively to limit the number of false positives. 🎀
simplecaps(str1, str2)
Checks the capitalisation of two strings isn't wildly different.
This will pick up many false positives, so don't be a slave to it 🙈. It is useful for identifying translations that don't start with a capital letter (upper-case letter) when they should, or those that do when they shouldn't. It will also highlight sentences that have extra capitals; depending on the capitalisation convention of your language, you might want to change these to Title Case, or change them all to normal sentence case.
simpleplurals(str1, str2)
Checks for English style plural(s) for you to review.
This test will extract any message that contains words with a final "(s)" in the source text. You can then inspect the message, to check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, making the English style messy. In this case, they often revert to the plural form. This test allows an editor to check that the plurals used are correct. Be aware that this test may create a number of false positives. 😭
For languages with no plural forms (only one noun form) this test will simply test that nothing like "(s)" was used in the translation. 🌈
singlequoting(str1, str2)
Checks whether singlequoting is consistent between the two strings.
The same as doublequoting but checks for the ' character. Because this is used in contractions like it's and in possessive forms like user's, this test can output spurious errors if your language doesn't use such forms. If a quote appears at the end of a sentence in the translation, i.e. '., this might not be detected properly by the check. 💅
spellcheck(str1, str2)
Checks words that don't pass a spell check.
This test will check for misspelled words in your translation. The test first checks for misspelled words in the original (usually English) text, and adds those to an exclusion list. The advantage of this exclusion is that many words that are specific to the application will not raise errors e.g. program names, brand names, function names. 🧚
The checker works with PyEnchant. You need to have PyEnchant installed as well as a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you have specified the --language option. ⭐
The pofilter error that gets created is honestly so useful: it lists the misspelled word plus the suggestions from the spell checker, so you can spot exactly which word is acting up and pick a replacement. Slay. 💅
startcaps(str1, str2)— Checks that the message starts with the correct capitalisation. After stripping whitespace and common punctuation characters, it checks that the first remaining character is correctly capitalised. So if the sentence starts with an upper-case letter in the source and the translation does not, boom: error. This check is entirely disabled for many languages that don’t make a distinction between upper and lower case. Contact us if this is not yet disabled for your language.startpunc(str1, str2)— Checks whether punctuation at the beginning of the strings matches. Operates likeendpunc, but you’ll probably see fewer errors. ✨startwhitespace(str1, str2)— Checks whether whitespace at the beginning of the strings matches. Same idea asendwhitespace, but you’ll see fewer errors.tabs(str1, str2)— Checks whether tabs are consistent between the two strings. Counts the number of\\ttab markers and reports an error if they differ.unchanged(str1, str2)— Checks whether a translation is basically identical to the original string. This is here to catch when the translation isn’t just a copy of the English original. Sometimes that copy-paste is exactly what you want, but other times you’ll detect words that should have been translated. 👀untranslated(str1, str2)— Checks whether a string has been translated at all. This check is really only useful if you want to extract untranslated strings so they can be translated independently of the main work.urls(str1, str2)— Checks that URLs are not translated. It checks only basic URLs (http,ftp,mailtoetc.), not all URIs (e.g.afp,smb,file). Generally you don’t want to translate URLs, unless they are example URLs (http://your_server.com/filename.html). If the URL is for configuration information, then you need to query the developers about placing configuration information in PO files. It shouldn’t really be there, unless it is very clearly marked: such information should go into a configuration file.validchars(str1, str2)— Checks that only characters specified as valid appear in the translation. Often during character conversion to and from UTF-8 you get some strange characters appearing in your translation. This test gives you a simple way to try to identify such errors. It only runs if you specify the--validcharsfilecommand line option. This file contains all the characters that are valid in your language. You must use UTF-8 encoding for the characters in the file. If the test finds any characters not in your valid characters file, then it will print the character together with its Unicode value (e.g.002B).variables(str1, str2)— Checks whether variables of various forms are consistent between the two strings. It makes sure variables that appear in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice. It does not at the moment cope with variables that use the reordering syntax of Gettext PO files.xmltags(str1, str2)— Checks that XML/HTML tags have not been translated. This check finds the number of tags in the source string and checks that the same number are in the translation. If the counts don’t match, then either the tag is missing or it was mistakenly translated by the translator, and both of those are errors. The check ignores tags or things that look like tags that cover the whole string, e.g.<Error>, but will produce false positives for things likeAn <Error> occurred, as hereErrorshould be translated. It also will allow translation of thealtattribute in e.g.<img src="bob.png" alt="Image description">or similar translatable attributes in OpenOffice.org help files. 🫶
class translate.filters.checks.StandardUnitChecker(checkerconfig=None, excludefilters=None, limitfilters=None, errorhandler=None)— The standard checks for common checks on translation units.categories— Categories where each checking function falls into. Function names are used as keys, categories are the values.property checker_name— Extract checker name, for example'mozilla'fromMozillaChecker.filteraccelerators_by_list(str1, acceptlist=None)— Filter out accelerators fromstr1.get_ignored_filters()— Return checker’s additional filters for current language.getfilters(excludefilters=None, limitfilters=None)— Returns a dictionary of available filters, including/excluding those in the given lists.hassuggestion(unit)— Checks if there is at least one suggested translation for this unit. If a message has a suggestion (an alternate translation stored inalt-transunits in XLIFF and.pendingfiles in PO), then these will be extracted. This is used by Pootle and is probably only useful inpofilterwhen using XLIFF files.isfuzzy(unit)— Check if the unit has been marked fuzzy. If a message is marked fuzzy in the PO file, then it is extracted. Note this is different from--fuzzyand--nofuzzyoptions which specify whether tests should be performed against messages marked fuzzy.isreview(unit)— Check if the unit has been marked review. If you have made use of the ‘review’ flags in your translations:
# (review) reason for review
# (pofilter) testname: explanation for translatorThen if a message is marked for review in the PO file, it will be extracted. Note this is different from --review and --noreview options which specify whether tests should be performed against messages already marked as under review.
nplurals(unit)— Checks for the correct number of noun forms for plural translations. This uses the plural information in the language module of the Translate Toolkit. This is the same as the Gettextnpluralvalue. It will check that the number of plurals required is the same as the number supplied in your translation.run_filters(unit, categorised=False)— Run all the tests in this suite.
Return type Dictionary
Returns Content of the dictionary is as follows:
{'testname': { 'message': message_or_exception, 'category': failure_category } }static run_test(test, unit)— Runs the given test on the given unit. Note that this can raise aFilterFailureas part of normal operation.setconfig(config)— Sets the accelerator list.setsuggestionstore(store)— Sets the filename that a checker should use for evaluating suggestions.
class translate.filters.checks.TeeChecker(checkerconfig=None, excludefilters=None, limitfilters=None, checkerclasses=None, errorhandler=None, languagecode=None)— A Checker that controls multiple checkers.categories = {}— Categories where each checking function falls into. Function names are used as keys, categories are the values.getfilters(excludefilters=None, limitfilters=None)— Returns a dictionary of available filters, including/excluding those in the given lists.run_filters(unit, categorised=False)— Run all the tests in the checker’s suites.setsuggestionstore(store)— Sets the filename that a checker should use for evaluating suggestions.
class translate.filters.checks.TermChecker(**kwargs)— A term-checking bestie, no cap.accelerators(str1, str2)— Checks whether accelerators are consistent between the two strings. This test is capable of checking the different types of accelerators that are used in different projects, like Mozilla or KDE. The test will pick up accelerators that are missing and ones that shouldn’t be there. Seeacceleratorson the localization guide for a full description on accelerators.acronyms(str1, str2)— Checks that acronyms that appear are unchanged. If an acronym appears in the original, this test will check that it appears in the translation. Translating acronyms is a language decision, but many languages leave them unchanged. In that case this test is useful for tracking down translations of the acronym and correcting them. 🎀blank(str1, str2)— Checks whether a translation is totally blank. 💀
This one checks if a translation has lowkey been turned into blanks — like, literally nothing but spaces. That's a whole different beast from untranslated, which is completely empty. Why do we care? Because if something's been "translated" as " ", it'll fool basically every tool out there into thinking the job's done. 💀
brackets(str1, str2)
Makes sure the bracket count in both strings matches. If ([{ or }]) show up in the original, this checks the same number of them show up in your translation. No sneaky missing ) on your watch, bestie. 🎀
categories
The categories that each checking function falls into — function names are the keys, categories are the values.
property checker_name
Yanks out the checker name, like 'mozilla' out of MozillaChecker.
compendiumconflicts(str1, str2)
Sniffs out Gettext compendium conflicts (#-#-#-#-#). When you use msgcat to build a PO compendium, it'll drop #-#-#-#-# into any entries that don't agree with each other. And if that compendium gets used later in a message merge, those conflicts just, like, manifest in your translations. This test pulls them out fast so you can fix them. ✨
credits(str1, str2)
Catches messages that are translation credits instead of actual translations. Some projects have a consistent way of shouting out their translators — a unit or two where you fill in your name and maybe your contact deets. This test lets you find those units easily and check they're filled in right, and it also switches off the other tests that would wrongly fire on them (like urls, emails, etc.). so thoughtful tbh 🫶
doublequoting(str1, str2)
Checks whether doublequoting is consistent between the two strings. It's looking at double quotes " to make sure you've got the same number in the original and in the translated string. And it's not clueless — it knows several languages use different quoting characters, so it'll test for those instead. 😌
doublespacing(str1, str2)
Flags bad double-spaces by comparing against the original. Basically it'll catch if you've got [space][space] when the original doesn't, or the original has it and your translation ghosts it. Some of these are spurious, and how you fix them is entirely up to your language's conventions. 🦋
doublewords(str1, str2)
Looks for repeated words in the translation. Stuff like "the the", "a a" gets highlighted by this test. Usually these are just typos that need fixing. Some languages do legitimately repeat words in their structure though — in that case, ignore those instances or just switch this test off. no cap 💅
emails(str1, str2)
Makes sure emails aren't translated. Generally you should not be translating email addresses — this check looks to see that addresses like info@example.com haven't been translated. There are of course some cases where you should translate the address, but generally you shouldn't.
endpunc(str1, str2)
Checks whether the punctuation at the end of the strings matches. This makes sure the ending of your translation has the same punctuation as the original. E.g. if it ends in :[space] then so should yours. It's genuinely useful for making sure you've got ellipses […] in all your translations, not just three sad separate full-stops. 🌸
You might also pick up some errors in the original — feel free to keep your translation and notify the programmers. In some languages, characters like ? or ! are always preceded by a space, e.g. [space]? — do what your language customs dictate. Other false positives you'll notice are, for example, if through changes in word-order you add "), etc. at the end of the sentence. Do not change these: your language word-order takes precedence.
Also, it must be noted: if you're tempted to leave out [full-stop] or [colon], or add [full-stop] to a sentence, these were often done for a reason, e.g. a list where fullstops make it look cluttered. So at first match them with the English, and make changes once the program is actually being used. ⭐
This check is aware of several language conventions for punctuation characters — like the custom question marks for Greek and Arabic, Devanagari Danda, full-width punctuation for CJK languages, etc. And support for your language can be added easily if it's not there yet. iconic tbh 👑
endwhitespace(str1, str2)
Checks whether the whitespace at the end of the strings matches. Works the same as endpunc but it's only concerned with whitespace. This filter is especially useful for strings that'll evidently be followed by another string in the program, e.g. [Password: ] or [Enter your username: ]. That whitespace is an inherent part of the string, so this filter makes sure you don't miss those important-but-otherwise-invisible spaces! 😭
If your language uses full-width punctuation (like Chinese), the visual spacing in the character might be enough without adding an extra space. 💜
escapes(str1, str2)
Checks whether escaping is consistent between the two strings. It looks at escapes such as \\n and \u0000 to make sure that if they exist in the original string, you've got them in the translation too.
filepaths(str1, str2)
Checks that file paths haven't been translated. Like, paths such as /home/user1 should stay exactly as they are. Generally you don't translate a file path — unless it's being used as an example, e.g. your_user_name/path/to/filename.conf. 🦋
filteraccelerators_by_list(str1, acceptlist=None)
Filters out accelerators from str1.
functions(str1, str2)
Checks that function names aren't translated. So function names like rgb() or getEntity.Name() stay pristine. periodt.
get_ignored_filters()
Returns the checker's additional filters for the current language.
getfilters(excludefilters=None, limitfilters=None)
Returns a dictionary of available filters, including/excluding the ones in the lists you give it.
kdecomments(str1, str2)
Checks that no KDE style comments appear in the translation. KDE style translator comments appear in PO files as _: followed by a newline, then comment\n. New translators often translate the comment (bless them), so this test tries to identify the instances where that's happened. 🙈
long(str1, str2)
Checks whether a translation is way longer than the original string. It's most useful in the special case where the translation is multiple characters long while the source text is only 1 character long. Otherwise it uses a general ratio that catches very big differences but is set conservatively to keep the false positives down. 🔥
musttranslatewords(str1, str2)
Checks that words configured as definitely translatable don't appear in the translation. So if, for instance, in your language you decide you must translate 'OK', then this test flags any occurrences of 'OK' in the translation when it showed up in the source string. You have to specify a file containing all of the must-translate words using --musttranslatefile. 📋
newlines(str1, str2)
Checks whether newlines are consistent between the two strings. It counts the number of \n newlines (and variants such as \r\n) and reports an error if they differ.
notranslatewords(str1, str2)
Checks that words configured as untranslatable do appear in the translation too. Loads of brand names shouldn't be translated, and this test lets you easily make sure words like Word, Excel, Impress, Calc, etc. aren't translated. You have to specify a file containing all of the no-translate words using --notranslatefile. 👑
numbers(str1, str2)
Checks whether numbers of various forms are consistent between the two strings. You'll see errors where you've either written the number out in full or converted it to a digit in your translation. Changes in order will trigger this error too. 🔢
options(str1, str2)
Checks that command line options aren't translated. In messages that contain command line options, such as --help, this test makes sure they stay untranslated. Sure, these could be translated in the future if programs build a mechanism to allow it, but currently they aren't translated. If the option has a parameter, e.g. --file=FILE, then the test checks that the parameter has been translated. 💅
printf(str1, str2)
Checks whether printf format strings match.
...and if the printf formatting variables aren't identical, bestie, that's an error, periodt. Printf statements are what programs use to format output in a human-readable way — they're basically placeholders for variable data. They let you specify string lengths, string padding, number padding, precision, all that good stuff. They generally look like this: %d, %5.2f, %100s, etc. The test can also handle variable reordering using the %1$s syntax. The variables' type and the details following the data get tested to make sure they're strictly identical — but they may be reordered, so don't panic about that part. 💅
SEE ALSO: printf Format String
puncspacing(str1, str2)
Checks for bad spacing after punctuation.
So like, if the original has [full-stop][space], this test makes sure your translation didn't yeet the space away. It also checks [comma], [colon], etc. Some languages literally don't use spaces after common punctuation marks — especially where full-width punctuation marks are used — and this check takes that into account. 🫶
purepunc(str1, str2)
Checks that strings which are purely punctuation don't get changed.
It extracts strings like + or -, because bestie, those usually should not be touched. 🙈
pythonbraceformat(str1, str2)
Checks whether python brace format strings match. Simple as that, no notes. ✨
run_filters(unit, categorised=False)
Does some optimisation by caching a bunch of the unit's data for the benefit of run_test(). Very giving, very helpful. 💖
run_test(test, unit)
Runs the given test on the given unit.
Heads up: this can raise a FilterFailure as part of normal operation. Like, that's not a bug, that's just how it goes, ngl.
sentencecount(str1, str2)
Checks that the number of sentences in both strings match.
It adds up the number of sentences to see that the count is the same between the original and the translated string. Okay so you may not always want to use this test — like if you find you often need to reformat your translation because the original is badly expressed, or because the structure of your language works better that way. Do what works best for your language: it's the meaning of the original you want to convey, not the exact way it was written in the English. Periodt. 👑
setconfig(config)
Sets the accelerator list. Short and sweet. 🎀
setsuggestionstore(store)
Sets the filename that a checker should use for evaluating suggestions.
short(str1, str2)
Checks whether a translation is much shorter than the original string.
This one's most useful in that special case where the translation is 1 character long while the source text is multiple characters long. Otherwise we use a general ratio that catches very big differences but is set conservatively to limit the number of false positives. 🤏
simplecaps(str1, str2)
Checks that the capitalisation of two strings isn't wildly different.
Fair warning: this will pick up many false positives, so don't be a slave to it, bestie. It's useful for spotting translations that don't start with a capital letter (upper-case letter) when they should, or those that do when they shouldn't. It'll also highlight sentences that have extra capitals — and depending on your language's capitalisation convention, you might want to change those to Title Case, or change them all to normal sentence case. Your call. 💅
simpleplurals(str1, str2)
Checks for English-style plural(s) for you to review.
This test extracts any message containing words with a final "(s)" in the source text. Then you can inspect the message and check that the correct plural form has been used for your language. In some languages, plurals are made by adding text at the beginning of words, which makes the English style messy — in that case people often just revert to the plural form. This test lets an editor check that the plurals used are correct. Just be aware it may create a number of false positives. 😭
For languages with no plural forms (only one noun form), this test will simply test that nothing like "(s)" was used in the translation.
singlequoting(str1, str2)
Checks whether singlequoting is consistent between the two strings.
Same as doublequoting but it checks for the ' character. Because that character is used in contractions like it's and in possessive forms like user's, this test can output spurious errors if your language doesn't use such forms. Also, if a quote appears at the end of a sentence in the translation — i.e. '. — this might not get detected properly by the check. 😬
spellcheck(str1, str2)
Checks words that don't pass a spell check.
This test checks for misspelled words in your translation. It first checks for misspelled words in the original (usually English) text and adds those to an exclusion list — which is iconic, because it means loads of words specific to the application won't raise errors, e.g. program names, brand names, function names. 🧚
The checker works with PyEnchant. You need PyEnchant installed plus a dictionary for your language (for example, one of the Hunspell or aspell dictionaries). This test will only work if you've specified the --language option.
The pofilter error that's created lists the misspelled word, plus the suggestions returned from the spell checker. That makes it super easy for you to identify the word and pick a replacement. ✨
startcaps(str1, str2)
Checks that the message starts with the correct capitalisation.
After stripping whitespace and common punctuation characters, it checks that the first remaining character is correctly capitalised. So if the sentence starts with an upper-case letter and the translation doesn't, you get an error. 🔥
This check is entirely disabled for many languages that don't make a distinction between upper and lower case. Contact us if it's not yet disabled for your language — we got you.
startpunc(str1, str2)
Checks whether punctuation at the beginning of the strings matches.
Operates as endpunc, but you'll probably see fewer errors. 💜
startwhitespace(str1, str2)
Checks whether whitespace at the beginning of the strings matches.
As in endwhitespace, but you'll see fewer errors.
tabs(str1, str2)
Checks whether tabs are consistent between the two strings.
It counts the number of \t tab markers and reports an error if they differ. Counting is hard, apparently. 😤
unchanged(str1, str2)
Checks whether a translation is basically identical to the original string.
This checks whether the translation isn't just a copy of the English original. Sometimes that's exactly what you want — but other times you'll catch words that should have been translated. 🕵️
untranslated(str1, str2)
Checks whether a string has been translated at all.
Honestly this check is really only useful if you want to extract untranslated strings so they can be translated independently of the main work.
urls(str1, str2)
Checks that URLs are not translated.
This only checks basic URLs (http, ftp, mailto etc.) — not all URIs (e.g. afp, smb, file). Generally you don't want to translate URLs, unless they're example URLs (http://your_server.com/filename.html). And if the URL is there for configuration information, then you need to query the developers about putting configuration information in PO files — it shouldn't really be there, unless it's very clearly marked: such info should go into a configuration file. 🚩
validchars(str1, str2)
Checks that only characters specified as valid appear in the translation.
Okay so often, during character conversion to and from UTF-8, you get some strange characters appearing in your translation. This test is a simple way to try and identify those errors.
This test will only run if you specify the --validcharsfile command line option. That file contains all the characters that are valid in your language, and you must use UTF-8 encoding for the characters in the file. 📝
If the test finds any characters not in your valid characters file, it'll print the character together with its Unicode value (e.g. 002B).
variables(str1, str2)
Checks whether variables of various forms are consistent between the two strings.
This makes sure that variables appearing in the original also appear in the translation. It can handle variables from projects like KDE or OpenOffice. It does not at the moment cope with variables that use the reordering syntax of Gettext PO files. 🤷♀️
xmltags(str1, str2)
Checks that XML/HTML tags have not been translated. Tags are not content, bestie — leave them alone! 🎀
So yeah — this check counts the tags in the source string and makes sure the translation has the same number. If the counts don't match, something's off: either a tag went missing, or the translator went and translated a tag (which, bestie, is an error both ways 👀).
The check ignores tags — and tag-looking things — that swallow the whole string, like <Error>. But ngl it will give you false positives for stuff like An <Error> occurred, because in that case Error actually should be translated. It also lets you translate the alt attribute in things like <img src="bob.png" alt="Image description">, or similar translatable attributes in OpenOffice.org help files. Slay, very accommodating.
class translate.filters.checks.TranslationChecker(checkerconfig=None,
excludefilters=None, limitfilters=None, errorhandler=None)A checker that hands the source and target strings to the checks instead of the whole unit. It's giving speed — it's a little faster and it makes testing way simpler ✨
categories— the categories each checking function falls into. Function names are the keys, categories are the values.checker_name(property) — pulls out the checker name, like'mozilla'fromMozillaChecker.filteraccelerators_by_list(str1, acceptlist=None)— filters the accelerators out ofstr1.get_ignored_filters()— returns the checker's extra filters for the current language.getfilters(excludefilters=None, limitfilters=None)— returns a dictionary of the available filters, including/excluding whatever's in the lists you pass.run_filters(unit, categorised=False)— does some optimisation by caching bits of the unit forrun_test()'s benefit.run_test(test, unit)— runs the given test on the given unit. Heads up: this can raise aFilterFailureas part of normal operation, so don't panic 😭setconfig(config)— sets the accelerator list.setsuggestionstore(store)— sets the filename a checker should use for evaluating suggestions.
class translate.filters.checks.UnitChecker(checkerconfig=None,
excludefilters=None, limitfilters=None, errorhandler=None)The parent Checker class — it does the checking using whatever functions the derived classes make available 🫶
categories— the categories each checking function falls into. Function names are the keys, categories are the values.checker_name(property) — pulls out the checker name, like'mozilla'fromMozillaChecker.filteraccelerators_by_list(str1, acceptlist=None)— filters the accelerators out ofstr1.get_ignored_filters()— returns the checker's extra filters for the current language.getfilters(excludefilters=None, limitfilters=None)— returns a dictionary of the available filters, including/excluding those in the given lists.run_filters(unit, categorised=False)— runs all the tests in this suite.
Return type: Dictionary
Returns: the dictionary is shaped like this —
{'testname': { 'message': message_or_exception, 'category': failure_category } }run_test(test, unit)(static) — runs the given test on the given unit. Again, it can raise aFilterFailureas part of normal operation.setconfig(config)— sets the accelerator list.setsuggestionstore(store)— sets the filename a checker should use for evaluating suggestions.
translate.filters.checks.batchruntests(pairs)Runs tests on a whole batch of string pairs 🔥
translate.filters.checks.intuplelist(pair, list)Tests whether pair == (a,b,c) is in list, but it treats None entries in list as wildcards — and those wildcards are only allowed in positions "a" and "c". Shortcut behaviour: it only bothers looking at "c" if "b" has already matched. Efficient queen 💅
translate.filters.checks.runtests(str1, str2, ignorelist=())Verifies that the tests pass for a pair of strings.
translate.filters.checks.tagname(string)Returns the name of the XML/HTML tag in string.
translate.filters.checks.tagproperties(strings, ignore)Returns all the properties in the XML/HTML tag string as (tagname, propertyname, propertyvalue) — but it ignores whichever combinations you list in ignore.
🎀 decoration 🎀
functions to get decorative/informative text out of strings… (their words, and honestly? iconic.)
translate.filters.decoration.countaccelerators(accelmarker,
acceptlist=None)returns a function that counts the number of accelerators marked with the given marker.
translate.filters.decoration.findaccelerators(str1, accelmarker,
acceptlist=None)returns all the accelerators and their locations in str1 marked with a given marker.
translate.filters.decoration.findmarkedvariables(str1, startmarker,
endmarker, ignorelist=[])returns all the variables and their locations in str1 marked with a given marker.
translate.filters.decoration.getaccelerators(accelmarker,
acceptlist=None)returns a function that gets a list of accelerators marked using accelmarker.
translate.filters.decoration.getemails(str1)Returns the email addresses that are in a string 📧
translate.filters.decoration.getfunctions(str1)returns the functions() that are in a string, while ignoring the trailing punctuation in the given parameter.
translate.filters.decoration.getnumbers(str1)Returns any numbers that are in the string 🔢
translate.filters.decoration.geturls(str1)Returns the URIs in a string.
translate.filters.decoration.getvariables(startmarker, endmarker)returns a function that gets a list of variables marked using startmarker and endmarker.
translate.filters.decoration.ispurepunctuation(str1)Checks whether the string is entirely punctuation. (All vibes, no words 💀)
translate.filters.decoration.isvalidaccelerator(accelerator,
acceptlist=None)returns whether the given accelerator character is valid.
Parameters
- accelerator (character) — a character to be checked for accelerator validity
- acceptlist (String) — a list of characters that are permissible as accelerators
Return type: Boolean
Returns: True if the supplied character is an acceptable accelerator
translate.filters.decoration.puncend(str1, punctuation)Returns all the punctuation from the end of the string.
translate.filters.decoration.puncstart(str1, punctuation)Returns all the punctuation from the start of the string.
translate.filters.decoration.spaceend(str1)Returns all the whitespace from the end of the string.
translate.filters.decoration.spacestart(str1)Returns all the whitespace from the start of the string.
🧚 helpers 🧚
a set of helper functions for filters… (the little guys doing the real work, fr fr)
translate.filters.helpers.countmatch(str1, str2, countstr)Checks whether countstr occurs the same number of times in str1 and str2.
translate.filters.helpers.countsmatch(str1, str2, countlist)Checks whether each element in countlist occurs the same number of times in str1 and str2.
translate.filters.helpers.filtercount(str1, func)Returns the number of characters in str1 that pass func.
translate.filters.helpers.filtertestmethod(testmethod, strfilter)Returns a version of testmethod that operates on filtered strings using strfilter.
translate.filters.helpers.funcmatch(str1, str2, func, *args)Returns whether the result of func is the same for str1 and str2.
translate.filters.helpers.funcsmatch(str1, str2, funclist)Checks whether the results of each func in funclist match for str1 and str2.
translate.filters.helpers.multifilter(str1, strfilters, *args)Passes str1 through a list of filters.
translate.filters.helpers.multifiltertestmethod(testmethod, strfilters)Returns a version of testmethod that operates on filtered strings using strfilter.
👑 pofilter 👑
Perform quality checks on Gettext PO, XLIFF and TMX localization files. Your faves 💖
Snippet files get created whenever a test fails. You can examine them, correct them, and merge them back into the originals using pomerge.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/pofilter.html for examples and usage instructions, and http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/pofilter_tests.html for full descriptions of all the tests. Go read them, iykyk 📚
class translate.filters.pofilter.FilterOptionParser(formats)A specialized Option Parser for filter tools… (yes, a whole bespoke option parser, main character energy)
add_option(Option)
add_option(opt_str, ..., kwarg=val, ...) -> None
build_checkerconfig(options)— prepares the checker config from the given options. It's mainly factored out for the sake of unit tests.check_values(values: Values, args: [string])—-> (values : Values, args : [string])
Checks that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new — whatever you like, honestly). The default implementation just returns the passed-in values; subclasses may override it as desired.
checkoutputsubdir(options, subdir)— checks whethersubdirunderoptions.outputneeds to be created, and creates it if neccessary (their spelling, not mine 💅).define_option(option)— defines the given option, replacing an existing one with the same short name if neccessary…destroy()— declares that you're done with this OptionParser. It cleans up reference cycles so the OptionParser (and every object it references) can be garbage-collected promptly. After you calldestroy(), the OptionParser is unusable — so like, don't try it 🙈disable_interspersed_args()— okay so this one flips the parser into "stop at the first non-option" mode. Reach for it when you've got a command processor that runs another command with its own options, and you want to make absolutely sure those options don't get confused with yours. no cross-contamination bestie 🫶enable_interspersed_args()— the opposite vibe: parsing does not stop at the first non-option, so you can intersperse switches with command arguments. This is the default behavior, periodt. See alsodisable_interspersed_args()and the class documentation description of the attributeallow_interspersed_args.error(msg: string)— prints a usage message that incorporatesmsgtostderrand then exits. ⚠️ and if you override this in a subclass, it should not return — it should either exit or raise an exception. like, pick one, don't just stand there 💀finalizetempoutputfile(options, outputfile, fulloutputpath)— writes the temp outputfile to its final destination. the glow-up moment ✨format_manpage()— returns a formatted manpage.static getformathelp(formats)— makes a nice help string for describing formats… 🎀static getfullinputpath(options, inputpath)— gets the full path to an input file.static getfulloutputpath(options, outputpath)— gets the full path to an output file.getfulltemplatepath(options, templatepath)— gets the full path to a template file.getoutputname(options, inputname, outputformat)— gets an output filename based on the input filename.getoutputoptions(options, inputpath, templatepath)— works out which output format and processor method to use… 🔥getpassthroughoptions(options)— get the options required to pass to the filtermethod…gettemplatename(options, inputname)— gets an output filename based on the input filename. (yes, same description asgetoutputname, the manpage said it, not me 🙈)static getusageman(option)— returns the usage string for the given option.static getusagestring(option)— returns the usage string for the given option.static isexcluded(options, inputpath)— checks if this path has been excluded.static isrecursive(fileoption, filepurpose='input')— checks iffileoptionis a recursive file.isvalidinputname(inputname)— checks if this is a valid input filename.static mkdir(parent, subdir)— makes a subdirectory (recursively if neccessary). 🗂️ (yes "neccessary" is spelled like that in the source, leave her alone)static openinputfile(options, fullinputpath)— opens the input file.static openoutputfile(options, fulloutputpath)— opens the output file.opentemplatefile(options, fulltemplatepath)— opens the template file (if required).static opentempoutputfile(options, fulloutputpath)— opens a temporary output file.parse_args(args=None, values=None)— parses the command line options, handling implicit input/output args. the main character of the parser ngl 👑static parse_noinput(option, opt, value, parser, *args, **kwargs)— this sets an option toTrue, but also sets input to-to prevent an error. sneaky little rescue mission 🦋print_help(file: file = stdout)— print an extended help message, listing all options and any help text provided with them, tofile(defaultstdout). 💖print_manpage(file=None)— outputs a manpage for the program using the help information.print_usage(file: file = stdout)— print the usage message for the current program (self.usage) tofile(defaultstdout). Any occurrence of the string"%prog"inself.usageis replaced with the name of the current program (basename ofsys.argv[0]). Does nothing ifself.usageis empty or not defined. like, it literally just vibes and returns 😭print_version(file: file = stdout)— print the version message for this program (self.version) tofile(defaultstdout). As withprint_usage(), any occurrence of"%prog"inself.versionis replaced by the current program's name. Does nothing ifself.versionis empty or undefined.processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)— process an individual file.recurseinputfilelist(options)— use a list of files, and find a common base directory for them.recurseinputfiles(options)— recurse through directories and return files to be processed.recursiveprocess(options)— recurse through directories and process files.run()— parses the arguments, and runsrecursiveprocesswith the resulting options. the whole plot in one line 🎬set_usage(usage=None)— sets the usage string — ifusagenot given, usesgetusagestringfor each option.seterrorleveloptions()— sets the errorlevel options.setformats(formats, usetemplates)— sets the format options using the given format dictionary.
Parameters formats (Dictionary or iterable) —
The dictionary keys should be:
- Single strings (or 1-tuples) containing an input format (if not
usetemplates) - Tuples containing an input format and template format (if
usetemplates) - Formats can be
Noneto indicate what to do with standard input
The dictionary values should be tuples of outputformat (string) and processor method.
setmanpageoption()— creates a manpage option that allows the optionparser to generate a manpage.setprogressoptions()— sets the progress options.static splitext(pathname)— splitspathnameinto name and ext, and removes the extsep.
Parameters pathname (string) — A file path
Returns root, ext
Return type tuple
splitinputext(inputpath)— splits aninputpathinto name and extension.splittemplateext(templatepath)— splits atemplatepathinto name and extension.templateexists(options, templatepath)— returns whether the given template exists…warning(msg, options=None, exc_info=None)— print a warning message incorporatingmsgtostderr.
translate.filters.pofilter.runfilter(inputfile, outputfile, templatefile, checkfilter=None) — reads in inputfile, filters using checkfilter, writes to outputfile. 💅
🧚 prefilters
Filters that strings can be passed through before certain tests. lil pre-game routine ✨
translate.filters.prefilters.filteraccelerators(accelmarker)
Returns a function that filters accelerators marked using accelmarker from a strings.
Parameters accelmarker (string) — Accelerator marker character
Return type Function
Returns fn(str1, acceplist=None)
translate.filters.prefilters.filtervariables(startmarker, endmarker, varfilter)
Returns a function that filters variables marked using startmarker and endmarker from a string.
Parameters
startmarker(string) — Start of variable markerendmarker(string) — End of variable markervarfilter(Function) —fn(variable, startmarker, endmarker)
Return type Function
Returns fn(str1)
translate.filters.prefilters.filterwordswithpunctuation(str1)
Goes through a list of known words that have punctuation and removes the punctuation from them. 🎀
translate.filters.prefilters.removekdecomments(str1)
Remove KDE-style PO comments.
KDE comments start with _:[space] and end with a literal \n. Example:
"_: comment\n"translate.filters.prefilters.varname(variable, startmarker, endmarker)
Variable filter that returns the variable name without the marking punctuation.
NOTE: Currently this function simply returns variable unchanged, no matter what marker’s are set to. 🙈
Return type String
Returns Variable name with the supplied startmarker and endmarker removed.
translate.filters.prefilters.varnone(variable, startmarker, endmarker)
Variable filter that returns an empty string.
Return type String
Returns Empty string
🌸 spelling
An API to provide spell checking for use in checks or elsewhere.
🌈 lang
Classes that represent languages and provides language-specific information.
All classes inherit from the parent class called common.
The type of data includes:
- Language codes
- Language name
- Plurals
- Punctuation transformation
- etc.
🦋 af
This module represents the Afrikaans language.
SEE ALSO: Afrikaans_language
class translate.lang.af.af(code)
This class represents Afrikaans.
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages. ⭐classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as source language).classmethod capsstart(text)— modify this for the indefinite article ('n).classmethod character_iter(text)— returns an iterator over the characters intext.classmethod characters(text)— returns a list of characters intext.checker = None— a language specific checker instance (seefilters.checks).
This doesn't need to be supplied, but will be used if it exists. 🔥
code = ''— the ISO 639 language code, probs with a country specifier or some other little modifier tagged on. Examples bestie:kmpt_BRsr_YU@Latn
commonpunc =`'.,;:!?-@#$%^*_()[]{}/\\\'`"<>'`— the punctuation marks that are common in English and honestly in most languages that use latin script ✨ethiopicpunc = '።፤፣'— punctuation used by several Ethiopic languages 💜fullname = ''— the full (English) name of this language, periodt.
Dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests = {}— a dictionary of tests to ignore in some or all checkers, no cap.
Keys are checker names and values are lists of names for the ignored tests in that checker. There's a special 'all' checker name you can use to say "ignore these tests in literally every checker".
Listed checkers have to be lowercase strings for the checker name, like "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker — iykyk 🫶
indicpunc = '।॥॰'— punctuation marks used by several Indic languages 🌸invertedpunc = '¿¡'— the inverted punctuation that sometimes shows up at the start of sentences in Spanish, Asturian, Galician, and Catalan 👑classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of length length 🎀listseperator = ', '— the string used to separate lists of textual elements. Most languages can probs stick with the default comma, but Arabic and some Asian languages might wanna override this one.miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian 💅mozilla_pluralequation = '0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
⚠️0is not a valid value — it must be overridden! Any positive integer is valid (it should probably be between1and6).
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language 🔢numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use.classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms.
This one's used for PO files, to fill into the header.
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {}— a dictionary of punctuation transformation rules thatpunctranslate()can use 💫classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation =`'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»…±°¹²³·©®×£¥€'`— okay so we include a whole bunch of punctuation types here, simply because this is only meant to determine if something is punctuation. Hopefully we catch some languages that might not be represented with modules. Most languages won't need to override this — it's giving "covers everything already" energy 🦋quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— the different quotation marks used by various languages 🎀rtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for example 🌙classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend = '.!?…'— these marks can indicate a sentence end. Once again, we're trying to account for many languages. Most langauges won't need to override this ⭐classmethod sentences(text, strip=True)— returns a list of sentences in text.specialchars = 'ëïêôûáéíóúý'— characters used by the language that might not be easy to type with common keyboard layouts 😭validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list 🔥validdoublewords = ['']— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words in text.classmethod words(text)— returns a list of words in text.
And then there's this absolute unit of a mapping, translate.lang.af.cyr2lat — Cyrillic to Latin letters for transliteration in Afrikaans, bestie, look at her go 💅
translate.lang.af.cyr2lat = {'Ё': 'Jo', 'А': 'A', 'Б': 'B', 'В': 'W',
'Г': 'G', 'Д': 'D', 'ДЖ': 'Dj', 'Е': 'Je', 'ЕЙ': 'Ei', 'Ж': 'Zj', 'З':
'Z', 'И': 'I', 'Й': 'J', 'К': 'K', 'Л': 'L', 'М': 'M', 'Н': 'N', 'О':
'O', 'П': 'P', 'Р': 'R', 'С': 'S', 'Т': 'T', 'У': 'Oe', 'Ф': 'F', 'Х':
'Ch', 'Ц': 'Ts', 'Ч': 'Tj', 'Ш': 'Sj', 'Щ': 'Sjtsj', 'Ъ': '', 'Ы': 'I',
'Ь': '', 'Э': 'E', 'Ю': 'Joe', 'Я': 'Ja', 'а': 'a', 'б': 'b', 'в': 'w',
'г': 'g', 'д': 'd', 'дж': 'dj', 'е': 'je', 'ей': 'ei', 'ж': 'zj', 'з':
'z', 'и': 'i', 'й': 'j', 'к': 'k', 'л': 'l', 'м': 'm', 'н': 'n', 'о':
'o', 'п': 'p', 'р': 'r', 'с': 's', 'т': 't', 'у': 'oe', 'ф': 'f', 'х':
'ch', 'ц': 'ts', 'ч': 'tj', 'ш': 'sj', 'щ': 'sjtsj', 'ъ': '', 'ы': 'i',
'ь': '', 'э': 'e', 'ю': 'joe', 'я': 'ja', 'ё': 'jo'}That's the mapping of Cyrillic to Latin letters for transliteration in Afrikaans, fr fr 🧚
translate.lang.af.tranliterate_cyrillic(text) — converts Cyrillic text to Latin according to the AWS transliteration rules. Slay.
🌈 am
This module represents the Amharic language 🇪🇹
SEE ALSO: Amharic_language
class translate.lang.am.am(code) — this class represents Amharic. Let's get into her whole vibe check 👑
CJKpunc = '。、，；！？「」『』【】'— punctuation marks used in certain circumstances with CJK languages 🌸classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language).classmethod capsstart(text)— determines whether the text starts with a capital letter 🔠classmethod character_iter(text)— returns an iterator over the characters in text.classmethod characters(text)— returns a list of characters in text.checker = None— a language specific checker instance (seefilters.checks).
You don't need to supply this, but it'll be used if it exists 😌
code = ''— the ISO 639 language code, possibly with a country specifier or other modifier.
Examples:
kmpt_BRsr_YU@Latn
commonpunc =`'.,;:!?-@#$%^*_()[]{}/\\\'`"<>'`— punctuation marks common in English and most languages that use latin script ✨ethiopicpunc = '።፤፣'— punctuation marks used by several Ethiopic languages 💜fullname = ''— the full (English) name of this language.
Dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers 🫶
Keys are checker names and values are lists of names for the ignored tests in that checker. That special 'all' checker name means the tests must be ignored in all the checkers.
Listed checkers have to be lowercase strings for the checker name — like "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker.
indicpunc = '।॥॰'— punctuation marks used by several Indic languages.invertedpunc = '¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan 👑classmethod length_difference(length)— returns an estimate to a likely change in length relative to an English string of length length 🎀listseperator = '፣ '— the string used to separate lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this.miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian 💅mozilla_pluralequation = '0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
⚠️0is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between1and6).
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language 🔢numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can vibe with 🔢classmethod numstart(text)— figures out whether text starts with a numeric value, yes or nopluralequation = '0'— the plural equation used to pick which plural form you get
This one gets filled into the header for PO files, bestie.
SEE ALSO: the Gettext manual, and translate.lang.data
puncdict = {',': '፣', '.': '።', ';': '፤'}— a dict of punctuation transformation rules thatpunctranslate()can use 💅classmethod punctranslate(text)— converts the punctuation in a string following the rules of the languagepunctuation =
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'We literally just throw a ton of punctuation types in here, because the only job is answering "is this a punctuation mark or not". Hopefully this catches some languages that don't even have modules yet 🫶 Most languages won't need to override this at all.
quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— all the different quotation marks various languages use ✨rtlpunc = '،؟؛÷'— these punctuation marks get used by Arabic and Persian, for exampleclassmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in textsentenceend = '።!?…'— these marks can signal a sentence end. Again, we're trying to cover a lot of languages. Most langauges won't need to override this 🌸classmethod sentences(text, strip=True)— returns a list of sentences in textspecialchars = ''— characters this language uses that aren't exactly easy to type on common keyboard layouts 🎀validaccel = None— characters that can work as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as the user can reach them from their keyboard in a single keystroke — but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list 💜validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words in textclassmethod words(text)— returns a list of words in text
🌙 ar
This module represents the Arabic language, and honestly? Iconic.
SEE ALSO: Arabic_language
class translate.lang.ar.ar(code) — this class represents Arabic ✨
CJKpunc = '。、，；！？「」『』【】'— punctuation marks used in certain circumstances with CJK languagesclassmethod alter_length(text)— converts the given string by adding or removing characters, as an estimate of translation length (with English assumed as the source language)classmethod capsstart(text)— determines whether text starts with a capital letterclassmethod character_iter(text)— returns an iterator over the characters in textclassmethod characters(text)— returns a list of characters in textchecker = None— a language-specific checker instance (seefilters.checks).
You don't have to supply this, but if it exists, it gets used 👑
code = ''— the ISO 639 language code, possibly with a country specifier or other modifier.
Examples:
kmpt_BRsr_YU@Latn
- **
commonpunc =`'.,;:!?-@#$%^*_()[]{}/\\\'`"<>'`— the punctuation marks that are common in English and most languages using the latin script ethiopicpunc = '።፤፣'— punctuation marks used by several Ethiopic languagesfullname = ''— the full (English) name of this language.
Dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn? 🙈
ignoretests = {'all': ['acronyms', 'simplecaps', 'startcaps']}— a dictionary of tests to ignore in some or all checkers.
Keys are checker names and values are lists of names for the ignored tests inside that checker. There's a special 'all' checker name you can use to say "these tests must be ignored in all the checkers".
The listed checker names have to be lowercase strings, like "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker 💅
indicpunc = '।॥॰'— punctuation marks used by several Indic languagesinvertedpunc = '¿¡'— the inverted punctuation sometimes used at the start of sentences in Spanish, Asturian, Galician, and Catalanclassmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of length lengthlistseperator = '، '— this string separates lists of textual elements.
Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override it 🍓
miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgianmozilla_pluralequation = '0'— for languages whose plural formula in Mozilla is different from the standard Gettext onenplurals = 0— the number of plural forms this language has.
0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the language's rulesnumbertuple = (('0', '٠'), ('1', '١'), ('2', '٢'), ('3', '٣'), ('4', '٤'), ('5', '٥'), ('6', '٦'), ('7', '٧'), ('8', '٨'), ('9', '٩'))— a tuple of number transformation rules thatnumbertranslate()can use 🔥classmethod numstart(text)— determines whether text starts with a numeric valuepluralequation = '0'— the plural equation for selecting plural forms.
This gets used for PO files, to fill into the header.
SEE ALSO: the Gettext manual, and translate.lang.data
puncdict = {',': '،', ';': '؛', '?': '؟'}— a dict of punctuation transformation rules thatpunctranslate()can useclassmethod punctranslate(text)— converts the punctuation in a string according to the language's rulespunctuation =
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'We include a whole bunch of punctuation types here, simply because this is only meant to determine whether something is punctuation. Hopefully we catch some languages that might not be represented with modules. Most languages won't need to override this 🫶
quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— the different quotation marks used by various languagesrtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for exampleclassmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in textsentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end. Once again, we're trying to account for many languages. Most langauges won't need to override this 💜classmethod sentences(text, strip=True)— returns a list of sentences in textspecialchars = ''— characters used by the language that might not be easy to input with common keyboard layoutsvalidaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list ⭐validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words in textclassmethod words(text)— returns a list of words in text
🐅 bn
This module represents the Bengali language, no cap.
SEE ALSO: Bengali_language
class translate.lang.bn.bn(code) — this class represents Bengali 🌸
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languagesclassmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language)classmethod capsstart(text)— determines whether the text starts with a capital letterclassmethod character_iter(text)— gives you an iterator over every character intext, one at a time, like a little character parade 🎀classmethod characters(text)— gives you a plain list of the characters intext, so you can index it and be dramatic about it 💅checker = None— a language-specific checker instance (peepfilters.checks). It doesn’t have to be supplied, but if it exists it will be used, periodt.code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples:kmpt_BRsr_YU@Latn
commonpunc— punctuation marks that are common in English and most languages using Latin script:
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc = '።፤፣'— punctuation marks used by several Ethiopic languages. Cute and specific 🦋fullname = ''— the full (English) name of this language. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers. Keys are checker names, values are lists of ignored test names in that checker. The special'all'checker name means “ignore these in every checker.” Listed checker names must be lowercase strings, like"mozilla"forMozillaCheckeror"libreoffice"forLibreOfficeChecker.indicpunc = '।॥॰'— punctuation marks used by several Indic languages. Iconic little marks ✨invertedpunc = '¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. Yes, the upside-down vibes are intentional 🙃classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of lengthlength.listseperator = ', '— this string separates lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override it. (Yes, the attribute name is spelledlistseperator— do not come for me, I’m copying the source 💀)miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian. The rest is just miscellaneous punctuation catching flights 🌈mozilla_pluralequation = '0'— this is for languages whose Mozilla plural formula differs from the standard Gettext one.nplurals = 0— the number of plural forms of this language.
0is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between1and6).
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple— a tuple of number transformation rules that can be used bynumbertranslate():
numbertuple = (('0', '০'), ('1', '১'), ('2', '২'), ('3', '৩'), ('4', '৪'), ('5', '৫'), ('6', '৬'), ('7', '৭'), ('8', '৮'), ('9', '৯'))classmethod numstart(text)— determines whether the text starts with a numeric value. Number detector says bestie, this starts with digits 🔥pluralequation = '0'— the plural equation for selection of plural forms. This is used for PO files, filling it into the header.
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {'.\n': '।\n', '. ': '। '}— a dictionary of punctuation transformation rules that can be used bypunctranslate().classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language. Punctuation glow-up, basically ✨punctuation— we include many types of punctuation here because this is only meant to determine whether something is punctuation. Hopefully this catches some languages that might not be represented with modules. Most languages won’t need to override this:
punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are the different quotation marks used by various languages. Quote collection is lowkey a flex 🎀rtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— gives you an iterator over the sentences intext.sentenceend = '।!?…'— these marks can indicate a sentence end. Once again, we try to account for many languages. Most langauges won’t need to override this.classmethod sentences(text, strip=True)— gives you a list of sentences intext. Sentence slay, separated and ready 💅specialchars = ''— characters used by the language that might not be easy to type with common keyboard layouts.validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X whereXis the accelerator. These can include combining diacritics as long as they are accessible from the user’s keyboard in a single keystroke, but normally they would be at least precomposed characters. All characters, lower and upper, are included in the list. Keyboard-accessibility queen 👑validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— gives you an iterator over the words intext. Word-by-word, no cap 🫶classmethod words(text)— gives you a list of the words intext.
🌺 code_or
This module represents the Odia language.
SEE ALSO: Odia_language
💜 translate.lang.code_or.code_or(code)
This class represents Odia.
CJKpunc = '。、，；！？「」『』【】'— punctuation marks used in certain circumstances with CJK languages.classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language). Length-shifting magic, but make it linguistic ✨classmethod capsstart(text)— determines whether the text starts with a capital letter. Capital-letter check, iconic and simple.classmethod character_iter(text)— returns an iterator over the characters intext.classmethod characters(text)— returns a list of characters intext.checker = None— a language-specific checker instance (seefilters.checks). This doesn’t need to be supplied, but will be used if it exists.code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples:kmpt_BRsr_YU@Latn
commonpunc— punctuation marks common in English and most languages that use Latin script:
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc = '።፤፣'— punctuation marks used by several Ethiopic languages.fullname = ''— the full (English) name of this language. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers. Keys are checker names and values are lists of ignored test names in the checker. A special'all'checker name tells the toolkit to ignore those tests in all checkers. Listed checker names must be lowercase strings, for example"mozilla"forMozillaCheckeror"libreoffice"forLibreOfficeChecker.indicpunc = '।॥॰'— punctuation marks used by several Indic languages.invertedpunc = '¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan.classmethod length_difference(length)— returns an estimate of a likely change in length relative to an English string of lengthlength.listseperator = ', '— this string separates lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this.miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian.mozilla_pluralequation = '0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
0is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between1and6).
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple = ()— a tuple of number transformation rules that can be used bynumbertranslate(). For this class it’s empty, and that’s valid — no number-swap rules loaded here.classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms. This is used for PO files to fill into the header.
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {'.\n': '।\n', '. ': '। '}— a little dictionary of punctuation transformation rules thatpunctranslate()can use. fr fr, it's just a lookup table for swapping punctuation around 💅classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language. like a little punctuation glow-up ✨punctuation— the big ol' pile of everything we consider punctuation:
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'
and the tea ☕ is: we include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won't need to override this 🫶
quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— the different quotation marks used by various languages. iconic little curly things 🎀rtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for example. right-to-left gang rise up 🌙classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences intext. one sentence at a time, no overwhelm 💆sentenceend = '।!?…'— marks that can indicate a sentence end. once again we're trying to account for many languages. Most langauges won't need to override this 👑classmethod sentences(text, strip=True)— returns a list of sentences intext. same energy, but as a list 💫specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts. keyboard struggle is real 😭validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they are accessible from the user's keyboard in a single keystroke, but normally they would be at least precomposed characters. All characters, lower and upper, are included in the list 🔑validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words. (yes bestie, the description says dictionary even though the default is a list — we don't make the rules 🙈)classmethod word_iter(text)— returns an iterator over the words intext. word by word, slay by slay 🦋classmethod words(text)— returns a list of words intext. periodt.
✨ common
This module contains all the common features for languages. it's giving "shared DNA" 🧬
Supported features:
- language code (
km,af) - language name (
Khmer,Afrikaans) - Plurals
- Number of plurals (
nplurals) - Plural equation
- Number of plurals (
pofiltertests to ignore
Segmentation:
- characters
- words
- sentences
Punctuation:
- End of sentence
- Start of sentence
- Middle of sentence
- Quotes
- single
- double
- Valid characters
- Accelerator characters
- Special characters
- Direction (rtl or ltr)
TODOs and Ideas for possible features:
- Language-Team information
- Segmentation
- phrases
lowkey a vision board at this point 💭
🦋 translate.lang.common.Common(code)
This class is the common parent class for all language classes. the mother of all language classes, literally 👑
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages 🀄classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as source language). it's a length vibe check 📏classmethod capsstart(text)— determines whether the text starts with a capital letter. uppercase girlie detector 🔠classmethod character_iter(text)— returns an iterator over the characters intext.classmethod characters(text)— returns a list of characters intext.checker = None— a language specific checker instance (seefilters.checks). This doesn't need to be supplied, but will be used if it exists. optional but make it cute ✨code = ''— the ISO 639 language code, possibly with a country specifier or other modifier.
Examples:
kmpt_BRsr_YU@Latn
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — punctuation marks that are common in English and most languages that use latin script. the standard-issue punctuation starter pack 🎒ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages 🌍fullname = ''— the full (English) name of this language.
Dialect codes should have the form of:
KhmerPortugese (Brazil)TODO: sr_YU@Latn?
ignoretests = {}— dictionary of tests to ignore in some or all checkers.
Keys are checker names and values are list of names for the ignored tests in the checker. A special 'all' checker name can be used to tell that the tests must be ignored in all the checkers.
Listed checkers to ignore tests on must be lowercase strings for the checker name, for example "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker. lowercase only, bestie, we don't do shouting 💅
indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages 🪷invertedpunc = '¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. upside-down and thriving 🙃classmethod length_difference(length)— returns an estimate to a likely change in length relative to an English string of lengthlength. 🔮listseperator = ', '— this string is used to separate lists of textual elements. Most languages probably can stick with the default comma, but Arabic and some Asian languages might want to override this. (yes, it's spelledlistseperator— do not come for her 🙈)miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian. little dot, big responsibilities 🫶mozilla_pluralequation = '0'— for languages that have a different plural formula in Mozilla than the standard one in Gettext. Mozilla said "not today" 😤nplurals = 0— the number of plural forms of this language.
⚠️ 0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6). don't ship the zero, I'm begging 😭
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language. ١٢٣ era 🔢numbertuple = ()— a tuple of number transformation rules that can be used bynumbertranslate().classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms.
This is used for PO files to fill into the header.
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {}— a dictionary of punctuation transformation rules that can be used bypunctranslate().classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language ✨punctuation— the whole punctuation buffet:
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'
We include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won't need to override this 💜
quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are different quotation marks used by various languages. quote variety is a lifestyle 🎀rtlpunc = '،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example 🌙classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences intext.sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end. Once again we try to account for many languages. Most langauges won't need to override this 🌈classmethod sentences(text, strip=True)— returns a list of sentences intext.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts ⌨️validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they are accessible from the user's keyboard in a single keystroke, but normally they would be at least precomposed characters. All characters, lower and upper, are included in the list 👑validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words intext.classmethod words(text)— returns a list of words intext. 🦋
🌸 data
This module stores information and functionality that relates to plurals. plural lore, basically 🧚
translate.lang.data.cldr_plural_categories = ['zero', 'one', 'two', 'few', 'many', 'other']— list of plural tags generated from CLDR 44.0.1 using ‐https://github.com/WeblateOrg/language-data✨translate.lang.data.expansion_factors = {'af': 0.1, 'ar': -0.09, 'es': 0.21, 'fr': 0.28, 'it': 0.2}— source to target string length expansion factors. how much chonkier your translation's gonna get 📈💖
translate.lang.data.languages = {'ach': ('Acholi', 2, 'n > 1'), 'af':
('Afrikaans', 2, '(n != 1)'), 'ak': ('Akan', 2, 'n > 1'), 'am':
('Amharic', 2, 'n > 1'), 'an': ('Aragonese', 2, '(n != 1)'), 'anp':
('Angika', 2, '(n != 1)'), 'ar': ('Arabic', 6, 'n==0 ? 0 : n==1 ? 1 :
n==2 ? 2 : n%100>=3 && n%100<=10 ? 3 : n%100>=11 ? 4 : 5'), 'arn':
('Mapudungun; Mapuche', 2, 'n > 1'), 'as': ('Assamese', 2, '(n != 1)'),
'ast': ('Asturian; Bable; Leonese; Asturleonese', 2, '(n != 1)'), 'ay':
('Aymará', 1, '0'), 'az': ('Azerbaijani', 2, '(n != 1)'), 'be':
('Belarusian', 3, 'n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 &&
(n%100<10 || n%100>=20) ? 1 : 2'), 'bg': ('Bulgarian', 2, '(n != 1)'),
'bn': ('Bengali', 2, '(n != 1)'), 'bn_BD': ('Bengali (Bangladesh)', 2,
'(n != 1)'), 'bn_IN': ('Bengali (India)', 2, '(n != 1)'), 'bo':
('Tibetan', 1, '0'), 'br': ('Breton', 2, 'n > 1'), 'brx': ('Bodo', 2,
'(n != 1)'), 'bs': ('Bosnian', 3, 'n%10==1 && n%100!=11 ? 0 : n%10>=2 &&
n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2'), 'ca': ('Catalan;
Valencian', 2, '(n != 1)'), 'ca@valencia': ('Catalan; Valencian
(Valencia)', 2, '(n != 1)'), 'cgg': ('Chiga', 1, '0'), 'cs': ('Czech',
3, '(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2'), 'csb': ('Kashubian', 3, 'n==1
? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2'), 'cy':
('Welsh', 2, '(n==2) ? 1 : 0'), 'da': ('Danish', 2, '(n != 1)'), 'de':
('German', 2, '(n != 1)'), 'doi': ('Dogri', 2, '(n != 1)'), 'dz':
('Dzongkha', 1, '0'), 'el': ('Greek, Modern (1453-)', 2, '(n != 1)'),
'en': ('English', 2, '(n != 1)'), 'en_GB': ('English (United Kingdom)',
2, '(n != 1)'), 'en_ZA': ('English (South Africa)', 2, '(n != 1)'),
'eo': ('Esperanto', 2, '(n != 1)'), 'es': ('Spanish; Castilian', 2, '(n
!= 1)'), 'es_AR': ('Argentinean Spanish', 2, '(n != 1)'), 'et':
('Estonian', 2, '(n != 1)'), 'eu': ('Basque', 2, '(n != 1)'), 'fa':
('Persian', 2, 'n > 1'), 'ff': ('Fulah', 2, '(n != 1)'), 'fi':
('Finnish', 2, '(n != 1)'), 'fil': ('Filipino; Pilipino', 2, '(n > 1)'),
'fo': ('Faroese', 2, '(n != 1)'), 'fr': ('French', 2, '(n > 1)'), 'fur':
('Friulian', 2, '(n != 1)'), 'fy': ('Frisian', 2, '(n != 1)'), 'ga':
('Irish', 5, 'n==1 ? 0 : n==2 ? 1 : (n>2 && n<7) ? 2 :(n>6 && n<11) ? 3
: 4'), 'gd': ('Gaelic; Scottish Gaelic', 4, '(n==1 || n==11) ? 0 : (n==2
|| n==12) ? 1 : (n > 2 && n < 20) ? 2 : 3'), 'gl': ('Galician', 2, '(n
!= 1)'), 'gu': ('Gujarati', 2, '(n != 1)'), 'gun': ('Gun', 2, '(n >
1)'), 'ha': ('Hausa', 2, '(n != 1)'), 'he': ('Hebrew', 2, '(n != 1)'),
'hi': ('Hindi', 2, '(n != 1)'), 'hne': ('Chhattisgarhi', 2, '(n != 1)'),
'hr': ('Croatian', 3, '(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 &&
(n%100<10 || n%100>=20) ? 1 : 2)'), 'ht': ('Haitian; Haitian Creole', 2,
'(n != 1)'), 'hu': ('Hungarian', 2, '(n != 1)'), 'hy': ('Armenian', 1,
'0'), 'ia': ('Interlingua (International Auxiliary Language
Association)', 2, '(n != 1)'), 'id': ('Indonesian', 1, '0'), 'is':
('Icelandic', 2, '(n != 1)'), 'it': ('Italian', 2, '(n != 1)'), 'ja':
('Japanese', 1, '0'), 'jbo': ('Lojban', 1, '0'), 'jv': ('Javanese', 2,
'(n != 1)'), 'ka': ('Georgian', 1, '0'), 'kab': ('Kabyle', 2, '(n !=
1)'), 'kk': ('Kazakh', 2, 'n != 1'), 'kl': ('Greenlandic', 2, '(n !=
1)'), 'km': ('Central Khmer', 1, '0'), 'kn': ('Kannada', 2, '(n != 1)'),
'ko': ('Korean', 1, '0'), 'kok': ('Konkani', 2, '(n != 1)'), 'ks':
('Kashmiri', 2, '(n != 1)'), 'ku': ('Kurdish', 2, '(n != 1)'), 'kw':
('Cornish', 4, '(n==1) ? 0 : (n==2) ? 1 : (n == 3) ? 2 : 3'), 'ky':
('Kirghiz; Kyrgyz', 2, 'n != 1'), 'lb': ('Luxembourgish; Letzeburgesch',
2, '(n != 1)'), 'ln': ('Lingala', 2, '(n > 1)'), 'lo': ('Lao', 1, '0'),
'lt': ('Lithuanian', 3, '(n%10==1 && n%100!=11 ? 0 : n%10>=2 &&
(n%100<10 || n%100>=20) ? 1 : 2)'), 'lv': ('Latvian', 3, '(n%10==1 &&
n%100!=11 ? 0 : n != 0 ? 1 : 2)'), 'mai': ('Maithili', 2, '(n != 1)'),
'me': ('Montenegrin', 3, 'n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4
&& (n%100<10 || n%100>=20) ? 1 : 2'), 'mfe': ('Morisyen', 2, '(n > 1)'),
'mg': ('Malagasy', 2, '(n > 1)'), 'mi': ('Maori', 2, '(n > 1)'), 'mk':
('Macedonian', 2, '(n==1 || n%10==1 ? 0 : 1)'), 'ml': ('Malayalam', 2,
'(n != 1)'), 'mn': ('Mongolian', 2, '(n != 1)'), 'mni': ('Meithei
(Manipuri)', 2, '(n != 1)'), 'mnk': ('Mandinka', 3, '(n==0 ? 0 : n==1 ?
1 : 2)'), 'mr': ('Marathi', 2, '(n != 1)'), 'ms': ('Malay', 1, '0'),
'mt': ('Maltese', 4, '(n==1 ? 0 : n==0 || ( n%100>1 && n%100<11) ? 1 :
(n%100>10 && n%100<20 ) ? 2 : 3)'), 'my': ('Burmese', 1, '0'), 'nah':
('Nahuatl languages', 2, '(n != 1)'), 'nap': ('Neapolitan', 2, '(n !=
1)'), 'nb': ('Bokmål, Norwegian; Norwegian Bokmål', 2, '(n != 1)'),
'ne': ('Nepali', 2, '(n != 1)'), 'nl': ('Dutch; Flemish', 2, '(n !=
1)'), 'nn': ('Norwegian Nynorsk; Nynorsk, Norwegian', 2, '(n != 1)'),
'nqo': ("N'Ko", 2, '(n > 1)'), 'nso': ('Pedi; Sepedi; Northern Sotho',
2, '(n != 1)'), 'oc': ('Occitan (post 1500)', 2, '(n > 1)'), 'or':
('Odia', 2, '(n != 1)'), 'pa': ('Panjabi; Punjabi', 2, '(n != 1)'),
'pap': ('Papiamento', 2, '(n != 1)'), 'pl': ('Polish', 3, '(n==1 ? 0 :
n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2)'), 'pms':
('Piemontese', 2, '(n != 1)'), 'ps': ('Pushto; Pashto', 2, '(n != 1)'),
'pt': ('Portuguese', 2, '(n != 1)'), 'pt_BR': ('Portuguese (Brazil)', 2,
'(n > 1)'), 'rm': ('Romansh', 2, '(n != 1)'), 'ro': ('Romanian', 3,
'(n==1 ? 0 : (n==0 || (n%100 > 0 && n%100 < 20)) ? 1 : 2)'), 'ru':
('Russian', 3, '(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 &&
(n%100<10 || n%100>=20) ? 1 : 2)'), 'rw': ('Kinyarwanda', 2, '(n !=
1)'), 'sa': ('Sanskrit', 3, '(n==1 ? 0 : n==2 ? 1 : 2)'), 'sah':
('Yakut', 1, '0'), 'sat': ('Santali', 2, '(n != 1)'), 'scn':
('Sicilian', 2, '(n != 1)'), 'sco': ('Scots', 2, '(n != 1)'), 'sd':
('Sindhi', 2, '(n != 1)'), 'se': ('Northern Sami', 2, '(n != 1)'), 'si':
('Sinhala; Sinhalese', 2, '(n != 1)'), 'sk': ('Slovak', 3, '(n==1) ? 0 :
(n>=2 && n<=4) ? 1 : 2'), 'sl': ('Slovenian', 4, '(n%100==1 ? 0 :
n%100==2 ? 1 : n%100==3 || n%100==4 ? 2 : 3)'), 'so': ('Somali', 2, '(n
!= 1)'), 'son': ('Songhai languages', 1, '0'), 'sq': ('Albanian', 2, '(n
!= 1)'), 'sr': ('Serbian', 3, '(n%10==1 && n%100!=11 ? 0 : n%10>=2 &&
n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2)'), 'st': ('Sotho, Southern',
2, '(n != 1)'), 'su': ('Sundanese', 1, '0'), 'sv': ('Swedish', 2, '(n !=
1)'), 'sw': ('Swahili', 2, '(n != 1)'), 'szl': ('Silesian', 3, '(n==1 ?
0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2)'), 'ta':
('Tamil', 2, '(n != 1)'), 'te': ('Telugu', 2, '(n != 1)'), 'tg':
('Tajik', 1, '0'), 'th': ('Thai', 1, '0'), 'ti': ('Tigrinya', 2, '(n >
1)'), 'tk': ('Turkmen', 2, '(n != 1)'), 'tr': ('Turkish', 2, '(n !=
1)'), 'tt': ('Tatar', 1, '0'), 'ug': ('Uighur; Uyghur', 1, '0'), 'uk':
('Ukrainian', 3, '(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 &&
(n%100<10 || n%100>=20) ? 1 : 2)'), 'ur': ('Urdu', 2, '(n != 1)'), 'uz':
('Uzbek', 2, '(n > 1)'), 've': ('Venda', 2, '(n != 1)'), 'vi':
('Vietnamese', 1, '0'), 'wa': ('Walloon', 2, '(n > 1)'), 'wo': ('Wolof',
2, '(n != 1)'), 'yo': ('Yoruba', 2, '(n != 1)'), 'yue': ('Yue', 1, '0'),
'zh_CN': ('Chinese (China)', 1, '0'), 'zh_HK': ('Chinese (Hong Kong)',
1, '0'), 'zh_TW': ('Chinese (Taiwan)', 1, '0'), 'zu': ('Zulu', 2, '(n !=
1)')}okay bestie, this is THE dictionary 🌈 — the big one, the language data map. Your language code is the dict key (and yeah, it can totally have country codes and modifiers in it, no biggie). The value is a tuple: (Full name in English from iso-codes, nplurals, plural equation). Three things, in that order, always.
🫶 real talk, listen up: those English names are not for user-facing places. Like, ever. It should always get passed through the function returned fromtr_lang(), or at the very least through_fix_language_name(). No exceptions, iykyk.
💖 translate.lang.data.normalize(string, normal_form='NFC')
Returns you a unicode string in its normalized form ✨ — it's giving tidy-up-your-unicode era.
Parameters
string– the string getting normalized 💅normal_form–NFC(that's the default, bestie),NFD,NFKC,NFKD
Returns Normalized string 🎀
translate.lang.data.scripts = {'Beng': ['bn', 'mni'], 'Deva': ['anp',
'bho', 'brx', 'doi', 'hi', 'kfy', 'kok', 'mai', 'mr', 'sa', 'sat'],
'Gujr': ['gu'], 'Khmr': ['km'], 'Knda': ['kn'], 'Laoo': ['lo'], 'Mlym':
['ml'], 'Mymr': ['my', 'shn'], 'Orya': ['or'], 'Sind': ['sd'], 'Taml':
['ta'], 'Tibt': ['bo'], 'assamese': ['as'], 'chinese': ['yue'],
'perso-arabic': ['ks']}Dictionary of scripts data 🦋. The keys are ISO 15924 script codes, plus script names for the scripts that are missing from the standard (yes, those get a whole different vibe of key). The value is a list of codes for languages that use that script.
This one's mainly here to alter how some checks behave — the accelerators check is the example, so like, the accelerators one, for instance 🔥.
🌸 translate.lang.data.simplercode(code)
This attempts to simplify the given language code by ignoring country codes, for example. Genuinely just vibes-based trimming 🎀
SEE ALSO:
- http://www.rfc-editor.org/rfc/bcp/bcp47.txt
- http://www.rfc-editor.org/rfc/rfc4646.txt
- http://www.rfc-editor.org/rfc/rfc4647.txt
- http://www.w3.org/International/articles/language-tags/
⭐ translate.lang.data.simplify_to_common(language_code)
Simplify language code to the most commonly used form for the language, stripping country information for languages that tend not to be localized differently for different countries. Basically: if nobody actually splits it by country, we don't keep the country. Periodt 💅
✨ de ✨
This module represents the German language 🇩🇪. Iconic, honestly.
SEE ALSO: German_language
💜 class translate.lang.de.de(code)
This class represents German. Say hi 👑
CJKpunc
CJKpunc = '。、，；！？「」『』【】'These punctuation marks are used in certain circumstances with CJK languages. Not a German thing, but she's holding them for a friend 🫶
classmethod alter_length(text) Converts the given string by adding or removing characters as an estimation of translation length (with English assumed as source language). It's the "how long is this gonna be in the other language, roughly" check ✨
classmethod capsstart(text) Determines whether the text starts with a capital letter. Detective work, but make it uppercase 🔍
classmethod character_iter(text) Returns an iterator over the characters in text. Lazy, efficient, main character energy 🦋
classmethod characters(text) Returns a list of characters in text. Same thing but she brought a list instead of an iterator 💅
checker = None A language specific checker instance (see filters.checks). You don't need to supply one — it's not a requirement — but it will be used if it exists. Optional but make it slay 🌈
code = '' The ISO 639 language code, possibly with a country specifier or other modifier.
Examples:
km
pt_BR
sr_YU@Latncommonpunc
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'These punctuation marks are common in English and most languages that use latin script. Like, the everyday punctuation starter pack 💀
ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages. 🎀fullname = ''— the full (English) name of this language, bestie. Dialect codes should have the form of:KhmerPortugese (Brazil)TODO: sr_YU@Latn?
ignoretests = {'all': ['simplecaps']}— dictionary of tests to ignore in some or all checkers. Keys are checker names and values are list of names for the ignored tests in the checker. A special ‘all’ checker name can be used to tell that the tests must be ignored in all the checkers. Listed checkers to ignore tests on must be lowercase strings for the checker name, for example “mozilla” for MozillaChecker or “libreoffice” for LibreOfficeChecker. ✨indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages.invertedpunc = '¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. lowkey iconic 💅classmethod length_difference(length)— returns an estimate to a likely change in length relative to an English string of length length.listseperator = ', '— this string is used to separate lists of textual elements. Most languages probably can stick with the default comma, but Arabic and some Asian languages might want to override this. (yes, the attribute name really is spelledlistseperator, don't come for me 😭)miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian.mozilla_pluralequation = '0'— this of languages that has different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.0is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).- SEE ALSO:
translate.lang.data
- SEE ALSO:
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple = ()— a tuple of number transformation rules that can be used bynumbertranslate().classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms. This is used for PO files to fill into the header.- SEE ALSO: Gettext manual,
translate.lang.data
- SEE ALSO: Gettext manual,
puncdict = {}— a dictionary of punctuation transformation rules that can be used bypunctranslate().classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation =— we include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won’t need to override this. 🦋
punctuation =
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are different quotation marks used by various languages.rtlpunc = '،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end. Once again we try to account for many languages. Most langauges won’t need to override this.classmethod sentences(text, strip=True)— returns a list of sentences in text.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts.validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they are accessible from the users keyboard in a single keystroke, but normally they would be at least precomposed characters. All characters, lower and upper, are included in the list.validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words in text.classmethod words(text)— returns a list of words in text.
🌸 el
This module represents the Greek language. 🇬🇷
SEE ALSO:
Greek_language
🫶 class translate.lang.el.el(code)
This class represents Greek. ✨
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages.classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as source language).classmethod capsstart(text)— determines whether the text starts with a capital letter.classmethod character_iter(text)— returns an iterator over the characters in text.classmethod characters(text)— returns a list of characters in text.checker = None— a language specific checker instance (seefilters.checks). This doesn’t need to be supplied, but will be used if it exists.code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples:kmpt_BRsr_YU@Latn
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — these punctuation marks are common in English and most languages that use latin script. 💖ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages.fullname = ''— the full (English) name of this language. Dialect codes should have the form of:KhmerPortugese (Brazil)TODO: sr_YU@Latn?
ignoretests = {}— dictionary of tests to ignore in some or all checkers. Keys are checker names and values are list of names for the ignored tests in the checker. A special ‘all’ checker name can be used to tell that the tests must be ignored in all the checkers. Listed checkers to ignore tests on must be lowercase strings for the checker name, for example “mozilla” for MozillaChecker or “libreoffice” for LibreOfficeChecker.indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages.invertedpunc = '¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan.classmethod length_difference(length)— returns an estimate to a likely change in length relative to an English string of length length.listseperator = ', '— this string is used to separate lists of textual elements. Most languages probably can stick with the default comma, but Arabic and some Asian languages might want to override this.miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian.mozilla_pluralequation = '0'— this of languages that has different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.0is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).- SEE ALSO:
translate.lang.data
- SEE ALSO:
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple = ()— a tuple of number transformation rules that can be used bynumbertranslate().classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms. This is used for PO files to fill into the header.- SEE ALSO: Gettext manual,
translate.lang.data
- SEE ALSO: Gettext manual,
puncdict = {';': '·', '?': ';'}— a dictionary of punctuation transformation rules that can be used bypunctranslate(). 🎀classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation =— we include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won’t need to override this.
punctuation =
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are different quotation marks used by various languages.rtlpunc = '،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example. 💅classmethod sentence_iter(text, strip=True)— hands you back an iterator over the sentences in text.sentenceend = '.!;…'— the marks that can mean "sentence ends here, bestie." Once again we're out here trying to cover so many languages at once 🌍 — and most languages literally won't need to override this one.classmethod sentences(text, strip=True)— same deal but the list version: returns a list of sentences in text.specialchars = ''— characters the language uses that might not be easy to type on a common keyboard layout. 🎀validaccel = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890αβγδεζηθικλμνξοπρστυφχψωΑΒΓΔΕΖΗΘΙΚΛΜΝΞΟΠΡΣΤΥΦΧΨΩ'— the characters that are allowed to be accelerators (access keys), i.e. Alt+X where X is the accelerator. 🔑 These can include combining diacritics as long as they're reachable from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All the characters are in the list — lower and upper.validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of those words. 👀classmethod word_iter(text)— iterator over the words in text. 🦋classmethod words(text)— and the list version: a list of words in text.
es 🫶
This module represents the Spanish language.
NOTE: since it only has special-case code for initial inverted punctuation, it could also be used for Asturian, Galician, or Catalan. 😌
class translate.lang.es.es(code) — this class is Spanish, no notes.
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages. ✨classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language).classmethod capsstart(text)— figures out whether the text starts with a capital letter. A vs a, iykyk.classmethod character_iter(text)— iterator over the characters in text.classmethod characters(text)— a list of characters in text. 🍓checker = None— a language-specific checker instance (seefilters.checks). You don't have to supply it, but if it exists, it gets used. 👑code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples:
km
pt_BR
sr_YU@Latncommonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — the punctuation marks that are common in English and most languages using the latin script.ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages.fullname = ''— the full (English) name of this language. 💖 Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests = {}— a dictionary of tests to ignore in some or all checkers. Keys are checker names, values are lists of names for the ignored tests in that checker. There's a special'all'checker name you can use to say the tests must be ignored in all the checkers. The checker names you list must be lowercase strings — for example"mozilla"forMozillaCheckeror"libreoffice"forLibreOfficeChecker. 📋indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages.invertedpunc = '¿¡'— the inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. ¡iconic!classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of length length.listseperator = ', '— this string separates lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override it. (yes, that spelling is on purpose, don't come for me 💀)miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian. 🎀mozilla_pluralequation = '0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6). ⚠️SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language. 🔢numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use.classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation used to select plural forms. This gets used for PO files, to fill into the header. 🌸
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— implements some extra features for inverted punctuation. 🙈punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'` — we include so many types of punctuation here, simply because this is only meant to determine whether something is punctuation. Hopefully we catch some languages that might not be represented with modules. Most languages won't need to override this. 💅quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— all the different quotation marks used by various languages. 💬rtlpunc = '،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example.sentenceend = '.!?…։؟।。！？።۔'— marks that can indicate a sentence end. Again, we try to account for many languages. Most languages won't need to override this. ⭐classmethod sentence_iter(text, strip=True)— iterator over the sentences in text.classmethod sentences(text, strip=True)— a list of sentences in text.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts. 🎀validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list.validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words. 👀classmethod word_iter(text)— iterator over the words in text.classmethod words(text)— a list of words in text. 🦋
factory 🏭
This module provides a factory to instantiate language classes. Slay, honestly.
translate.lang.factory.get_all_languages() — return all language classes. all of them. every single one. 👑
translate.lang.factory.getlanguage(code) — this returns a language class.
Parameters
- code — the ISO 639 language code
fa 🌸
This module represents the Persian language.
SEE ALSO: Persian_language
class translate.lang.fa.fa(code) — this class represents Persian. 🫶
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages. ✨classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language).classmethod capsstart(text)— determines whether the text starts with a capital letter.classmethod character_iter(text)— iterator over the characters in text.classmethod characters(text)— a list of characters in text.checker = None— a language-specific checker instance (seefilters.checks). Doesn't need to be supplied, but will be used if it exists. 👑code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples:
km
pt_BR
sr_YU@Latncommonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — these punctuation marks are common in English and most languages that use the latin script.ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages.fullname = ''— the full (English) name of this language. 💖 Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests={'all': ['simplecaps', 'startcaps']}— your lil' skip-list of tests, either per checker or for literally everybody 🫶
Keys are checker names, and the values are lists of the names of the ignored tests in that checker. There's a special 'all' checker name you can use to say the tests must be ignored in all the checkers.
The checkers you list must be lowercase strings for the checker name — for example "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker.
indicpunc='।॥॰'— the punctuation marks several Indic languages use. respect the classics ✨invertedpunc='¿¡'— inverted punctuation, sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. the little upside-down besties 🙃classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of length length.listseperator='، '— the string used to separate lists of textual elements.
Most languages can probably just stick with the default comma, but Arabic and some Asian languages might want to override this. (Yes, it's spelled listseperator. That's the name. We love her anyway 💅)
miscpunc='…±°¹²³·©®×£¥€'— the odds-and-ends pile of punctuation.
Fun fact: the middle dot (·) is used by Greek and Georgian. iykyk ⭐
mozilla_pluralequation='0'— for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals=0— the number of plural forms of this language.
🚨0is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between1and6).
- SEE ALSO:
translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple— a tuple of number transformation rules thatnumbertranslate()can use. For this one it's the whole digit-swap situation:
(('0', '٠'), ('1', '١'), ('2', '٢'), ('3', '٣'), ('4', '٤'),
('5', '٥'), ('6', '٦'), ('7', '٧'), ('8', '٨'), ('9', '٩'),
('0', '۰'), ('1', '۱'), ('2', '۲'), ('3', '۳'), ('4', '۴'),
('5', '۵'), ('6', '۶'), ('7', '۷'), ('8', '۸'), ('9', '۹'))classmethod numstart(text)— figures out whether the text starts with a numeric value.pluralequation='0'— the plural equation used to select plural forms.
This is what gets filled into the header for PO files. header duty 🎀
- SEE ALSO: Gettext manual,
translate.lang.data
puncdict={',': '،', ';': '؛', '?': '؟'}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— implements “French” quotation marks. très chic 🥖punctuation— the big ol' grab-bag of everything punctuation-ish:
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'We include many types of punctuation here simply because this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won't need to override this. lowkey a safety net 🦋
quotes='‘’‛“”„‟′″‴‵‶‷‹›«»'— these are the different quotation marks used by various languages.rtlpunc='،؟؛÷'— punctuation marks used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend='.!?…։؟।。！？።۔'— these marks can indicate a sentence end.
Once again we try to account for many languages. Most langauges won't need to override this. covering all the bases fr fr 💖
classmethod sentences(text, strip=True)— returns a list of sentences in text. Same energy assentence_iter(), just the listy version ✨specialchars=''— characters used by the language that might not be easy to input with common keyboard layouts.validaccel=None— the characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator.
These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they would be at least precomposed characters. All characters, lower and upper, are included in the list.
validdoublewords=[]— some languages allow double words in certain cases. This is a dictionary of such words. 👑classmethod word_iter(text)— returns an iterator over the words in text.classmethod words(text)— returns a list of words in text. word girl summer 🌈
🧚 fi
This module represents the Finnish language. moi moi 🇫🇮
✨ class translate.lang.fi.fi(code) ✨
This class represents Finnish. Let's get into her whole personality:
CJKpunc='。、，；！？「」『』【】'— punctuation marks used in certain circumstances with CJK languages.classmethod alter_length(text)— converts the given string by adding or removing characters, as an estimation of translation length (with English assumed as the source language).classmethod capsstart(text)— determines whether the text starts with a capital letter.classmethod character_iter(text)— returns an iterator over the characters in text.classmethod characters(text)— returns a list of characters in text.checker=None— a language-specific checker instance (seefilters.checks).
This doesn't need to be supplied, but it will be used if it exists.
code=''— the ISO 639 language code, possibly with a country specifier or other modifier.
Examples:
km
pt_BR
sr_YU@Latncommonpunc— the punctuation marks that are common in English and most languages that use the latin script:
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc='።፤፣'— punctuation marks used by several Ethiopic languages.fullname=''— the full (English) name of this language.
Dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
🙈 yeah the TODO is literally in there, we're not hiding it
ignoretests={}— dictionary of tests to ignore in some or all checkers.
Keys are checker names and values are lists of names for the ignored tests in the checker. A special 'all' checker name can be used to say that the tests must be ignored in all the checkers.
Listed checkers to ignore tests on must be lowercase strings for the checker name, for example "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker.
indicpunc='।॥॰'— punctuation marks used by several Indic languages.invertedpunc='¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan.classmethod length_difference(length)— returns an estimate to a likely change in length relative to an English string of length length.listseperator=', '— this string is used to separate lists of textual elements.
Most languages probably can stick with the default comma, but Arabic and some Asian languages might want to override this.
miscpunc='…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian.mozilla_pluralequation='0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals=0— the number of plural forms of this language.
💀0is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between1and6).
- SEE ALSO:
translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple=()— a tuple of number transformation rules that can be used bynumbertranslate().classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation='0'— the plural equation for selection of plural forms.
This is used for PO files to fill into the header.
- SEE ALSO: Gettext manual,
translate.lang.data
puncdict={}— a dictionary of punctuation transformation rules that can be used bypunctranslate().classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language. note: this one is the plain version, no French quotes here 🎀punctuation— same enormous punctuation pile as ever:
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'We include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won't need to override this. ⭐
quotes='‘’‛“”„‟′″‴‵‶‷‹›«»'— these are different quotation marks used by various languages.rtlpunc='،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences intext. 🫶sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end. Once again we’re trying to cover a whole bunch of languages; most languages won’t need to override this. ✨classmethod sentences(text, strip=True)— returns a list of sentences intext. 💖specialchars = ''— characters used by the language that might not be easy to type with common keyboard layouts. 🎀validaccel = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890äöÄÖ'— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they’re accessible from the user’s keyboard in a single keystroke, but normally they’d be at least precomposed characters. All characters, lower and upper, are included in the list. 🔥validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words. 🦋classmethod word_iter(text)— returns an iterator over the words intext. 🌸classmethod words(text)— returns a list of words intext. 💅
fr 🎀
This module represents the French language. slay. 💖
SEE ALSO: French_language
✨ class translate.lang.fr.fr(code) ✨
This class represents French. 👑
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages. 🀄classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as source language). 📏classmethod capsstart(text)— determines whether the text starts with a capital letter. 🔠classmethod character_iter(text)— returns an iterator over the characters intext. 🧚classmethod characters(text)— returns a list of characters intext. ✨checker = None— a language-specific checker instance (seefilters.checks). This doesn’t need to be supplied, but it will be used if it exists. 🕵️code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples:kmpt_BRsr_YU@Latn
commonpunc— these punctuation marks are common in English and most languages that use latin script. The exact value:
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages. 🌍fullname = ''— the full (English) name of this language. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests = {}— dictionary of tests to ignore in some or all checkers. Keys are checker names and values are lists of names for the ignored tests in the checker. A special'all'checker name can be used to say the tests must be ignored in all checkers. Listed checkers to ignore tests on must be lowercase strings for the checker name — for example"mozilla"forMozillaCheckeror"libreoffice"forLibreOfficeChecker. 🫡indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages. 🪷invertedpunc = '¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. 🙃classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of length length. 📐listseperator = ', '— this string is used to separate lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this. 📝miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian. 🎯mozilla_pluralequation = '0'— for languages that have a different plural formula in Mozilla than the standard one in Gettext. 🦎nplurals = 0— the number of plural forms of this language.
0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language. 🔢numbertuple = ()— a tuple of number transformation rules that can be used bynumbertranslate(). 🧮classmethod numstart(text)— determines whether the text starts with a numeric value. 🔟pluralequation = '0'— the plural equation for selection of plural forms. This is used for PO files to fill into the header. SEE ALSO: Gettext manual,translate.lang.datapuncdict = {'!': '\xa0!', '#': '\xa0#', ':': '\xa0:', ';': '\xa0;', '?': '\xa0?'}— a dictionary of punctuation transformation rules that can be used bypunctranslate(). 🎀classmethod punctranslate(text)— implements some extra features for quotation marks. Known shortcomings:%and$are not touched yet for fear of variables.- Double spaces might be introduced.
punctuation— we include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won’t need to override this. The exact value:
punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are different quotation marks used by various languages. 💬rtlpunc = '،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example. 🌙classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences intext. 🫶sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end. Once again we try to account for many languages. Most languages won’t need to override this. ✨classmethod sentences(text, strip=True)— returns a list of sentences intext. 💖specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts. 🎀validaccel = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890éÉ'— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they’re accessible from the user’s keyboard in a single keystroke, but normally they’d be at least precomposed characters. All characters, lower and upper, are included in the list. 🔥validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words. 🦋classmethod word_iter(text)— returns an iterator over the words intext. 🌸classmethod words(text)— returns a list of words intext. 💅
gu 🦋
This module represents the Gujarati language. 💚
SEE ALSO: Gujarati_language
✨ class translate.lang.gu.gu(code) ✨
This class represents Gujarati. 👑
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages. 🀄classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as source language). 📏classmethod capsstart(text)— determines whether the text starts with a capital letter. 🔠classmethod character_iter(text)— returns an iterator over the characters intext. 🧚classmethod characters(text)— returns a list of characters intext. ✨checker = None— a language-specific checker instance (seefilters.checks). This doesn’t need to be supplied, but it will be used if it exists. 🕵️code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples:kmpt_BRsr_YU@Latn
commonpunc— these punctuation marks are common in English and most languages that use latin script. The exact value:
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages. 🌍fullname = ''— the full (English) name of this language. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers. 🫡
Keys are checker names, and the values are lists of names for the tests you want ignored in that particular checker. There's a little cheat code though: the special 'all' checker name tells everything to ignore those tests in every single checker, no questions asked. 🫶
Checkers listed for test-ignoring have to be lowercase strings for the checker name — like "mozilla" for MozillaChecker, or "libreoffice" for LibreOfficeChecker. lowercase only, bestie, iykyk. 💅
indicpunc = '।॥॰'— punctuation marks several Indic languages use. 🪷invertedpunc = '¿¡'— the flipped punctuation that sometimes kicks off a sentence in Spanish, Asturian, Galician, and Catalan. it's giving upside-down slay ✨classmethod length_difference(length)— hands you an estimate of how much the length will likely change compared to an English string of length length.listseperator = ', '— the string used to separate lists of textual elements. most languages can just chill with the default comma, but Arabic and some Asian languages might want to override this one.miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian. tiny dot, big responsibilities. 🎀mozilla_pluralequation = '0'— this one's for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms in this language.
0 is not a valid value — you have to override it. any positive integer is valid (and honestly it should probably be between 1 and 6).- SEE ALSO:
translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string following the rules of the language.numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use. 🧮classmethod numstart(text)— figures out whether the text starts with a numeric value.pluralequation = '0'— the plural equation used to select plural forms.
This is what gets filled into the header of PO files. 📝
- SEE ALSO: Gettext manual,
translate.lang.data
puncdict = {}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation =— we throw a whole bunch of punctuation types in here, simply because this is only meant to determine if something is punctuation. hopefully we catch some languages that might not be represented with modules. most languages won't need to override this. 🌈
punctuation =
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— the different quotation marks various languages use. so many little curly friends 🦋rtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end. once again we're trying to account for many languages (yes, the typo'd "langauges" energy is real 🌸). most languages won't need to override this.classmethod sentences(text, strip=True)— returns a list of sentences in text.specialchars = ''— characters the language uses that might not be easy to type on common keyboard layouts.validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. these can include combining diacritics as long as they're reachable from the user's keyboard in a single keystroke — but normally they'd be at least precomposed characters. all characters, lower and upper, are included in the list. ⌨️validdoublewords = []— some languages allow double words in certain cases. this is a dictionary of such words. 👀classmethod word_iter(text)— returns an iterator over the words in text.classmethod words(text)— returns a list of words in text.
🌸 he 🌸
This module represents the Hebrew language. ✡️💙
- SEE ALSO:
Hebrew_language
translate.lang.he.he(code)
This class represents Hebrew.
CJKpunc = '。、，；！？「」『』【】'— punctuation marks used in certain circumstances with CJK languages. yes, even here, don't ask, just vibe 🎐classmethod alter_length(text)— converts the given string by adding or removing characters, as an estimate of translation length (English is assumed to be the source language).classmethod capsstart(text)— determines whether the text starts with a capital letter. 🔠classmethod character_iter(text)— returns an iterator over the characters in text.classmethod characters(text)— returns a list of characters in text.checker = None— a language-specific checker instance (seefilters.checks).
You don't have to supply this, but it'll be used if it exists. 🕵️♀️
code = ''— the ISO 639 language code, possibly with a country specifier or other modifier.
Examples:
km
pt_BR
sr_YU@Latncommonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — the punctuation marks that are common in English and most languages using the Latin script.ethiopicpunc = '።፤፣'— punctuation marks several Ethiopic languages use. 🌍fullname = ''— the full (English) name of this language.
Dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests = {'all': ['acronyms', 'simplecaps', 'startcaps']}— a dictionary of tests to ignore in some or all checkers. 🙈
Keys are checker names, values are the list of names for the ignored tests in that checker. The special 'all' checker name means the tests must be ignored in all the checkers.
Listed checkers to ignore tests on must be lowercase strings for the checker name, for example "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker.
indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages.invertedpunc = '¿¡'— inverted punctuation, sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan.classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of length length. 📏listseperator = ', '— this string separates lists of textual elements. most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this.miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian. iconic little dot honestly 💫mozilla_pluralequation = '0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).- SEE ALSO:
translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use.classmethod numstart(text)— determines whether the text starts with a numeric value. 🔢pluralequation = '0'— the plural equation for selection of plural forms.
This gets used for PO files, to fill into the header.
- SEE ALSO: Gettext manual,
translate.lang.data
puncdict = {}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation =— we include many types of punctuation here, simply since this is only meant to determine whether something is punctuation. hopefully we catch some languages that might not be represented with modules. most languages won't need to override this.
punctuation =
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are the different quotation marks used by various languages. curly quotes nation rise up 🎀rtlpunc = '،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end 👑 once again we're trying to account for a whole bunch of languages, because of course we are. Most languages won't need to override this, ngl.classmethod sentences(text, strip=True)— returns a list of sentences intext. Yes bestie, it just... gives you the sentences back 💅specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts 🎀validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they would be at least precomposed characters. All characters, lower and upper, are included in the list ✨validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words 🦋classmethod word_iter(text)— returns an iterator over the words intext. Lazy girl math, but for words 💜classmethod words(text)— returns a list of words intext. Same energy, but it wants everything upfront 🔥
🌸 hy
This module represents the Armenian language.
SEE ALSO: Armenian_language
class translate.lang.hy.hy(code) — this class represents Armenian. Main character energy fr fr 👑
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages. Like, very specific circumstances 💅classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language) 🎀classmethod capsstart(text)— determines whether the text starts with a capital letter. Detective mode: activated 🔍classmethod character_iter(text)— returns an iterator over the characters intext✨classmethod characters(text)— returns a list of characters intext. Iconic, no notes 🌟checker = None— a language specific checker instance (seefilters.checks). This doesn't need to be supplied, but it will be used if it exists 🫶code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples, because we love a demo:
km
pt_BR
sr_YU@Latncommonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — these punctuation marks are common in English and most languages that use latin script 🌈ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages 🦋fullname = ''— the full (English) name of this language. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
(yes, that last one is literally a TODO in the list, we're keeping it, don't come for me 😭)
ignoretests = {'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers. Keys are checker names and values are lists of names for the ignored tests in the checker. A special ‘all’ checker name can be used to tell that the tests must be ignored in all the checkers. Listed checkers to ignore tests on must be lowercase strings for the checker name — for example"mozilla"forMozillaChecker, or"libreoffice"forLibreOfficeChecker💖indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages ❤️🔥invertedpunc = '¿¡'— the little inverted punctuation marks that sometimes open a sentence in Spanish, Asturian, Galician, and Catalan 🫶classmethod length_difference(length)— gives you an estimate of how much a translation is likely to grow or shrink compared to an English string of length length ✨listseperator = ', '— the string used to separate lists of textual elements. Most languages can absolutely just keep the default comma, but Arabic and some Asian languages might want to override this 💅miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian, no cap.mozilla_pluralequation = 'n!=1 ? 1 : 0'— for languages that have a different plural formula in Mozilla than the standard one in Gettext 👑nplurals = 0— the number of plural forms of this language.- 🚨
0is not a valid value — it must be overridden. Any positive integer is valid (and it should probably be between1and6). - SEE ALSO:
translate.lang.data
- 🚨
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language 🔢numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use.classmethod numstart(text)— figures out whether the text starts with a numeric value.pluralequation = '0'— the plural equation used to select plural forms. This is what gets filled into the header of PO files 🎀- SEE ALSO: Gettext manual,
translate.lang.data
- SEE ALSO: Gettext manual,
puncdict = {'!': '՜', '.': '։', ':': '՝', '?': '՞'}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language ✨punctuation— we include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages that might not be represented with modules. Most languages won't need to override this:
punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»…±°¹²³·©®×£¥€։՝՜՞'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are the different quotation marks used by various languages 🦋rtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text 🌸sentenceend = '։՝՜…'— these marks can indicate a sentence end. Once again we're trying to account for many languages. Most languages won't need to override this.classmethod sentences(text, strip=True)— returns a list of sentences in text.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts ⌨️validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list 💖validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words in text.classmethod words(text)— returns a list of words in text.
🌈 identify
This module is literally the language-detective of the family — it contains functions for identifying languages based on language models 🕵️♀️
🍓 ja
This module represents the Japanese language 🇯🇵
SEE ALSO: Japanese_language
class translate.lang.ja.ja(code)
This class represents Japanese. 日本語の時間だよ〜 ✨
CJKpunc = '。、，；！？「」『』【】'— punctuation marks used in certain circumstances with CJK languages.classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language) 📏classmethod capsstart(text)— determines whether the text starts with a capital letter.classmethod character_iter(text)— returns an iterator over the characters in text.classmethod characters(text)— returns a list of characters in text.checker = None— a language-specific checker instance (seefilters.checks). You don't have to supply this, but it will be used if it exists 👑code = ''— the ISO 639 language code, possibly with a country specifier or other modifier.
Examples:
km
pt_BR
sr_YU@Latncommonpunc— these punctuation marks are common in English and most languages that use the latin script:
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc = '።፤፣'— punctuation marks used by several Ethiopic languages.fullname = ''— the full (English) name of this language.
Dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— a dictionary of tests to ignore in some or all checkers. Keys are checker names and values are the list of names of the ignored tests in that checker. You can use the special'all'checker name to say the tests must be ignored in all the checkers.- Listed checkers to ignore tests on must be lowercase strings for the checker name — for example
"mozilla"for MozillaChecker or"libreoffice"for LibreOfficeChecker 💅
- Listed checkers to ignore tests on must be lowercase strings for the checker name — for example
indicpunc = '।॥॰'— punctuation marks used by several Indic languages.invertedpunc = '¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan.classmethod length_difference(length)— returns an estimate of a likely change in length relative to an English string of length length.listseperator = '、､，,'— the string used to separate lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this.miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian.mozilla_pluralequation = '0'— for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.- 🚨
0is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between1and6). - SEE ALSO:
translate.lang.data
- 🚨
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple = ()— a tuple of number transformation rules that can be used bynumbertranslate().classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms. This is used for PO files to fill into the header.- SEE ALSO: Gettext manual,
translate.lang.data
- SEE ALSO: Gettext manual,
puncdict = {',\n': '、\n', ', ': '、', '.\n': '。\n', '. ': '。'}— a dictionary of punctuation transformation rules that can be used bypunctranslate().classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation— we include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won't need to override this:
punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are the different quotation marks used by various languages.rtlpunc = '،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend = '。｡！？!?…'— these marks can indicate a sentence end. Once again we try to account for many languages. Most languages won't need to override this.classmethod sentences(text, strip=True)— returns a list of sentences in text.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts 🎀validaccel=None— the chars that can be used as accelerators, aka access keys, aka theXin Alt+X 💅. These can include combining diacritics as long as they're reachable from the user's keyboard in a single keystroke — but normally they'd at least be precomposed characters. All the characters, lower and upper, are included in the list. Periodt.validdoublewords=[]— some languages allow double words in certain cases, and this is a dictionary of such words. 🫶classmethod word_iter(text)— hands you an iterator over the words intext. ✨classmethod words(text)— hands you a list of the words intext, no iterator energy, just the list. 🎀
🦋 km
This module represents the Khmer language.
SEE ALSO: Khmer_language
class translate.lang.km.km(code) — this class represents Khmer. 👑
And here's everything it's carrying:
CJKpunc='。、，；！？「」『』【】'— these punctuation marks get used in certain circumstances with CJK languages. 🀄classmethod alter_length(text)— converts the given string by adding or removing characters, as an estimation of translation length (with English assumed as the source language). So like, a length vibe check. 📏classmethod capsstart(text)— determines whether the text starts with a capital letter.classmethod character_iter(text)— returns an iterator over the characters intext.classmethod characters(text)— returns a list of the characters intext. 💜checker = None— a language-specific checker instance (seefilters.checks). It doesn't need to be supplied, but it will be used if it exists. Lowkey optional, highkey useful.code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. For example:
km
pt_BR
sr_YU@Latncommonpunc—'.,;:!?-@#$%^*_()[]{}/\\\'`"<>'— these punctuation marks are common in English and most languages that use latin script.ethiopicpunc='።፤፣'— these punctuation marks are used by several Ethiopic languages.fullname = ''— the full (English) name of this language. 🌈 Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers. 🔥 Keys are checker names, values are lists of names of the ignored tests in that checker. A special'all'checker name tells you the tests must be ignored in all the checkers. Listed checkers to ignore tests on must be lowercase strings for the checker name — for example"mozilla"forMozillaChecker, or"libreoffice"forLibreOfficeChecker. iykyk.indicpunc='।॥॰'— these punctuation marks are used by several Indic languages.invertedpunc='¿¡'— inverted punctuation, sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. 🙃khmerpunc='។៕៖៘'— these marks are only used for Khmer. exclusive 💅classmethod length_difference(length)— returns an estimate of a likely change in length, relative to an English string of lengthlength.listseperator=', '— this string separates lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this. (yes, that's the attribute name, don't come for me 🫶)miscpunc='…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian. cute.mozilla_pluralequation='n!=1 ? 1 : 0'— for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
🚨 0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language. 🔢numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use.classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms. This is used for PO files, to fill into the header. 📄
SEE ALSO: Gettext manual, translate.lang.data
puncdict={'!': '\xa0!', '.': '\xa0។', ':': '\xa0៖', '?': '\xa0?'}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language. ✨punctuation—'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»…±°¹²³·©®×£¥€។៕៖៘'— we include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won't need to override this. 🎀quotes='‘’‛“”„‟′″‴‵‶‷‹›«»'— the different quotation marks used by various languages.rtlpunc='،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences intext.sentenceend='!?…។៕៘'— these marks can indicate a sentence end. Once again we're trying to account for many languages. Most languages won't need to override this. 💖classmethod sentences(text, strip=True)— returns a list of sentences intext.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts. the struggle 😭validaccel=None— the characters that can be used as accelerators (access keys), i.e. Alt+X whereXis the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list.validdoublewords=[]— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words intext.classmethod words(text)— returns a list of the words intext. 🦋
🌸 kn
This module represents the Kannada language.
SEE ALSO: Kannada_language
class translate.lang.kn.kn(code) — this class represents Kannada. 👑
CJKpunc='。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages.classmethod alter_length(text)— converts the given string by adding or removing characters, as an estimation of translation length (with English assumed as the source language).classmethod capsstart(text)— determines whether the text starts with a capital letter. 🔤classmethod character_iter(text)— returns an iterator over the characters intext.classmethod characters(text)— returns a list of the characters intext.checker = None— a language specific checker instance (seefilters.checks). This doesn't need to be supplied, but it will be used if it exists. 🫶code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples:
km
pt_BR
sr_YU@Latncommonpunc—'.,;:!?-@#$%^*_()[]{}/\\\'`"<>'— these punctuation marks are common in English and most languages that use latin script.ethiopicpunc='።፤፣'— these punctuation marks are used by several Ethiopic languages.fullname = ''— the full (English) name of this language. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers. Keys are checker names, values are lists of names of the ignored tests in the checker. 💀 A special'all'checker name can be used to say the tests must be ignored in all the checkers. Listed checkers to ignore tests on must be lowercase strings for the checker name — for example"mozilla"forMozillaChecker, or"libreoffice"forLibreOfficeChecker.indicpunc='।॥॰'— these punctuation marks are used by several Indic languages.invertedpunc='¿¡'— inverted punctuation, sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan.classmethod length_difference(length)— returns an estimate to a likely change in length, relative to an English string of lengthlength. 📐listseperator = ', '— the string that glues lists of text bits together. Ngl most languages can just vibe with the default comma, but Arabic and some Asian languages are gonna wanna override this one. (And yes, the attribute really is spelledlistseperator— that's not a typo on my end, don't come for me 💅)miscpunc = '…±°¹²³·©®×£¥€'— the odds-and-ends punctuation pile. That middle dot (·) is doing main character work for Greek and Georgian, bestie.mozilla_pluralequation = '0'— for languages whose Mozilla plural formula is different from the standard Gettext one. 🦋nplurals = 0— how many plural forms this language has.
Heads up, bestie:0is not a valid value — it must be overridden. Any positive integer is fine (and fr fr it should probably be between1and6).
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language. Numbers but make it localized ✨numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can snack on.classmethod numstart(text)— figures out whether the text starts with a numeric value. Like, does it begin with digits, yes or no 🎀pluralequation = '0'— the plural equation used to select plural forms.
This is the one that gets filled into the header of PO files. 👑
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— converts the punctuation in a string according to the language's rules. Swaps your.for a。, that kind of slay.punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'` — okay so we shoved so many kinds of punctuation in here, simply because this attribute only exists to answer "is this thing punctuation?" Hoping we catch some languages that might not even have modules yet 🫶 Most languages won't need to override this.quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— all the different quotation marks various languages are out here using. Iconic variety, tbh.rtlpunc = '،؟؛÷'— the punctuation marks used by Arabic and Persian, for example. Right-to-left legends 🌙classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences intext. Lazy, on-demand sentence energy 💜sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate the end of a sentence. Once again we're trying to cover as many languages as possible; most languages won't need to override this.classmethod sentences(text, strip=True)— returns a list of sentences intext. Same vibe as above, just all at once instead of one at a time.specialchars = ''— characters the language uses that might not be easy to type on common keyboard layouts. The struggle 😭validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're reachable from the user's keyboard in a single keystroke — but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list. No cap. 🔥validdoublewords = []— some languages allow double words in certain cases, and this is the dictionary of such words. Double trouble, but make it linguistics.classmethod word_iter(text)— returns an iterator over the words intext. Word by word, no rush 🧚classmethod words(text)— returns a list of words intext. All of them at once, periodt.
🌸 ko
This module represents the Korean language. 🇰🇷
SEE ALSO: Korean_language
💖 class translate.lang.ko.ko(code)
This class represents Korean.
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks get used in certain circumstances with CJK languages. CJK punctuation is a whole aesthetic, honestly.classmethod alter_length(text)— massages the given string by adding or removing characters, as an estimation of translation length (with English assumed as the source language). It's giving "ballpark figure" energy.classmethod capsstart(text)— determines whether the text starts with a capital letter. Simple. Effective. Slay.classmethod character_iter(text)— returns an iterator over the characters intext.classmethod characters(text)— returns a list of characters intext.checker = None— a language-specific checker instance (seefilters.checks).
You don't have to supply this, but it'll get used if it exists. Optional but iconic.
code = ''— the ISO 639 language code, possibly with a country specifier or some other modifier.
Examples:
kmpt_BRsr_YU@Latn
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — the punctuation marks that are common in English and most languages using the latin script. The default girlies.ethiopicpunc = '።፤፣'— punctuation marks used by several Ethiopic languages. 🌈fullname = ''— the full (English) name of this language.
Dialect codes should have this form:
- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers.
Keys are checker names and values are the list of names of the ignored tests inside that checker. There's a special 'all' checker name you can use to say the tests must be ignored across every checker.
Any checker names you list must be lowercase strings — like "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker. Lowercase or it doesn't count, bestie 💅
indicpunc = '।॥॰'— punctuation marks used by several Indic languages. 🌺invertedpunc = '¿¡'— the inverted punctuation sometimes used at the start of sentences in Spanish, Asturian, Galician, and Catalan. Upside-down and thriving 🙃classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of lengthlength.listseperator = ', '— the string used to separate lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might wanna override this.miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian, in case you were wondering why it's here.mozilla_pluralequation = '0'— for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
0is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between1and6).
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language. ⭐numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use.classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation used to select plural forms.
This is used for PO files, to fill into the header.
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'` — we include many types of punctuation here, simply since this is only meant to determine whether something is punctuation. Hopefully we catch some languages that might not be represented with modules. Most languages won't need to override this.quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— the different quotation marks used by various languages.rtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences intext.sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end. Once again we try to account for many languages; most languages won't need to override this.classmethod sentences(text, strip=True)— returns a list of sentences intext.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts.validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list. 🎀validdoublewords = []— some languages allow double words in certain cases; this is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words intext.classmethod words(text)— returns a list of words intext. 🦋words(text)(classmethod) — hands you back a list of the words in text. that's it, that's the whole job bestie 💅
🌸 mr 🌸
this module represents the Marathi language, ok and it ate fr.
SEE ALSO: Marathi_language
👑 class translate.lang.mr.mr(code)
this class represents Marathi. main character energy only.
CJKpunc='。、，；！？「」『』【】'— these punctuation marks get used in certain circumstances with CJK languages. yes we're keeping them around even here, no cap.alter_length(text)(classmethod) — converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language). it's giving "how much longer/shorter will this get" energy ✨capsstart(text)(classmethod) — determines whether the text starts with a capital letter. detective work but make it typography 🕵️♀️character_iter(text)(classmethod) — returns an iterator over the characters in text.characters(text)(classmethod) — returns a list of characters in text. sibling energy to the one above, iykyk.checker=None— a language-specific checker instance (seefilters.checks). this doesn't need to be supplied, but it will be used if it exists. optional icon behavior 💖code=''— the ISO 639 language code, possibly with a country specifier or other modifier.
examples, so you know the vibe:
km
pt_BR
sr_YU@Latncommonpunc— these punctuation marks are common in English and most languages that use latin script:
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc='።፤፣'— these punctuation marks are used by several Ethiopic languages. 🫶fullname=''— the full (English) name of this language.
dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests={'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers.
keys are checker names and values are lists of names for the ignored tests in that checker. a special 'all' checker name can be used to tell you that the tests must be ignored in all the checkers.
listed checkers to ignore tests on must be lowercase strings for the checker name, like "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker. lowercase era, periodt.
indicpunc='।॥॰'— these punctuation marks are used by several Indic languages.invertedpunc='¿¡'— inverted punctuation, sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. upside-down slay 🙃length_difference(length)(classmethod) — returns an estimate to a likely change in length relative to an English string of length length.listseperator=', '— this string is used to separate lists of textual elements. most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this. (yes it's spelledlistseperator, don't come for me, that's literally the name 💀)miscpunc='…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian.mozilla_pluralequation='0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals=0— the number of plural forms of this language.
⚠️ 0 is not a valid value — it must be overridden. any positive integer is valid (it should probably be between 1 and 6). so like… please actually set it bestie, don't leave it at zero 😭
SEE ALSO: translate.lang.data
capsstart(text)(classmethod) — checks whether the text starts with a capital letter, bestie. Simple vibe check, big payoff 🎀character_iter(text)(classmethod) — hands you an iterator over the characters in text, one lil guy at a time ✨characters(text)(classmethod) — same energy but you get a whole list of the characters in text back 🦋checker— a language-specific checker instance (peepfilters.checksfor the deets). You don't have to supply it, but if it's there, it gets used. No pressure, it just slays when present 💅code— the ISO 639 language code, possibly with a country specifier or other modifier. Empty string by default. Examples of what this looks like:
km
pt_BR
sr_YU@Latncommonpunc— punctuation marks that are common in English and most languages using latin script:
.,;:!?-@#$%^*_()[]{}/\\\'`"<>ethiopicpunc— the punctuation squad used by several Ethiopic languages:
።፤፣fullname— the full (English) name of this language. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests— a dictionary of tests to ignore in some or all checkers 🙈
Keys are checker names and values are lists of names for the ignored tests in that checker. There's a special 'all' checker name you can use to say "hey, ignore these tests in every checker." Default:
{'all': ['accelerators', 'simplecaps', 'startcaps']}Listed checkers to ignore tests on must be lowercase strings for the checker name — for example "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker.
indicpunc— punctuation marks used by several Indic languages:
।॥॰invertedpunc— the flipped punctuation sometimes used at the start of sentences in Spanish, Asturian, Galician, and Catalan. Yes, upside down on purpose, it's giving icon behavior:
¿¡length_difference(length)(classmethod) — returns an estimate of the likely change in length relative to an English string of length length. Math era 🔥listseperator— the string used to separate lists of textual elements. Default is', '. Most languages can probably stick with the default comma, but Arabic and some Asian languages might wanna override this. (And yes, it's spelledlistseperatorin the source — we don't fix what isn't ours 😭)miscpunc— the miscellaneous punctuation pile. Fun fact: the middle dot (·) is used by Greek and Georgian:
…±°¹²³·©®×£¥€mozilla_pluralequation— for languages that have a different plural formula in Mozilla than the standard one in Gettext. Default is'0'.nplurals— the number of plural forms of this language. Defaults to0, and listen:
0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6). So yeah, don't leave it at zero, bestie 💀SEE ALSO: translate.lang.data
numbertranslate(text)(classmethod) — converts the numbers in a string according to the rules of the language 🔢numbertuple— a tuple of number transformation rules thatnumbertranslate()can use. Defaults to().numstart(text)(classmethod) — determines whether the text starts with a numeric value. Detective work, but make it numbers 👑pluralequation— the plural equation for selection of plural forms. Default is'0'. This is what gets used for PO files to fill into the header.
SEE ALSO: Gettext manual, translate.lang.data
puncdict— a dictionary of punctuation transformation rules thatpunctranslate()can use. Key/value pairs of "this mark" → "that mark", and it defaults to{}.punctranslate(text)(classmethod) — converts the punctuation in a string according to the rules of the language ✨punctuation— the whole punctuation buffet. Yes, we include many types here, simply because this is only meant to determine if something is punctuation. Hopefully we catch some languages that might not be represented with modules at all. Most languages won't need to override this:
.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€quotes— the different quotation marks various languages use. Look at them go 🎀
‘’‛“”„‟′″‴‵‶‷‹›«»rtlpunc— punctuation marks used by Arabic and Persian, for example:
،؟؛÷sentence_iter(text, strip=True)(classmethod) — returns an iterator over the sentences in text 📖sentenceend— the marks that can indicate a sentence end. Once again we're trying to account for a whole bunch of languages here; most languages won't need to override this.sentences(text, strip=True)(classmethod) — returns a list of sentences in text. Same deal assentence_iter(), just, like, collected 🌸specialchars— characters used by the language that might not be easy to input with common keyboard layouts. Defaults to empty.validaccel— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator 💅
These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke — but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list. Default is None.
validdoublewords— some languages allow double words in certain cases. This is a dictionary of such words. Defaults to[]. (The docs say "dictionary," the default is a list — we're just passing the info along, no cap 🫶)word_iter(text)(classmethod) — returns an iterator over the words in text ✨words(text)(classmethod) — returns a list of words in text. Team "just give me the list already" 🦋
And that's the whole Punjabi module, alphabetized and accounted for, every last attribute present and correct 💖
Next up: more language modules, because this package has so many of them and we're literally only getting started. Stay tuned bestie, the language tour continues 🌈
classmethod capsstart(text)— literally just tells you iftextkicks off with a capital letter. That's it, that's the whole vibe check 💅classmethod character_iter(text)— hands you back an iterator that walks over every character intext, one lil guy at a time 🎀classmethod characters(text)— same energy but it gives you a full-on list of the characters intext. No iterating required ✨checker = None— a language-specific checker instance (peepfilters.checksfor the deets). You don't have to supply one, but bestie, if it exists it absolutely gets used 🫶code = ''— the ISO 639 language code, and it can have a country specifier or some other modifier tacked on. Iconic examples:
km
pt_BR
sr_YU@Latncommonpunc— these punctuation marks are common in English and most languages that use latin script. The exact string is:
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc = '።፤፣'— punctuation marks used by several Ethiopic languages 🌸fullname = ''— the full (English) name of this language. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— a dictionary of tests to ignore in some or all checkers 🔥
The keys are checker names and the values are lists of names for the ignored tests in that checker. There's a special 'all' checker name you can use to say "these tests must be ignored in all the checkers."
The checker names you list to ignore tests on must be lowercase strings, like "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker. Lowercase or it's not gonna hit, iykyk.
indicpunc = '।॥॰'— punctuation marks used by several Indic languages ⭐invertedpunc = '¿¡'— the inverted punctuation sometimes used at the start of sentences in Spanish, Asturian, Galician, and Catalan. It's giving upside-down era 🙃classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string oflengthlength 📏listseperator = ', '— this string separates lists of textual elements. Most languages can probably just stick with the default comma, but Arabic and some Asian languages might want to override this. (And yes, that's how it's spelled — we don't fix names around here, we respect them 💜)miscpunc = '…±°¹²³·©®×£¥€'— a whole little grab bag of misc punctuation. Ngl the middle dot (·) is used by Greek and Georgian 👑mozilla_pluralequation = '0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
0 is not a valid value — it must be overridden. Any positive integer is valid (and it should probably be between 1 and 6).SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string following the rules of the language 🔢numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use.classmethod numstart(text)— tells you whethertextstarts with a numeric value.pluralequation = '0'— the plural equation used to select plural forms. This is what gets filled into the header for PO files 💖
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {'.\n': '।\n', '. ': '। '}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language ✨punctuation— we include so many types of punctuation here, simply because this is only meant to determine whether something counts as punctuation. Hopefully we catch some languages that might not be represented with modules at all. Most languages won't need to override this. The full value, no cap:
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— the different quotation marks various languages use 🎀rtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for example 🌙classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences intext.sentenceend = '।!?…'— these marks can indicate a sentence end. Once again we're out here trying to account for many languages. Most languages won't need to override this 🦋classmethod sentences(text, strip=True)— returns a list of the sentences intext.specialchars = ''— characters used by the language that might not be easy to type with common keyboard layouts ⌨️validaccel = None— the characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list 💅validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words intext.classmethod words(text)— returns a list of the words intext📝
🎨 poedit
Functions to manage Poedit's language features.
NOTE: The ISO 639 maps are from Poedit's isocode.cpp (v1.4.2) so we match the currently released versions of Poedit. Gotta stay in sync bestie 🫶
translate.lang.poedit.dialects
{'Chinese': {'CHINA': 'zh_CN', 'None': 'zh_CN', 'TAIWAN': 'zh_TW'}, 'English': {'None': 'en', 'SOUTH AFRICA': 'en_ZA', 'UNITED KINGDOM': 'en_GB'}, 'Portuguese': {'BRAZIL': 'pt_BR', 'None': 'pt', 'PORTUGAL': 'pt'}}Language dialects based on ISO 3166 country names — 'None' is the default fallback 🔥
translate.lang.poedit.isocode(language, country=None)
Returns a language code for the given Poedit language name.
Poedit uses language and country names in the PO header entries:
X-Poedit-LanguageX-Poedit-Country
This function converts the supplied language name into the required ISO 639 code. If needed — like, in the case of dialects — the country name gets used to create an xx_YY style dialect code ✨
Parameters
- language (String) — Language name
- country (String) — Country name
Returns
ISO 639 language code
Return type
String
translate.lang.poedit.lang_codes
{'aa': 'Afar', 'ab': 'Abkhazian', 'ae': 'Avestan', 'af': 'Afrikaans', 'am': 'Amharic', 'ar': 'Arabic', 'as': 'Assamese', 'ay': 'Aymara', 'az': 'Azerbaijani', 'ba': 'Bashkir', 'be': 'Belarusian', 'bg': 'Bulgarian', 'bh': 'Bihari', 'bi': 'Bislama', 'bn': 'Bengali', 'bo': 'Tibetan', 'br': 'Breton', 'bs': 'Bosnian', 'ca': 'Catalan', 'ce': 'Chechen', 'ch': 'Chamorro', 'co': 'Corsican', 'cs': 'Czech', 'cu': 'Church Slavic', 'cv': 'Chuvash', 'cy': 'Welsh', 'da': 'Danish', 'de': 'German', 'dz': 'Dzongkha', 'el': 'Greek', 'en': 'English', 'eo': 'Esperanto', 'es': 'Spanish', 'et': 'Estonian', 'eu': 'Basque', 'fa': 'Persian', 'fi': 'Finnish', 'fj': 'Fijian', 'fo': 'Faroese', 'fr': 'French', 'fur': 'Friulian', 'fy': 'Frisian', 'ga': 'Irish', 'gd': 'Gaelic', 'gl': 'Galician', 'gn': 'Guarani', 'gu': 'Gujarati', 'ha': 'Hausa', 'he': 'Hebrew', 'hi': 'Hindi', 'ho': 'Hiri Motu', 'hr': 'Croatian', 'hu': 'Hungarian', 'hy': 'Armenian', 'hz': 'Herero', 'ia': 'Interlingua', 'id': 'Indonesian', 'ie': 'Interlingue', 'ik': 'Inupiaq', 'is': 'Icelandic', 'it': 'Italian', 'iu': 'Inuktitut', 'ja': 'Japanese', 'jw': 'Javanese', 'ka': 'Georgian', 'ki': 'Kikuyu', 'kj': 'Kuanyama', 'kk': 'Kazakh', 'kl': 'Kalaallisut', 'km': 'Khmer', 'kn': 'Kannada', 'ko': 'Korean', 'ks': 'Kashmiri', 'ku': 'Kurdish', 'kv': 'Komi', 'kw': 'Cornish', 'ky': 'Kyrgyz', 'la': 'Latin', 'lb': 'Letzeburgesch', 'ln': 'Lingala', 'lo': 'Lao', 'lt': 'Lithuanian', 'lv': 'Latvian', 'mg': 'Malagasy', 'mh': 'Marshall', 'mi': 'Maori', 'mk': 'Macedonian', 'ml': 'Malayalam', 'mn': 'Mongolian', 'mo': 'Moldavian', 'mr': 'Marathi', 'ms': 'Malay', 'mt': 'Maltese', 'my': 'Burmese', 'na': 'Nauru', 'nb': 'Norwegian Bokmal', 'ne': 'Nepali', 'ng': 'Ndonga', 'nl': 'Dutch', 'nn': 'Norwegian Nynorsk', 'nr': 'Ndebele, South', 'nv': 'Navajo', 'ny': 'Chichewa; Nyanja', 'oc': 'Occitan', 'om': '(Afan) Oromo', 'or': 'Oriya', 'os': 'Ossetian; Ossetic', 'pa': 'Panjabi', 'pi': 'Pali', 'pl': 'Polish', 'ps': 'Pashto, Pushto', 'pt': 'Portuguese', 'qu': 'Quechua', 'rm': 'Rhaeto-Romance', 'rn': 'Rundi', 'ro': 'Romanian', 'ru': 'Russian', 'rw': 'Kinyarwanda', 'sa': 'Sanskrit', 'sc': 'Sardinian', 'sd': 'Sindhi', 'se': 'Northern Sami', 'sg': 'Sangro', 'sh': 'Serbo-Croatian', 'si': 'Sinhalese', 'sk': 'Slovak', 'sl': 'Slovenian', 'sm': 'Samoan', 'sn': 'Shona', 'so': 'Somali', 'sq': 'Albanian', 'sr': 'Serbian', 'ss': 'Siswati', 'st': 'Sesotho', 'su': 'Sundanese', 'sv': 'Swedish', 'sw': 'Swahili', 'ta': 'Tamil', 'te': 'Telugu', 'tg': 'Tajik', 'th': 'Thai', 'ti': 'Tigrinya', 'tk': 'Turkmen', 'tl': 'Tagalog', 'tn': 'Setswana', 'to': 'Tonga', 'tr': 'Turkish', 'ts': 'Tsonga', 'tt': 'Tatar', 'tw': 'Twi', 'ty': 'Tahitian', 'ug': 'Uighur', 'uk': 'Ukrainian', 'ur': 'Urdu', 'uz': 'Uzbek', 'vi': 'Vietnamese', 'vo': 'Volapuk', 'wa': 'Walloon', 'wo': 'Wolof', 'xh': 'Xhosa', 'yi': 'Yiddish', 'yo': 'Yoruba', 'za': 'Zhuang', 'zh': 'Chinese', 'zu': 'Zulu'}ISO369 codes and names as used by Poedit. Mostly these are identical to ISO 639, but there are some differences — so don't just assume, okay? 🙈
translate.lang.poedit.lang_names = {'(Afan) Oromo': 'om', 'Abkhazian':
'ab', 'Afar': 'aa', 'Afrikaans': 'af', 'Albanian': 'sq', 'Amharic':
'am', 'Arabic': 'ar', 'Armenian': 'hy', 'Assamese': 'as', 'Avestan':
'ae', 'Aymara': 'ay', 'Azerbaijani': 'az', 'Bashkir': 'ba', 'Basque':
'eu', 'Belarusian': 'be', 'Bengali': 'bn', 'Bihari': 'bh', 'Bislama':
'bi', 'Bosnian': 'bs', 'Breton': 'br', 'Bulgarian': 'bg', 'Burmese':
'my', 'Catalan': 'ca', 'Chamorro': 'ch', 'Chechen': 'ce', 'Chichewa;
Nyanja': 'ny', 'Chinese': 'zh', 'Church Slavic': 'cu', 'Chuvash': 'cv',
'Cornish': 'kw', 'Corsican': 'co', 'Croatian': 'hr', 'Czech': 'cs',
'Danish': 'da', 'Dutch': 'nl', 'Dzongkha': 'dz', 'English': 'en',
'Esperanto': 'eo', 'Estonian': 'et', 'Faroese': 'fo', 'Fijian': 'fj',
'Finnish': 'fi', 'French': 'fr', 'Frisian': 'fy', 'Friulian': 'fur',
'Gaelic': 'gd', 'Galician': 'gl', 'Georgian': 'ka', 'German': 'de',
'Greek': 'el', 'Guarani': 'gn', 'Gujarati': 'gu', 'Hausa': 'ha',
'Hebrew': 'he', 'Herero': 'hz', 'Hindi': 'hi', 'Hiri Motu': 'ho',
'Hungarian': 'hu', 'Icelandic': 'is', 'Indonesian': 'id', 'Interlingua':
'ia', 'Interlingue': 'ie', 'Inuktitut': 'iu', 'Inupiaq': 'ik', 'Irish':
'ga', 'Italian': 'it', 'Japanese': 'ja', 'Javanese': 'jw',
'Kalaallisut': 'kl', 'Kannada': 'kn', 'Kashmiri': 'ks', 'Kazakh': 'kk',
'Khmer': 'km', 'Kikuyu': 'ki', 'Kinyarwanda': 'rw', 'Komi': 'kv',
'Korean': 'ko', 'Kuanyama': 'kj', 'Kurdish': 'ku', 'Kyrgyz': 'ky',
'Lao': 'lo', 'Latin': 'la', 'Latvian': 'lv', 'Letzeburgesch': 'lb',
'Lingala': 'ln', 'Lithuanian': 'lt', 'Macedonian': 'mk', 'Malagasy':
'mg', 'Malay': 'ms', 'Malayalam': 'ml', 'Maltese': 'mt', 'Maori': 'mi',
'Marathi': 'mr', 'Marshall': 'mh', 'Moldavian': 'mo', 'Mongolian': 'mn',
'Nauru': 'na', 'Navajo': 'nv', 'Ndebele, South': 'nr', 'Ndonga': 'ng',
'Nepali': 'ne', 'Northern Sami': 'se', 'Norwegian Bokmal': 'nb',
'Norwegian Nynorsk': 'nn', 'Occitan': 'oc', 'Oriya': 'or', 'Ossetian;
Ossetic': 'os', 'Pali': 'pi', 'Panjabi': 'pa', 'Pashto, Pushto': 'ps',
'Persian': 'fa', 'Polish': 'pl', 'Portuguese': 'pt', 'Quechua': 'qu',
'Rhaeto-Romance': 'rm', 'Romanian': 'ro', 'Rundi': 'rn', 'Russian':
'ru', 'Samoan': 'sm', 'Sangro': 'sg', 'Sanskrit': 'sa', 'Sardinian':
'sc', 'Serbian': 'sr', 'Serbo-Croatian': 'sh', 'Sesotho': 'st',
'Setswana': 'tn', 'Shona': 'sn', 'Sindhi': 'sd', 'Sinhalese': 'si',
'Siswati': 'ss', 'Slovak': 'sk', 'Slovenian': 'sl', 'Somali': 'so',
'Spanish': 'es', 'Sundanese': 'su', 'Swahili': 'sw', 'Swedish': 'sv',
'Tagalog': 'tl', 'Tahitian': 'ty', 'Tajik': 'tg', 'Tamil': 'ta',
'Tatar': 'tt', 'Telugu': 'te', 'Thai': 'th', 'Tibetan': 'bo',
'Tigrinya': 'ti', 'Tonga': 'to', 'Tsonga': 'ts', 'Turkish': 'tr',
'Turkmen': 'tk', 'Twi': 'tw', 'Uighur': 'ug', 'Ukrainian': 'uk', 'Urdu':
'ur', 'Uzbek': 'uz', 'Vietnamese': 'vi', 'Volapuk': 'vo', 'Walloon':
'wa', 'Welsh': 'cy', 'Wolof': 'wo', 'Xhosa': 'xh', 'Yiddish': 'yi',
'Yoruba': 'yo', 'Zhuang': 'za', 'Zulu': 'zu'}Reversed lang_codes — that's the entire gloss on it, bestie. It's the reverse lookup of that big gorgeous dict above, and yes, it deserved its own little moment ✨
🦋 si
This module represents the Sinhala language 🌸
SEE ALSO: Sinhala_language
💜 class translate.lang.si.si(code)
This class represents Sinhala. Obsessed with her, honestly.
CJKpunc='。、，；！？「」『』【】'— these punctuation marks get used in certain circumstances with CJK languages. Not her usual vibe but she keeps them on hand 💅classmethod alter_length(text)— converts the given string by adding or removing characters, as an estimation of translation length (with English assumed as the source language). It's a length vibe check, not a promise 🔥classmethod capsstart(text)— determines whether the text starts with a capital letter. Capital-checking icon behavior 👑classmethod character_iter(text)— returns an iterator over the characters intext. One lil character at a time, no cap 🦋classmethod characters(text)— returns a list of characters intext. Same energy, but as a list, periodt ✨checker=None— a language specific checker instance (peepfilters.checks). This doesn't need to be supplied, but it will be used if it exists 💫code=''— the ISO 639 language code, possibly with a country specifier or other modifier. Like, examples:
km
pt_BR
sr_YU@Latncommonpunc—'.,;:!?-@#$%^*_()[]{}/\\\'`"<>'— these punctuation marks are common in English and most languages that use latin script. The whole squad 🫶ethiopicpunc='።፤፣'— these punctuation marks are used by several Ethiopic languages 🌈fullname=''— the full (English) name of this language. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests={'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers 😭 Keys are checker names and values are lists of names for the ignored tests in that checker. A special'all'checker name can be used to say the tests must be ignored in all the checkers. Listed checkers to ignore tests on must be lowercase strings for the checker name, for example"mozilla"forMozillaCheckeror"libreoffice"forLibreOfficeChecker💀indicpunc='।॥॰'— these punctuation marks are used by several Indic languages 💖invertedpunc='¿¡'— inverted punctuation, sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. It's giving upside down but make it fashion 🙈classmethod length_difference(length)— returns an estimate of a likely change in length relative to an English string of length length. Estimation queen 🎀listseperator=', '— this string is used to separate lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this ⭐miscpunc='…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian. The more you know 🌸mozilla_pluralequation='0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext 👑nplurals=0— the number of plural forms of this language.
Careful, bestie:0is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between1and6) 💅 SEE ALSO:translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language. Math but multilingual 🔥numbertuple=()— a tuple of number transformation rules that can be used bynumbertranslate()✨classmethod numstart(text)— determines whether the text starts with a numeric value. Digit detection, no cap 💜pluralequation='0'— the plural equation for selection of plural forms. This is used for PO files, to fill into the header 🎀
SEE ALSO: Gettext manual, translate.lang.data
puncdict={}— a dictionary of punctuation transformation rules that can be used bypunctranslate()🦋classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language. Glow-up for your commas 💫punctuation— the big one:
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'We include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won't need to override this 🫶
quotes='‘’‛“”„‟′″‴‵‶‷‹›«»'— these are the different quotation marks used by various languages. Quote collection: iconic 🌈rtlpunc='،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example 🧚classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences intext💖sentenceend='.!?…։؟।。！？።۔'— these marks can indicate a sentence end. Once again we try to account for many languages. Most languages won't need to override this 😭classmethod sentences(text, strip=True)— returns a list of sentences intext. Same thing but as a list, periodt ✨specialchars=''— characters used by the language that might not be easy to input with common keyboard layouts. Keyboard-struggle representation 🎀validaccel=None— characters that can be used as accelerators (access keys), i.e. Alt+X whereXis the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list 👑validdoublewords=[]— some languages allow double words in certain cases. The docs call this a dictionary of such words (the default is an empty list[], and honestly the source says both, so). 🙈classmethod word_iter(text)— returns an iterator over the words intext🦋classmethod words(text)— returns a list of words intext. Listified words, slay 💅
🌸 st
This module represents the Southern Sotho language 🫶
💜 class translate.lang.st.st(code)
This class represents Southern Sotho. Main character energy fr fr.
CJKpunc='。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages ✨classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language) 🔥classmethod capsstart(text)— determines whether the text starts with a capital letter 👑classmethod character_iter(text)— returns an iterator over the characters intext🦋classmethod characters(text)— returns a list of characters intext🌸checker=None— a language specific checker instance (seefilters.checks) 💫checker = None— a language-specific checker instance (peepfilters.checks). You don't have to supply it, but if it exists, it gets used. ✨code = ''— the ISO 639 language code, possibly with a country specifier or other modifier attached. Examples, bestie:
km
pt_BR
sr_YU@Latncommonpunc— these punctuation marks are common in English and most languages that use latin script. 💅
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc = '።፤፣'— these punctuation marks get used by several Ethiopic languages. 🎀fullname = ''— the full (English) name of this language.
Dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests = {}— a dictionary of tests to ignore in some or all checkers. 🔥
Keys are checker names and the values are lists of names for the ignored tests in that checker. There's a special 'all' checker name you can use to say the tests must be ignored in all the checkers.
Listed checkers to ignore tests on must be lowercase strings for the checker name, for example "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker.
indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages. 🌸invertedpunc = '¿¡'— inverted punctuation, sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. 🙈classmethod length_difference(length)— returns an estimate to a likely change in length relative to an English string of length length.listseperator = ', '— this string is used to separate lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this.
Yes, it really is spelled listseperator. That's the attribute name, not a typo on my end. 🫶
miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian.mozilla_pluralequation = '0'— this one's for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).
- SEE ALSO:
translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple = ()— a tuple of number transformation rules that can be used bynumbertranslate().classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms.
This is used for PO files, to fill into the header.
- SEE ALSO: Gettext manual,
translate.lang.data
puncdict = {}— a dictionary of punctuation transformation rules that can be used bypunctranslate().classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation— we include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won't need to override this. 🦋
punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are the different quotation marks used by various languages. 🎀rtlpunc = '،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end. Once again we try to account for many languages. Most langauges won't need to override this.classmethod sentences(text, strip=True)— returns a list of sentences in text.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts.validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list.validdoublewords = ['o', 'le', 'ba']— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words in text.classmethod words(text)— returns a list of words in text.
🌈 sv 🌈
This module represents the the Swedish language. 🇸🇪 (okay it literally says "the the" in the source, I'm just the messenger 💀)
- SEE ALSO:
Swedish_language
🫶 class translate.lang.sv.sv(code) 🫶
This class represents Swedish. Let's go through its whole closet, no skips. 👑
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages.classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as source language).classmethod capsstart(text)— determines whether the text starts with a capital letter.classmethod character_iter(text)— returns an iterator over the characters in text.classmethod characters(text)— returns a list of characters in text.checker = None— a language specific checker instance (seefilters.checks). This doesn't need to be supplied, but will be used if it exists. ✨code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples:
km
pt_BR
sr_YU@Latncommonpunc— these punctuation marks are common in English and most languages that use latin script. 💅
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages.fullname = ''— the full (English) name of this language.
Dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests = {}— dictionary of tests to ignore in some or all checkers.
Keys are checker names and values are list of names for the ignored tests in the checker. A special 'all' checker name can be used to tell that the tests must be ignored in all the checkers.
Listed checkers to ignore tests on must be lowercase strings for the checker name, for example "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker.
indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages. 🌸invertedpunc = '¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan.classmethod length_difference(length)— returns an estimate to a likely change in length relative to an English string of length length.listseperator = ', '— this string is used to separate lists of textual elements. Most languages probably can stick with the default comma, but Arabic and some Asian languages might want to override this.miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian.mozilla_pluralequation = '0'— this of languages that has different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).
- SEE ALSO:
translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple = ()— a tuple of number transformation rules that can be used bynumbertranslate().classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms. This is used for PO files to fill into the header.- SEE ALSO: Gettext manual,
translate.lang.data
- SEE ALSO: Gettext manual,
puncdict = {}— a dictionary of punctuation transformation rules that can be used bypunctranslate().classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation— bestie this string is long on purpose: it's every character we treat as punctuation, periodt. The whole point is just to answer "is this thing punctuation or not," so we cast a huge net in the hopes of catching languages that don't even have a module yet. Most languages won't ever need to override this:
punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes— the whole silly little family of quotation marks that different languages use 💅
quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'rtlpunc— punctuation that Arabic and Persian (for example) use, iykyk 🫶
rtlpunc = '،؟؛÷'classmethod sentence_iter(text, strip=True)— hands you back an iterator over the sentences intext✨sentenceend— the marks that can signal "yep, that's the end of a sentence." Again we're trying to cover as many languages as we can; most languages won't need to override it:
sentenceend = '.!?…։؟।。！？።۔'classmethod sentences(text, strip=True)— same vibes, but returns a list of the sentences intext🎀specialchars— characters this language uses that are lowkey a pain to type on a normal keyboard layout:
specialchars = ''validaccel— the characters allowed as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as the user can reach them from their keyboard in a single keystroke, but normally they'd at least be precomposed characters. Both lowercase and uppercase are in the list 🔥
validaccel = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890åäöÅÄÖ'validdoublewords— some languages allow double words in certain cases, and this is a dictionary of exactly those words:
validdoublewords = []classmethod word_iter(text)— iterator over the words intext🦋classmethod words(text)— list of the words intext🌸
ta 🐅
This module represents the Tamil language.
SEE ALSO: Tamil_language
class translate.lang.ta.ta(code) ✨
This class represents Tamil.
CJKpunc— punctuation marks that get used in certain circumstances with CJK languages:
CJKpunc = '。、，；！？「」『』【】'classmethod alter_length(text)— transforms the given string by adding or removing characters, as an estimate of translation length (with English assumed as the source language) 📏classmethod capsstart(text)— tells you whether the text starts with a capital letter 🔠classmethod character_iter(text)— iterator over the characters intext🧚classmethod characters(text)— list of the characters intext💖checker = None— a language-specific checker instance (peepfilters.checks). You don't have to supply this, but it'll be used if it exists 🎀code = ''— the ISO 639 language code, possibly with a country specifier or some other modifier. Examples:
km
pt_BR
sr_YU@Latncommonpunc— the punctuation marks that are common in English and most languages using the latin script:
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'ethiopicpunc— punctuation marks used by several Ethiopic languages:
ethiopicpunc = '።፤፣'fullname = ''— the full (English) name of this language 👑. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— a dictionary of tests to ignore in some or all checkers. Keys are checker names, values are the lists of names of the tests ignored in that checker. The special'all'checker name says "ignore these tests in every checker." Listed checker names have to be lowercase strings, like"mozilla"forMozillaCheckeror"libreoffice"forLibreOfficeChecker🙈indicpunc— punctuation marks used by several Indic languages:
indicpunc = '।॥॰'invertedpunc— the upside-down punctuation that sometimes shows up at the start of sentences in Spanish, Asturian, Galician, and Catalan:
invertedpunc = '¿¡'classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of length length 📐listseperator = ', '— the string used to separate lists of textual elements. Most languages can probably just stick with the default comma, but Arabic and some Asian languages might want to override it 🌈miscpunc— assorted punctuation; the middle dot (·) is used by Greek and Georgian:
miscpunc = '…±°¹²³·©®×£¥€'mozilla_pluralequation = '0'— for languages whose plural formula in Mozilla differs from the standard Gettext one:
mozilla_pluralequation = '0'nplurals = 0— the number of plural forms this language has 🔢
0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language 💜numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use:
numbertuple = ()classmethod numstart(text)— tells you whether the text starts with a numeric value 🍓pluralequation = '0'— the plural equation used to select plural forms. This is what gets filled into the header of PO files. SEE ALSO: Gettext manual,translate.lang.datapuncdict = {}— a dictionary of punctuation transformation rules thatpunctranslate()can use:
puncdict = {}classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language ✨punctuation— the big one again, same energy as before: we include a ton of punctuation types simply because this is only meant to determine whether something is punctuation. Hopefully we catch some languages that might not be represented with modules — most languages won't need to override this:
punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes— the different quotation marks used by various languages 🎀
quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'rtlpunc— punctuation marks used by Arabic and Persian, for example 💅
rtlpunc = '،؟؛÷'classmethod sentence_iter(text, strip=True)— iterator over the sentences intext🦋sentenceend— marks that can indicate a sentence end. Once again we're trying to account for many languages; most langauges won't need to override this ⭐
sentenceend = '.!?…։؟।。！？።۔'classmethod sentences(text, strip=True)— list of the sentences intext🌸specialchars— characters the language uses that might not be easy to type with common keyboard layouts:
specialchars = ''validaccel = None— characters usable as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're reachable from the user's keyboard in a single keystroke, but normally they'd at least be precomposed characters. All characters, lower and upper, are included in the list 👑
validaccel = Nonevaliddoublewords = []— some languages allow double words in certain cases; this is a dictionary of such words:
validdoublewords = []classmethod word_iter(text)— iterator over the words intext🧚classmethod words(text)— list of the words intext💖
team 🫶
Module to guess the language ISO code based on the 'Language-Team' entry in the header of a Gettext PO file.
okay so bestie, we’re still deep in translate.lang.team and here’s the absolute unit of a dict that maps language codes to contact snippets 🫶
translate.lang.team.LANG_TEAM_CONTACT_SNIPPETS = {'af':
('i18n@af.org.za', 'Petri Jooste'), 'am': ('@geez.org',), 'ar':
('arabeyes.org', 'Arabeyes'), 'as': ('assam@mm.assam-glug.org',), 'ast':
('@softastur.org', 'launchpad.net/~ubuntu-l10n-ast',
'softast-xeneral@lists.sourceforge.net', 'Softastur'), 'az':
('linuxaz@azerimal.net', 'gnome@azitt.com', 'gnome@azətt.com'), 'az_IR':
('az-ir@lists.sharif.edu',), 'be': ('i18n@mova.org', 'i18n@tut.by',
'mozilla_byx@poczta.fm'), 'be@latin':
('translation-team-be-latin@lists', 'be-latin.open-tran.eu'), 'bg':
('dict@fsa-bg.org', 'dict@linux.zonebg.com'), 'bn':
('gnome-translation@bengalinux.org', 'core@bengalinux.org',
'ankur-bd-l10n@googlegroups.com', 'redhat-translation@bengalinux.org'),
'bn_IN': ('anubad@lists.ankur.org.in',), 'br': ('drouizig@drouizig.org',
'brenux@free.fr', 'tradgnome@softcatala.net', 'fedora@softcatala.org'),
'bs': ('lokal@linux.org.ba', 'lokal@lugbih.org'), 'ca':
('@softcatala.org',), 'crh':
('tilde-birlik-tercime@lists.sourceforge.net',), 'cs':
('fedora-cs-list@redhat.com', 'cs-users@lists.fedoraproject.org',
'debian-l10n-czech@lists.debian.org',
'kde-czech-apps@lists.sourceforge.net', 'kde-czech-apps@lists.sf.net',
'translations.cs@gnupg.cz'), 'cy': ('gnome-cy@lists.linux.org.uk',
'gnome-cy@pengwyn.linux.org.uk', 'gnome-cy@www.linux.org',
'gnome-cy@www.linux.org.uk', 'cy@pengwyn.linux.org.uk'), 'da':
('dansk@dansk-gruppen.dk', 'dansk@klid.dk', 'sslug-locale@sslug.dk'),
'de': ('gnome-de@gnome.org', 'debian-l10n-german@lists.debian.org'),
'dz': ('pgeyleg@dit.gov.bt', 'pgyeleg@dit.gov.bt'), 'el':
('debian-l10n-greek@lists.debian.org', 'i18ngr@lists.hellug.gr',
'i18n@hellug.gr', 'nls@tux.hellug.gr', 'team@gnome.gr',
'team@lists.gnome.gr', 'users@el.openoffice.org'), 'en@shaw':
('ubuntu-l10n-en-shaw@launchpad.net',
'ubuntu-l10n-en-shaw@lists.launchpad.net'), 'en_AU':
('trans@six-by-nine.com.au',), 'en_CA': ('adamw@gnome.org',
'adamw@freebsd.org'), 'en_GB': ('kde-en-gb@kde.me.uk',), 'eo':
('eo-tradukado@lists.tuxfamily.org',
'debian-l10n-esperanto@lists.debian.org',
'ubuntu-l10n-eo@lists.launchpad.net', 'eo-tradukado.tuxfamily.org'),
'es': ('pgsql-es-ayuda@postgresql.org',
'debian-l10n-spanish@lists.debian.org', 'gnome-es@gnome.org',
'traductores@es.gnome.org'), 'et': ('gnome-et@linux.ee',
'kde-et@linux.ee', 'linux-ee@lists.eenet.ee', 'linux-et@lists.eenet.ee',
'et-gnome@linux.ee', 'linux-ee@eenet.ee'), 'eu':
('debian-l10n-basque@lists.debian.org',
'debian-l10n-eu@lists.debian.org', 'itzulpena@euskalgnu.org',
'gnome@euskalgnu.org', 'librezale@librezale.org',
'linux-eu@chanae.alphanet.ch'), 'fa': ('farsi@lists.sharif.edu',
'Farsiweb.info'), 'fi': ('debian-l10n-finnish@lists.debian.org',
'gnome-fi-laatu@lists.sourceforge.net', 'laatu@lokalisointi.org',
'lokalisointi-laatu@linux-aktivaattori.org', 'laatu@gnome.fi',
'yast-trans-fi@kotoistaminen.novell.fi'), 'fr':
('debian-l10n-french@lists.debian.org', 'gnomefr@traduc.org',
'kde-francophone@kde.org', 'traduc@traduc.org',
'pgsql-fr-generale@postgresql.org', 'rpm-fr@livna.org'), 'ga':
('gaeilge-gnulinux@lists.sourceforge.net',
'gaeilge-a@listserv.heanet.ie'), 'gl': ('trasno@ceu.fi.udc.es',
'gnome@g11n.net', 'gpul-traduccion@ceu.fi.udc.es',
'proxecto@trasno.net', 'trasno@gpul.org'), 'gu':
('indianoss-gujarati@lists.sourceforge.net',), 'he':
('debian-hebrew-common@lists.alioth.debian.org',
'kde-il@yahoogroups.com', 'fedora-he-list@redhat.com',
'mdk-hebrew@iglu.org.il'), 'hi':
('indlinux-hindi-gnome@lists.sourceforge.net',
'indlinux-hindi@lists.sourceforge.net'), 'hr': ('translator-shop.org',
'lokalizacija@linux.hr'), 'hu':
('debian-l10n-hungarian@lists.debian.org', 'gnome@fsf.hu',
'gnome@gnome.hu', 'magyar@lists.linux.hu'), 'id': ('@id.gnome.org',
'@gnome.linux.or.id', 'mdk-id@yahoogroups.com', 'linux.or.id',
'gnome@i15n.org'), 'io': ('gnome-ido@lists.mterry.name',), 'is':
('gnome@techattack.nu', 'kde-isl@mmedia.is', 'kde-isl@molar.is'), 'it':
('debian-l10n-italian@lists.debian.org', 'traduzioni@itpug.org',
'fedora-trans-it@redhat.com', 'tp@lists.linux.it'), 'ja':
('debian-doc@debian.or.jp', 'debian-japanese@lists.debian.org',
'gnome-translation@gnome.gr.jp', 'translation@gnome.gr.jp',
'jpug-doc@ml.postgresql.jp'), 'ka': ('geognome@googlegroups.com',
'Ubuntu-Georgian-Translators@googlegroups.com'), 'kk':
('kk_KZ@googlegroups.com',), 'km': ('@khmeros.info',), 'kn':
('debian-l10n-kannada@lists.debian.org',), 'ko':
('gnome-kr-hackers@list.kldp.net', 'gnome-kr-hackers@lists.kldp.net',
'gnome-kr-translation@lists.kldp.net', 'pgsql-kr@postgresql.or.kr',
'hangul-hackers@lists.kldp.net', 'debian-l10n-korean@lists.debian.org',
'gnome-kr-translation@lists.sourceforge.net'), 'ks':
('ks-gnome-trans-commits@lists.code.indlinux.net',), 'ku':
('gnu-ku-wergerandin@lists.sourceforge.net',), 'ky':
('i18n-team-ky-kyrgyz@lists.sourceforge.net', 'ky-li@mail.ru'), 'la':
('gnome-latin-list@gnome.org',), 'li': ('li@gnome.org',), 'lt':
('gimp-lt@lists.akl.lt', 'gnome-lt@lists.akl.lt',
'gnome-lt@lists.gnome.org', 'komp_lt@konferencijos.lt'), 'lv':
('lata-l10n@googlegroups.com', 'lata-i18n@groups.google.com',
'locale@laka.lv', 'll10nt@os.lv'), 'mai': ('maithili.sf.net',), 'mg':
('i18n-malagasy-gnome@gnome.org',), 'mi': ('maori@nzlinux.org.nz',),
'mk': ('gnomk-main@lists.sourceforge.net', 'lug@lists.linux.net.mk',
'mkde-l10n@lists.sourceforge.net', 'ossm-members@hedona.on.net.mk'),
'ml': ('smc-discuss@googlegroups.com',), 'mn': ('openmn-',
'openmn.org'), 'ms': ('gabai-penyumbang@lists.sourceforge.net',
'gabai-penyumbang@lists.sf.net', 'kedidiemas@yahoogroups.com'), 'nb':
('i18n-nb@lister.ping.uio.no',), 'nds':
('nds-lowgerman@lists.sourceforge.net',), 'ne': ('info@mpp.org.np',),
'nl': ('debian-l10n-dutch@lists.debian.org', 'vertaling@nl.gnome.org',
'vertaling@vrijschrift.org', 'nl@vrijschrift.org',
'vertaling@nl.linux.org', 'vertaling@nl.li.org'), 'nn':
('i18n-nn@lister.ping.uio.no',), 'nso': ('sepedi@translate.org.za',),
'or': ('oriya-group@lists.sarovar.org', 'oriya-it@googlegroups.com'),
'pa': ('punjabi-l10n@users.sf.net', 'fedora-pa-list@redhat.com',
'punjabi-users@lists.sf.net', 'punjabi-l10n@lists.sourceforge.net',
'punlinux-i18n@lists.sourceforge.net'), 'pl': ('gnomepl@aviary.pl',
'debian-l10n-polish@lists.debian.org', 'gnome-l10n@lists.aviary.pl',
'translators@gnomepl.org'), 'ps': ('pathanisation@googelgroups.com',),
'pt': ('fedora-trans-pt@redhat.org', 'gnome_pt@yahoogroups.com',
'traduz@debianpt.org', 'traduz@debian.pt'), 'pt_BR':
('gnome-l10n-br@listas.cipsga.org.br', 'gnome-pt_br-list@gnome.org',
'fedora-docs-br@redhat.com', 'fedora-trans-pt-br@redhat.com',
'ldp-br@bazar.conectiva.com.br', 'pgbr-dev@postgresql.org.br',
'pgbr-dev@listas.postgresql.org.br',
'debian-l10n-portuguese@lists.debian.org'), 'ro':
('fedora-ro@googlegroups.com', 'gnomero-list@lists.sourceforge.net',
'debian-l10n-romanian@lists.debian.org'), 'ru':
('pgsql-rus@yahoogroups.com', 'debian-l10n-russian@lists.debian.org',
'gnupg-ru@gnupg.org'), 'scn': ('l10n@cademiasiciliana.org',), 'sk':
('sk-i18n@lists.linux.sk', 'kde-sk@linux.sk'), 'sl':
('gnome-si@googlegroups.com',), 'sq':
('gnome-albanian-perkthyesit@lists.sourceforge.net',
'debian-l10n-albanian@lists.debian.org'), 'sr': ('@prevod.org',
'serbiangnome-lista@nongnu.org'), 'sv':
('debian-l10n-swedish@lists.debian.org', 'tp-sv@listor.tp-sv.se'), 'ta':
('gnome-tamil-translation@googlegroups.com',
'tamilinix@yahoogroups.com', 'Ubuntu-l10n-tam@lists.ubuntu.com',
'tamil-DI@yahoogroups.com'), 'te': ('localisation@swecha.org',
'indlinux-telugu@lists.sourceforge.net'), 'th': ('l10n@opentle.org',
'thai-l10n@googlegroup.com', 'thailang@buraphalinux.org',
'thai-l10n@googlegroups.com', 'l10n.opentle.org'), 'tk':
('kakilikgroup@yahoo.com',), 'tl': ('debian-tl@banwa.upm.edu.ph',),
'tr': ('debian-l10n-turkish@lists.debian.org', 'gnome-turk@gnome.org',
'gnu-tr-u12a@lists.sourceforge.net', 'turkce@pardus.org.tr'), 'tt':
('tatarish.l10n@gmail.com',), 'ug': ('gnome-uighur@yahoogroups.com',),
'uk': ('linux@linux.org.ua',), 'ur': ('l10n@urduweb.org',
'urdu.scs.gift@gmail.com'), 've': ('venda@translate.org.za',), 'vi':
('gnomevi-list@lists.sourceforge.net', 'vi-VN@googlegroups.com'), 'wa':
('linux-wa@',), 'xh': ('xh-translate@ubuntu.com',
'xhosa@translate.org.za', 'xhosa@ubuntu.com'), 'zh_CN':
('i18n-translation@lists.linux.net.cn', 'i18n-zh@googlegroups.com',
'translation-team-zh-cn@lists.sourceforge.net',
'i18n-zh@googlegroup.com'), 'zh_TW': ('zh-l10n@lists.linux.org.tw',
'chinese-l10n@googlegroups.com', 'community@linuxhall.org',
'zh-l10n@linux.org.tw'), 'zu': ('zulu@translate.org.za',)}like, this is the ultimate group chat directory for language teams — every language code gets a little bundle of contact info that can be used to uniquely identify the language 💖 it’s giving “who you gonna call” but make it l10n 👻📧
next up:
translate.lang.team.guess_language(team_string)
this one guesses the language of a PO file based on the Language-Team entry. (and yes, the original manpage says “Gueses” — typo and all, no cap 😭💀)
🌸 te 🌸
this module is literally all about the Telugu language, bestie 🇮🇳✨
SEE ALSO: Telugu_language
class translate.lang.te.te(code)
this class represents Telugu. periodt. 👑
CJKpunc='。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages. (yes, even though this is Telugu, the attribute is there, iykyk 🫶)alter_length(text)(classmethod) — converts the given string by adding or removing characters as an estimation of translation length (with English assumed as source language). it’s giving “how long is this gonna be in English tho” energy 📏✨capsstart(text)(classmethod) — determines whether the text starts with a capital letter. 🔠character_iter(text)(classmethod) — returns an iterator over the characters intext. one by one, no cap 🧚characters(text)(classmethod) — returns a list of characters intext. same vibes, but list-shaped 🎀checker=None— a language specific checker instance (seefilters.checks). this doesn’t need to be supplied, but will be used if it exists. lowkey optional, highkey useful if you got one 👀code=''— the ISO 639 language code, possibly with a country specifier or other modifier. examples:kmpt_BRsr_YU@Latn
commonpunc='.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — these punctuation marks are common in English and most languages that use latin script. the whole punctuation starter pack fr fr 💅ethiopicpunc='።፤፣'— these punctuation marks are used by several Ethiopic languages. not Telugu’s usual vibe, but it’s in the class, so we list it 🫡fullname=''— the full (English) name of this language. dialect codes should have the form of:- Khmer
- Portugese (Brazil) (yes, the manpage spells it “Portugese” — we keep it exactly, no editing 💀)
- TODO: sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— a lil dictionary of tests to skip in some checkers or, like, literally all of them. The keys are checker names and the values are lists of test names to ignore for that checker. There's a special'all'checker name you can use to say "bestie, skip these tests in every checker." And heads up: the checker names you list have to be lowercase strings — so"mozilla"forMozillaChecker,"libreoffice"forLibreOfficeChecker, iykyk.indicpunc = '।॥॰'— punctuation marks that a bunch of Indic languages use. 🪷invertedpunc = '¿¡'— the upside-down punctuation that shows up at the start of sentences in Spanish, Asturian, Galician, and Catalan. it's giving "¿qué?" ✨classmethod length_difference(length)— gives you an estimate of how much the length will likely change compared to an English string of length length.listseperator = ', '— the string used to separate lists of textual elements. Most languages can honestly just keep the default comma, but Arabic and some Asian languages might wanna override this one.miscpunc = '…±°¹²³·©®×£¥€'— assorted punctuation. fun fact: the middle dot (·) is used by Greek and Georgian. 🫶mozilla_pluralequation = '0'— for languages that have a different plural formula in Mozilla than the standard one Gettext uses.nplurals = 0— the number of plural forms this language has.
🚨 0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use.classmethod numstart(text)— figures out whether the text starts with a numeric value or not.pluralequation = '0'— the plural equation used to select plural forms. This is what gets filled into the header of PO files. 🎀
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'` — okay so we throw a lot of punctuation types in here, simply because this is only meant for determining whether something counts as punctuation. The hope is we catch some languages that might not even have modules yet. Most languages won't need to override this. 💅quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— the different quotation marks various languages use.rtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate the end of a sentence. Again, we're trying to cover as many languages as possible here. Most languages won't need to override this. (yes the original says "langauges", we don't judge 💀)classmethod sentences(text, strip=True)— returns a list of sentences in text.specialchars = ''— characters the language uses that might not be easy to type on common keyboard layouts.validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics, as long as they're reachable from the user's keyboard in a single keystroke — but normally they'd be at least precomposed characters. Both lowercase and uppercase characters are included in the list. ⌨️validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words. (yes, the docstring says dictionary even though the default is a list — iconic inconsistency, we keep it 💀)classmethod word_iter(text)— returns an iterator over the words in text.classmethod words(text)— returns a list of words in text.
🐘 th 🐘
this module represents the Thai language. สวัสดีค่ะ ✨
SEE ALSO: Thai_language
class translate.lang.th.th(code) — this class represents Thai.
CJKpunc = '。、，；！？「」『』【】'— punctuation marks used in certain circumstances with CJK languages.classmethod alter_length(text)— converts the given string by adding or removing characters, as an estimation of translation length (with English assumed as the source language).classmethod capsstart(text)— determines whether the text starts with a capital letter.classmethod character_iter(text)— returns an iterator over the characters in text.classmethod characters(text)— returns a list of characters in text.checker = None— a language-specific checker instance (seefilters.checks). You don't have to supply this, but it'll be used if it exists.code = ''— the ISO 639 language code, possibly with a country specifier or some other modifier.
Examples:
km
pt_BR
sr_YU@Latncommonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — the punctuation marks that are common in English and most languages using the Latin script.ethiopicpunc = '።፤፣'— punctuation marks used by several Ethiopic languages.fullname = ''— the full (English) name of this language.
Dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO: sr_YU@Latn?
ignoretests = {'all': ['sentencecount', 'simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers. The keys are checker names, and the values are lists of the names of the ignored tests for that checker. The special'all'checker name tells you the tests must be ignored in all the checkers. Listed checkers to ignore tests on must be lowercase strings for the checker name — for example"mozilla"forMozillaCheckeror"libreoffice"forLibreOfficeChecker.indicpunc = '।॥॰'— punctuation marks used by several Indic languages.invertedpunc = '¿¡'— inverted punctuation, sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan.classmethod length_difference(length)— returns an estimate to a likely change in length relative to an English string of length length.listseperator = ', '— the string used to separate lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this.miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian.mozilla_pluralequation = '0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
🚨 0 is not a valid value — it must be overridden. Any positive integer is valid (it should probably be between 1 and 6).
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use.classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms. This is used for PO files, to fill into the header. 🎀
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {'. ': ' '}— a dictionary of punctuation transformation rules thatpunctranslate()can use.classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language.punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'` — we include many types of punctuation here, simply since this is only meant to determine if something is punctuation. Hopefully we catch some languages which might not be represented with modules. Most languages won't need to override this.quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are the different quotation marks used by various languages.rtlpunc = '،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend = '.!?…։؟।。！？።۔'— these marks can mean "sentence ends here". Once again we're out here trying to cover like a million languages at once. Most languages will not need to override this, fr.classmethod sentences(text, strip=True)— gives you back a list of the sentences chilling inside text.specialchars = ''— the characters this language uses that are lowkey hard to type on your average keyboard layout.validaccel = None— the characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're actually reachable from the user's keyboard in a single keystroke — but normally they'd be precomposed characters at minimum. All characters, lower and upper, are included in the list. 💅validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words in text.classmethod words(text)— returns a list of the words in text.
🫶 SEE ALSO: Uyghur_language
🦋 ur — the Urdu module, bestie
This module represents the Urdu language.
🌸 SEE ALSO: Urdu_language
✨ class translate.lang.ur.ur(code)
This class represents Urdu. Iconic.
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks get used in certain circumstances with CJK languages.classmethod alter_length(text)— converts the given string by adding or removing characters, as an estimate of translation length (with English assumed as the source language).classmethod capsstart(text)— figures out whether the text starts with a capital letter.classmethod character_iter(text)— returns an iterator over the characters in text.classmethod characters(text)— returns a list of the characters in text.checker = None— a language-specific checker instance (seefilters.checks).
You don't have to supply this, but it gets used if it exists.
code = ''— the ISO 639 language code, possibly with a country specifier or other modifier.
Examples:
km
pt_BR
sr_YU@Latncommonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — these punctuation marks are common in English and most languages that use latin script.ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages.fullname = ''— the full (English) name of this language.
Dialect codes should have the form of:
- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers.
Keys are checker names and values are lists of names of the ignored tests in that checker. A special 'all' checker name can be used to say the tests must be ignored in all the checkers.
The listed checker names you ignore tests on must be lowercase strings, for example "mozilla" for MozillaChecker or "libreoffice" for LibreOfficeChecker.
indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages.invertedpunc = '¿¡'— the upside-down punctuation bestie, sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. it's giving "question but make it fashion" ✨classmethod length_difference(length)— hands you an estimate of the likely change in length compared to an English string of length. lowkey a glow-up prediction for your string 💅listseperator = '، '— the string used to separate lists of textual elements. most languages can literally just keep the default comma, but Arabic and some Asian languages might wanna override this one. (yes, that's "seperator" spelled like that in the source — we don't fix, we report 🫶)miscpunc = '…±°¹²³·©®×£¥€'— the miscellaneous punctuation pile. fun fact: the middle dot (·) is used by Greek and Georgian 🌸mozilla_pluralequation = '0'— for languages that have a different plural formula in Mozilla than the standard one in Gettext. iykyknplurals = 0— the number of plural forms of this language.
0 is not a valid value — it must be overridden. any positive integer is valid (and it should probably be between 1 and 6, ngl). 😭
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language. numbers but localized, periodt 🔥numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use. an empty tuple by default, so nothing's happening until you fill it in.classmethod numstart(text)— determines whether the text starts with a numeric value. yes/no, does it begin with a number 👑pluralequation = '0'— the plural equation used to select plural forms.
this is used for PO files, to fill into the header. 🎀
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {',': '،', '.': '۔', ';': '؛', '?': '؟'}— a dictionary of punctuation transformation rules thatpunctranslate()can use. it's giving "find and replace but make it multilingual" ✨classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language. tiny swaps, big vibes 💫punctuation
punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'we include so many types of punctuation here, simply because this is only meant to determine whether something is punctuation. hopefully we catch some languages that might not be represented with modules at all. most languages won't need to override this — it's already an icon 💅
quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— all the different quotation marks used by various languages. the quote multiverse 🌈rtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for example. right-to-left besties stand up 🫶classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text. lazy, on demand, main character energy ⭐sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end. once again we're trying to account for many languages (and yes, "langauges" is how it's spelled in the source 😭). most languages won't need to override this.classmethod sentences(text, strip=True)— returns a list of sentences in text. the eager version of the above.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts. empty by default, so no stress 🍓validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. these can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. all characters, lower and upper, are included in the list. no cap 👑validdoublewords = []— some languages allow double words in certain cases. this is a dictionary of such words. (yes, it's a list default but the doc calls it a dictionary — we keep it exactly as written 🙈)classmethod word_iter(text)— returns an iterator over the words in text. one word at a time, no rush 🦋classmethod words(text)— returns a list of words in text. all of them, right now, iconic.
🌸 vi
this module represents the Vietnamese language. xin chào bestie 💖
SEE ALSO: Vietnamese_language
class translate.lang.vi.vi(code)
this class represents Vietnamese. let's get into it 👇
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks are used in certain circumstances with CJK languages. situational punctuation, honestly valid 🌸classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language). it's a little stretch/squish for your text 💅classmethod capsstart(text)— determines whether the text starts with a capital letter. capital or not, that's the whole question ✨classmethod character_iter(text)— returns an iterator over the characters in text. one lil guy at a time 🎀classmethod characters(text)— returns a list of characters in text. the whole squad, listed.checker = None— a language-specific checker instance (seefilters.checks). this doesn't need to be supplied, but it will be used if it exists. optional but make it count 🫶code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. examples, because we love specifics:
km
pt_BR
sr_YU@Latncommonpunc
commonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>'these punctuation marks are common in English and most languages that use the latin script. the everyday crew 💫
ethiopicpunc = '።፤፣'— these punctuation marks are used by several Ethiopic languages. niche but iconic 🔥fullname = ''— the full (English) name of this language. dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
(and yes, "Portugese" is spelled that way in the original — we report, we don't edit 😭)
ignoretests = {}— a dictionary of tests to ignore in some or all checkers. keys are checker names and values are lists of names of the ignored tests in that checker. a special'all'checker name can be used to say the tests must be ignored in all the checkers. listed checkers to ignore tests on must be lowercase strings for the checker name — for example"mozilla"forMozillaCheckeror"libreoffice"forLibreOfficeChecker. lowercase or it won't hit, periodt 👑indicpunc = '।॥॰'— these punctuation marks are used by several Indic languages. love that for them 🌸invertedpunc = '¿¡'— inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan. the ✨prequel✨ marks.classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of length. vibe check for your string's future size 💅listseperator = ', '— the string used to separate lists of textual elements. most languages can probably stick with the default comma, but Arabic and some Asian languages might want to override this. (again: "seperator", ship it 🫶)miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian. cute AND functional.mozilla_pluralequation = 'n!=1 ? 1 : 0'— for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
0 is not a valid value — it must be overridden. any positive integer is valid (it should probably be between 1 and 6). 😭
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language.numbertuple = ()— a tuple of number transformation rules that can be used bynumbertranslate(). empty until you say otherwise.classmethod numstart(text)— determines whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selection of plural forms.
this is used for PO files, to fill into the header.
SEE ALSO: Gettext manual, translate.lang.data
puncdict = {'!': ' !', '#': ' #', ':': ' :', ';': ' ;'}— a dictionary of punctuation transformation rules that can be used bypunctranslate(). look at those lil spaces sneaking in 🦋classmethod punctranslate(text)— implements some extra features for quotation marks.
known shortcomings, and we're being honest about them 🫶:
%and$are not touched yet, for fear of variables- double spaces might be introduced
punctuation
punctuation = '.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'we include many types of punctuation here, simply since this is only meant to determine if something is punctuation. hopefully we catch some languages which might not be represented with modules. most languages won't need to override this. it's already doing the most and we love it 💅
quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— these are different quotation marks used by various languages. a whole quote wardrobe 🌈rtlpunc = '،؟؛÷'— these punctuation marks are used by Arabic and Persian, for example. right-to-left rep 🫶classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend = '.!?…։؟।。！？።۔'— these marks can indicate a sentence end. once again we try to account for many languages (spelled "langauges" in the source — iconic typo, no notes 😭). most languages won't need to override this.classmethod sentences(text, strip=True)— returns a list of sentences in text.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts. empty for now, no pressure 🍓validaccel = None— the characters that can be used as accelerators (aka access keys, like Alt+X where X is the accelerator) 💅 these can include combining diacritics as long as they're reachable from the user's keyboard in a single keystroke, but ngl normally they'd be at least precomposed characters. All characters, lower and upper, are in the list.validdoublewords = []— some languages allow double words in certain cases, and this is the dictionary of those words 🫶classmethod word_iter(text)— returns an iterator over the words in text.classmethod words(text)— returns a list of words in text.
🀄 zh
This module is the whole vibe for the Chinese language — both traditional and simplified, no cap 🇨🇳✨
SEE ALSO: Chinese_language
class translate.lang.zh.zh(code) — this class represents Chinese, bestie 💖
CJKpunc = '。、，；！？「」『』【】'— these punctuation marks get used in certain circumstances with CJK languages. Iconic little squiggles honestly 🎀classmethod alter_length(text)— converts the given string by adding or removing characters as an estimation of translation length (with English assumed as the source language) 📏classmethod capsstart(text)— tells you whether the text starts with a capital letter.classmethod character_iter(text)— returns an iterator over the characters in text.classmethod characters(text)— returns a list of characters in text.checker = None— a language-specific checker instance (peepfilters.checks). You don't have to supply it, but it'll be used if it exists 👀code = ''— the ISO 639 language code, possibly with a country specifier or other modifier. Examples, served fresh:
km
pt_BR
sr_YU@Latncommonpunc = '.,;:!?-@#$%^*_()[]{}/\\\'"<>'` — the punctuation marks that are common in English and most languages using the latin script.ethiopicpunc = '።፤፣'— punctuation marks used by several Ethiopic languages 🌍fullname = ''— the full (English) name of this language. Dialect codes should have the form of:- Khmer
- Portugese (Brazil)
- TODO:
sr_YU@Latn?
ignoretests = {'all': ['simplecaps', 'startcaps']}— dictionary of tests to ignore in some or all checkers 🔇 Keys are checker names and values are lists of names for the ignored tests in that checker. A special'all'checker name tells you the tests must be ignored in all the checkers. Listed checkers to ignore tests on have to be lowercase strings for the checker name — e.g."mozilla"forMozillaCheckeror"libreoffice"forLibreOfficeChecker.indicpunc = '।॥॰'— punctuation marks used by several Indic languages.invertedpunc = '¿¡'— the inverted punctuation sometimes used at the beginning of sentences in Spanish, Asturian, Galician, and Catalan 🙃classmethod length_difference(length)— returns an estimate of the likely change in length relative to an English string of length length.listseperator = '、'— this string separates lists of textual elements. Most languages can probably stick with the default comma, but Arabic and some Asian languages might wanna override this one 🍓miscpunc = '…±°¹²³·©®×£¥€'— the middle dot (·) is used by Greek and Georgian, iykyk.mozilla_pluralequation = '0'— this is for languages that have a different plural formula in Mozilla than the standard one in Gettext.nplurals = 0— the number of plural forms of this language.
⚠️ 0 is not a valid value — it must be overridden. Any positive integer is valid (and it should probably be between 1 and 6) 💀
SEE ALSO: translate.lang.data
classmethod numbertranslate(text)— converts the numbers in a string according to the rules of the language 🔢numbertuple = ()— a tuple of number transformation rules thatnumbertranslate()can use.classmethod numstart(text)— tells you whether the text starts with a numeric value.pluralequation = '0'— the plural equation for selecting plural forms. This is what gets filled into the header of PO files 📝
SEE ALSO: Gettext manual, translate.lang.data
puncdict— a dictionary of punctuation transformation rules thatpunctranslate()can use:
{'!\n': '！\n', '! ': '！', '% ': '%', '.\n': '。\n',
'. ': '。', ':\n': '：\n', ': ': '：', ';\n': '；\n', '; ': '；',
'?\n': '？', '? ': '？'}classmethod punctranslate(text)— converts the punctuation in a string according to the rules of the language ✨punctuation— we include so many types of punctuation here, simply because this is only meant to determine whether something is punctuation. Hopefully we catch some languages that might not be represented with modules 🫶 Most languages won't need to override this:
'.,;:!?-@#$%^*_()[]{}/\\\'`"<>‘’‛“”„‟′″‴‵‶‷‹›«»¿¡،؟؛÷。、，；！？「」『』【】।॥॰።፤፣…±°¹²³·©®×£¥€'quotes = '‘’‛“”„‟′″‴‵‶‷‹›«»'— the different quotation marks used by various languages 🎀rtlpunc = '،؟؛÷'— punctuation marks used by Arabic and Persian, for example.classmethod sentence_iter(text, strip=True)— returns an iterator over the sentences in text.sentenceend = '。！？!?…'— these marks can indicate a sentence end. Once again we're trying to account for many languages. Most langauges won't need to override this.classmethod sentences(text, strip=True)— returns a list of sentences in text.specialchars = ''— characters used by the language that might not be easy to input with common keyboard layouts ⌨️validaccel = None— characters that can be used as accelerators (access keys), i.e. Alt+X where X is the accelerator. These can include combining diacritics as long as they're accessible from the user's keyboard in a single keystroke, but normally they'd be at least precomposed characters. All characters, lower and upper, are included in the list 💅validdoublewords = []— some languages allow double words in certain cases. This is a dictionary of such words.classmethod word_iter(text)— returns an iterator over the words in text.classmethod words(text)— returns a list of words in text.
🧩 misc
Miscellaneous modules for translate — including modules for backward compatibility with pre-2.3 versions of Python 🦋
🔑 dictutils
Implements a case-insensitive (on keys) dictionary and an order-sensitive dictionary. Bestie, your keys literally do not care about case ✨
class translate.misc.dictutils.cidict(fromdict=None)
clear()— remove all items from the dict 🧹copy()— return a shallow copy of the dict.classmethod fromkeys(iterable, value=None, /)— create a new dictionary with keys from iterable and values set to value.get(key, default=None)— return the value for key if key is in the dictionary, else default.items()— return a set-like object providing a view on the dict's items.keys()— return a set-like object providing a view on the dict's keys.pop(k[, d]) -> v— remove specified key and return the corresponding value.- If the key is not found, return the default if given; otherwise, raise a
KeyError😭
- If the key is not found, return the default if given; otherwise, raise a
popitem()— remove and return a(key, value)pair as a 2-tuple.- Pairs are returned in LIFO (last-in, first-out) order. Raises
KeyErrorif the dict is empty.
- Pairs are returned in LIFO (last-in, first-out) order. Raises
setdefault(key, default=None, /)— insert key with a value of default if key is not in the dictionary.- Return the value for key if key is in the dictionary, else default.
update(updatedict) -> None— update from a dictionary.D.update(E) -> None.Update D from E: for k in E.keys(): D[k] = E[k].
values()— return an object providing a view on the dict's values.
🔍 file_discovery
translate.misc.file_discovery.get_abs_data_filename(path_parts, basedirs=None)
Get the absolute path to the given file- or directory name in the current running application's data directory 📂
Parameters
- path_parts (list) – the path parts that can be joined by
os.path.join().
🧵 multistring
Supports a hybrid Unicode string that can also have a list of alternate strings in the strings attribute 👑
class translate.misc.multistring.multistring(string='')
capitalize()— return a capitalized version of the string.- More specifically, make the first character have upper case and the rest lower case.
casefold()— return a version of the string suitable for caseless comparisons.center(width, fillchar=' ', /)— return a centered string of length width.- Padding is done using the specified fill character (default is a space).
count()— tells you how many times the substringsubshows up inS[start:end], non-overlapping only (so"aaa".count("aa")is1, bestie, no cap).
The optional start and end args work exactly like slice notation, iykyk.
encode(encoding='utf-8', errors='strict')— turns the string into bytes using whatever codec is registered for encoding.- encoding — the encoding to encode in. yes, that's the whole vibe.
- errors — the error handling scheme for encoding errors. default is
'strict', which means encoding errors raise aUnicodeEncodeError(💀). other valid values:'ignore','replace','xmlcharrefreplace', plus literally any other name registered withcodecs.register_errorthat can handleUnicodeEncodeErrors.
endswith()—Trueif the string ends with the suffix you asked about,Falseotherwise. simple as.- suffix — a string, or a tuple of strings to try. tuple = try them all, queen.
- start — optional start position. default: start of the string.
- end — optional stop position. default: end of the string.
expandtabs(tabsize=8)— returns a copy where every tab character got expanded into spaces ✨
If tabsize isn't given, a tab size of 8 characters is assumed.
find()— returns the lowest index inSwhere substringsubis found, as long assubis contained withinS[start:end].
The optional start and end args work like slice notation. Returns -1 on failure (not an exception, she's chill like that).
format(*args, **kwargs)— returns a formatted version of the string, using substitutions from args and kwargs. The substitutions are identified by braces ({and}). iconic behavior.format_map(mapping, /)— returns a formatted version of the string, using substitutions from mapping. Same deal, substitutions identified by braces ({and}).index()— returns the lowest index inSwhere substringsubis found, such thatsubis contained withinS[start:end].
Optional start and end args are interpreted like slice notation. Raises ValueError when the substring isn't found — so yeah, this one will make it a whole thing 😭
isalnum()—Trueif the string is alpha-numeric,Falseotherwise.
A string is alpha-numeric if all characters in it are alpha-numeric and there's at least one character in the string. (empty string? not invited to the party.)
isalpha()—Trueif the string is alphabetic,Falseotherwise.
Alphabetic = all characters are alphabetic and there's at least one character in the string.
isascii()—Trueif all characters in the string are ASCII,Falseotherwise.
ASCII characters have code points in the range U+0000-U+007F. And the empty string is ASCII too — slay, loophole.
isdecimal()—Trueif the string is a decimal string,Falseotherwise.
Decimal string = all characters are decimal and there's at least one character in the string.
isdigit()—Trueif the string is a digit string,Falseotherwise.
Digit string = all characters are digits and there's at least one character in the string.
isidentifier()—Trueif the string is a valid Python identifier,Falseotherwise.
Pro tip straight from the page: call keyword.iskeyword(s) to test whether string s is a reserved identifier, like "def" or "class". 🧚
islower()—Trueif the string is a lowercase string,Falseotherwise.
A string is lowercase if all cased characters in it are lowercase and there's at least one cased character in the string.
isnumeric()—Trueif the string is a numeric string,Falseotherwise.
Numeric = all characters are numeric and there's at least one character in the string.
isprintable()—Trueif all characters in the string are printable,Falseotherwise.
A character is printable if repr() may use it in its output. that's the whole test, periodt.
isspace()—Trueif the string is a whitespace string,Falseotherwise.
Whitespace string = all characters are whitespace and there's at least one character in the string.
istitle()—Trueif the string is a title-cased string,Falseotherwise.
In a title-cased string, upper- and title-case characters may only follow uncased characters, and lowercase characters may only follow cased ones. rules are rules 💅
isupper()—Trueif the string is an uppercase string,Falseotherwise.
Uppercase = all cased characters are uppercase and there's at least one cased character in the string.
join(iterable, /)— concatenates any number of strings 🫶
The string whose method you called gets inserted between each given string, and the result comes back as a brand new string.
Example: '.'.join(['ab', 'pq', 'rs']) -> 'ab.pq.rs'
ljust(width, fillchar=' ', /)— returns a left-justified string of length width.
Padding uses the specified fill character (default is a space).
lower()— returns a copy of the string converted to lowercase. lowercase era.lstrip(chars=None, /)— returns a copy of the string with leading whitespace removed.
If chars is given and not None, remove characters in chars instead.
static maketrans()— returns a translation table usable forstr.translate(). (note thestatic, bestie, she's built different)
With only one argument, it must be a dictionary mapping Unicode ordinals (integers) or characters to Unicode ordinals, strings or None. Character keys will then be converted to ordinals. With two arguments, they must be strings of equal length, and in the resulting dictionary each character in x will be mapped to the character at the same position in y. With a third argument, it must be a string, whose characters will be mapped to None in the result. three modes, all valid 🌈
partition(sep, /)— partitions the string into three parts using the given separator.
It searches for the separator in the string. If the separator is found, you get a 3-tuple containing the part before the separator, the separator itself, and the part after it.
If the separator isn't found, you get a 3-tuple containing the original string and two empty strings. no drama, just empties.
removeprefix(prefix, /)— returns astrwith the given prefix string removed if present.
If the string starts with the prefix string, return string[len(prefix):]. Otherwise, return a copy of the original string.
removesuffix(suffix, /)— returns astrwith the given suffix string removed if present.
If the string ends with the suffix string and that suffix isn't empty, return string[:-len(suffix)]. Otherwise, return a copy of the original string.
replace(old, new, count=None)— returns a copy with all occurrences of substring old replaced by new.- count — maximum number of occurrences to replace.
-1(the default value) means replace all occurrences.
- count — maximum number of occurrences to replace.
If the optional count argument is given, only the first count occurrences are replaced.
rfind()— returns the highest index inSwhere substringsubis found, such thatsubis contained withinS[start:end].
Optional start and end args work like slice notation. Returns -1 on failure.
rindex()— returns the highest index inSwhere substringsubis found, such thatsubis contained withinS[start:end].
Optional start and end args work like slice notation. Raises ValueError when the substring isn't found. yeah, she's the dramatic sibling of rfind() 🙈
rjust(width, fillchar=' ', /)— returns a right-justified string of length width.
Padding uses the specified fill character (default is a space).
rpartition(sep, /)— partitions the string into three parts using the given separator.
This one searches for the separator in the string starting at the end. If the separator is found, you get a 3-tuple containing the part before the separator, the separator itself, and the part after it.
If the separator isn't found, you get a 3-tuple containing two empty strings and the original string — mirror image of partition(), cute.
rsplit(sep=None, maxsplit=-1)— returns a list of the substrings in the string, using sep as the separator string.- sep — the separator used to split the string.
When set to None (the default value), it will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.
maxsplit— maximum number of splits.-1(which is the default value) means… no limit, go off bestie.
Splitting starts at the end of the string and works its way to the front. Backwards energy, we love it.
🌸 rstrip(chars=None, /)
Returns a copy of the string with the trailing whitespace removed. Cleanup on aisle right-side 🧹
If chars is given and it's not None, remove characters in chars instead. Like, you pick what gets chopped.
⭐ split(sep=None, maxsplit=-1)
Returns a list of the substrings in the string, using sep as the separator string. Basically it's the "cut this up" button ✂️
sep— the separator used to split the string.
When set to None (the default value), it will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result. No ghosty empty entries, periodt.
maxsplit— maximum number of splits.-1(the default value) means no limit.
Splitting starts at the front of the string and works to the end.
Note,str.split()is mainly useful for data that has been intentionally delimited. With natural text that includes punctuation, consider using the regular expression module. Fr fr, don't come forsplit()when your data has commas and vibes in it — that's a regex job 💅
🦋 splitlines(keepends=False)
Returns a list of the lines in the string, breaking at line boundaries. Line-by-line, main character energy 📋
Line breaks are not included in the resulting list unless keepends is given and true. So the newlines only come along if you explicitly invited them 🎀
🍓 startswith()
Returns True if the string starts with the specified prefix, False otherwise.
prefix— a string or a tuple of strings to try. (Yes, a whole tuple, she's flexible 💅)start— optional start position. Default: start of the string.end— optional stop position. Default: end of the string.
🌈 strip(chars=None, /)
Returns a copy of the string with leading and trailing whitespace removed. Both ends, symmetrical queen 👑
If chars is given and it's not None, remove characters in chars instead.
💜 swapcase()
Converts uppercase characters to lowercase and lowercase characters to uppercase. Full flip, no notes 🔄
🧚 title()
Returns a version of the string where each word is titlecased.
More specifically, words start with uppercased characters and all remaining cased characters have lower case. Very "Every Word Capitalized" of her ✨
👑 translate(table, /)
Replaces each character in the string using the given translation table. Character glow-up, one at a time 💫
table— translation table, which must be a mapping of Unicode ordinals to Unicode ordinals, strings, orNone.
The table must implement lookup/indexing via __getitem__, for instance a dictionary or list. If this operation raises LookupError, the character is left untouched. Characters mapped to None are deleted — gone, poof, no trace 💀
🎀 upper()
Returns a copy of the string converted to uppercase. LOUDER 📢
⭐ zfill(width, /)
Pads a numeric string with zeros on the left, to fill a field of the given width. Zeroes on the left like a little number costume 🎀
The string is never truncated. Nope, she refuses.
🧰 optrecurse
class translate.misc.optrecurse.ManHelpFormatter(indent_increment=0, max_help_position=0, width=80, short_first=1)
format_option_strings(option)— returns a comma-separated list of option strings & metavariables.
class translate.misc.optrecurse.ManPageOption(*opts, **attrs)
take_action(action, dest, opt, value, values, parser)— atake_actionthat can handle manpage as well as standard actions. Multi-talented, we stan 🫶
class translate.misc.optrecurse.RecursiveOptionParser(formats, usetemplates=False, allowmissingtemplate=False, description=None)
A specialized Option Parser for recursing through directories. Down the rabbit hole we go 🐇
add_option(Option)add_option(opt_str, ..., kwarg=val, ...) -> Nonecheck_values(values: Values, args: [string])—-> (values : Values, args : [string])
Check that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new – whatever you like). Default implementation just returns the passed-in values; subclasses may override as desired. Do whatever, bestie, it's your parser 💅
checkoutputsubdir(options, subdir)— checks to see ifsubdirunderoptions.outputneeds to be created, creates if neccessary.define_option(option)— defines the given option, replacing an existing one of the same short name if neccessary…destroy()— declare that you are done with thisOptionParser. This cleans up reference cycles so theOptionParser(and all objects referenced by it) can be garbage-collected promptly. After callingdestroy(), theOptionParseris unusable. She's retired, don't call her back 😭disable_interspersed_args()— sets parsing to stop on the first non-option. Use this if you have a command processor which runs another command that has options of its own and you want to make sure these options don't get confused. Boundary setting, iconic 🔥enable_interspersed_args()— sets parsing to not stop on the first non-option, allowing interspersing switches with command arguments. This is the default behavior. See alsodisable_interspersed_args()and the class documentation description of the attributeallow_interspersed_args.error(msg: string)— print a usage message incorporating 'msg' to stderr and exit. If you override this in a subclass, it should not return – it should either exit or raise an exception. No casual returns allowed 🚫finalizetempoutputfile(options, outputfile, fulloutputpath)— write the temp output file to its final destination. Home at last 💖format_manpage()— returns a formatted manpage.static getformathelp(formats)— make a nice help string for describing formats…static getfullinputpath(options, inputpath)— gets the full path to an input file.static getfulloutputpath(options, outputpath)— gets the full path to an output file.getfulltemplatepath(options, templatepath)— gets the full path to a template file.getoutputname(options, inputname, outputformat)— gets an output filename based on the input filename.getoutputoptions(options, inputpath, templatepath)— works out which output format and processor method to use…getpassthroughoptions(options)— get the options required to pass to thefiltermethod…gettemplatename(options, inputname)— gets an output filename based on the input filename. (Yes, this one says the same thing asgetoutputname— that's what the page says, don't come for me 🙈)static getusageman(option)— returns the usage string for the given option.static getusagestring(option)— returns the usage string for the given option.static isexcluded(options, inputpath)— checks if this path has been excluded. Blocked 🚫static isrecursive(fileoption, filepurpose='input')— checks iffileoptionis a recursive file.isvalidinputname(inputname)— checks if this is a valid input filename.static mkdir(parent, subdir)— makes a subdirectory (recursively if neccessary).static openinputfile(options, fullinputpath)— opens the input file.static openoutputfile(options, fulloutputpath)— opens the output file.opentemplatefile(options, fulltemplatepath)— opens the template file (if required).static opentempoutputfile(options, fulloutputpath)— opens a temporary output file.parse_args(args=None, values=None)— parses the command line options, handling implicit input/output args. Sneaky helpful 🎀print_help(file: file = stdout)— print an extended help message, listing all options and any help text provided with them, to 'file' (defaultstdout).print_manpage(file=None)— outputs a manpage for the program using the help information. Meta! ✨print_usage(file: file = stdout)— print the usage message for the current program (self.usage) to 'file' (defaultstdout). Any occurrence of the string "%prog" inself.usageis replaced with the name of the current program (basename ofsys.argv[0]). Does nothing ifself.usageis empty or not defined.print_version(file: file = stdout)— print the version message for this program (self.version) to 'file' (defaultstdout). As withprint_usage(), any occurrence of "%prog" inself.versionis replaced by the current program's name. Does nothing ifself.versionis empty or undefined.processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)— process an individual file.recurseinputfilelist(options)— use a list of files, and find a common base directory for them.recurseinputfiles(options)— recurse through directories and return files to be processed.recursiveprocess(options)— recurse through directories and process files.run()— parses the arguments, and runsrecursiveprocesswith the resulting options…set_usage(usage=None)— sets the usage string – ifusagenot given, usesgetusagestringfor each option.seterrorleveloptions()— sets the errorlevel options.setformats(formats, usetemplates)— sets the format options using the given format dictionary.
Parameters
formats (Dictionary or iterable) –
The dictionary keys should be:
- Single strings (or 1-tuples) holding an input format (only when not
usetemplates) - Tuples holding an input format plus a template format (when
usetemplatesis on) - Formats can be
None— that's the little signal for "this one works off standard input, bestie"
And the dictionary values? Those are tuples of outputformat (a string) and the processor method. Simple as.
setmanpageoption() — builds a manpage option so the optionparser can literally generate a whole manpage for you. ✨
setprogressoptions() — sets the progress options. That's it, that's the whole gig. 🎀
static splitext(pathname)
Splits pathname into a name and an ext, and — key detail — removes the extsep while it's at it.
- Parameters: pathname (
string) – a file path - Returns:
root, ext - Return type:
tuple
splitinputext(inputpath)
Splits an inputpath into name and extension. Very demure, very mindful.
splittemplateext(templatepath)
Splits a templatepath into name and extension. Same energy, different path. 💅
templateexists(options, templatepath)
Returns whether the given template exists… (yes, the original really does trail off like that. she's mysterious. 🦋)
warning(msg, options=None, exc_info=None)
Prints a warning message to stderr, with msg baked right into it.
🧚 ourdom
A module that hands us modified DOM functionality, customised for our needs. No cap.
Heads up, bestie: anyone usingourdomneeds to make sure no code is still reaching for classes directly out ofminidom— stuff likeminidom.Element,minidom.Document, or methods such asminidom.parseString— because the functionality we provide here will not be in those objects. Like, at all. 💀
class translate.misc.ourdom.Document
- property
documentElement— the top-level element of this document. main character energy 👑 - property
firstChild— first child node, orNone. - property
lastChild— last child node, orNone. - property
localName— namespace-local name of this node.
class translate.misc.ourdom.Element(tagName, namespaceURI=None, prefix=None, localName=None)
- property
attributes— theNamedNodeMapof attributes on the element. - property
firstChild— first child node, orNone. getAttribute(attname)— returns the value of the specified attribute.
So like: it gives you the value of the element's attribute named attname, as a string. If the element doesn't have such an attribute you get an empty string back. But also — plot twist — an explicitly given attribute value can also be an empty string, so use the hasAttribute method to tell those two cases apart. iykyk 🔥
getElementsByTagName(name)— returns all descendant elements with the given tag name.
That's the list of all descendant elements (not just direct children, bestie) with the specified tag name.
hasAttribute(name)— checks whether the element has an attribute with the specified name.
Returns True if the element has an attribute with that specified name. Otherwise? False. Boom. 💅
- property
lastChild— last child node, orNone. - property
localName— namespace-local name of this element. writexml(writer, indent, addindent, newl)— writes an XML element to a file-like object.
Writes the element out to the writer object, which has to provide a write method — think a file or a StringIO object, that kind of thing.
class translate.misc.ourdom.ExpatBuilderNS(options=None)
createParser()— creates a new namespace-handling parser. ✨getParser()— returns the parser object, creating a new one if it's needed. very "if not exists, make one" of her.install(parser)— inserts the namespace-handlers onto the parser.parseFile(file)— parses a document from a file object, returning the document node.parseString(string)— parses a document from a string, returning the document node.reset()— frees all the data structures used during DOM construction. clean slate vibes 🌸start_namespace_decl_handler(prefix, uri)— pushes this namespace declaration onto our storage.
translate.misc.ourdom.getElementsByTagName_helper(parent, name, dummy=None)
A reimplementation of getElementsByTagName as an iterator. 🦋
Note that this is not compatible withgetElementsByTagName, which returns a list — so the class below exposes this throughyieldElementsByTagNameinstead. Important. Don't mix them up, bestie.
translate.misc.ourdom.getnodetext(node)
Returns the node's text by iterating through the child nodes. one by one, deliberately, like a queen. 👑
translate.misc.ourdom.parse(file, parser=None, bufsize=None)
Parses a file into a DOM, by filename or by file object.
translate.misc.ourdom.parseString(string, parser=None)
Parses a file into a DOM from a string.
translate.misc.ourdom.searchElementsByTagName_helper(parent, name, onlysearch)
Limits the search to within tags occurring in onlysearch. stay in your lane, tags. 💅
translate.misc.ourdom.writexml_helper(self, writer, indent='', addindent='', newl='')
A replacement for writexml that formats output like your typical XML files. Nodes get indented — but text nodes, where whitespace can be significant, do not get indented. lowkey the most important rule here, ngl. 🫶
🍓 progressbar
Progress bar utilities for reporting feedback on how far along an application is. So you're never just staring at a silent terminal like 😭.
class translate.misc.progressbar.DotsProgressBar
An ultra-simple progress indicator that just writes a dot for each action.
show(verbosemessage)— shows a dot for progress :-). iconic, minimal, ate.
class translate.misc.progressbar.HashProgressBar(*args, **kwargs)
A ProgressBar which knows how to go back to the beginning of the line.
show(verbosemessage)— displays the progress bar.
class translate.misc.progressbar.MessageProgressBar(*args, **kwargs)
A ProgressBar that just writes out the messages, with no progress display at all.
show(verbosemessage)— displays the progress bar.
class translate.misc.progressbar.NoProgressBar
An invisible indicator that does literally nothing. ghost mode. 👻
show(verbosemessage)— shows nothing for progress :-).
class translate.misc.progressbar.ProgressBar(minValue=0, maxValue=100, totalWidth=50)
A plain progress bar that doesn't know very much about output.
show(verbosemessage)— displays the progress bar.
class translate.misc.progressbar.VerboseProgressBar(*args, **kwargs)
show(verbosemessage)— displays the progress bar.
💜 quote
String processing utilities for extracting strings with all sorts of different delimiters.
translate.misc.quote.entitydecode(source, name2codepoint)
Decodes source using entities from name2codepoint.
Parameters:
- source (
unicode) – source string to decode - name2codepoint (
dict()) – dictionary mapping entity names (without the leading&or the trailing;) to code points
translate.misc.quote.entityencode(source, codepoint2name)
Encodes source using entities from codepoint2name.
Parameters:
- source (
unicode) – source string to encode - codepoint2name (
dict()) – dictionary mapping code points to entity names (without the leading&or the trailing;)
translate.misc.quote.escapecontrols(source)
Escapes control characters in the given string. byeee weird invisible characters 💅
translate.misc.quote.extract(source, startdelim, enddelim, escape=None, startinstring=False, allowreentry=True)
Extracts a doublequote-delimited string from a string, allowing for backslash-escaping. Returns a tuple of (quoted string with quotes, still in string at end).
translate.misc.quote.extractwithoutquotes(source, startdelim, enddelim, escape=None, startinstring=False, includeescapes=True, allowreentry=True)
Extracts a doublequote-delimited string from a string, allowing for backslash-escaping. Oh, and includeescapes can also be a function that takes the whole escaped string and returns the replaced version. fancy 🎀
translate.misc.quote.find_all(searchin, substr)
Returns a list of locations where substr occurs in searchin. The locations are not allowed to overlap — so no double-dipping, bestie.
translate.misc.quote.htmlentitydecode(source)
Decodes source using HTML entities, e.g. © -> ©.
Parameters: source (unicode) – source string to decode
translate.misc.quote.htmlentityencode(source)
Encodes source using HTML entities, e.g. © -> ©.
Parameters: source (unicode) – source string to encode
translate.misc.quote.java_utf8_properties_encode(source)
Encodes source in the escaped-unicode encoding used by java utf-8 .properties files. 🌈
translate.misc.quote.javapropertiesencode(source)
Encodes source in the escaped-unicode encoding used by Java .properties files.
translate.misc.quote.mozillaescapemarginspaces(source)
Escapes leading and trailing spaces for Mozilla .properties files. those sneaky margin spaces were doing too much anyway.
translate.misc.quote.propertiesdecode(source)
Decodes source from the escaped-unicode encoding used by .properties files.
Java uses Latin1 by default, and Mozilla uses UTF-8 by default.
And here's the lore, bestie: since the.decode("unicode-escape")routine decodes everything, and we very much do not want that, we reimplemented the algorithm fromPython Objects/unicode.cin Python and modified it to retain escaped control characters. real ones know. 🔥
⭐ wsgi
Wrapper to launch the bundled CherryPy server.
translate.misc.wsgi.launch_server(host, port, app, **kwargs)
Uses the cheroot WSGI server — a multithreaded, scalable server. go off. 👑
🦋 xml_helpers
Helper functions for working with XML.
translate.misc.xml_helpers.clear_content(node)
Removes XML node content.
Unlike clear() this one is not removing attributes. read that twice, bestie.
translate.misc.xml_helpers.expand_closing_tags(elem)
Changes the value of empty XML tags to an empty string.
This changes lxml behavior so these tags render as <tag></tag> instead of <tag />. tiny difference, huge personality. ✨
translate.misc.xml_helpers.getText(node, xml_space='preserve')
Extracts the plain text content out of the given node. 🫶
…and yeah, that method checks the xml:space attribute of whatever node you hand it, plus there's an optional default you can pass for when this node doesn't specify anything. Slay, moving on 🫶
translate.misc.xml_helpers.getXMLlang(node)— grabs thexml:langattribute onnode. Easy.translate.misc.xml_helpers.getXMLspace(node, default=None)— grabs thexml:spaceattribute onnode, with an optionaldefault.translate.misc.xml_helpers.namespaced(namespace, name)— returnsnamein Clark notation inside the namespace you gave it.
Like, for example, namespaced(“source”) in an XLIFF document could totally return:
{urn:oasis:names:tc:xliff:document:1.1}sourceThis is genuinely needed all over lxml, no cap.
translate.misc.xml_helpers.normalize_space(text: str)— normalizes the text you pass soxml:space="default"actually gets implemented properly.translate.misc.xml_helpers.normalize_xml_space(node, xml_space: str, remove_start: bool = False)— normalizes spaces following the node'sxml:space, or alternatively thexml_spaceparameter you pass in.translate.misc.xml_helpers.reindent(elem, level: int = 0, indent: str = ' ', max_level: int = 4, skip: list[str] | None = None, toplevel=True, leaves: list[str] | None = None)— adjusts the indentation so it matches the spec ✨ Every nested tag gets identified by theindentstring, up tomax_leveldeep, and it can possibly skip tags that are listed inskip. (The signature also carriestoplevelandleaves.)translate.misc.xml_helpers.setXMLlang(node, lang)— sets thexml:langattribute onnode.translate.misc.xml_helpers.setXMLspace(node, value)— sets thexml:spaceattribute onnode.translate.misc.xml_helpers.string_xpath = string()— returns a non-normalized string in the node subtree.translate.misc.xml_helpers.string_xpath_normalized = normalize-space()— returns a (space) normalized string in the node subtree.translate.misc.xml_helpers.valid_chars_only(text: str) -> str— keeps libxml from literally crashing on unexpected chars. Protective queen behaviour 💅translate.misc.xml_helpers.validate_char(char: str) -> bool— figures out if a char is valid for XML, based onxmlIsChar_chfrom ‐ https://github.com/GNOME/libxml2/blob/master/include/libxml/chvalid.h.translate.misc.xml_helpers.xml_preserve_ancestors = ancestor-or-self::*[attribute::xml:space='preserve']— all ancestors withxml:space='preserve'.translate.misc.xml_helpers.xml_space_ancestors = ancestor-or-self::*/attribute::xml:space— all thexml:spaceattributes hanging out in the ancestors.
🔎 search
Services for searching and matching of text. It's giving search engine energy but for translations 🦋
🧚 lshtein
A class that calculates a similarity based on the Levenshtein distance.
SEE ALSO: Levenshtein_distance
If it's available, python-Levenshtein gets used, which gives way better performance since it's implemented natively. Faster is better, periodt.
translate.search.lshtein.distance(a, b, stopvalue=0)— same functionality aspython_distance. Uses the fast C version if we detected it earlier.
Heads up bestie: this does not support arbitrary sequence types — only string types. No cap.
translate.search.lshtein.native_distance(a, b, stopvalue=0)— same functionality aspython_distance. Uses the fast C version if we detected it earlier.
Same caveat, literally: no arbitrary sequence types, only string types 💀
translate.search.lshtein.python_distance(a, b, stopvalue=-1)— calculates the distance for use in similarity calculation. The Python version.
🎯 match
Class to perform translation memory matching from a store of translation units.
class translate.search.match.matcher(store, max_candidates=10, min_similarity=75, max_length=70, comparer=None, usefuzzy=False)
A class that does the matching and stores all the configuration for the matching process. Main character energy honestly 👑
buildunits(candidates)— builds a list of units that conform to the base API, with the score chilling in the comment.extendtm(units, store=None, sort=True)— extends the memory with extra unit(s).
Parameters:
units– the units to add to the TM.store– optional store where some metadata can be retrieved from and associated with each unit.sort– optional parameter you can set toFalseto supress sorting of the candidates list. You should probably only use this inmatcher.inittm().
static getstartlength(min_similarity, text)— calculates the minimum length we're interested in. The extra fat is there because we don't use plain character distance only.getstoplength(min_similarity, text)— calculates a length beyond which we're just not interested anymore. Again, extra fat because we don't use plain character distance only.inittm(stores, reverse=False)— initialises the memory for later use. We use simple base units for speedup.matches(text)— returns a list of possible matches for the given source text.
Parameters:
text(String) – the text that will be searched for in the translation memory.
Return type: list
Returns: a list of units with the source and target strings from the translation memory. If self.addpercentage is True (which is the default), the match quality is given as a percentage in the notes.
setparameters(max_candidates=10, min_similarity=75, max_length=70)— sets the parameters without reinitialising the tm. Lowkey important: if a parameter isn't specified, it gets set to the default — it is not ignored 🙈usable(unit)— returns whether this translation unit is usable for TM.
translate.search.match.sourcelen(unit)
Returns the length of the source string. That's it, that's the whole vibe.
class translate.search.match.terminologymatcher(store, max_candidates=10, min_similarity=75, max_length=500, comparer=None)
A matcher with settings specifically for terminology matching. Note the max_length=500 glow-up ✨
buildunits(candidates)— builds a list of units that conform to the base API, with the score in the comment.extendtm(units, store=None, sort=True)— extends the memory with extra unit(s).
Parameters:
units– the units to add to the TM.store– optional store where some metadata can be retrieved from and associated with each unit.sort– optional parameter that can be set toFalseto supress sorting of the candidates list. This should probably only be used inmatcher.inittm().
getstartlength(min_similarity, text)— calculates the minimum length we're interested in. The extra fat is because we don't use plain character distance only.getstoplength(min_similarity, text)— calculates a length beyond which we're not interested. Extra fat, same reason: we don't use plain character distance only.inittm(store)— normal initialisation, but it converts all source strings to lower case. Down bad for lowercase, honestly.matches(text)— normal matching after convertingtextto lower case. Then it replaces with the original unit so comments etc. are retained 🎀setparameters(max_candidates=10, min_similarity=75, max_length=70)— sets the parameters without reinitialising the tm. If a parameter isn't specified, it's set to the default, not ignored.usable(unit)— returns whether this translation unit is usable for terminology.
translate.search.match.unit2dict(unit)
Converts a pounit to a simple dict structure for use over the web. Web-friendly realness 🌈
📚 terminology
A class that does terminology matching.
🌐 services
translate.services is part of the translate toolkit. It provides network services for interacting with the toolkit. Networking era 🦋
📡 tmserver
A translation memory server using tmdb for storage, communicating with clients using JSON over HTTP.
class translate.services.tmserver.TMServer(tmdbfile, tmfiles, max_candidates=3, min_similarity=75, max_length=1000, prefix='', source_lang=None, target_lang=None)
A RESTful JSON TM server. That's the whole flex 💅
💾 storage
Classes that represent various storage formats for localization. The closet of this toolkit, but make it organized.
🧱 base
Base classes for storage interfaces.
class translate.storage.base.DictStore(unitclass=None, encoding=None)
Attributes, bestie:
Extensions = None— a list of file extentions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human usable name of this store type.UnitClass— alias ofTranslationUnit.
Methods:
add_unit_to_index(unit)— adds a unit to the source and location idexes.addsourceunit(source)— adds and returns a new unit with the given source string.
Return type: TranslationUnit
addunit(unit)— appends the given unit to the object's list of units.
Fr fr, always use this method instead of trying to modify the list manually. Don't be that person 😭
Parameters:
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from the text, either using thechardetlib or by trying to decode the file.static fallback_detection(text)— simple detection based on BOM, for whenchardetisn't available. Backup plan iconique.findid(id)— finds the unit with the matching id by checkingid_index.findunit(source)— finds the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)— finds the units with the given source string.
Return type: TranslationUnit or None
getids()— returns a list of unit ids.
...okay so we're still deep in TranslationUnit land and there's a lot left, bestie, but every single one of these is a lil baddie and we're gonna give them all their moment 💅
getprojectstyle()— gets the project type for this store. that's it, that's the whole vibe ✨getsourcelanguage()— gets the source language for this store 🌸gettargetlanguage()— gets the target language for this store 🎀getunits()— returns a list of all the units in this store. the whole squad 👑isempty()— returnsTrueif the object doesn't contain any translation units. empty store = empty vibes frmakeindex()— indexes the items in this store. ngl at least.sourceindexshould be useful 🔥merge_on(property) — the matching criterion to use when merging on.- Returns — the default matching criterion for all the subclasses.
- Return type —
string
parse(data)— parser to process the given source string.parsefile(storefile)(classmethod) — reads the given file (or opens the given filename) and parses back to an object. it literally reads it back into existence, iconicparsestring(storestring)(classmethod) — converts the string representation back to an object.remove_unit_from_index(unit)— removes a unit from source andlocatonindexes. (yes bestie that's how it's spelled in the source, we do not fix, we respect 🫶)removeunit(unit)— removes the given unit to the object's list of units.
okay this one is IMPORTANT: you always use this method, never ever try to modify the list manually yourself. hands off the list, bestie 🙈
- Parameters — unit (
TranslationUnit) – the unit that will be added.
require_index()— makes sure the source index exists. manifesting it into being ✨save()— save to the file that data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write to.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates if format can store suggestions and alternative translation for a unit.translate(source)— returns the translated string for a given source string.- Return type —
StringorNone
- Return type —
unit_iter()— iterator over all the units in this store. go off iterator, go off 💫
🦋 class translate.storage.base.DictUnit(source=None)
adderror(errorname, errortext)— adds an error message to this unit.- Parameters
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
- Parameters
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. know your limits, king 💀
addlocations(location)— adds a location or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. it's giving deprecated, iykyk 😭
addnote(text, origin=None, position='append')— adds a note (comment).- Parameters
text(string) – usually just a sentence or two.origin(string) – specifies who/where the comment comes from.origincan be one of the following text strings: -'translator'-'developer','programmer','source code'(synonyms)
- Parameters
buildfromunit(unit)(classmethod) — builds a native unit from a foreign unit.
preserving as much information as possible. we're not losing a single crumb 🍓
getcontext()(static) — gets the message context.geterrors()(static) — gets all error messages.- Return type —
Dictionary
- Return type —
getid()— a unique identifier for this unit.- Return type —
string - Returns — an identifier for this unit that is unique in the store.
- Return type —
derived classes should override this in a way that guarantees a unique identifier for each unit in the store. no duplicates allowed, periodt 💅
getlocations()(static) — a list of source code locations.- Return type —
List
- Return type —
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all notes about this unit.
it will probably be freeform text or something reasonable that can be synthesised by the format. it should not include location comments (see getlocations()). keep those separate bestie 🎀
gettargetlen()— returns the length of the target string.- Return type —
Integer
- Return type —
NOTE: plural forms might be combined. don't come for me about the math 🙈
getunits()— this unit in a list. just the one. a party of one 💜getvalue()— returns dictionary for serialization.hasplural()(static) — tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— used to see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. it's giving legacy code ngl 😭
isfuzzy()(static) — indicates whether this unit is fuzzy.isheader()(static) — indicates whether this unit is a header.isobsolete()(static) — indicate whether a unit is obsolete.isreview()(static) — indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated.
this should be used to distinguish real units for translation from header, obsolete, binary or other blank units. real ones only ✨
istranslated()— indicates whether this unit is translated.
this should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does). don't do the shortcut, bestie 🫶
makeobsolete()— makes a unit obsolete. it's over for that unit fr 💀markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.- Parameters
needsreview– defaults toTrue.explanation– adds an optional explanation as a note.
- Parameters
merge(otherunit, overwrite=False, comments=True, authoritative=False)— do basic format agnostic merging.multistring_to_rich(mulstring)— convert a multistring to a list of "rich" string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator's notes. sorry translator bestie, they're gone 🌸rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.rich_source(property)
SEE ALSO:rich_to_multistring(),multistring_to_rich().
rich_target(property)
SEE ALSO:rich_to_multistring(),multistring_to_rich().
rich_to_multistring(elem_list)(classmethod) — convert a "rich" string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setid(value, unitid=None)— sets the unique identified for this unit.
only implemented if format allows ids independant from other unit properties like source or context. (typos intact, we stan consistency 💅)
unit_iter()— iterator that only returns this unit. one unit. just the one. main character energy 👑
💀 exception translate.storage.base.ParseError(inner_exc)
add_note(object, /)—Exception.add_note(note)– add a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb)– setself.__traceback__totband returnself.
🌈 class translate.storage.base.TranslationStore(unitclass=None, encoding=None)
base class for stores for multiple translation units of type UnitClass. the mothership of stores, lowkey iconic 🧚
Extensions = None— a list of file extentions associated with this store typeMimetypes = None— a list of MIME types associated with this store typeName = 'Base translation store'— the human usable name of this store typeUnitClass— the class of units that will be instantiated and used by this class
alias of TranslationUnit
add_unit_to_index(unit)— add a unit to source and locationidexes. (spelling is canon, do not touch 🫶)addsourceunit(source)— add and returns a new unit with the given source string.
Return type: TranslationUnit 💖
addunit(unit)
Appends the unit you hand it to the object's list of units 🫶
And listen — always use this method instead of trying to poke the list yourself, bestie. No manual list surgery, ok?
Parameters
unit(TranslationUnit) — the unit that's getting added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]
Tries to sniff out a file encoding from text, either by leaning on the chardet lib or just by straight-up attempting to decode the file 🔍
static fallback_detection(text)
A lil' simple detection based on the BOM, for when chardet isn't available. Backup plan energy ✨
findid(id)
Finds the unit with a matching id by checking id_index.
findunit(source)
Finds the unit that has the given source string.
Return type: TranslationUnit or None
findunits(source)
Finds the units with the given source string.
Return type: TranslationUnit or None
getids()
Returns a list of unit ids 🎀
getprojectstyle()
Gets the project type for this store.
getsourcelanguage()
Gets the source language for this store.
gettargetlanguage()
Gets the target language for this store.
getunits()
Returns a list of all the units in this store.
isempty()
Returns True if the object doesn't contain any translation units. Empty-store check, iconic 💅
makeindex()
Indexes the items in this store. At the very least .sourceindex should be useful.
property merge_on
The matching criterion to use when merging on.
Returns: the default matching criterion for all the subclasses.
Return type: string
parse(data)
The parser that processes the given source string.
classmethod parsefile(storefile)
Reads the given file (or opens the given filename) and parses it back into an object.
classmethod parsestring(storestring)
Converts the string representation back into an object.
remove_unit_from_index(unit)
Removes a unit from the source and location indexes.
removeunit(unit)
Removes the given unit from the object's list of units.
Same deal as adding — always use this method rather than messing with the list manually, k?
Parameters
unit(TranslationUnit) — the unit that will be added.
require_index()
Makes sure the source index actually exists. Gotta have it 🌸
save()
Saves to the file that data was originally read from — if that's available.
savefile(storefile)
Writes the string representation to the given file (or filename).
serialize(out)
Converts to a bytes representation that can be parsed back with parsestring(). The out argument should be an open file-like object to write to.
setprojectstyle(project_style)
Sets the project type for this store.
setsourcelanguage(sourcelanguage)
Sets the source language for this store.
settargetlanguage(targetlanguage)
Sets the target language for this store.
suggestions_in_format = False
Indicates whether the format can store suggestions and alternative translations for a unit.
translate(source)
Returns the translated string for a given source string.
Return type: String or None
unit_iter()
An iterator over all the units in this store. Go off, loop queen 👑
class translate.storage.base.TranslationUnit(source=None)
The base class for translation units 🧚
Our whole concept of a translation unit is heavily influenced by XLIFF — so yeah, most of the method and variable names are borrowed straight from XLIFF terminology. Iykyk.
A translation unit is made of this:
- A source string. That's the original translatable text.
- A target string. That's the translation of the source.
- Zero or more notes on the unit. Notes are typically comments from a translator about the unit, or comments coming from the source code.
- Zero or more locations. Locations show where in the original source code this unit came from.
- Zero or more errors. Some tools (eg.
pofilter) can run checks on translations and spit out error messages.
adderror(errorname, errortext)
Adds an error message to this unit.
Parameters
errorname(string) — a single word to id the error.errortext(string) — the text describing the error.
addlocation(location)
Adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it 🙈
addlocations(location)
Adds a location, or a list of locations.
NOTE: most classes shouldn't need to implement this — they should implement TranslationUnit.addlocation() instead.
WARNING: this method might be removed in future. Heads up, bestie 💀
addnote(text, origin=None, position='append')
Adds a note (aka a comment) 📝
Parameters
text(string) — usually just a sentence or two.origin(string) — says who/where the comment comes from.origincan be one of these text strings:'translator','developer','programmer','source code'(those last three are synonyms).
classmethod buildfromunit(unit)
Builds a native unit from a foreign unit — preserving as much information as possible 🫶
static getcontext()
Gets the message context.
static geterrors()
Gets all the error messages.
Return type: Dictionary
getid()
A unique identifier for this unit.
Return type: string
Returns: an identifier for this unit that's unique in the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. No duplicates allowed, periodt ✨
static getlocations()
A list of source code locations.
Return type: List
NOTE: shouldn't be implemented if the format doesn't support it 🙈
getnotes(origin=None)
Returns all the notes about this unit.
It'll probably be freeform text, or something reasonable that the format can synthesise. It should not include location comments — for those, see getlocations().
gettargetlen()
Returns the length of the target string.
Return type: Integer
NOTE: plural forms might be combined.
getunits()
This unit, in a list. Just the one. Solo main character energy 👑
static hasplural()
Tells you whether or not this specific unit has plural strings.
infer_state()
An empty method that should be overridden in subclasses to infer the current state(_n) of the unit from its current state.
isblank()
Used to check whether this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather than get rid of it.
static isfuzzy()
Indicates whether this unit is fuzzy.
static isheader()
Indicates whether this unit is a header.
static isobsolete()
Indicates whether a unit is obsolete.
static isreview()
Indicates whether this unit needs review.
istranslatable()
Indicates whether this unit can be translated.
This should be used to separate real translation units from headers, obsolete units, binary units, or other blank units 🌈
istranslated()
Indicates whether this unit is translated.
Use this rather than deducing it from .target, so other classes can implement more functionality (like XLIFF does). Trust the method, not the vibe check 💅
makeobsolete()
Makes a unit obsolete.
markfuzzy(value=True)
Marks the unit as fuzzy, or not fuzzy.
markreviewneeded(needsreview=True, explanation=None)
Marks the unit to indicate whether it needs review.
Parameters
needsreview— defaults toTrue.explanation— adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)
Does basic, format-agnostic merging 🤝
multistring_to_rich(mulstring)
Converts a multistring into a list of "rich" string trees 🌳
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)
Removes all the translator's notes.
rich_parsers = []
A list of functions to use for parsing a string into a rich string tree.
property rich_source
💅 SEE ALSO:rich_to_multistring(),multistring_to_rich(). (same little duo as above, no cap 🫶)
property rich_target — the target-side rich property, serving the exact same vibes ✨
💅 SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list) — converts a “rich” string tree into a multistring, and it's honestly so satisfying to watch 🦋
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context) — sets the message context. Context is that main character energy for your strings 💖
setid(value) — sets the unique identifier for this unit 🎀
Key caveat bestie: this is only implemented if the format allows ids to be independant from other unit properties like source or context.
unit_iter() — an iterator that only returns this unit. Like, one unit. Just the one. It's giving solo era 🌸
🌈 benchmark
class translate.storage.benchmark.TranslateBenchmarker(test_dir, storeclass) — a whole class to aid in benchmarking Translate Toolkit stores. Speedrun those stores, fr fr 🔥
clear_test_dir() — removes the given directory. Bye bestie 👋
create_sample_files(num_dirs, files_per_dir, strings_per_file, source_words_per_string, target_words_per_string) — creates sample files for benchmarking. All those knobs, all that control ✨
parse_files(file_dir=None) — parses all the files in the test directory into memory 🧠
parse_placeables() — parses placeables.
🎀 bundleprojstore
class translate.storage.bundleprojstore.BundleProjectStore(fname) — represents a translate project bundle (zip archive). It's a zip file full of project files, iconic 📦
append_file(afile, fname, ftype='trans', delete_orig=False) — appends the given file to the project with the given filename, marked to be of type ftype ('src', 'trans', 'tgt').
- Parameters
delete_orig– ifTrue, as set byconvert_forward(),afileis deleted after appending, if possible.
⚠️ NOTE: for this implementation, the appended file will be deleted from disk ifdelete_origisTrue. Like, actually gone. Be so serious about this one 😭
cleanup() — cleans up our mess: removes temporary files. Cleaning arc 🧹
get_file(fname) — retrieves a project file (source, translation or target file) from the project archive 💖
get_filename_type(fname) — gets the type of file ('src', 'trans', 'tgt') with the given name.
get_proj_filename(realfname) — tries to find a project file name for the given real file name. Bestie detective work 🔍
load(zipname) — loads the bundle project from the zip file of the given name 🦋
remove_file(fname, ftype=None) — removes the file with the given project name from the project.
save(filename=None) — saves all project files to the bundle zip file ✨
property sourcefiles — read-only access to self._sourcefiles.
property targetfiles — read-only access to self._targetfiles.
property transfiles — read-only access to self._transfiles.
update_file(pfname, infile) — updates the file with the given project file name with the contents of infile.
- Returns: the results from
BundleProjStore.append_file().
exception translate.storage.bundleprojstore.InvalidBundleError — the exception that gets thrown when a bundle is not it 💀
add_note(object, /) — Exception.add_note(note) – add a note to the exception 📝
with_traceback(object, /) — Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
🦋 catkeys
Manage the Haiku catkeys translation format 💖
The Haiku catkeys format is the translation format used for localisation of the Haiku operating system 🎀
It's a bilingual base class derived format, with CatkeysFile and CatkeysUnit providing file-level and unit-level access. The file format is described here: http://www.haiku-os.org/blog/pulkomandy/2009-09-24_haiku_locale_kit_translator_handbook 👑
💜 Implementation
The implementation covers the full requirements of a catkeys file. These files are simple Tab Separated Value (TSV) files that can be read by Microsoft Excel and other spreadsheet programs. They use the .txt extension, which does make it more difficult to automatically identify such files 😭
The dialect of the TSV files is specified by CatkeysDialect.
🌸 Encoding
The files are UTF-8 encoded ✨
🌈 Header
CatkeysHeader provides header management support.
🧚 Escaping
catkeys seem to escape things like in C++ (strings are just extracted from the source code unchanged, it seems).
Functions allow for _escape() and _unescape().
class translate.storage.catkeys.CatkeysDialect — describes the properties of a catkeys generated TAB-delimited file 🎀
class translate.storage.catkeys.CatkeysFile(inputfile=None, **kwargs) — a catkeys translation memory file 💅
Extensions = ['catkeys']— a list of file extentions associated with this store typeMimetypes = ['application/x-catkeys']— a list of MIME types associated with this store typeName = 'Haiku catkeys file'— the human usable name of this store typeUnitClass— alias ofCatkeysUnit
add_unit_to_index(unit) — add a unit to source and location idexes 🔍
addsourceunit(source) — add and return a new unit with the given source string.
- Return type:
TranslationUnit
addunit(unit) — append the given unit to the object's list of units.
- This method should always be used rather than trying to modify the list manually, bestie 🙈
- Parameters:
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str] — try to detect a file encoding from text, using either the chardet lib or by trying to decode the file 🔥
static fallback_detection(text) — simple detection based on BOM in case chardet is not available.
findid(id) — find unit with matching id by checking id_index.
findunit(source) — find the unit with the given source string.
- Return type:
TranslationUnitorNone
findunits(source) — find the units with the given source string.
- Return type:
TranslationUnitorNone
getids() — return a list of unit ids 🎀
getprojectstyle() — get the project type for this store.
getsourcelanguage() — get the source language for this store.
gettargetlanguage() — get the target language for this store.
getunits() — return a list of all units in this store ✨
isempty() — return True if the object doesn't contain any translation units.
makeindex() — indexes the items in this store. At least .sourceindex should be useful 🦋
property merge_on — the matching criterion to use when merging on.
- Returns: the default matching criterion for all the subclasses.
- Return type: string
parse(input) — parse the given file or file source string 💖
classmethod parsefile(storefile) — reads the given file (or opens the given filename) and parses back to an object.
classmethod parsestring(storestring) — convert the string representation back to an object.
remove_unit_from_index(unit) — remove a unit from source and locaton indexes 👋
removeunit(unit) — remove the given unit from the object's list of units.
- This method should always be used rather than trying to modify the list manually.
- Parameters:
unit(TranslationUnit) – the unit that will be added.
require_index() — make sure source index exists.
save() — save to the file that data was originally read from, if available 💾
savefile(storefile) — write the string representation to the given file (or filename).
serialize(out) — converts to a bytes representation that can be parsed back using parsestring(). out should be an open file-like objects to write to.
setprojectstyle(project_style) — set the project type for this store.
setsourcelanguage(sourcelanguage) — set the source language for this store.
settargetlanguage(newlang) — set the target language for this store.
suggestions_in_format = False — indicates if format can store suggestions and alternative translation for a unit.
translate(source) — return the translated string for a given source string.
- Return type: String or
None
unit_iter() — iterator over all the units in this store 🌈
class translate.storage.catkeys.CatkeysHeader(header=None) — a catkeys translation memory header 👑
setchecksum(checksum) — set the checksum for the file.
settargetlanguage(newlang) — set a human readable target language 💬
class translate.storage.catkeys.CatkeysUnit(source=None) — a catkeys translation memory unit 🎀
adderror(errorname, errortext) — adds an error message to this unit.
- Parameters
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
addlocation(location) — add one location to the list of locations 📍
NOTE: bestie, this shouldn't be implemented if the format doesn't support it 💅
addlocations(location)— adds a location, or a whole list of locations, no big deal.
NOTE: most classes shouldn't need to implement this — they should implement TranslationUnit.addlocation() instead, okurrr? 🫶
WARNING: this method might get removed in the future, so like… don't get too attached 💀
addnote(text, origin=None, position='append')— adds a note (a comment, basically) to the unit.
Parameters:
- text (string) — usually just a sentence or two, keep it cute ✨
- origin (string) — says who/where the comment comes from. origin can be one of these text strings:
'translator','developer','programmer','source code'(those last three are synonyms, iykyk)
classmethod buildfromunit(unit)— builds a native unit out of a foreign one, preserving as much information as possible 🧚property dict— gets the dictionary of values for a catkeys line.getcontext()— gets the message context.getdict()— gets the dictionary of values for a catkeys line.static geterrors()— gets all the error messages.
Return type: Dictionary
getid()— a unique identifier for this unit 🌟
Return type: string
Returns an identifier for this unit that's unique in the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store — like, actually guarantee it, no cap 👑
static getlocations()— a list of source code locations.
Return type: List
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all the notes about this unit 🎀
It'll probably be freeform text, or something reasonable that the format can synthesise. It should not include location comments (those live in getlocations()).
gettargetlen()— returns the length of the target string.
Return type: Integer
NOTE: plural forms might be combined. 😭
getunits()— this unit, in a list. that's it, that's the whole vibe.static hasplural()— tells you whether this specific unit has plural strings.infer_state()— an empty method that should be overridden in sub-classes, so it can infer the current state(_n) of the unit from its current state.isblank()— lets you see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. 🙈
static isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete.static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated.
Use this to tell actual translation units apart from header, obsolete, binary or other blank units 💅
istranslated()— indicates whether this unit is translated.
Use this instead of deducing it from .target, so other classes can implement more functionality (the way XLIFF does) ✨
makeobsolete()— makes a unit obsolete.markfuzzy(present=True)— marks the unit as fuzzy, or not fuzzy. your call 🦋markreviewneeded(needsreview=True, explanation=None)— marks the unit to show whether it needs review.
Parameters:
- needsreview — defaults to
True. - explanation — adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic, format-agnostic merging 🫶multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees:
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator's notes. bye bye 👋rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a "rich" string tree into a multistring, iconic behaviour honestly:
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setdict(newdict)— sets the dictionary of values for a catkeys line.
Parameters:
- newdict (
Dict) — a new dictionary with catkeys line elements.
setid(value)— sets the unique identifier for this unit.
Only implemented if the format allows ids independent from other unit properties like source or context 🌸
unit_iter()— an iterator that only returns this unit. short king energy.
And the catkeys constants, living their best life at module level:
translate.storage.catkeys.FIELDNAMES = ['source', 'context', 'comment', 'target']— field names for a catkeys TU 💖translate.storage.catkeys.FIELDNAMES_HEADER = ['version', 'language', 'mimetype', 'checksum']— field names for the catkeys headertranslate.storage.catkeys.FIELDNAMES_HEADER_DEFAULTS = {'checksum': '', 'language': '', 'mimetype': '', 'version': '1'}— default or minimum header entries for a catkeys file
cpo
Classes that hold units of .po files (pounit) or entire files (pofile) 🐦
Gettext-style .po (or .pot) files are used in translations for KDE, GNOME and many other projects — like, everybody's using them.
This uses libgettextpo from the gettext package. Any version before 0.17 will at least cause some subtle bugs or may not work at all 😬 Developers might want to have a look at gettext-tools/libgettextpo/gettext-po.h from the gettext package for the public API of the library.
translate.storage.cpo.get_libgettextpo_version() — returns the libgettextpo version.
Return type: three-value tuple
Returns: libgettextpo version in the following format:: (major version, minor version, subminor version)
translate.storage.cpo.lsep = ' ' — separator for #: entries.
class translate.storage.cpo.po_error_handlerclass translate.storage.cpo.po_filetranslate.storage.cpo.po_file_t— apo_file_trepresents a PO file.class translate.storage.cpo.po_filepostranslate.storage.cpo.po_filepos_t— apo_filepos_trepresents the position in a PO file.class translate.storage.cpo.po_iteratortranslate.storage.cpo.po_iterator_t— apo_iterator_trepresents an iterator through a PO file.class translate.storage.cpo.po_messagetranslate.storage.cpo.po_message_t— apo_message_trepresents a message in a PO file.class translate.storage.cpo.po_xerror_handler
class translate.storage.cpo.pofile(inputfile=None, noheader=False, **kwargs)
Extensions = ['po', 'pot']— a list of file extensions associated with this store typeMimetypes = ['text/x-gettext-catalog', 'text/x-gettext-translation', 'text/x-po', 'text/x-pot']— a list of MIME types associated with this store typeName = 'Gettext PO file'— the human-usable name of this store typeUnitClass— alias ofpounit
Methods, let's go:
add_unit_to_index(unit)— adds a unit to the source and location indexes.addsourceunit(source)— adds and returns a new unit with the given source string.
Return type: TranslationUnit
addunit(unit, new=True)— appends the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually, bestie. no touching the list directly 🙅♀️
Parameters:
- unit (
TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from the text, using either the chardet lib or by trying to decode the file.static fallback_detection(text)— simple detection based on BOM in case chardet isn't available.findid(id)— finds the unit with the matching id by checkingid_index.findunit(source)— finds the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)— finds the units with the given source string.
Return type: TranslationUnit or None
getheaderplural()— returns thenpluralandpluralvalues from the header.getids()— returns a list of unit ids.getprojectstyle()— returns the project based on the information in the header.
okay so bestie, the project gets figured out in this exact sequence — no skipping, no improvising, periodt:
- Use the
'X-Project-Style'entry in the header. - Use the
'Report-Msgid-Bug-To'entry. - Use the
'X-Accelerator'entry. - Use the Project ID.
- Analyse the file itself (not yet implemented — yeah, it's giving "coming soon" 😭💀).
getsourcelanguage()
Gets the source language for this store. That's it, that's the vibe 💅
gettargetlanguage()
Returns the target language based on the info chillin' in the header.
The target language is determined in this sequence, bestie — follow the order:
- Use the
'Language'entry in the header. - Poedit's custom headers.
- Analysing the
'Language-Team'entry.
getunits()
Returns a list of all the units in this store. All of them. No crumbs left behind ✨
header()
Returns the header element, or None. Only the first element is allowed to be a header (main character energy only 👑). Note that this could still return an empty header element, if present — so like, don't be surprised if it's giving empty but valid 🫶
init_headers(charset='UTF-8', encoding='8bit', **kwargs)
Sets default values for po headers. Cute little defaults, slay.
isempty()
Returns True if the object doesn't contain any translation units. Empty store = empty vibes 🍓
makeheader(**kwargs)
Create a header for the given filename.
Check .makeheaderdict() for the info on parameters — it's all over there, iykyk.
makeheaderdict(charset='CHARSET', encoding='ENCODING', project_id_version=None, pot_creation_date=None, po_revision_date=None, last_translator=None, language_team=None, mime_version=None, plural_forms=None, report_msgid_bugs_to=None, **kwargs)
Create a header dictionary with useful defaults. Look at all those kwargs, we love a flexible icon 💖
pot_creation_date can be None (current date) or a value (datetime or string). po_revision_date can be None (form), False (= pot_creation_date), True (= now), or a value (datetime or string).
Returns: Dictionary with the header items. Return type: dict of strings.
makeindex()
Indexes the items in this store. At least .sourceindex should be useful. Get that index together, bestie 🔥
property merge_on
The matching criterion to use when merging on. Literally the "are you the same person" check for merging 💅
mergeheaders(otherstore)
Merges another header with this header.
This header is assumed to be the template — so like, it's the blueprint, the head honcho, the main character 👑
parse(input)
Parser to process the given source string. Feed it the string, it does the thing.
classmethod parsefile(storefile)
Reads the given file (or opens the given filename) and parses back to an object. File goes in, object comes out — the way that reverse-uno slays 🦋
parseheader()
Parses the PO header and returns the interpreted values as a dictionary. Header in, dictionary out, no cap.
classmethod parsestring(storestring)
Convert the string representation back to an object. String representation, we're turning you back into a real one 🌸
remove_unit_from_index(unit)
Remove a unit from source and location indexes. Bye bestie, you're off the index list 💅
removeduplicates(duplicatestyle='merge')
Make sure each msgid is unique; merge comments etc from duplicates into the original. Dedupe era 🔥
removeunit(unit)
Remove the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually — don't touch the list yourself, bestie, that's how you break stuff 😭
Parameters: unit (TranslationUnit) – The unit that will be added. (yes, it says "added", we just report what the docs say 🫶)
require_index()
Make sure source index exists. Like reminding your friend to exist at the group hang — necessary 💜
save()
Save to the file that data was originally read from, if available. If there's a home file, we go back to it 🏠
savefile(storefile)
Write the string representation to the given file (or filename). Save it to wherever, periodt.
serialize(out)
Converts to a bytes representation that can be parsed back using parsestring(). out should be an open file-like objects to write to.
setprojectstyle(project_style)
Set the project in the header.
Parameters: project_style (str) – the new project.
setsourcelanguage(sourcelanguage)
Set the source language for this store. Source language = set, done, slay 💅
settargetlanguage(lang)
Set the target language in the header.
This removes any custom Poedit headers if they exist — yes, they get evicted, sorry not sorry 🙈
Parameters: lang (str) – the new target language code.
suggestions_in_format = False
Indicates if format can store suggestions and alternative translation for a unit. It's False, so like... it cannot, bestie 💀
translate(source)
Return the translated string for a given source string.
Return type: String or None.
unit_iter()
Iterator over all the units in this store. Every unit, one by one, no crumbs 💖
updatecontributor(name, email=None)
Add contribution comments if necessary. Give the contributors their flowers 🌸
updateheader(add=False, **kwargs)
Updates the fields in the PO style header.
This will create a header if add == True. So like, flip the add switch and the header gets born ✨
updateheaderplural(nplurals, plural)
Update the Plural-Form PO header. Plurals getting their update moment 💅
🌈 class translate.storage.cpo.pounit(source=None, encoding='utf-8', gpo_message=None)
Okay so this is the cpo pounit class, and it is stacked with methods. Let's go 🫶
CPO_ENC = 'utf-8'
fixed encoding that is always used for cPO structure (self._gpo_message). Always. 'utf-8'. No negotiating 🎀
adderror(errorname, errortext)
Adds an error message to this unit. Errors get logged, bestie 💀
addlocation(location)
Add one location to the list of locations.
NOTE: Shouldn't be implemented if the format doesn't support it.
addlocations(location)
Add a location or a list of locations.
NOTE: Most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: This method might be removed in future.
So like, use with caution — it's giving deprecated energy 😭
addnote(text, origin=None, position='append')
Adds a note (comment). Sweet little annotations 💖
Parameters:
text(string) – Usually just a sentence or two.origin(string) – Specifies who/where the comment comes from. Origin can be one of the following text strings:'translator';'developer','programmer','source code'(synonyms).
classmethod buildfromunit(unit, encoding=None)
Build a native unit from a foreign unit, preserving as much information as possible. Glow-up but make it lossless 🌈
getcontext()
Get the message context. Context = obtained ✨
geterrors()
Get all error messages. All of them. The whole error collection 💀
getid()
The unique identifier for this unit according to the conventions in .mo files. That's the unit's government name 🫶
getlocations()
A list of source code locations.
Return type: List.
NOTE: Shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)
Returns all notes about this unit.
It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see getlocations()). Notes = notes, locations = locations, keep 'em separated bestie 💅
gettargetlen()
Returns the length of the target string.
Return type: Integer.
NOTE: Plural forms might be combined.
getunits()
This unit in a list. Like, a list of one. It's giving solo main character energy 👑
hasplural()
Tells whether or not this specific unit has plural strings. Plural check: iconic, or not 💅
infer_state()
Empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state. Parents leave it blank, kids do the work — generational tradition 😭
isblank()
Used to see if this unit has no source or target string.
NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
isfuzzy()
Indicates whether this unit is fuzzy. Fuzzy = soft-launch translation 🌸
isheader()
Indicates whether this unit is a header. Header check incoming ✨
isobsolete()
Indicate whether a unit is obsolete. Old news, bestie 💀
isreview()
Indicates whether this unit needs review. Review era 👀
istranslatable()
Indicates whether this unit can be translated.
This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. Translational icon vs. not-translational icon 💅
istranslated()
Indicates whether this unit is translated.
This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does). Don't just peek at .target and assume — use the method, bestie 🫶
makeobsolete()
Make a unit obsolete. Aging it out, periodt.
markfuzzy(present=True)
Marks the unit as fuzzy or not. Toggle the fuzz ✨
markreviewneeded(needsreview=True, explanation=None)
Marks the unit to indicate whether it needs review. Adds an optional explanation as a note. Review flag: raised, explanation: optional 💅
merge(otherpo, overwrite=False, comments=True, authoritative=False)
Merges the otherpo (with the same msgid) into this one.
Overwrite non-blank self.msgstr only if overwrite is True; merge comments only if comments is True.
(authoritative is in the signature too, bestie — authoritative=False and we're not out here inventing behaviour for it 🫶)
property msgidcomment
Extract KDE style msgid comments from the unit.
Return type: String.
Returns: Returns the extracted msgidcomments found in this unit's msgid. The way that it extracts — obsessed 🦋
multistring_to_rich(mulstring)
Convert a multistring to a list of "rich" string trees. Rich string trees, fancy fancy ✨🌳
Last code snippet from that TranslationUnit moment, bestie — literal output, no edits:
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— Remove all the translator’s notes. 🧹rich_parsers = []— A list of functions to use for parsing a string into a rich string tree.property rich_source- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
property rich_target- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
classmethod rich_to_multistring(elem_list)— Convert a “rich” string tree to a multistring. ✨
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— Set the message context.setid(value)— Sets the unique identified for this unit. 💅- only implemented if format allows ids independant from other unit properties like source or context
unit_iter()— Iterator that only returns this unit.
🍓 csvl10n
classes that hold units of comma-separated values (.csv) files (csvunit) or entire files (csvfile) for use with localisation. 🎀
class translate.storage.csvl10n.DefaultDialect
class translate.storage.csvl10n.csvfile(inputfile=None, fieldnames=None, encoding='auto')
This class represents a .csv file with various lines. The default format contains three columns: location, source, target. 🌈
Extensions = ['csv']— A list of file extentions associated with this store type.Mimetypes = ['text/comma-separated-values', 'text/csv']— A list of MIME types associated with this store type.Name = 'Comma Separated Value'— The human usable name of this store type.UnitClass— alias ofcsvunit.add_unit_to_index(unit)— Add a unit to source and location idexes.addsourceunit(source)— Add and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— Append the given unit to the object’s list of units.- This method should always be used rather than trying to modify the list manually.
- Parameters
unit(TranslationUnit) – The unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— Try to detect a file encoding from text, using either the chardet lib or by trying to decode the file. 🔍static fallback_detection(text)— Simple detection based on BOM in case chardet is not available.findid(id)— Find unit with matching id by checkingid_index.findunit(source)— Find the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— Find the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
getids()— Return a list of unit ids.getprojectstyle()— Get the project type for this store.getsourcelanguage()— Get the source language for this store.gettargetlanguage()— Get the target language for this store.getunits()— Return a list of all units in this store.isempty()— ReturnTrueif the object doesn’t contain any translation units.makeindex()— Indexes the items in this store. At least.sourceindexshould be useful.property merge_on— The matching criterion to use when merging on.- Returns: The default matching criterion for all the subclasses.
- Return type:
string
parse(csvsrc, sample_length=1024)— Parser to process the given source string.classmethod parsefile(storefile)— Reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— Convert the string representation back to an object.remove_unit_from_index(unit)— Remove a unit from source and locaton indexes.removeunit(unit)— Remove the given unit to the object’s list of units.- This method should always be used rather than trying to modify the list manually.
- Parameters
unit(TranslationUnit) – The unit that will be added.
require_index()— Make sure source index exists.save()— Save to the file that data was originally read from, if available. 💾savefile(storefile)— Write the string representation to the given file (or filename).serialize(out)— Write to file.setprojectstyle(project_style)— Set the project type for this store.setsourcelanguage(sourcelanguage)— Set the source language for this store.settargetlanguage(targetlanguage)— Set the target language for this store.suggestions_in_format = False— Indicates if format can store suggestions and alternative translation for a unit.translate(source)— Return the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— Iterator over all the units in this store.
class translate.storage.csvl10n.csvunit(source=None)
add_spreadsheet_escapes(source, target)— Add common spreadsheet escapes to two strings.adderror(errorname, errortext)— Adds an error message to this unit. 🚨- Parameters
errorname(string) – A single word to id the error.errortext(string) – The text describing the error.
- Parameters
addlocation(location)— Add one location to the list of locations.- NOTE: Shouldn’t be implemented if the format doesn’t support it.
addlocations(location)— Add a location or a list of locations.- NOTE: Most classes shouldn’t need to implement this, but should rather implement
TranslationUnit.addlocation(). - WARNING: This method might be removed in future.
- NOTE: Most classes shouldn’t need to implement this, but should rather implement
addnote(text, origin=None, position='append')— Adds a note (comment). 📝- Parameters
text(string) – Usually just a sentence or two.origin(string) – Specifies who/where the comment comes from.- Origin can be one of the following text strings:
‘translator’‘developer’,‘programmer’,‘source code’(synonyms)
- Origin can be one of the following text strings:
- Parameters
classmethod buildfromunit(unit)— Build a native unit from a foreign unit.- Preserving as much information as possible.
getcontext()— Get the message context.static geterrors()— Get all error messages.- Return type:
Dictionary
- Return type:
getid()— A unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.
- Return type:
getlocations()— A list of source code locations.- Return type:
List - NOTE: Shouldn’t be implemented if the format doesn’t support it.
- Return type:
getnotes(origin=None)— Returns all notes about this unit.- It will probably be freeform text or something reasonable that can be synthesised by the format.
- It should not include location comments (see
getlocations()).
gettargetlen()— Returns the length of the target string.- Return type:
Integer - NOTE: Plural forms might be combined.
- Return type:
getunits()— This unit in a list.static hasplural()— Tells whether or not this specific unit has plural strings.infer_state()— Empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— Used to see if this unit has no source or target string.- NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
isfuzzy()— Indicates whether this unit is fuzzy.static isheader()— Indicates whether this unit is a header.static isobsolete()— Indicate whether a unit is obsolete.static isreview()— Indicates whether this unit needs review.istranslatable()— Indicates whether this unit can be translated.
This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. lowkey essential, don't skip it 💅
istranslated()— tells you whether this unit is translated or not ✨makeobsolete()— flips a unit to obsolete, bye bestie 👋markfuzzy(value=True)— marks the unit as fuzzy, or un-fuzzies it if you passvalueas falsy. yes that's a word now 🫶markreviewneeded(needsreview=True, explanation=None)— marks the unit to say whether it needs review 👀- Parameters
needsreview— defaults toTrue.explanation— optionally drops in an explanation as a note.
- Parameters
match_header()— checks if this unit might be a header 🎀merge(otherunit, overwrite=False, comments=True, authoritative=False)— does the basic, format-agnostic merge thing fr fr 🧚multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees, no cap ✨
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]remove_spreadsheet_escapes(source, target)— strips the common spreadsheet escapes out of two strings 💀removenotes(origin=None)— nukes all the translator's notes. all of them. gone 🗑️rich_parsers = []— a list of functions used to parse a string into a rich string tree 🌈property rich_source— the rich version of the source 🦋
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target— the rich version of the target 🦋
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a "rich" string tree back into a multistring 🔁
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(value)— sets the message context 🎯setid(value)— sets the unique id for this unit. only actually implemented if the format lets ids be independent from other unit properties like source or context 😭unit_iter()— an iterator that only ever returns this one unit. main character energy, honestly 👑
🗂️ directory 🌸
This module provides functionality to work with directories 💖
class translate.storage.directory.Directory(dir=None)— this class represents a directory 📁file_iter()— iterator over(dir, filename)for all files in this directory.getfiles()— returns a list of(dir, filename)tuples for all the file names in this directory.getunits()— a list of all the units in all the files in this directory.scanfiles()— populates the internal file data.unit_iter()— iterator over all the units in all the files in this directory ✨
🧚 dtd 💅
Classes that hold units of .dtd files (dtdunit) or entire files (dtdfile).
These are the specific .dtd files used for localisation by Mozilla 🦋
🌸 Specifications
The following info is provided by Mozilla:
There is a grammar for entity definitions, which isn't really precise, as the spec says. There's no formal specification for DTD files, it's just "whatever makes this work" basically. The whole piece is clearly not the strongest point of the xml spec.
XML elements are allowed in entity values. A bunch of things that are "allowed" will just break the resulting document, and Mozilla forbids these in their DTD parser 💀
🌸 Dialects
There are two dialects, bestie:
- Regular DTD
- Android DTD
Both dialects are similar, but the Android DTD uses some particular escapes that regular DTDs simply don't have 👀
🌸 Escaping in regular DTD
In DTD, characters are usually escaped inside the entities. To make translation easier, some of those escaped characters get unescaped when reading from — or converting — the DTD, and then they get escaped again when saving, or converting to a DTD 🔁
In regular DTD the following characters are usually or sometimes escaped:
- The
%character is escaped using%or%or% - The
"character is escaped using" - The
'character is escaped using'(partial roundtrip) - The
&character is escaped using& - The
<character is escaped using<(not yet implemented) - The
>character is escaped using>(not yet implemented)
Besides those, there are a lot of escapes for a huge number of characters. These escapes usually take the form &#NUMBER; where NUMBER is the numerical code for the character 🎀
There are a few particularities in DTD escaping. Some of the escapes aren't implemented yet, either because they're not really necessary or because implementing them is too hard 😭
A special case is the ' escaping using ', which doesn't give you a full roundtrip conversion — this is on purpose, to support some special Mozilla DTD files 🌟
Also, the " character is never escaped when the previous character is = (so the sequence =" is present in the string). That's to avoid escaping the " that indicates an attribute assignment — for example in a href attribute for an a tag in HTML (anchor tag) 💅
🌸 Escaping in Android DTD
It has the same escapes as in regular DTD, plus these ones:
- The
'character is escaped using'or'oru0027 - The
"character is escaped using"
🌸 The goods
translate.storage.dtd.accesskeysuffixes = ('.accesskey', '.accessKey', '.akey')— Accesskey Suffixes: entries with this suffix may be combined with labels ending inlabelsuffixesinto accelerator notation 👑class translate.storage.dtd.dtdfile(inputfile=None, android=False)— a.dtdfile made up ofdtdunits🎀Extensions = None— a list of file extensions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human-usable name of this store type.UnitClass— alias ofdtdunit.add_unit_to_index(unit)— adds a unit to the source and location indexes.addsourceunit(source)— adds and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— appends the given unit to the object's list of units. always use this instead of trying to mess with the list by hand, fr fr 🙈- Parameters:
unit(TranslationUnit) – the unit that will be added.
- Parameters:
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, either using thechardetlib or by just trying to decode the file 🔍static fallback_detection(text)— simple detection based on BOM, in casechardetisn't available.findid(id)— finds the unit with a matching id by checkingid_index.findunit(source)— finds the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— finds the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
getids()— returns a list of unit ids.getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store.isempty()— returnsTrueif the object doesn't contain any translation units 💀makeindex()— indexes the items in this store. at minimum.sourceindexshould be useful ✨property merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(dtdsrc)— reads the source code of a dtd file in and includes them asdtdunitsinself.units.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back into an object 🫶classmethod parsestring(storestring)— converts the string representation back into an object.remove_unit_from_index(unit)— removes a unit from the source and location indexes.removeunit(unit)— removes the given unit from the object's list of units. again: always use this instead of editing the list manually, iykyk 🙈- Parameters:
unit(TranslationUnit) – the unit that will be added.
- Parameters:
require_index()— makes sure the source index exists.save()— saves to the file that the data was originally read from, if that's available 💾
savefile(storefile)— writes the string representation out to whatever file (or filename) you hand it. 💾serialize(out)— writes the content to a file.setprojectstyle(project_style)— sets the project type for this store. ✨setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— tells you whether the format can actually store suggestions and alternative translations for a unit. (Spoiler: this one says no 🙈)translate(source)— returns the translated string for a given source string.
Return type: String or None
unit_iter()— iterator over all the units in this store. 🌀
🦋 class translate.storage.dtd.dtdunit(source='', android=False)
An entity definition straight outta a DTD file (plus any comments that came along for the ride). Obsessed with how tidy these are, ngl. 🎀
adderror(errorname, errortext)— adds an error message to this unit.
Parameters:
- errorname (string) — a single word that ids the error.
- errortext (string) — the text describing the error.
addlocation(location)— sets the entity to the given "location". 📍addlocations(location)— adds a location, or a whole list of locations.
NOTE: most classes shouldn't need to implement this — they should implement TranslationUnit.addlocation() instead, fr.
WARNING: this method might get yeeted in the future. 💀
addnote(text, origin=None, position='append')— adds a note (a comment, basically).
Parameters:
- text (string) — usually just a sentence or two.
- origin (string) — says who/where the comment comes from. Origin can be one of these text strings: -
'translator'-'developer','programmer','source code'(synonyms, bestie, pick whichever)
classmethod buildfromunit(unit)— builds a native unit out of a foreign unit, preserving as much info as humanly possible. 🫶static getcontext()— gets the message context.static geterrors()— gets all the error messages.
Return type: Dictionary
getid()— a unique identifier for this unit.
Return type: string
Returns: an identifier for this unit that's unique within the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. No duplicates, periodt. 👑
getlocations()— returns the entity as a location (identifier).getnotes(origin=None)— returns all notes about this unit.
It'll probably be freeform text or something reasonable that the format can synthesise. It should not include location comments (those live in getlocations(), iykyk).
getoutput()— converts the dtd entity back into string form. 🔁gettargetlen()— returns the length of the target string.
Return type: Integer
NOTE: plural forms might be combined.
getunits()— this unit, in a list. Just the one. It's giving solo main character energy. ✨static hasplural()— tells you whether this specific unit has plural strings.infer_state()— an empty method that sub-classes should override to infer the currentstate(_n)of the unit from its current state.isblank()— returns whether thisdtdunitdoesn't actually have an entity definition.static isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete.static isreview()— indicates whether this unit needs review. 👀istranslatable()— indicates whether this unit can be translated.
Use this to tell actual translation units apart from headers, obsolete ones, binary ones, or other blank units. Super important, no cap.
istranslated()— indicates whether this unit is translated.
Use this instead of deducing it from .target, so other classes can implement extra functionality (like XLIFF does, bestie, it's built different).
makeobsolete()— makes a unit obsolete. bye boo 👋markfuzzy(value=True)— marks the unit as fuzzy, or un-fuzzy.markreviewneeded(needsreview=True, explanation=None)— marks the unit to say whether it needs review.
Parameters:
- needsreview — defaults to
True. - explanation — adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does a basic, format-agnostic merge. 🫱🫲multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees. Literally so satisfying:
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]parse(dtdsrc)— reads the first dtd element from the source code into this object, and returnslinesprocessed. 📥removenotes(origin=None)— removes all the translator's notes.rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a "rich" string tree back into a multistring. 🧚
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setid(new_id)— sets the unique identifier for this unit.
Only implemented if the format allows ids that are independent from other unit properties like source or context.
property source— gets the unquoted source string.property target— gets the unquoted target string.unit_iter()— an iterator that only returns this unit. one girl, one iterator 💅
🌈 translate.storage.dtd.labelsuffixes = ('.label', '.title')
Label suffixes: entries with this suffix can be combined with accesskeys found in entries ending with accesskeysuffixes. Cute little tag-team situation. 🎀
💜 translate.storage.dtd.quoteforandroid(source)
Escapes a line for Android DTD files. 📱
🌸 translate.storage.dtd.quotefordtd(source)
Quotes and escapes a line for regular DTD files.
⭐ translate.storage.dtd.removeinvalidamps(name, value)
Finds and removes ampersands that are not part of an entity definition.
Okay so listen: a stray & in a DTD file can absolutely break an application's ability to parse the file. In Mozilla localisation this is a huge deal — these can break the parsing of files used in XUL and therefore break the whole interface rendering. 😭 Tracking the problem down is genuinely so difficult, so by removing potential broken ampersands and warning the users, we can ensure the output DTD will always be parsable. Iconic problem-solving behaviour, honestly.
Parameters:
- name (String) — entity name
- value (String) — entity text value
Return type: String
Returns: entity value without the bad ampersands ✨
🦋 translate.storage.dtd.unquotefromandroid(source)
Unquotes a quoted Android DTD definition.
🦋 translate.storage.dtd.unquotefromdtd(source)
Unquotes a quoted dtd definition.
🔥 _factory_classes
Py2exe can't find stuff that we import dynamically, so we have this file just for the sake of the Windows installer — so it can easily pick up all the stuff we need and make sure it actually makes it into the installer. Not me solving a whole packaging problem with one file 🙈
👑 factory
factory methods to build real storage objects that conform to base.py.
translate.storage.factory.getclass(storefile, localfiletype=None, ignore=None, classes=None, classes_str=None, hiddenclasses=None)— a factory that returns the applicable class for the type of file presented. Specifyignoreto ignore some part at the back of the name (like.gz). 🗂️translate.storage.factory.getobject(storefile, localfiletype=None, ignore=None, classes=None, classes_str=None, hiddenclasses=None)— a factory that returns a usable object for the type of file presented.
Parameters:
- storefile (file or str or TranslationStore) — File object or file name.
Specify ignore to ignore some part at the back of the name (like .gz).
translate.storage.factory.supported_files()— returns data about all supported files.
Returns: list of type that include (name, extensions, mimetypes)
Return type: list
🎀 fpo
Classes for the support of Gettext .po and .pot files.
This implementation assumes that cpo is working. Don't use this directly, bestie — it can be used once cpo has been established to work. 💖
translate.storage.fpo.lsep = ' '— separator for#:entries.class translate.storage.fpo.pofile(inputfile=None, noheader=False, **kwargs)— a.pofile containing various units. 🗃️Extensions = ['po', 'pot']— the file extensions that belong to this store type, bestie.Mimetypes = ['text/x-gettext-catalog', 'text/x-gettext-translation', 'text/x-po', 'text/x-pot']— the MIME types associated with this store type. iykyk.Name = 'Gettext PO file'— the human-usable name of this store type. iconic label behavior.UnitClass— alias ofpounit. simple, clean, slay.add_unit_to_index(unit)— adds a unit to the source and location indexes.addsourceunit(source)— adds and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— appends the given unit to the object’s list of units.- This method should always be used rather than trying to modify the list manually. no cap.
- Parameters:
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, using either thechardetlib or by trying to decode the file. detective mode.static fallback_detection(text)— simple detection based on BOM in casechardetis not available.findid(id)— finds the unit with matchingidby checkingid_index. indexed and unbothered.findunit(source)— finds the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— finds the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
getheaderplural()— returns thenpluralandpluralvalues from the header. plural girlies rise.getids()— returns a list of unit ids.getprojectstyle()— returns the project based on information in the header.- The project is determined in the following sequence:
- Use the
X-Project-Styleentry in the header. - Use
Report-Msgid-Bug-Toentry - Use the
X-Acceleratorentry - Use the Project ID
- Analyse the file itself (not yet implemented)
getsourcelanguage()— gets the source language for this store.gettargetlanguage()— returns the target language based on information in the header.- The target language is determined in the following sequence:
- Use the
Languageentry in the header. - Poedit’s custom headers.
- Analysing the
Language-Teamentry.
getunits()— returns a list of all units in this store.header()— returns the header element, orNone. Only the first element is allowed to be a header. Note that this could still return an empty header element, if present. mysterious but valid.init_headers(charset='UTF-8', encoding='8bit', **kwargs)— sets default values for po headers.isempty()— returnsTrueif the object doesn’t contain any translation units.makeheader(**kwargs)— creates a header for the given filename. Check.makeheaderdict()for information on parameters.makeheaderdict(charset='CHARSET', encoding='ENCODING', project_id_version=None, pot_creation_date=None, po_revision_date=None, last_translator=None, language_team=None, mime_version=None, plural_forms=None, report_msgid_bugs_to=None, **kwargs)— creates a header dictionary with useful defaults.pot_creation_datecan beNone(current date) or a value (datetime or string).po_revision_datecan beNone(form),False(=pot_creation_date),True(= now), or a value (datetime or string).- Returns: Dictionary with the header items
- Return type:
dictof strings
makeindex()— indexes the items in this store. At least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.mergeheaders(otherstore)— merges another header with this header. This header is assumed to be the template.parse(input)— parses the given file or file source string.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object.parseheader()— parses the PO header and returns the interpreted values as a dictionary.classmethod parsestring(storestring)— converts the string representation back to an object.remove_unit_from_index(unit)— removes a unit from source and location indexes.removeduplicates(duplicatestyle='merge')— makes sure eachmsgidis unique ; merge comments etc from duplicates into original.removeunit(unit)— remove the given unit to the object’s list of units. (yes, “to” — the page said what it said 💀)- This method should always be used rather than trying to modify the list manually.
- Parameters:
unit(TranslationUnit) – the unit that will be added.
require_index()— makes sure source index exists.save()— save to the file that data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— writes content to file.setprojectstyle(project_style)— sets the project in the header.- Parameters:
project_style(str) – the new project
- Parameters:
setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(lang)— sets the target language in the header.- This removes any custom Poedit headers if they exist. heads up, bestie.
- Parameters:
lang(str) – the new target language code
suggestions_in_format = False— indicates if format can store suggestions and alternative translation for a unit.translate(source)— returns the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— iterator over all the units in this store.updatecontributor(name, email=None)— adds contribution comments if necessary.updateheader(add=False, **kwargs)— updates the fields in the PO style header.- This will create a header if
add == True.
- This will create a header if
updateheaderplural(nplurals, plural)— updates thePlural-FormPO header.
🧚 class translate.storage.fpo.pounit(source=None, **kwargs)
adderror(errorname, errortext)— adds an error message to this unit.addlocation(location)— adds a location tosourcecommentsin the PO unit.- Parameters:
location(String) – text location e.g.file.c:23does not include#:
- Parameters:
addlocations(location)— adds a location or a list of locations.- NOTE: Most classes shouldn’t need to implement this, but should rather implement
TranslationUnit.addlocation(). - WARNING: This method might be removed in future.
- NOTE: Most classes shouldn’t need to implement this, but should rather implement
addnote(text, origin=None, position='append')— this is modeled on the XLIFF method. Seexliff.py::xliffunit.addnote. crossover episode.classmethod buildfromunit(unit)— builds a native unit from a foreign unit, preserving as much information as possible.getcontext()— gets the message context.geterrors()— gets all error messages.getid()— returns a unique identifier for this unit.getlocations()— gets a list of locations fromsourcecommentsin the PO unit.- rtype:
List; return: A list of the locations with#:stripped
- rtype:
getnotes(origin=None)— returns comments based onoriginvalue (programmer, developer, source code and translator).gettargetlen()— returns the length of the target string.- Return type:
Integer - NOTE: Plural forms might be combined.
- Return type:
getunits()— this unit in a list. tiny but mighty.hasmarkedcomment(commentmarker)— checks whether the given comment marker is present as# (commentmarker)…hasplural()— returns whether thispounitcontains plural strings…hastypecomment(typecomment)— checks whether the given type comment is present.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— used to see if this unit has no source or target string.- NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
isfuzzy()— indicates whether this unit is fuzzy.isheader()— indicates whether this unit is a header.isobsolete()— indicate whether a unit is obsolete.isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated. the final vibe check.
...This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. Like, don't @ me, that's literally the whole point 💅
istranslated()— tells you whether this unit is translated or not. And bestie, please use this instead of trying to deduce it from.target— that's so we can let other classes (looking at you, XLIFF 👀) implement extra functionality without breaking stuff.makeobsolete()— makes this unit obsolete. bye babe 🫶markfuzzy(present=True)— marks the unit as fuzzy, or un-fuzzy if you pass something falsy.markreviewneeded(needsreview=True, explanation=None)— flags the unit to show whether it needs review, and if you want you can toss in an optionalexplanationthat gets added as a note. receipts! 🧾merge(otherpo, overwrite=False, comments=True, authoritative=False)— merges theotherpo(same msgid, obvs) into this one. Deets: it only overwrites a non-blankself.msgstrifoverwriteisTrue, and it only merges comments ifcommentsisTrue.multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees ✨
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— nukes all the translator's notes (a.k.a. other comments). clean slate energy 🧹rich_parsers = []— a list of functions used for parsing a string into a rich string tree.property rich_source— the rich version of the source. SEE ALSO:rich_to_multistring(),multistring_to_rich().property rich_target— the rich version of the target. SEE ALSO:rich_to_multistring(),multistring_to_rich().classmethod rich_to_multistring(elem_list)— converts a "rich" string tree back into a multistring. the reverse glow-up 🔁
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setid(value)— sets the unique id for this unit. heads up: this is only actually implemented if the format lets ids be independent from other unit properties like source or context 🤷♀️settypecomment(typecomment, present=True)— changes whether a given typecomment is present.property target— returns the unescapedmsgstr.unit_iter()— an iterator that only returns this one unit. solo main character energy 👑
🌸 html 🌸
module for parsing html files for translation — yes, we do html now, slay 💅
class translate.storage.html.POHTMLParser(inputfile=None, callback=None)
EMPTY_HTML_ELEMENTS = ['area', 'base', 'br', 'col', 'embed', 'hr', 'img', 'input', 'link', 'meta', 'param', 'source', 'track', 'wbr']— an empty element is one that can't have any child nodes (i.e. nested elements or text nodes). In HTML, using a closing tag on an empty element is usually invalid. Reference —https://developer.mozilla.org/en-US/docs/Glossary/Empty_elementExtensions = None— a list of file extensions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human-usable name of this store type.TRANSLATABLE_ATTRIBUTES = ['abbr', 'alt', 'lang', 'summary', 'title', 'value']— text from these HTML attributes gets extracted as translation units. heads up though: thecontentattribute of meta tags is a special case 🫠TRANSLATABLE_ELEMENTS = ['address', 'article', 'aside', 'blockquote', 'caption', 'dd', 'dt', 'div', 'figcaption', 'footer', 'header', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'li', 'main', 'nav', 'option', 'p', 'pre', 'section', 'td', 'th', 'title']— these HTML elements (tags) get extracted as translation units, unless they don't have any translatable text content. And if one translatable element is embedded inside another, the outer unit gets split into the parts before and after the inner unit. nesting drama, handled ✂️TRANSLATABLE_METADATA = ['description', 'keywords']— document metadata from meta elements with these names gets extracted as translation units. Reference —https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta/nameUnitClass— alias ofhtmlunit.
methods, bestie 🎀
add_unit_to_index(unit)— adds a unit to the source and location indexes.addsourceunit(source)— adds and returns a new unit with the given source string. Return type:TranslationUnitaddunit(unit)— appends the given unit to the object's list of units. always use this instead of poking the list manually, fr fr. Parameters:unit(TranslationUnit) – the unit that will be added.close()— handles any buffered data.detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from the text, either using thechardetlib or by just trying to decode the file. 🔍do_encoding(htmlsrc)— returns the html text properly encoded based on a charset.static fallback_detection(text)— a simple detection based on the BOM, for whenchardetisn't available. the backup plan 💅feed(data)— feeds data to the parser. call it as often as you want, with as little or as much text as you want (may include ‘n’).findid(id)— finds the unit with the matching id by checkingid_index.findunit(source)— finds the unit with the given source string. Return type:TranslationUnitorNonefindunits(source)— finds the units with the given source string. Return type:TranslationUnitorNoneget_starttag_text()— returns the full source of the start tag: ‘<…>’.getids()— returns a list of unit ids.getpos()— returns the current line number and offset.getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store.guess_encoding(htmlsrc)— returns the encoding of the html text. we do this by looking for ‘charset=’ inside a meta tag, iykyk 🕵️♀️handle_charref(name)— handles entries in the form&#NNNN;, e.g.⃡.handle_entityref(name)— handles named entities of the form&aaaa;, e.g.’.isempty()— returnsTrueif the object doesn't contain any translation units. empty store = sad store 😭makeindex()— indexes the items in this store. at minimum.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on. Returns: the default matching criterion for all the subclasses. Return type:stringparse(htmlsrc)— parser that processes the given source string.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back to an object.classmethod parsestring(storestring)— converts the string representation back into an object.remove_unit_from_index(unit)— removes a unit from the source and location indexes.removeunit(unit)— removes the given unit from the object's list of units. again: always use this instead of editing the list by hand, bestie. Parameters:unit(TranslationUnit) – the unit that will be added.require_index()— makes sure the source index exists.reset()— resets this instance. loses all unprocessed data. gone, reduced to atoms 💀save()— saves to the file that data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like object to write to.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates whether the format can store suggestions and alternative translations for a unit.translate(source)— returns the translated string for a given source string. Return type:StringorNoneunit_iter()— iterator over all the units in this store.
class translate.storage.html.htmlfile(inputfile=None, callback=None)
EMPTY_HTML_ELEMENTS— bestie, this is the list of void elements: the ones that literally cannot have child nodes (no nested elements, no text nodes, nada). In HTML, slapping a closing tag on an empty element is usually invalid, so don't. Reference ‐ https://developer.mozilla.org/en-US/docs/Glossary/Empty_element
EMPTY_HTML_ELEMENTS = ['area', 'base', 'br', 'col', 'embed',
'hr', 'img', 'input', 'link', 'meta', 'param', 'source', 'track',
'wbr']Extensions = None— a list of file extentions (spelled exactly like that in the source, we don't gatekeep) associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human usable name of this store type. Like, the name a human would actually use.TRANSLATABLE_ATTRIBUTES— text from these HTML attributes gets extracted as translation units. 🌸 Heads up though: thecontentattribute ofmetatags is a special case.
TRANSLATABLE_ATTRIBUTES = ['abbr', 'alt', 'lang', 'summary',
'title', 'value']TRANSLATABLE_ELEMENTS— these HTML elements (tags) get extracted as translation units, unless they're lacking translatable text content. And the juicy bit: if one translatable element is embedded inside another, the outer translation unit gets split into the parts before and after the inner translation unit. 🪓
TRANSLATABLE_ELEMENTS = ['address', 'article', 'aside',
'blockquote', 'caption', 'dd', 'dt', 'div', 'figcaption',
'footer', 'header', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'li',
'main', 'nav', 'option', 'p', 'pre', 'section', 'td', 'th',
'title']TRANSLATABLE_METADATA— document metadata frommetaelements with these names gets extracted as translation units. Reference ‐ https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta/name
TRANSLATABLE_METADATA = ['description', 'keywords']UnitClass— alias ofhtmlunit. That's it, that's the whole vibe.add_unit_to_index(unit)— add a unit to the source and location idexes. (Their spelling, not mine 💅)addsourceunit(source)— add and return a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— append the given unit to the object's list of units. Lowkey important: this method should always be used rather than trying to modify the list manually. Don't be that person. 🙈- Parameters:
unit(TranslationUnit) – the unit that will be added.
- Parameters:
close()— handle any buffered data. Flush it, babe.detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from text, using either thechardetlib or by just... trying to decode the file. Feral but it works.do_encoding(htmlsrc)— return the HTML text properly encoded based on a charset. ✨static fallback_detection(text)— simple detection based on BOM, for whenchardetisn't available. The backup bestie.feed(data)— feed data to the parser. Call this as often as you want, with as little or as much text as you want (may include ‘n’). Snack-sized or whole-meal, parser doesn't care.findid(id)— find the unit with a matchingidby checkingid_index.findunit(source)— find the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— find the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
get_starttag_text()— return full source of the start tag: ‘<…>’.getids()— return a list of unit ids.getpos()— return the current line number and offset.getprojectstyle()— get the project type for this store.getsourcelanguage()— get the source language for this store.gettargetlanguage()— get the target language for this store.getunits()— return a list of all units in this store.guess_encoding(htmlsrc)— returns the encoding of the HTML text. How? We look for ‘charset=’ within ametatag. Sherlock who? 🔍handle_charref(name)— handle entries in the form&#NNNN;e.g.⃡.handle_entityref(name)— handle named entities of the form&aaaa;e.g.’.isempty()— returnTrueif the object doesn't contain any translation units.makeindex()— indexes the items in this store. At least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(htmlsrc)— parser to process the given source string.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— convert the string representation back to an object.remove_unit_from_index(unit)— remove a unit from the source and locaton indexes. (Again, their spelling 😭 we love them anyway.)removeunit(unit)— remove the given unit from the object's list of units. Same deal asaddunit: always use this method rather than trying to modify the list manually, fr fr.- Parameters:
unit(TranslationUnit) – the unit that will be added.
- Parameters:
require_index()— make sure the source index exists. Like checking your bag before you leave. 🎀reset()— reset this instance. Loses all unprocessed data. ⚠️ Yes, all of it.save()— save to the file that data was originally read from, if available.savefile(storefile)— write the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like object to write to.setprojectstyle(project_style)— set the project type for this store.setsourcelanguage(sourcelanguage)— set the source language for this store.settargetlanguage(targetlanguage)— set the target language for this store.suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit.translate(source)— return the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— iterator over all the units in this store.
🦋 class translate.storage.html.htmlunit(source=None)
A unit of translatable/localisable HTML content. This is the little guy that actually holds your strings. 💖
adderror(errorname, errortext)— adds an error message to this unit.- Parameters:
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
- Parameters:
addlocation(location)— add one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. No fake features, bestie.
addlocations(location)— add a location or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. 👀 Don't get too attached.
addnote(text, origin=None, position='append')— adds a note (comment).- Parameters:
text(string) – usually just a sentence or two.origin(string) – specifies who/where the comment comes from.origincan be one of the following text strings: - ‘translator’ - ‘developer’, ‘programmer’, ‘source code’ (synonyms)
- Parameters:
classmethod buildfromunit(unit)— build a native unit from a foreign unit, preserving as much information as possible. Immigration but make it translation. 🛂static getcontext()— get the message context.static geterrors()— get all error messages.- Return type:
Dictionary
- Return type:
getid()— a unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store.
- Return type:
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. No duplicates, no cap. 👑
getlocations()— a list of source code locations.- Return type:
List
- Return type:
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all notes about this unit. It'll probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()).gettargetlen()— returns the length of the target string.- Return type:
Integer
- Return type:
NOTE: plural forms might be combined.
getunits()— this unit, in a list. One-item list, main character energy.static hasplural()— tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— used to see if this unit has no source or target string.
🎀 NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()— tells you whether this unit is fuzzy ✨static isheader()— tells you whether this unit is a header 🏷️static isobsolete()— tells you whether a unit is obsolete 💀static isreview()— tells you whether this unit needs review 👀istranslatable()— tells you whether this unit can be translated 🫶
This is the one you use to tell actual real-deal units apart from headers, obsolete stuff, binary bits, or other blank units. iykyk.
istranslated()— tells you whether this unit is translated 🌍
Use this instead of trying to deduce it from .target, bestie — that way other classes can implement more functionality (like XLIFF does) without you guessing wrong.
makeobsolete()— makes a unit obsolete 🪦markfuzzy(value=True)— marks the unit as fuzzy, or not fuzzy, depending onvalue🌫️markreviewneeded(needsreview=True, explanation=None)— marks the unit to show whether it needs review 🔍
Parameters
needsreview— defaults toTrue.explanation— adds an optional explanation as a note 📝
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic, format-agnostic merging 🤝multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees 🌳
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator's notes 🧹rich_parsers = []— a list of functions to use for parsing a string into a rich string tree 🧚property rich_source— the rich source 🦋- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
property rich_target— the rich target 💜- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
classmethod rich_to_multistring(elem_list)— converts a "rich" string tree into a multistring 🔄
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context 🧭setid(value)— sets the unique identifier for this unit 🆔
Only implemented if the format allows ids independent of other unit properties like source or context.
unit_iter()— an iterator that only returns this unit 🙈
🌸 ical 🌸
Class that manages iCalender files for translation 🗓️
iCalendar files follow the RFC2445 specification, no cap.
The iCalendar specification uses the following naming conventions:
- Component: an event, journal entry, timezone, etc 🎪
- Property: a property of a component: summary, description, start time, etc 🔖
- Attribute: an attribute of a property, e.g. language 🗣️
The following are localisable in this implementation:
- VEVENT component:
SUMMARY,DESCRIPTION,COMMENTandLOCATIONproperties ✨
While other items could be localised, this isn't seen as important until use cases arise. In such a case, simply adjusting the component.name and property.name lists to include these will allow expanded localisation. Slay.
🌈 LANGUAGE Attribute
While the iCalendar format allows items to have a language attribute, this is not used. The reason being that for most of the items that we localise, they're only allowed to occur zero or once. So summary would ideally be present in multiple languages in one file, but the format doesn't allow such multiple entries. This is unfortunate as it prevents the creation of a single multilingual iCalendar file 😭
🦋 Future Format Support
As this format uses vobject, which supports various formats including vCard, it's possible to expand this format to understand those too if needed 👑
💖 class translate.storage.ical.icalfile(inputfile=None, **kwargs)
An ical file 🗂️
Extensions = None— a list of file extentions associated with this store type 📎Mimetypes = None— a list of MIME types associated with this store type 🏷️Name = 'Base translation store'— the human usable name of this store type 🫶UnitClass— alias oficalunit🔁add_unit_to_index(unit)— adds a unit to source and location idexes 🗃️addsourceunit(source)— adds and returns a new unit with the given source string 💫
Return type: TranslationUnit
addunit(unit)— appends the given unit to the object's list of units ➕
This method should always be used rather than trying to modify the list manually, fr fr.
Parameters
unit(TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, using either thechardetlib or by trying to decode the file 🕵️static fallback_detection(text)— simple detection based on BOM in casechardetisn't available 🧯findid(id)— finds a unit with a matching id by checkingid_index🔎findunit(source)— finds the unit with the given source string 🎯
Return type: TranslationUnit or None
findunits(source)— finds the units with the given source string 🎯🎯
Return type: TranslationUnit or None
getids()— returns a list of unit ids 🆔getprojectstyle()— gets the project type for this store 🎨getsourcelanguage()— gets the source language for this store 🗣️gettargetlanguage()— gets the target language for this store 🌍getunits()— returns a list of all units in this store 📋isempty()— returnsTrueif the object doesn't contain any translation units 🫙makeindex()— indexes the items in this store. At least.sourceindexshould be useful 🗂️property merge_on— the matching criterion to use when merging on 🤝
Returns: the default matching criterion for all the subclasses.
Return type: string
parse(input)— parses the given file or file source string 📥classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object 📂classmethod parsestring(storestring)— converts the string representation back to an object 🔁remove_unit_from_index(unit)— removes a unit from source and locaton indexes 🗑️removeunit(unit)— removes the given unit from the object's list of units ➖
This method should always be used rather than trying to modify the list manually.
Parameters
unit(TranslationUnit) — the unit that will be added.
require_index()— makes sure the source index exists ✅save()— saves to the file that data was originally read from, if available 💾savefile(storefile)— writes the string representation to the given file (or filename) 📝serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like object to write to 🧬setprojectstyle(project_style)— sets the project type for this store 🎨setsourcelanguage(sourcelanguage)— sets the source language for this store 🗣️settargetlanguage(targetlanguage)— sets the target language for this store 🌍suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit 💭translate(source)— returns the translated string for a given source string 🔄
Return type: String or None
unit_iter()— an iterator over all the units in this store 🚶♀️
🎀 class translate.storage.ical.icalunit(source=None, **kwargs)
An ical entry that is translatable 🗓️✨
adderror(errorname, errortext)— adds an error message to this unit 🚨
Parameters
errorname(string) — a single word to id the error.errortext(string) — the text describing the error.
addlocation(location)— adds one location to the list of locations 📍
NOTE: Shouldn't be implemented if the format doesn't support it.
addlocations(location)— adds a location or a list of locations 📍📍
NOTE: Most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: This method might be removed in future. 💀
addnote(text, origin=None, position='append')— adds a note (comment) 📝
Parameters
text(string) — usually just a sentence or two.
origin(string) — okay so this tells you who/where the comment comes from. Origin can be one of these exact text strings:'translator','developer','programmer','source code'(the last three are synonyms, iykyk). 💬
classmethod buildfromunit(unit)
Builds a native unit from a foreign unit. It preserves as much information as possible — no cap. 🫶
static getcontext()
Gets the message context. 💌
static geterrors()
Gets all the error messages. Return type: Dictionary. 😭
getid()
A unique identifier for this unit. Return type: string.
Returns: an identifier for this unit that is unique in the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. 👑
getlocations()
A list of source code locations. Return type: List.
NOTE: Shouldn’t be implemented if the format doesn’t support it. 🙈
getnotes(origin=None)
Returns all notes about this unit. It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see getlocations()). 📝
gettargetlen()
Returns the length of the target string. Return type: Integer.
NOTE: Plural forms might be combined. 🌀
getunits()
This unit in a list. ✨
static hasplural()
Tells whether or not this specific unit has plural strings. 🔢
infer_state()
Empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state. 🧠
isblank()
Used to see if this unit has no source or target string.
NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. 💀
static isfuzzy()
Indicates whether this unit is fuzzy. 🌫️
static isheader()
Indicates whether this unit is a header. 🏷️
static isobsolete()
Indicate whether a unit is obsolete. 🪦
static isreview()
Indicates whether this unit needs review. 👀
istranslatable()
Indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. ✅
istranslated()
Indicates whether this unit is translated. This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does). 🌍
makeobsolete()
Make a unit obsolete. rip bestie 🪦
markfuzzy(value=True)
Marks the unit as fuzzy or not. Default is value=True. 🌫️
markreviewneeded(needsreview=True, explanation=None)
Marks the unit to indicate whether it needs review.
Parameters:
needsreview— Defaults toTrue. 👀explanation— Adds an optional explanation as a note. 📝
merge(otherunit, overwrite=False, comments=True, authoritative=False)
Do basic format-agnostic merging. 🧬
multistring_to_rich(mulstring)
Convert a multistring to a list of “rich” string trees. 🌳
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)
Remove all the translator’s notes. 🧹
rich_parsers = []
A list of functions to use for parsing a string into a rich string tree. 🧚
property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich(). 🔗
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich(). 🔗
classmethod rich_to_multistring(elem_list)
Convert a “rich” string tree to a multistring. 🌈
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)
Set the message context. 💬
setid(value)
Sets the unique identifier for this unit. Only implemented if the format allows ids independent from other unit properties like source or context. 🆔
unit_iter()
Iterator that only returns this unit. ☝️
🌸 ini 🌸
Class that manages .ini files for translation. 🎀
# a comment ; a comment
[Section] a = a string b : a stringtranslate.storage.ini.Dialect
Base class for differentiating dialect options and functions. 🗣️
translate.storage.ini.DialectDefault
That’s the whole vibe — no extra description here, bestie. ✨
translate.storage.ini.DialectInno
Same energy — just the class. 💅
translate.storage.ini.inifile(inputfile=None, dialect='default', **kwargs)
An INI file. 🎀
Extensions = None— A list of file extensions associated with this store type. 📎Mimetypes = None— A list of MIME types associated with this store type. 🏷️Name = 'Base translation store'— The human usable name of this store type. 👑UnitClass— alias ofiniunit. 🔁add_unit_to_index(unit)— Add a unit to source and location indexes. 📍addsourceunit(source)— Add and returns a new unit with the given source string. Return type:TranslationUnit. 🆕addunit(unit)— Append the given unit to the object’s list of units. This method should always be used rather than trying to modify the list manually.
Parameters:
unit(TranslationUnit) — The unit that will be added. 🫶
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— Try to detect a file encoding fromtext, using either thechardetlib or by trying to decode the file. 🕵️♀️static fallback_detection(text)— Simple detection based on BOM in casechardetis not available. 🧨findid(id)— Find unit with matching id by checkingid_index. 🔎findunit(source)— Find the unit with the given source string. Return type:TranslationUnitorNone. 🎯findunits(source)— Find the units with the given source string. Return type:TranslationUnitorNone. 🎯getids()— Return a list of unit ids. 🆔getprojectstyle()— Get the project type for this store. 🏗️getsourcelanguage()— Get the source language for this store. 🌍gettargetlanguage()— Get the target language for this store. 🌍getunits()— Return a list of all units in this store. 📚isempty()— ReturnTrueif the object doesn’t contain any translation units. 🫙makeindex()— Indexes the items in this store. At least.sourceindexshould be useful. 🗂️property merge_on— The matching criterion to use when merging on.
Returns: The default matching criterion for all the subclasses. Return type: string. 💍
parse(input)— Parse the given file or file source string. 📥classmethod parsefile(storefile)— Reads the given file (or opens the given filename) and parses back to an object. 📂classmethod parsestring(storestring)— Convert the string representation back to an object. 🔁remove_unit_from_index(unit)— Remove a unit from source and location indexes. 🗑️removeunit(unit)— Remove the given unit to the object’s list of units. This method should always be used rather than trying to modify the list manually.
Parameters:
unit(TranslationUnit) — The unit that will be added. 🫶
require_index()— Make sure source index exists. ✅save()— Save to the file that data was originally read from, if available. 💾savefile(storefile)— Write the string representation to the given file (or filename). 📝serialize(out)— Converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write to. 🧬setprojectstyle(project_style)— Set the project type for this store. 🏗️setsourcelanguage(sourcelanguage)— Set the source language for this store. 🌍settargetlanguage(targetlanguage)— Set the target language for this store. 🌍suggestions_in_format = False— Indicates if format can store suggestions and alternative translation for a unit. 💡translate(source)— Return the translated string for a given source string. Return type:StringorNone. 🌐unit_iter()— Iterator over all the units in this store. 🔁
translate.storage.ini.iniunit(source=None, **kwargs)
A INI file entry. 🧚
adderror(errorname, errortext)
Drops an error message onto this unit, so it can ✨remember✨ what went wrong.
Parameters
errorname(string) — one single word that IDs the error.errortext(string) — the actual text describing what happened.
addlocation(location)
Adds exactly one location to the list of locations. 📍
NOTE: shouldn't be implemented if the format doesn't support it. If it can't hold locations, don't fake it, bestie.
addlocations(location)
Adds a location, or a whole list of locations, in one go.
NOTE: most classes shouldn't need to implement this — they should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. 👀 So maybe don't build your whole personality around it.
addnote(text, origin=None, position='append')
Adds a note (a comment, basically) to the unit.
Parameters
text(string) — usually just a sentence or two. Keep it cute, keep it short.origin(string) — says who/where the comment came from. Origin can be one of these text strings:'translator','developer','programmer','source code'(synonyms).position— defaults to'append'.
classmethod buildfromunit(unit)
Builds a native unit out of a foreign unit, preserving as much information as possible. It's giving "adopt don't shop" but for translation units. 🫶
static getcontext()
Gets the message context.
static geterrors()
Gets all the error messages.
Return type: Dictionary
getid()
A unique identifier for this unit. Like a little name tag but make it ✨unique✨.
Return type: string
Returns: an identifier for this unit that is unique in the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. No duplicates, no cap. 💅
getlocations()
A list of source code locations.
Return type: List
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)
Returns all the notes about this unit. 📝
It'll probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see getlocations()).
gettargetlen()
Returns the length of the target string.
Return type: Integer
NOTE: plural forms might be combined. So the number might be a lil' bit of a group effort.
getunits()
This unit, in a list. Just the one. Main character energy, honestly. 👑
static hasplural()
Tells you whether or not this specific unit has plural strings.
infer_state()
An empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state. It's literally a blank canvas, bestie.
isblank()
Used to see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. So it's lowkey on the way out.
static isfuzzy()
Indicates whether this unit is fuzzy. 🦋
static isheader()
Indicates whether this unit is a header.
static isobsolete()
Indicates whether a unit is obsolete. 💀
static isreview()
Indicates whether this unit needs review.
istranslatable()
Indicates whether this unit can be translated.
This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. So like, actual translatable content vs. the npc units.
istranslated()
Indicates whether this unit is translated.
This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does).
makeobsolete()
Makes a unit obsolete. Bye bye 👋
markfuzzy(value=True)
Marks the unit as fuzzy or not.
markreviewneeded(needsreview=True, explanation=None)
Marks the unit to show whether it needs review. 👀
Parameters
needsreview— defaults toTrue.explanation— adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)
Does basic, format-agnostic merging. Two units walk in, one unit walks out. 🚪✨
multistring_to_rich(mulstring)
Converts a multistring to a list of "rich" string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)
Removes all the translator's notes. Poof. 🪄
rich_parsers = []
A list of functions to use for parsing a string into a rich string tree.
property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)
Converts a "rich" string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)
Sets the message context.
setid(value)
Sets the unique identifier for this unit. 🏷️
Only implemented if the format allows ids independent from other unit properties like source or context.
unit_iter()
An iterator that only returns this unit. One and done.
translate.storage.ini.register_dialect(dialect)
A decorator that registers the dialect. Slay, decorators. ✨
🧚 jsonl10n
A class that manages JSON data files for translation.
JSON is an acronym for JavaScript Object Notation — it's an open standard designed for human-readable data interchange. Fr fr, it's everywhere.
JSON basic types:
- Number (integer or real)
- String (double-quoted Unicode with backslash escaping)
- Boolean (
trueorfalse) - Array (an ordered sequence of values, comma-separated and enclosed in square brackets)
- Object (a collection of key:value pairs, comma-separated and enclosed in curly braces)
null
Example:
{
"firstName": "John",
"lastName": "Smith",
"age": 25,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021"
},
"phoneNumber": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "fax",
"number": "646 555-4567"
}
]
}Todo:
- Handle
\uand other escapes in Unicode - Manage data type storage and conversion.
True–>"True"–>True
class translate.storage.jsonl10n.ARBJsonFile(inputfile=None, filter=None, **kwargs)
ARB JSON file. 🎀
See the following URLs for docs:
https://github.com/google/app-resource-bundle/wiki/ApplicationResourceBundleSpecificationhttps://docs.flutter.dev/development/accessibility-and-localization/internationalization#dart-tools
Extensions = None
A list of file extensions associated with this store type.
Mimetypes = None
A list of MIME types associated with this store type.
Name = 'Base translation store'
The human usable name of this store type.
UnitClass
An alias of ARBJsonUnit.
add_unit_to_index(unit)
Adds a unit to the source index and the location index. Two indexes, one unit, iconic behaviour.
addsourceunit(source)
Adds and returns a new unit with the given source string.
Return type: TranslationUnit
addunit(unit)
Appends the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually. Like, don't touch the list directly, bestie. 🙈
Parameters
unit(TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]
Tries to detect a file encoding from the text, using either the chardet lib or by trying to decode the file. Detective mode: activated. 🔍
static fallback_detection(text)
Simple detection based on BOM, in case chardet isn't available. Backup plan, but make it cute.
findid(id)
Finds the unit with the matching id by checking id_index.
findunit(source)
Finds the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)
Finds the units with the given source string.
Return type: TranslationUnit or None
getids()
Returns a list of unit ids.
getprojectstyle()
Gets the project type for this store.
getsourcelanguage()
Gets the source language for this store. 🌈
gettargetlanguage()
Gets the target language for this store.
getunits()
Returns a list of all the units in this store.
isempty()
Returns True if the object doesn't contain any translation units. Empty store = empty vibes. 🫙
makeindex()— okay so this indexes the items in this store. At least.sourceindexshould be useful, bestie. ✨property merge_on— the matching criterion you use when merging on. Returns: The default matching criterion for all the subclasses. Return type:stringparse(input)— parses the given file or file source string. 🫶classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object. 🌸classmethod parsestring(storestring)— converts the string representation back to an object. 🎀remove_unit_from_index(unit)— removes a unit from source and locaton indexes. 💅removeunit(unit)— removes the given unit to the object’s list of units. This method should always be used rather than trying to modify the list manually.unit(TranslationUnit) – The unit that will be added.
require_index()— makes sure source index exists. ✅save()— saves to the file that data was originally read from, if available. 💾savefile(storefile)— writes the string representation to the given file (or filename). 🦋serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write to. 🔥setprojectstyle(project_style)— sets the project type for this store. 🛠️setsourcelanguage(sourcelanguage)— sets the source language for this store. 🌍settargetlanguage(targetlanguage)— sets the target language for this store. 🌐suggestions_in_format = False— indicates if format can store suggestions and alternative translation for a unit. 💡translate(source)— returns the translated string for a given source string. Return type:String or Noneunit_iter()— iterator over all the units in this store. 🔁
🦋 class translate.storage.jsonl10n.ARBJsonUnit(source=None, item=None, notes=None, placeholders=None, metadata=None, **kwargs)
adderror(errorname, errortext)— adds an error message to this unit. 😭errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
addlocation(location)— adds one location to the list of locations.
NOTE: Shouldn’t be implemented if the format doesn’t support it.
addlocations(location)— adds a location or a list of locations.
NOTE: Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: This method might be removed in future.
addnote(text, origin=None, position='append')— adds a note (comment). 📝text(string) – usually just a sentence or two.origin(string) – specifies who/where the comment comes from. Origin can be one of the following text strings:‘translator’‘developer’,‘programmer’,‘source code’(synonyms)
classmethod buildfromunit(unit)— builds a native unit from a foreign unit. Preserving as much information as possible. 🧚static getcontext()— gets the message context. 🎯static geterrors()— gets all error messages. Return type:Dictionarygetid()— a unique identifier for this unit. Return type:stringReturns: an identifier for this unit that is unique in the store. Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. 👑getlocations()— a list of source code locations. Return type:List
NOTE: Shouldn’t be implemented if the format doesn’t support it.
getnotes(origin=None)— returns all notes about this unit. It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()). 💬gettargetlen()— returns the length of the target string. Return type:Integer
NOTE: Plural forms might be combined.
getunits()— this unit in a list. 📦getvalue()— returns dictionary for serialization. 🗂️static hasplural()— tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state. 🌀isblank()— used to see if this unit has no source or target string.
NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()— indicates whether this unit is fuzzy. 🌫️isheader()— indicates whether this unit is a header. 🏷️static isobsolete()— indicate whether a unit is obsolete. 💀static isreview()— indicates whether this unit needs review. 👀istranslatable()— indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. ✅istranslated()— indicates whether this unit is translated. This should be used rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does). 💖makeobsolete()— makes a unit obsolete. 🪦markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.needsreview– defaults toTrue.explanation– adds an optional explanation as a note. 🔍
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic format agnostic merging. 🧩multistring_to_rich(mulstring)— converts a multistring to a list of “rich” string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator’s notes. 🧹rich_parsers = []— a list of functions to use for parsing a string into a rich string tree. 🌱property rich_source— the rich source property. 💅
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target— the rich target property. 💅
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a “rich” string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context. 🎯setid(value, unitid=None)— sets the unique identified for this unit. only implemented if format allows ids independant from other unit properties like source or context. 🆔unit_iter()— iterator that only returns this unit. ☝️
🌈 class translate.storage.jsonl10n.BaseJsonUnit(source=None, item=None, notes=None, placeholders=None, **kwargs)
A JSON entry. 🍓
adderror(errorname, errortext)— adds an error message to this unit. 😭errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
addlocation(location)— adds one location to the list of locations.
NOTE: Shouldn’t be implemented if the format doesn’t support it.
addlocations(location)— adds a location or a list of locations.
NOTE: Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: This method might be removed in future.
addnote(text, origin=None, position='append')— adds a note (comment). 📝text(string) – usually just a sentence or two.origin(string) – specifies who/where the comment comes from. Origin can be one of the following text strings:‘translator’‘developer’,‘programmer’,‘source code’(synonyms)
classmethod buildfromunit(unit)— builds a native unit out of a foreign unit, preserving literally as much info as possible 🫶static getcontext()— gets the message context ✨static geterrors()— gets all the error messages.- Return type: Dictionary
getid()— a unique identifier for this unit, iykyk.- Return type: string
- Returns: an identifier for this unit that is unique in the store
- fr fr, derived classes should override this so it guarantees a unique identifier for each unit in the store 👑
getlocations()— a list of source code locations.- Return type: List
- NOTE: shouldn’t be implemented if the format doesn’t support it 🙈
getnotes(origin=None)— returns all notes about this unit. It’ll probably be freeform text or something reasonable that the format can synthesise. It should not include location comments — seegetlocations()for those 💅gettargetlen()— returns the length of the target string.- Return type: Integer
- NOTE: plural forms might be combined 🌀
getunits()— this unit, in a list. That’s it. One-item list energy ✨getvalue()— returns dictionary for serialization.static hasplural()— tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state 🧚isblank()— used to see if this unit has no source or target string.- NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. So like… it’s on thin ice 💀
static isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicate whether a unit is obsolete.static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units 🌈istranslated()— indicates whether this unit is translated. Use this rather than deducing it from.target, so that other classes can implement more functionality (as XLIFF does) 🫡makeobsolete()— make a unit obsolete. bye bestie 👋markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.- Parameters:
needsreview— defaults toTrue.explanation— adds an optional explanation as a note 📝
- Parameters:
merge(otherunit, overwrite=False, comments=True, authoritative=False)— do basic format agnostic merging.multistring_to_rich(mulstring)— convert a multistring to a list of “rich” string trees 🌳
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— remove all the translator’s notes 🧹rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
property rich_target- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
classmethod rich_to_multistring(elem_list)— convert a “rich” string tree to a multistring 💖
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— set the message context.setid(value, unitid=None)— sets the unique identifier for this unit. Only implemented if the format allows ids independent from other unit properties like source or context 🔑unit_iter()— iterator that only returns this unit. Solo main character energy, no ensemble cast 🌟
🦋 class translate.storage.jsonl10n.FlatJsonUnit(source=None, item=None, notes=None, placeholders=None, **kwargs)
Okay bestie, new class, same iconic energy — here’s everything it can do 💅
adderror(errorname, errortext)— adds an error message to this unit.- Parameters:
errorname(string) — a single word to id the error.errortext(string) — the text describing the error.
- Parameters:
addlocation(location)— add one location to the list of locations.- NOTE: shouldn’t be implemented if the format doesn’t support it 🙈
addlocations(location)— add a location or a list of locations.- NOTE: most classes shouldn’t need to implement this, but should rather implement
TranslationUnit.addlocation(). - > WARNING: this method might be removed in future. So like… don’t get too attached 😭
- NOTE: most classes shouldn’t need to implement this, but should rather implement
addnote(text, origin=None, position='append')— adds a note (comment).- Parameters:
text(string) — usually just a sentence or two.origin(string) — specifies who/where the comment comes from. Origin can be one of the following text strings:'translator','developer','programmer','source code'(synonyms) 🎀
- Parameters:
classmethod buildfromunit(unit)— builds a native unit from a foreign unit, preserving literally as much info as possible 🫶static getcontext()— gets the message context ✨static geterrors()— gets all the error messages.- Return type: Dictionary
getid()— a unique identifier for this unit.- Return type: string
- Returns: an identifier for this unit that is unique in the store
- Derived classes should override this so it guarantees a unique identifier for each unit in the store 👑
getlocations()— a list of source code locations.- Return type: List
- NOTE: shouldn’t be implemented if the format doesn’t support it 🙈
getnotes(origin=None)— returns all notes about this unit. It’ll probably be freeform text or something reasonable that the format can synthesise. It should not include location comments — seegetlocations()) 💅gettargetlen()— returns the length of the target string.- Return type: Integer
- NOTE: plural forms might be combined 🌀
getunits()— this unit, in a list. One-item list, still iconic ✨getvalue()— returns dictionary for serialization.static hasplural()— tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state 🧚isblank()— used to see if this unit has no source or target string.- NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. Yeah it’s giving deprecated 💀
static isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicate whether a unit is obsolete.static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units 🌈istranslated()— indicates whether this unit is translated. Use this rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does) 🫡makeobsolete()— make a unit obsolete. bye 👋markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.- Parameters:
needsreview— defaults toTrue.explanation— adds an optional explanation as a note 📝
- Parameters:
merge(otherunit, overwrite=False, comments=True, authoritative=False)— do basic format agnostic merging.multistring_to_rich(mulstring)— convert a multistring to a list of “rich” string trees 🌳
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]Ooh and look at that, each string got wrapped in its own cute little StringElem shell 💅 like lil matryoshka dolls, it's giving nesting energy fr fr.
removenotes(origin=None)— yeets all of the translator's notes. gone. bye. 💨rich_parsers = []— a list of functions you use to parse a string into a rich string tree ✨ (yes it starts out empty, you gotta fill it yourself, that's the vibe).- property
rich_source— the rich-string version of the source. lowkey the real one.
SEE ALSO:rich_to_multistring(),multistring_to_rich().
- property
rich_target— same deal but for the target side, bestie 🫶
SEE ALSO:rich_to_multistring(),multistring_to_rich().
- classmethod
rich_to_multistring(elem_list)— flips a "rich" string tree back into a multistring, no biggie:
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context. that's the whole thing. 🎀setid(value, unitid=None)— sets the unique identifier for this unit. heads up tho: this is only implemented if the format actually allows ids that are independent from other unit properties like source or context. so if your format is clingy, it just won't work 😭unit_iter()— an iterator that only returns this one unit. party of one 🎉 literally just the unit itself.
🦋 class translate.storage.jsonl10n.GoI18NJsonFile(inputfile=None, filter=None, **kwargs)
okay so this one is the go-i18n JSON file store, bestie. it's giving Go energy 🇬🇨✨
the docs live here, go look:
https://github.com/nicksnyder/go-i18n/tree/v1 ‐ https://pkg.go.dev/github.com/nicksnyder/go-i18nExtensions = None— a list of file extensions associated with this store type. (yes the original says "extentions", we don't gatekeep spelling here 💀)Mimetypes = None— a list of MIME types associated with this store type 🌸Name = 'Base translation store'— the human usable name of this store type, aka the glow-up label 👑UnitClass— alias ofGoI18NJsonUnit. it's the same class in a little outfit 💅add_unit_to_index(unit)— adds a unit to the source and location indexes. (original says "idexes" — iconic, we stan 🫶)addsourceunit(source)— adds and returns a new unit with the given source string.
Return type: TranslationUnit
addunit(unit)— appends the given unit to the object's list of units. bestie, always use this method, do not go poking at the list manually, that's not the vibe 🙈
Parameters: unit (TranslationUnit) – The unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from the text, either by using the chardet lib or by just trying to decode the file and seeing what happens 😭- static
fallback_detection(text)— simple detection based on BOM, for when chardet isn't available. the backup bestie 🫶 findid(id)— finds the unit with a matching id by checkingid_index.findunit(source)— finds the unit with the given source string.
Return type:TranslationUnitorNone
findunits(source)— finds the units (plural!!) with the given source string.
Return type:TranslationUnitorNone
getids()— returns a list of unit ids. a whole roster ✨getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store. iykyk 🌈getunits()— returns a list of all the units in this store.isempty()— returnsTrueif the object doesn't contain any translation units. empty bucket vibes 🪣makeindex()— indexes the items in this store. at minimum.sourceindexshould be useful, so like, use it 💅- property
merge_on— the matching criterion to use when merging on.
Returns: The default matching criterion for all the subclasses.
Return type: string
parse(input)— parses the given file or file source string.- classmethod
parsefile(storefile)— reads the given file (or opens the given filename) and parses it back to an object. - classmethod
parsestring(storestring)— converts the string representation back to an object. the reverse-uno card 🔄 remove_unit_from_index(unit)— removes a unit from the source and location indexes.removeunit(unit)— removes the given unit from the object's list of units. again: use this method, don't touch the list yourself, fr fr 🫶
Parameters: unit (TranslationUnit) – The unit that will be added. (yes, the docstring still says "added", we're keeping it, it's camp 💀)
require_index()— makes sure the source index exists. if it doesn't, it gets made. slay 🎀save()— saves to the file that the data was originally read from, if that file is available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like object to write to. bytes era ✨setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit. spoiler: this one's a no 🙈translate(source)— returns the translated string for a given source string.
Return type:StringorNone
unit_iter()— iterator over all the units in this store. every single one, no skips 👑
🧚 class translate.storage.jsonl10n.GoI18NJsonUnit(source=None, item=None, notes=None, placeholders=None, **kwargs)
the unit class for go-i18n JSON. this is where one single message lives its best life 💜
adderror(errorname, errortext)— adds an error message to this unit.
Parameters: - errorname (string) – A single word to id the error. - errortext (string) – The text describing the error.
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. no fake features, bestie 🚫
addlocations(location)— adds a location or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. so like… enjoy it while it lasts 😭
addnote(text, origin=None, position='append')— adds a note (a comment).
Parameters: - text (string) – Usually just a sentence or two. - origin (string) – Specifies who/where the comment comes from. Origin can be one of the following text strings:'translator','developer','programmer','source code'(synonyms).
- classmethod
buildfromunit(unit)— builds a native unit from a foreign unit, preserving as much information as possible. glow-up conversion, nothing lost 💅 - static
getcontext()— gets the message context. - static
geterrors()— gets all the error messages.
Return type: Dictionary
getid()— a unique identifier for this unit.
Return type: string
Returns: an identifier for this unit that is unique in the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. that's on you, not us 🫶
getlocations()— a list of source code locations.
Return type: List
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all the notes about this unit. it'll probably be freeform text or something reasonable that the format can synthesise. it should not include location comments (seegetlocations()). keep it separated bestie ✨gettargetlen()— returns the length of the target string.
Return type: Integer
NOTE: plural forms might be combined.
getunits()— this unit, in a list. one-item list, main character energy 👑getvalue()— returns a dictionary for serialization.- static
hasplural()— tells you whether or not this specific unit has plural strings. yes/no/maybe 🌸 infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state. it literally does nothing here, it's a placeholder with a dream 😭isblank()— used to see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. (their words, not ours 💀)
- static
isfuzzy()— indicates whether this unit is fuzzy. fuzzy wuzzy 🧸 - static
isheader()— indicates whether this unit is a header. - static
isobsolete()— indicates whether a unit is obsolete. it's over bestie 💀 - static
isreview()— indicates whether this unit needs review. 👀 istranslatable()— indicates whether this unit can be translated. vibe check: translatable or not 🎀
…and yeah, that's literally the whole point of it — you use this to tell real units meant for translation apart from header units, obsolete ones, binary ones, and all the other blank ones. Vibe check on your units, basically. 💅
istranslated()
Tells you whether this unit is translated or not.
Use this instead of trying to deduce it from .target yourself, okay? That way other classes can add more functionality (looking at you, XLIFF) and your code doesn't break. 🫶
makeobsolete()
Makes the unit obsolete. Bye bestie 👋
markfuzzy(value=True)
Marks the unit as fuzzy, or not fuzzy. Your call, value defaults to True.
markreviewneeded(needsreview=True, explanation=None)
Marks the unit to say whether it needs review.
Parameters
needsreview– defaults toTrue.explanation– adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)
Does a basic format-agnostic merge. No thoughts, just merging. ✨
multistring_to_rich(mulstring)
Converts a multistring to a list of "rich" string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)
Removes all the translator's notes. All of them. Gone. 🧹
rich_parsers = []
A list of functions to use for parsing a string into a rich string tree.
property rich_source
SEE ALSO: rich_to_multistring(), multistring_to_rich().
property rich_target
SEE ALSO: rich_to_multistring(), multistring_to_rich().
classmethod rich_to_multistring(elem_list)
Converts a "rich" string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)
Sets the message context. Context is main character energy. 👑
setid(value, unitid=None)
Sets the unique identifier for this unit.
Only actually implemented if the format allows ids that are independent of other unit properties like source or context. iykyk.
unit_iter()
An iterator that only returns this unit. Just the one. Solo trip. 🌟
class translate.storage.jsonl10n.GoI18NV2JsonFile(inputfile=None, filter=None, **kwargs)
go-i18n v2 JSON file. Obsessed with this format ngl.
See the following URLs for the docs:
https://github.com/nicksnyder/go-i18n ‐ https://pkg.go.dev/github.com/nicksnyder/go-i18n/v2
Extensions = None
A list of file extentions associated with this store type. (Yes, that's how it's spelled. Copying it exactly, bestie.)
Mimetypes = None
A list of MIME types associated with this store type.
Name = 'Base translation store'
The human usable name of this store type.
UnitClass
alias of GoI18NV2JsonUnit.
add_unit_to_index(unit)
Adds a unit to the source and location idexes. 🗂️
addsourceunit(source)
Adds and returns a new unit with the given source string.
Return type: TranslationUnit
addunit(unit)
Appends the given unit to the object's list of units.
You should always use this method rather than trying to modify the list manually. fr fr, don't touch the list.
Parameters
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]
Tries to detect a file encoding from text, either using the chardet lib or by just trying to decode the file. ✨
static fallback_detection(text)
Simple detection based on BOM, in case chardet isn't available. Backup plan behavior. 💜
findid(id)
Finds the unit with the matching id by checking id_index.
findunit(source)
Finds the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)
Finds the units with the given source string.
Return type: TranslationUnit or None
getids()
Returns a list of unit ids.
getprojectstyle()
Gets the project type for this store.
getsourcelanguage()
Gets the source language for this store.
gettargetlanguage()
Gets the target language for this store.
getunits()
Returns a list of all units in this store.
isempty()
Returns True if the object doesn't contain any translation units.
makeindex()
Indexes the items in this store. At minimum, .sourceindex should be useful.
property merge_on
The matching criterion to use when merging on.
Returns: the default matching criterion for all the subclasses. Return type: string
parse(input)
Parses the given file or file source string.
classmethod parsefile(storefile)
Reads the given file (or opens the given filename) and parses it back to an object.
classmethod parsestring(storestring)
Converts the string representation back to an object.
remove_unit_from_index(unit)
Removes a unit from the source and locaton indexes. (Again — spelling preserved exactly. 💅)
removeunit(unit)
Removes the given unit from the object's list of units.
You should always use this method rather than trying to modify the list manually.
Parameters
unit(TranslationUnit) – the unit that will be added.
require_index()
Makes sure the source index exists. Asserting dominance over your indexes. 🔥
save()
Saves to the file that data was originally read from, if available.
savefile(storefile)
Writes the string representation to the given file (or filename).
serialize(out)
Converts to a bytes representation that can be parsed back using parsestring(). out should be an open file-like object to write to.
setprojectstyle(project_style)
Sets the project type for this store.
setsourcelanguage(sourcelanguage)
Sets the source language for this store.
settargetlanguage(targetlanguage)
Sets the target language for this store.
suggestions_in_format = False
Indicates whether the format can store suggestions and alternative translations for a unit.
translate(source)
Returns the translated string for a given source string.
Return type: String or None
unit_iter()
Iterator over all the units in this store. 🦋
class translate.storage.jsonl10n.GoI18NV2JsonUnit(source=None, item=None, notes=None, placeholders=None, **kwargs)
adderror(errorname, errortext)
Adds an error message to this unit.
Parameters
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
addlocation(location)
Adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it.
addlocations(location)
Adds a location, or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. 😭 Don't get too attached.
addnote(text, origin=None, position='append')
Adds a note (comment).
Parameters
text(string) – usually just a sentence or two.origin(string) – specifies who/where the comment comes from. Origin can be one of the following text strings:'translator','developer','programmer','source code'(synonyms).
classmethod buildfromunit(unit)
Builds a native unit from a foreign unit.
Preserving as much information as possible. Glow-up without losing the lore. ✨
static getcontext()
Gets the message context.
static geterrors()
Gets all error messages.
Return type: Dictionary
getid()
A unique identifier for this unit.
Return type: string Returns: an identifier for this unit that is unique in the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. No duplicates, no cap. 🎀
getlocations()
A list of source code locations.
Return type: List
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)
Returns all notes about this unit.
It'll probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see getlocations()).
gettargetlen()
Returns the length of the target string.
Return type: Integer
NOTE: plural forms might be combined. Just so you know. 💀
getunits()
This unit, in a list. That's it. That's the list. 🍓
getvalue()
Returns dictionary for serialization.
static hasplural()— tells you whether this specific unit has plural strings or not. yes, per-unit, bestie 💅infer_state()— an empty method, literally just vibes, meant to be overridden in sub-classes so they can infer the currentstate(_n)of the unit from its currentstate. it's giving "template method, do it yourself" 🎀isblank()— used to check if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. 😭 so like, it's on thin ice, fr fr.
static isfuzzy()— indicates whether this unit is fuzzy 🌸static isheader()— indicates whether this unit is a header 👑static isobsolete()— indicates whether a unit is obsolete 💀static isreview()— indicates whether this unit needs review 🙈istranslatable()— indicates whether this unit can be translated.
This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. like, the whole "is this actually a snack or just a decoration" check ✨
istranslated()— indicates whether this unit is translated.
This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does). no cap, don't do the deduction thing 🫶
makeobsolete()— make a unit obsolete. bye bestie 💀markfuzzy(value=True)— marks the unit as fuzzy or not. defaultTrue.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.
Parameters
- needsreview — defaults to
True. - explanation — adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— do basic format agnostic merging. slay, merge across formats 💅multistring_to_rich(mulstring)— convert a multistring to a list of "rich" string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— remove all the translator's notes. poof, gone 🧚rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— convert a "rich" string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— set the message context.setid(value, unitid=None)— sets the unique identifier for this unit.
only implemented if format allows ids independant from other unit properties like source or context. iykyk 🔥
unit_iter()— iterator that only returns this unit. literally just the one, main character energy 👑
🦋 class translate.storage.jsonl10n.GoTextJsonFile(inputfile=None, filter=None, **kwargs)
gotext JSON file.
see the following URLs for doc:
https://pkg.go.dev/golang.org/x/text/cmd/gotext ‐ https://github.com/golang/text/tree/master/cmd/gotext/examples/extract/locales/en-US
Extensions = None— a list of file extentions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human usable name of this store type.UnitClass— alias ofGoTextJsonUnit.add_unit_to_index(unit)— add a unit to source and location idexes.addsourceunit(source)— add and returns a new unit with the given source string.
Return type: TranslationUnit
addunit(unit)— append the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually.
Parameters: unit (TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from text, using either the chardet lib or by trying to decode the file.static fallback_detection(text)— simple detection based on BOM in case chardet is not available. the backup plan, always good to have one 🫶findid(id)— find unit with matching id by checkingid_index.findunit(source)— find the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)— find the units with the given source string.
Return type: TranslationUnit or None
getids()— return a list of unit ids.getprojectstyle()— get the project type for this store.getsourcelanguage()— get the source language for this store.gettargetlanguage()— get the target language for this store.getunits()— return a list of all units in this store.isempty()— returnTrueif the object doesn't contain any translation units.makeindex()— indexes the items in this store. at least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.
Returns: the default matching criterion for all the subclasses.
Return type: string
parse(input)— parse the given file or file source string.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— convert the string representation back to an object.remove_unit_from_index(unit)— remove a unit from source and locaton indexes.removeunit(unit)— remove the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually.
Parameters: unit (TranslationUnit) – the unit that will be added.
save()— save to the file that data was originally read from, if available.savefile(storefile)— write the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back usingparsestring(). out should be an open file-like objects to write to.setprojectstyle(project_style)— set the project type for this store.setsourcelanguage(sourcelanguage)— set the source language for this store.settargetlanguage(targetlanguage)— set the target language for this store.suggestions_in_format = False— indicates if format can store suggestions and alternative translation for a unit.translate(source)— return the translated string for a given source string.
Return type: String or None
unit_iter()— iterator over all the units in this store.
🎀 class translate.storage.jsonl10n.GoTextJsonUnit(source=None, item=None, notes=None, placeholders=None, comment=None, message=None, meaning=None, key=None, fuzzy=None, position=None, **kwargs)
adderror(errorname, errortext)— adds an error message to this unit.
Parameters
- errorname (
string) – a single word to id the error. - errortext (
string) – the text describing the error.
addlocation(location)— add one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it.
addlocations(location)— add a location or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: this method might be removed in future. ⚠️ so like, don't get too attached.
addnote(text, origin=None, position='append')— adds a note (comment).
Parameters
- text (
string) – usually just a sentence or two. - origin (
string) – specifies who/where the comment comes from. Origin can be one of the following text strings: -'translator'-'developer','programmer','source code'(synonyms)
classmethod buildfromunit(unit)— build a native unit from a foreign unit.
Preserving as much information as possible. 💖
static getcontext()— get the message context.static geterrors()— get all error messages.
Return type: Dictionary
getid()— a unique identifier for this unit.
Return type: string
Returns an identifier for this unit that’s unique in the store ✨ Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. No duplicate IDs, bestie — that’s the whole vibe 💅
getlocations()
Gives you a list of source code locations 🗺️
Return type: List
💡 NOTE: Shouldn’t be implemented if the format doesn’t support it.
getnotes(origin=None)
Returns all the notes about this unit.
It’ll probably be freeform text or something reasonable that the format can synthesise. It should not include location comments (see getlocations()).
gettargetlen()
Returns the length of the target string.
Return type: Integer
💡 NOTE: Plural forms might be combined.
getunits()
This unit in a list. Literally just itself, wrapped in a list 🫶
getvalue()
Returns dictionary for serialization.
static hasplural()
Tells whether or not this specific unit has plural strings.
infer_state()
Empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state.
isblank()
Used to see if this unit has no source or target string.
💡 NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()
Indicates whether this unit is fuzzy.
static isheader()
Indicates whether this unit is a header.
static isobsolete()
Indicate whether a unit is obsolete.
static isreview()
Indicates whether this unit needs review.
istranslatable()
Indicates whether this unit can be translated.
This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.
istranslated()
Indicates whether this unit is translated.
This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does).
makeobsolete()
Make a unit obsolete.
markfuzzy(value=True)
Marks the unit as fuzzy or not.
markreviewneeded(needsreview=True, explanation=None)
Marks the unit to indicate whether it needs review.
Parameters:
needsreview– Defaults toTrue.explanation– Adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)
Do basic format agnostic merging.
multistring_to_rich(mulstring)
Convert a multistring to a list of “rich” string trees 🌳
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)
Remove all the translator’s notes.
rich_parsers = []
A list of functions to use for parsing a string into a rich string tree.
property rich_source
SEE ALSO: rich_to_multistring(), multistring_to_rich().
property rich_target
SEE ALSO: rich_to_multistring(), multistring_to_rich().
classmethod rich_to_multistring(elem_list)
Convert a “rich” string tree to a multistring ✨
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)
Set the message context.
setid(value, unitid=None)
Sets the unique identifier for this unit.
Only implemented if format allows ids independent from other unit properties like source or context.
unit_iter()
Iterator that only returns this unit.
class translate.storage.jsonl10n.I18NextFile(inputfile=None, filter=None, **kwargs)
A i18next v3 format, this is nested JSON with several additions. See https://www.i18next.com/ 🦋
Extensions = None— A list of file extentions associated with this store typeMimetypes = None— A list of MIME types associated with this store typeName = 'Base translation store'— The human usable name of this store typeUnitClass— alias ofI18NextUnit
add_unit_to_index(unit)
Add a unit to source and location idexes.
addsourceunit(source)
Add and returns a new unit with the given source string.
Return type: TranslationUnit
addunit(unit)
Append the given unit to the object’s list of units.
This method should always be used rather than trying to modify the list manually.
Parameters:
unit(TranslationUnit) – The unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]
Try to detect a file encoding from text, using either the chardet lib or by trying to decode the file.
static fallback_detection(text)
Simple detection based on BOM in case chardet is not available.
findid(id)
Find unit with matching id by checking id_index.
findunit(source)
Find the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)
Find the units with the given source string.
Return type: TranslationUnit or None
getids()
Return a list of unit ids.
getprojectstyle()
Get the project type for this store.
getsourcelanguage()
Get the source language for this store.
gettargetlanguage()
Get the target language for this store.
getunits()
Return a list of all units in this store.
isempty()
Return True if the object doesn’t contain any translation units.
makeindex()
Indexes the items in this store. At least .sourceindex should be useful.
property merge_on
The matching criterion to use when merging on.
Returns: The default matching criterion for all the subclasses.
Return type: string
parse(input)
Parse the given file or file source string.
classmethod parsefile(storefile)
Reads the given file (or opens the given filename) and parses back to an object.
classmethod parsestring(storestring)
Convert the string representation back to an object.
remove_unit_from_index(unit)
Remove a unit from source and locaton indexes.
removeunit(unit)
Remove the given unit to the object’s list of units.
This method should always be used rather than trying to modify the list manually.
Parameters:
unit(TranslationUnit) – The unit that will be added. (Yes, the original says “added” — but it’s the one being removed, bestie 😭💀)
require_index()
Make sure source index exists.
save()
Save to the file that data was originally read from, if available.
savefile(storefile)
Write the string representation to the given file (or filename).
serialize(out)
Converts to a bytes representation that can be parsed back using parsestring(). out should be an open file-like objects to write to.
setprojectstyle(project_style)
Set the project type for this store.
setsourcelanguage(sourcelanguage)
Set the source language for this store.
settargetlanguage(targetlanguage)
Set the target language for this store.
suggestions_in_format = False
Indicates if format can store suggestions and alternative translation for a unit.
translate(source)
Return the translated string for a given source string.
Return type: String or None
unit_iter()
Iterator over all the units in this store.
class translate.storage.jsonl10n.I18NextUnit(source=None, item=None, notes=None, placeholders=None, **kwargs)
A i18next v3 format, JSON with plurals. See https://www.i18next.com/ 🌸
adderror(errorname, errortext)
Adds an error message to this unit.
Parameters:
errorname(string) – A single word to id the error.errortext(string) – The text describing the error.
addlocation(location)
Add one location to the list of locations.
💡 NOTE: Shouldn’t be implemented if the format doesn’t support it.
addlocations(location)
Add a location or a list of locations.
💡 NOTE: Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation().
⚠️ WARNING: This method might be removed in future. 💀
addnote(text, origin=None, position='append')
Okay so this one adds a note (comment) 📝. Usually it’s just a little extra context, no big dramatic monologue.
- Parameters
text(string) — Usually just a sentence or two.origin(string) — Specifies who/where the comment comes from. Origin can be one of these text strings:'translator'; or'developer','programmer','source code'(those are synonyms).
classmethod buildfromunit(unit)
Builds a native unit from a foreign unit 🌍➡️🏠. Preserving as much information as possible.
static getcontext()
Gets the message context. Context girlies rise 🙌.
static geterrors()
Gets all error messages.
- Return type:
Dictionary
getid()
A unique identifier for this unit 🆔.
- Return type:
string - Returns: an identifier for this unit that is unique in the store
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. No duplicate-ID chaos allowed, bestie 💅.
getlocations()
A list of source code locations 📍.
- Return type:
List
NOTE: Shouldn’t be implemented if the format doesn’t support it.
getnotes(origin=None)
Returns all notes about this unit. It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see getlocations()). Keep the location gossip separate, tyvm 🫶.
gettargetlen()
Returns the length of the target string.
- Return type:
Integer
NOTE: Plural forms might be combined.
getunits()
This unit in a list. Just one lil unit, living its list life ✨.
getvalue()
Returns dictionary for serialization.
static hasplural()
Tells whether or not this specific unit has plural strings.
infer_state()
Empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state.
isblank()
Used to see if this unit has no source or target string.
NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()
Indicates whether this unit is fuzzy.
static isheader()
Indicates whether this unit is a header.
static isobsolete()
Indicate whether a unit is obsolete.
static isreview()
Indicates whether this unit needs review.
istranslatable()
Indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.
istranslated()
Indicates whether this unit is translated. This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does). Don’t guess from .target, okay? Let the class cook 🍳.
makeobsolete()
Make a unit obsolete.
markfuzzy(value=True)
Marks the unit as fuzzy or not.
markreviewneeded(needsreview=True, explanation=None)
Marks the unit to indicate whether it needs review.
- Parameters
needsreview— Defaults toTrue.explanation— Adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)
Do basic format agnostic merging.
multistring_to_rich(mulstring)
Convert a multistring to a list of “rich” string trees 🌳✨.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)
Remove all the translator’s notes.
rich_parsers = []
A list of functions to use for parsing a string into a rich string tree.
property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)
Convert a “rich” string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)
Set the message context.
setid(value, unitid=None)
Sets the unique identified for this unit. only implemented if format allows ids independant from other unit properties like source or context. Yes, “identified” and “independant” are exactly what the page says — we keep the lore 😭.
unit_iter()
Iterator that only returns this unit.
class translate.storage.jsonl10n.I18NextV4File(inputfile=None, filter=None, **kwargs)
A i18next v4 format, this is nested JSON with several additions.
Extensions = None— A list of file extentions associated with this store type.Mimetypes = None— A list of MIME types associated with this store type.Name = 'Base translation store'— The human usable name of this store type.UnitClass— alias ofI18NextV4Unit.
add_unit_to_index(unit)
Add a unit to source and location idexes.
addsourceunit(source)
Add and returns a new unit with the given source string.
- Return type:
TranslationUnit
addunit(unit)
Append the given unit to the object’s list of units.
This method should always be used rather than trying to modify the list manually.
- Parameters
unit(TranslationUnit) — The unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]
Try to detect a file encoding from text, using either the chardet lib or by trying to decode the file.
static fallback_detection(text)
Simple detection based on BOM in case chardet is not available.
findid(id)
Find unit with matching id by checking id_index.
findunit(source)
Find the unit with the given source string.
- Return type:
TranslationUnit or None
findunits(source)
Find the units with the given source string.
- Return type:
TranslationUnit or None
getids()
Return a list of unit ids.
getprojectstyle()
Get the project type for this store.
getsourcelanguage()
Get the source language for this store.
gettargetlanguage()
Get the target language for this store.
getunits()
Return a list of all units in this store.
isempty()
Return True if the object doesn’t contain any translation units.
makeindex()
Indexes the items in this store. At least .sourceindex should be useful.
property merge_on
The matching criterion to use when merging on.
- Returns: The default matching criterion for all the subclasses.
- Return type:
string
parse(input)
Parse the given file or file source string.
classmethod parsefile(storefile)
Reads the given file (or opens the given filename) and parses back to an object.
classmethod parsestring(storestring)
Convert the string representation back to an object.
remove_unit_from_index(unit)
Remove a unit from source and locaton indexes.
removeunit(unit)
Remove the given unit to the object’s list of units.
This method should always be used rather than trying to modify the list manually.
- Parameters
unit(TranslationUnit) — The unit that will be added.
require_index()
Make sure source index exists.
save()
Save to the file that data was originally read from, if available.
savefile(storefile)
Write the string representation to the given file (or filename).
serialize(out)
Converts to a bytes representation that can be parsed back using parsestring(). out should be an open file-like objects to write to.
setprojectstyle(project_style)
Set the project type for this store.
setsourcelanguage(sourcelanguage)
Set the source language for this store.
settargetlanguage(targetlanguage)
Set the target language for this store.
suggestions_in_format = False
Indicates if format can store suggestions and alternative translation for a unit.
translate(source)
Return the translated string for a given source string.
- Return type:
String or None unit_iter()— iterator over all the units in this store. Every single one, no cap. 💅
🌸 translate.storage.jsonl10n.I18NextV4Unit 🌸
class translate.storage.jsonl10n.I18NextV4Unit(source=None, item=None, notes=None, placeholders=None, **kwargs)This one is a i18next v4 format, JSON with plurals, and it’s giving translation-storage main character energy. See https://www.i18next.com/ for the lore. 🦋
adderror(errorname, errortext)— adds an error message to this unit.errorname(string) — a single word to id the error.errortext(string) — the text describing the error.
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn’t be implemented if the format doesn’t support it.
addlocations(location)— adds a location or a list of locations.
NOTE: most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: this method might be removed in future.
addnote(text, origin=None, position='append')— adds a note (comment).text(string) — usually just a sentence or two.origin(string) — specifies who/where the comment comes from. It can be one of these text strings:'translator';'developer','programmer','source code'(synonyms).- The signature also has
position='append', bestie. 🎀
classmethod buildfromunit(unit)— builds a native unit from a foreign unit, preserving as much information as possible. Iconic.static getcontext()— gets the message context.static geterrors()— gets all error messages.- Return type:
Dictionary
- Return type:
getid()— a unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store.
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.
- Return type:
getlocations()— a list of source code locations.- Return type:
List
- Return type:
NOTE: shouldn’t be implemented if the format doesn’t support it.
getnotes(origin=None)— returns all notes about this unit. It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()).gettargetlen()— returns the length of the target string.- Return type:
Integer
- Return type:
NOTE: plural forms might be combined.
getunits()— this unit in a list.getvalue()— returns dictionary for serialization.static hasplural()— tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— used to see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete.static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.istranslated()— indicates whether this unit is translated. This should be used rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does).makeobsolete()— makes a unit obsolete.markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.needsreview— defaults toTrue.explanation— adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic format agnostic merging. 🫶multistring_to_rich(mulstring)— converts a multistring to a list of “rich” string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator’s notes.rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source— the rich source property. SEE ALSO:rich_to_multistring(),multistring_to_rich().property rich_target— the rich target property. SEE ALSO:rich_to_multistring(),multistring_to_rich().classmethod rich_to_multistring(elem_list)— converts a “rich” string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setid(value, unitid=None)— sets the unique identified for this unit. Only implemented if the format allows ids that areindependantfrom other unit properties like source or context. 💀unit_iter()— iterator that only returns this unit.
🦋 translate.storage.jsonl10n.JsonFile 🦋
class translate.storage.jsonl10n.JsonFile(inputfile=None, filter=None, **kwargs)A JSON file. That’s the whole vibe. ✨
Extensions = None— a list of fileextentionsassociated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human usable name of this store type.UnitClass— alias ofFlatJsonUnit.add_unit_to_index(unit)— adds a unit to source and locationidexes.addsourceunit(source)— adds and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— appends the given unit to the object’s list of units. This method should always be used rather than trying to modify the list manually.unit(TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, using either the chardet lib or by trying to decode the file.static fallback_detection(text)— simple detection based on BOM in case chardet is not available.findid(id)— finds the unit with matching id by checkingid_index.findunit(source)— finds the unit with the given source string.- Return type:
TranslationUnit or None
- Return type:
findunits(source)— finds the units with the given source string.- Return type:
TranslationUnit or None
- Return type:
getids()— returns a list of unit ids.getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store.isempty()— returnsTrueif the object doesn’t contain any translation units.makeindex()— indexes the items in this store. At least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(input)— parses the given file or file source string.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— converts the string representation back to an object.remove_unit_from_index(unit)— removes a unit from source andlocatonindexes.removeunit(unit)— removes the given unit to the object’s list of units. This method should always be used rather than trying to modify the list manually.unit(TranslationUnit) — the unit that will be added. (Yes, the page says “added” here; I’m not making the rules. 😭)
require_index()— makes sure source index exists.save()— saves to the file that data was originally read from, if available. 🌈savefile(storefile)— writes the string representation to the given file (or filename). 💖serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write to. ✨setprojectstyle(project_style)— sets the project type for this store. 🎀setsourcelanguage(sourcelanguage)— sets the source language for this store. 🌸settargetlanguage(targetlanguage)— sets the target language for this store. 🦋suggestions_in_format = False— indicates if format can store suggestions and alternative translation for a unit. 💅translate(source)— returns the translated string for a given source string.- Return type:
String or None
- Return type:
unit_iter()— iterator over all the units in this store. 🫶
✨ class translate.storage.jsonl10n.JsonNestedFile(inputfile=None, filter=None, **kwargs) ✨
A JSON file with nested keys. iconic honestly.
Extensions = None— a list of file extentions associated with this store type. 😭Mimetypes = None— a list of MIME types associated with this store type. 🔥Name = 'Base translation store'— the human usable name of this store type. 👑UnitClass— alias ofJsonNestedUnit. 🌈add_unit_to_index(unit)— add a unit to source and location idexes. ⭐addsourceunit(source)— add and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— append the given unit to the object’s list of units.- this method should always be used rather than trying to modify the list manually.
- Parameters
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from text, using either the chardet lib or by trying to decode the file. 🧚static fallback_detection(text)— simple detection based on BOM in case chardet is not available. 💜findid(id)— find unit with matching id by checkingid_index. 🔍findunit(source)— find the unit with the given source string.- Return type:
TranslationUnit or None
- Return type:
findunits(source)— find the units with the given source string.- Return type:
TranslationUnit or None
- Return type:
getids()— return a list of unit ids. 🎀getprojectstyle()— get the project type for this store. 🌸getsourcelanguage()— get the source language for this store. 🦋gettargetlanguage()— get the target language for this store. ✨getunits()— return a list of all units in this store. 💖isempty()— returnTrueif the object doesn’t contain any translation units. 💅makeindex()— indexes the items in this store. At least.sourceindexshould be useful. 🫶property merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(input)— parse the given file or file source string. 🌟classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object. 📂classmethod parsestring(storestring)— convert the string representation back to an object. 🔁remove_unit_from_index(unit)— remove a unit from source and locaton indexes. 🧹removeunit(unit)— remove the given unit to the object’s list of units.- this method should always be used rather than trying to modify the list manually.
- Parameters
unit(TranslationUnit) – the unit that will be added.
require_index()— make sure source index exists. ✅save()— save to the file that data was originally read from, if available. 💾savefile(storefile)— write the string representation to the given file (or filename). 📝serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write to. 🧬setprojectstyle(project_style)— set the project type for this store. 🎨setsourcelanguage(sourcelanguage)— set the source language for this store. 🌍settargetlanguage(targetlanguage)— set the target language for this store. 🌎suggestions_in_format = False— indicates if format can store suggestions and alternative translation for a unit. 💡translate(source)— return the translated string for a given source string.- Return type:
String or None
- Return type:
unit_iter()— iterator over all the units in this store. 🔄
🦋 class translate.storage.jsonl10n.JsonNestedUnit(source=None, item=None, notes=None, placeholders=None, **kwargs) 🦋
A nested JSON entry. little nesting moment fr fr.
aderror(errorname, errortext)— adds an error message to this unit.- Parameters
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
- Parameters
addlocation(location)— add one location to the list of locations.
NOTE: Shouldn’t be implemented if the format doesn’t support it. 🙈
addlocations(location)— add a location or a list of locations.
NOTE: Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: This method might be removed in future. 💀
addnote(text, origin=None, position='append')— adds a note (comment).- Parameters
text(string) – usually just a sentence or two.origin(string) – specifies who/where the comment comes from. Origin can be one of the following text strings:'translator''developer','programmer','source code'(synonyms)
- Parameters
classmethod buildfromunit(unit)— build a native unit from a foreign unit. Preserving as much information as possible. 🧩static getcontext()— get the message context. 🗣️static geterrors()— get all error messages.- Return type:
Dictionary
- Return type:
getid()— a unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store. Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. 🆔
- Return type:
getlocations()— a list of source code locations.- Return type:
List
- Return type:
NOTE: Shouldn’t be implemented if the format doesn’t support it. 🗺️
getnotes(origin=None)— returns all notes about this unit. It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()). 📝gettargetlen()— returns the length of the target string.- Return type:
Integer
- Return type:
NOTE: Plural forms might be combined. 🔢
getunits()— this unit in a list. 📦getvalue()— returns dictionary for serialization. 🧾static hasplural()— tells whether or not this specific unit has plural strings. 🧮infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state. 🌀isblank()— used to see if this unit has no source or target string.
NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. 🫥
static isfuzzy()— indicates whether this unit is fuzzy. 🌫️static isheader()— indicates whether this unit is a header. 🏷️static isobsolete()— indicate whether a unit is obsolete. 🗑️static isreview()— indicates whether this unit needs review. 👀istranslatable()— indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. ✅istranslated()— indicates whether this unit is translated. This should be used rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does). 🌐makeobsolete()— make a unit obsolete. ⚰️markfuzzy(value=True)— marks the unit as fuzzy or not. 🌫️markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.
Parameters
needsreview– lowkey defaults toTrue. 💅explanation– adds an optional lil explanation as a note, bestie.
merge(otherunit, overwrite=False, comments=True, authoritative=False) — does the basic format-agnostic merging thing. no thoughts, just vibes ✨
multistring_to_rich(mulstring) — flips a multistring into a list of “rich” string trees. it's giving transformation:
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None) — yeets all the translator’s notes. gone, goodbye, periodt 🫶
rich_parsers = [] — a whole list of functions you use to parse a string into a rich string tree. the toolbox, basically 🧰
property rich_source — the rich source, obviously 💖
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target — and the rich target, matching energy 🌸
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list) — flips a “rich” string tree back into a multistring. the reverse slay:
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context) — sets the message context. cool cool cool.
setid(value, unitid=None) — sets the unique identifier for this unit. and like, it's only implemented if the format allows ids to be independent from other unit properties like source or context. iykyk 🔥
unit_iter() — an iterator that only returns this unit. just the one. main character energy 👑
🦋 class translate.storage.jsonl10n.WebExtensionJsonFile(inputfile=None, filter=None, **kwargs) 🦋
the WebExtension JSON file. that's the whole vibe.
see these URLs for the docs, bestie:
https://developer.chrome.com/extensions/i18n ‐ https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Internationalization
Extensions = None — a list of file extensions associated with this store type.
Mimetypes = None — a list of MIME types associated with this store type.
Name = 'Base translation store' — the human-usable name of this store type. iconic label tbh 🏷️
UnitClass — alias of WebExtensionJsonUnit.
add_unit_to_index(unit) — adds a unit to the source and location indexes. bookkeeping era 📚
addsourceunit(source) — adds and returns a new unit with the given source string.
Return type: TranslationUnit
addunit(unit) — appends the given unit to the object’s list of units.
this method should always be used rather than trying to modify the list manually. fr fr, don't touch the list yourself 🙈
Parameters:unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str] — tries to detect a file encoding from text, using either the chardet lib or by just trying to decode the file. detective mode 🔍
static fallback_detection(text) — simple detection based on BOM, for when chardet isn't available. the backup bestie 💜
findid(id) — finds the unit with a matching id by checking id_index. gotcha.
findunit(source) — finds the unit with the given source string.
Return type:TranslationUnitorNone
findunits(source) — finds the units with the given source string. yes, plural, we're messy 💅
Return type:TranslationUnitorNone
getids() — returns a list of unit ids.
getprojectstyle() — gets the project type for this store.
getsourcelanguage() — gets the source language for this store.
gettargetlanguage() — gets the target language for this store.
getunits() — returns a list of all units in this store. everything, all at once 🌈
isempty() — returns True if the object doesn’t contain any translation units. empty vessel behavior.
makeindex() — indexes the items in this store. at least .sourceindex should be useful. minimum effort, maximum usefulness ⭐
property merge_on — the matching criterion to use when merging on.
Returns: the default matching criterion for all the subclasses. Return type: string
parse(input) — parses the given file or file source string.
classmethod parsefile(storefile) — reads the given file (or opens the given filename) and parses it back to an object. no cap.
classmethod parsestring(storestring) — converts the string representation back to an object. round trip, slay ✈️
remove_unit_from_index(unit) — removes a unit from the source and location indexes. reverse bookkeeping 📚
removeunit(unit) — removes the given unit from the object’s list of units.
this method should always be used rather than trying to modify the list manually. again. don't. touch. the. list. 🙈
Parameters:unit(TranslationUnit) – the unit that will be added.
require_index() — makes sure the source index exists. manifesting it ✨
save() — saves to the file that data was originally read from, if available.
savefile(storefile) — writes the string representation to the given file (or filename).
serialize(out) — converts to a bytes representation that can be parsed back using parsestring(). out should be an open file-like object to write to. bytes btw 💀
setprojectstyle(project_style) — sets the project type for this store.
setsourcelanguage(sourcelanguage) — sets the source language for this store.
settargetlanguage(targetlanguage) — sets the target language for this store.
suggestions_in_format = False — indicates if the format can store suggestions and alternative translations for a unit. this one? no.
translate(source) — returns the translated string for a given source string.
Return type:StringorNone
unit_iter() — iterator over all the units in this store. every single one 🎀
🌸 class translate.storage.jsonl10n.WebExtensionJsonUnit(source=None, item=None, notes=None, placeholders=None, **kwargs) 🌸
your actual unit, bestie. let's go:
adderror(errorname, errortext) — adds an error message to this unit.
Parameters: -errorname(string) – a single word to id the error. -errortext(string) – the text describing the error.
addlocation(location) — adds one location to the list of locations.
NOTE: shouldn’t be implemented if the format doesn’t support it. we're not doing fake functionality around here.
addlocations(location) — adds a location or a list of locations.
NOTE: most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: this method might be removed in future. ⚠️ so like, don't get too attached.
addnote(text, origin=None, position='append') — adds a note (comment).
Parameters: -text(string) – usually just a sentence or two. -origin(string) – specifies who/where the comment comes from. origin can be one of these text strings: -'translator'-'developer','programmer','source code'(synonyms)
classmethod buildfromunit(unit) — builds a native unit from a foreign unit. and it preserves as much information as possible, king behavior 👑
static getcontext() — gets the message context.
static geterrors() — gets all error messages.
Return type: Dictionary
getid() — a unique identifier for this unit.
Return type: string Returns: an identifier for this unit that is unique in the store derived classes should override this in a way that guarantees a unique identifier for each unit in the store. no duplicates allowed, periodt.
getlocations() — a list of source code locations.
Return type: List NOTE: shouldn’t be implemented if the format doesn’t support it. same rule as before 🫶
getnotes(origin=None) — returns all the notes about this unit.
it will probably be freeform text or something reasonable that can be synthesised by the format. it should not include location comments (see getlocations()). keep 'em separate bestie.
gettargetlen() — returns the length of the target string.
Return type: Integer NOTE: plural forms might be combined. just so you know 💅
getunits() — this unit, in a list. a list of one. iconic.
getvalue() — returns dictionary for serialization. 🗂️
static hasplural() — tells you whether or not this specific unit has plural strings.
infer_state() — an empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state. placeholder energy.
isblank() — used to see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather than get rid of this. future plans, ok? 🔮
static isfuzzy() — indicates whether this unit is fuzzy. fuzzy wuzzy 🌸
static isheader()— tells you whether this unit is a header. that's it, that's the whole vibe 🎀static isobsolete()— indicates whether a unit is obsolete.static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can actually be translated.
this is the one to use when you need to separate the real units from headers, obsolete units, binary units, and other blank units. no cap, that's literally its job 🫶
istranslated()— indicates whether this unit is translated.
bestie, please use this instead of trying to deduce it from .target — that way other classes can implement extra functionality (like XLIFF does) 💅
makeobsolete()— makes a unit obsolete. iconic retirement behavior ✨markfuzzy(value=True)— marks the unit as fuzzy, or not fuzzy.value=Trueby default, obviously.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.
Parameters:
needsreview— defaults toTrue.explanation— adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic, format-agnostic merging. lowkey the workhorse 🔥multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees. it's giving transformation ✨
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator's notes. gone, deleted, not me crying about it 😭rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source—
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target—
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a "rich" string tree into a multistring. the reverse slay 💖
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setid(value, unitid=None)— sets the unique identifier for this unit.
only implemented if the format allows ids that are independent from other unit properties like source or context. so like, format-dependent, iykyk 🦋
unit_iter()— an iterator that only returns this unit. just the one. main character energy 👑
🌈 lisa
parent class for LISA standards (TMX, TBX, XLIFF). the whole girlie family 💅
🧚 class translate.storage.lisa.LISAfile(inputfile=None, sourcelanguage='en', targetlanguage=None, **kwargs)
a class representing a file store for one of the LISA file formats. so organized of her 🌸
Extensions = None— a list of file extensions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human-usable name of this store type.UnitClass— alias ofLISAunit.suggestions_in_format = False— indicates whether the format can store suggestions and alternative translations for a unit.
add_unit_to_index(unit)— adds a unit to the source and location indexes.addheader()— a method to be overridden to initialise headers, etc. (so like, do your own thing here ⭐)addsourceunit(source)— adds and returns a new unit with the given string as its first entry.addunit(unit, new=True)— appends the given unit to the object's list of units.
this method should always be used rather than trying to modify the list manually. fr fr, don't touch the list 🙈
Parameters:
unit(TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, using either the chardet lib or by trying to decode the file. detective mode 🔍static fallback_detection(text)— simple detection based on BOM in case chardet isn't available. the backup bestie 🫶findid(id)— finds the unit with the matching id by checkingid_index.findunit(source)— finds the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)— finds the units with the given source string. plural, because sometimes there's more than one 😌
Return type: TranslationUnit or None
getids()— returns a list of unit ids.getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store.initbody()— initialisesself.bodyso it never needs to be retrieved from the XML again. efficiency queen ✨isempty()— returnsTrueif the object doesn't contain any translation units.makeindex()— indexes the items in this store. at least.sourceindexshould be useful 💅property merge_on— the matching criterion to use when merging on.
Returns: the default matching criterion for all the subclasses.
Return type: string
namespaced(name)— returnsnamein Clark notation.
for example, namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourcethis is needed throughout lxml. iconic little helper 🌈
parse(xml)— populates this object from the given xml string.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back to an object.classmethod parsestring(storestring)— converts the string representation back to an object.remove_unit_from_index(unit)— removes a unit from the source and location indexes. bye bestie 👋removeunit(unit)— removes the given unit from the object's list of units.
this method should always be used rather than trying to modify the list manually. same rule as before, no DIY list surgery 🙈
Parameters:
unit(TranslationUnit) — the unit that will be added.
require_index()— makes sure the source index exists. accountability check ✅save()— saves to the file that data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out=None)— converts to a string containing the file's XML.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.translate(source)— returns the translated string for a given source string.
Return type: String or None
unit_iter()— iterator over all the units in this store. all of them. every single one 🎀
💜 class translate.storage.lisa.LISAunit(source, empty=False, **kwargs)
a single unit in the file. provisional work is done to make several languages possible. she's doing her best ✨
adderror(errorname, errortext)— adds an error message to this unit.
Parameters:
errorname(string) — a single word to id the error.errortext(string) — the text describing the error.
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. know your limits, bestie 💅
addlocations(location)— adds a location or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. so like, don't get too attached 😭
addnote(text, origin=None, position='append')— adds a note (comment).
Parameters:
text(string) — usually just a sentence or two.origin(string) — specifies who/where the comment comes from. origin can be one of the following text strings: -'translator'-'developer','programmer','source code'(synonyms)
(position='append' is the default, in case you were wondering 🦋)
classmethod buildfromunit(unit)— builds a native unit from a foreign unit.
preserving as much information as possible. respectful queen 👑
static createlanguageNode(lang, text, purpose=None)— returns an xmlElementset up with the given parameters to represent a single language entry. has to be overridden.getNodeText(languageNode, xml_space='preserve')— retrieves the term from the givenlanguageNode.static getcontext()— gets the message context.static geterrors()— gets all error messages.
Return type: Dictionary
getid()— a unique identifier for this unit.
Return type: string
Okay so this next bit returns an identifier for this unit that is unique in the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.
getlanguageNode(lang=None, index=None)— Retrieves alanguageNodeeither by language or by index.getlanguageNodes()— Returns a list of all nodes that contain per-language information.static getlocations()— A list of source code locations.- Return type:
List - NOTE: Shouldn’t be implemented if the format doesn’t support it.
- Return type:
getnotes(origin=None)— Returns all notes about this unit.
It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see getlocations()).
gettarget(lang=None)— Retrieves the “target” text (second entry), or the entry in the specified language, if it exists.gettargetlen()— Returns the length of the target string.- Return type:
Integer - NOTE: Plural forms might be combined.
- Return type:
getunits()— This unit in a list.static hasplural()— Tells whether or not this specific unit has plural strings.infer_state()— Empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— Used to see if this unit has no source or target string.- NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()— Indicates whether this unit is fuzzy.static isheader()— Indicates whether this unit is a header.static isobsolete()— Indicate whether a unit is obsolete.static isreview()— Indicates whether this unit needs review.istranslatable()— Indicates whether this unit can be translated.
This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.
istranslated()— Indicates whether this unit is translated.
This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does).
makeobsolete()— Make a unit obsolete.markfuzzy(value=True)— Marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— Marks the unit to indicate whether it needs review.- Parameters
needsreview— Defaults toTrue.explanation— Adds an optional explanation as a note.
- Parameters
merge(otherunit, overwrite=False, comments=True, authoritative=False)— Do basic format agnostic merging.multistring_to_rich(mulstring)— Convert a multistring to a list of “rich” string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]namespaced(name)— Returnsnamein Clark notation.
For example namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourceThis is needed throughout lxml.
removenotes(origin=None)— Remove all the translator’s notes.rich_parsers = []— A list of functions to use for parsing a string into a rich string tree.property rich_source- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
property rich_target- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
classmethod rich_to_multistring(elem_list)— Convert a “rich” string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— Set the message context.setid(value)— Sets the unique identified for this unit.
only implemented if format allows ids independant from other unit properties like source or context
settarget(target, lang='xx', append=False)— Sets the “target” string (second language), or alternatively appends to the list.unit_iter()— Iterator that only returns this unit.
🌸 markdown
Module for parsing Markdown files for translation.
The principles for extraction of translation units are as follows:
- Extract all content relevant for translation, at the cost of also including some formatting.
- One translation unit per paragraph.
- Keep formatting out of the translation units as much as possible. Exceptions include phrase emphasis and inline code. Use placeholders
{1},{2}, …, as needed. - Avoid HTML entities in the translation units. Use Unicode equivalents if possible.
White space within translation units is normalized, because the PO format does not preserve white space, and the translated Markdown content may have to be reflowed anyway.
class translate.storage.markdown.MarkdownFile(inputfile=None, callback=None, max_line_length=None)Extensions = None— A list of file extentions associated with this store type.Mimetypes = None— A list of MIME types associated with this store type.Name = 'Base translation store'— The human usable name of this store type.UnitClass— alias ofMarkdownUnit.add_unit_to_index(unit)— Add a unit to source and location idexes.addsourceunit(source)— Add and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— Append the given unit to the object’s list of units.
This method should always be used rather than trying to modify the list manually.
- Parameters:
unit(TranslationUnit) – The unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— Try to detect a file encoding fromtext, using either thechardetlib or by trying to decode the file.static fallback_detection(text)— Simple detection based on BOM in casechardetis not available.findid(id)— Find unit with matchingidby checkingid_index.findunit(source)— Find the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— Find the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
getids()— Return a list of unit ids.getprojectstyle()— Get the project type for this store.getsourcelanguage()— Get the source language for this store.gettargetlanguage()— Get the target language for this store.getunits()— Return a list of all units in this store.isempty()— ReturnTrueif the object doesn’t contain any translation units.makeindex()— Indexes the items in this store. At least.sourceindexshould be useful.property merge_on— The matching criterion to use when merging on.- Returns: The default matching criterion for all the subclasses.
- Return type:
string
parse(data)— Process the given source string (binary).classmethod parsefile(storefile)— Reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— Convert the string representation back to an object.remove_unit_from_index(unit)— Remove a unit from source and locaton indexes.removeunit(unit)— Remove the given unit to the object’s list of units.
This method should always be used rather than trying to modify the list manually.
- Parameters:
unit(TranslationUnit) – The unit that will be added.
require_index()— Make sure source index exists.save()— Save to the file that data was originally read from, if available.savefile(storefile)— Write the string representation to the given file (or filename).serialize(out)— Converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write to.setprojectstyle(project_style)— Set the project type for this store.setsourcelanguage(sourcelanguage)— Set the source language for this store.settargetlanguage(targetlanguage)— Set the target language for this store.suggestions_in_format = False— tells you whether this format can hold suggestions and alternative translations for a unit. Simple boolean, no cap. ✨
translate(source)
hands you back the translated string for whatever source string you pass in. That's the whole vibe.
- Return type: String or None
unit_iter()
an iterator that walks over, like, every unit in the store. all of them. 💅
🌸 class translate.storage.markdown.MarkdownUnit(source=None)
a single unit of translatable / localisable markdown content. yes bestie, markdown is content too. 🎀
adderror(errorname, errortext)
slaps an error message onto this unit.
Parameters:
errorname(string) — a single word that ids the error.errortext(string) — the text describing the error.
addlocation(location)
adds one location to the list of locations. easy.
NOTE: shouldn't be implemented if the format doesn't support it. fr, don't fake it.
addlocations(location)
adds a location, or a whole list of locations. multi-tasking icon.
NOTE: most classes shouldn't need to implement this — they should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. ⚠️ so, like, don't get too attached.
addnote(text, origin=None, position='append')
adds a note (a comment). 🗒️
Parameters:
text(string) — usually just a sentence or two.origin(string) — specifies who/where the comment comes from. origin can be one of these text strings:'translator','developer','programmer','source code'(those last ones are synonyms).
classmethod buildfromunit(unit)
builds a native unit out of a foreign unit, preserving as much information as possible. glow-up conversion. ✨
static getcontext()
gets the message context.
static geterrors()
gets all the error messages.
- Return type: Dictionary
getid()
a unique identifier for this unit. one of a kind, main character energy 👑
- Return type: string
- Returns: an identifier for this unit that is unique in the store
Derived classes should override this so it guarantees a unique identifier for each unit in the store. no duplicates allowed, periodt.
getlocations()
a list of source code locations.
- Return type: List
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)
returns all the notes about this unit. it'll probably be freeform text or something reasonable that the format can synthesise. it should not include location comments (see getlocations()). iykyk.
gettargetlen()
returns the length of the target string.
- Return type: Integer
NOTE: plural forms might be combined. heads up.
getunits()
this unit, but in a list. one-item list, but make it cute. 🍓
static hasplural()
tells you whether or not this specific unit has plural strings.
infer_state()
an empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state. placeholder behavior, lowkey.
isblank()
used to check if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()
indicates whether this unit is fuzzy.
static isheader()
indicates whether this unit is a header.
static isobsolete()
indicates whether a unit is obsolete. 👋
static isreview()
indicates whether this unit needs review.
istranslatable()
indicates whether this unit can be translated.
This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.
istranslated()
indicates whether this unit is translated.
This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (the way XLIFF does). don't be out here guessing. 🙈
makeobsolete()
makes a unit obsolete.
markfuzzy(value=True)
marks the unit as fuzzy or not fuzzy. toggle vibe. 🦋
markreviewneeded(needsreview=True, explanation=None)
marks the unit to show whether it needs review.
Parameters:
needsreview— defaults toTrue.explanation— adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)
does basic, format-agnostic merging. no format drama.
multistring_to_rich(mulstring)
converts a multistring into a list of "rich" string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)
removes all the translator's notes. yeet. 💀
rich_parsers = []
a list of functions used for parsing a string into a rich string tree.
property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)
converts a "rich" string tree into a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)
sets the message context.
setid(value)
sets the unique identifier for this unit.
only implemented if the format allows ids independent from other unit properties like source or context.
unit_iter()
an iterator that only returns this unit. just the one. 🫶
🌈 class translate.storage.markdown.TranslatingMarkdownRenderer(*args: Any, **kwargs: Any)
expand_placeholders(fragments: Iterable[mistletoe.markdown_renderer.Fragment]) -> Iterable[mistletoe.markdown_renderer.Fragment]
expands placeholder fragments, recursively. inception vibes. 🌀
classmethod insert_placeholder_markers(fragments: Iterable[mistletoe.markdown_renderer.Fragment]) -> Iterable[mistletoe.markdown_renderer.Fragment]
sets the text of the (top-level) placeholder fragments to "{n}". returns an ordered list of placeholders.
classmethod merge_adjacent_placeholders(fragments: Iterable[mistletoe.markdown_renderer.Fragment]) -> Iterable[mistletoe.markdown_renderer.Fragment]
replaces sequences of placeholders and whitespace with bigger placeholders. glow-up merge. ✨
remove_placeholder_markers(markdown: str, placeholders: Iterable[mistletoe.markdown_renderer.Fragment]) -> str
replaces placeholder markers in the given markdown with the actual placeholder content.
span_to_lines(tokens: Iterable[mistletoe.span_token.SpanToken], max_line_length: int) -> Iterable[str]
renders a sequence of span tokens to markdown, with translation.
classmethod trim_flanking_placeholders(fragments: Iterable[mistletoe.markdown_renderer.Fragment]) -> tuple[Iterable[mistletoe.markdown_renderer.Fragment], Iterable[mistletoe.markdown_renderer.Fragment], Iterable[mistletoe.markdown_renderer.Fragment]]
splits leading and trailing placeholders and whitespace, plus the main content, into separate lists. placeholders marked as important are kept with the main content. bougie but make it functional. 💅
🧚 mo
module for parsing Gettext .mo files for translation.
the coding of .mo files was produced from Gettext documentation, Python's msgfmt.py, and by observing and testing existing .mo files in the wild. fr fr, field research. 🔥
the hash algorithm is implemented for MO files — this should result in faster access of the MO file. the hash is optional for Gettext and is not needed for reading or writing MO files; in this implementation it is always on and does produce sometimes different results to Gettext in very small files. no cap, tiny files can be quirky. 😭
class translate.storage.mo.mofile(inputfile=None, **kwargs)
a class representing a .mo file. iconic container behavior. 👑
Extensions = ['mo', 'gmo']— a list of file extensions associated with this store typeMimetypes = ['application/x-gettext-catalog', 'application/x-mo']— a list of MIME types associated with this store typeName = 'Gettext MO file'— the human-usable name of this store typeUnitClass— alias ofmounit
add_unit_to_index(unit)
adds a unit to the source index and the location index. indexing slay. 🔍
addsourceunit(source)
adds and returns a new unit with the given source string.
- Return type: TranslationUnit
addunit(unit)
appends the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually. pls don't touch the list directly, bestie. 🙏
Parameters:
unit(TranslationUnit) — the unit that will be added.detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— bestie this one just tries to sniff out a file's encoding from the text, either by leaning on thechardetlib or by literally attempting to decode the thing. vibes-based forensics honestly 💅static fallback_detection(text)— the humble backup plan: a simple detection based on the BOM, for whenchardetisn't available. not glamorous but it shows up 🌸findid(id)— hunts down the unit with a matchingidby checkingid_index. like a little detective 🕵️♀️findunit(source)— finds the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— finds all the units with the given source string, plural girlies rise ✨- Return type:
TranslationUnitorNone
- Return type:
getheaderplural()— returns thenpluralandpluralvalues straight out of the header. the plural lore 📖getids()— returns a list of unit ids. all of them. every single one.getprojectstyle()— returns the project based on the info chillin' in the header.- The project is determined in this exact sequence, no skipping:
- Use the
'X-Project-Style'entry in the header. - Use
'Report-Msgid-Bug-To'entry - Use the
'X-Accelerator'entry - Use the Project ID
- Analyse the file itself (not yet implemented) 🙈
- Use the
- The project is determined in this exact sequence, no skipping:
getsourcelanguage()— gets the source language for this store. where it's from, basically 🌍gettargetlanguage()— returns the target language based on the info in the header.- The target language is determined in this sequence:
- Use the
'Language'entry in the header. - Poedit's custom headers.
- Analysing the
'Language-Team'entry.
- Use the
- The target language is determined in this sequence:
getunits()— returns a list of all the units in this store. the whole squad 🫶header()— returns the header element, orNone. Only the first element is allowed to be a header, okay? important rule. Note that this could still return an empty header element, if present. so like… it might be there, just hollow 💀init_headers(charset='UTF-8', encoding='8bit', **kwargs)— sets default values for po headers. the starter pack 🎀isempty()— returnsTrueif the object doesn't contain any translation units. empty like my schedule on a friday night 😭makeheader(**kwargs)— creates a header for the given filename.- Check
.makeheaderdict()for the deets on parameters, no cap.
- Check
makeheaderdict(charset='CHARSET', encoding='ENCODING', project_id_version=None, pot_creation_date=None, po_revision_date=None, last_translator=None, language_team=None, mime_version=None, plural_forms=None, report_msgid_bugs_to=None, **kwargs)— creates a header dictionary with useful defaults. iconic behaviour honestly 👑pot_creation_datecan beNone(current date) or a value (datetime or string) — so many options bestiepo_revision_datecan beNone(form),False(=pot_creation_date),True(=now), or a value (datetime or string)- Returns: Dictionary with the header items
- Return type:
dictof strings
makeindex()— indexes the items in this store. at least.sourceindexshould be useful. do it for the index girlies 🔥property merge_on— the matching criterion to use when merging on.- Returns: The default matching criterion for all the subclasses.
- Return type: string
mergeheaders(otherstore)— merges another header with this header.- This header is assumed to be the template. no pressure or anything 💅
parse(input)— parses the given file or file source string. feed it and it eats 🍓classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object. round trip slay ✨parseheader()— parses the PO header and returns the interpreted values as a dictionary.classmethod parsestring(storestring)— converts the string representation back to an object. the reverse uno card 🌈remove_unit_from_index(unit)— removes a unit from the source and location indexes. bye bestie 👋removeunit(unit)— removes the given unit to the object's list of units. (yes that's what it says, don't come for me 😭)- This method should always be used rather than trying to modify the list manually. seriously, don't touch it yourself.
- Parameters:
unit(TranslationUnit) – The unit that will be added.
require_index()— makes sure the source index exists. enforcing boundaries, love that for it 🫶save()— save to the file that data was originally read from, if available. only if available though, no promises 🌸savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— outputs a string representation of the MO data file. format the whole thing out 💫setprojectstyle(project_style)— sets the project in the header.- Parameters:
project_style(str) – the new project
- Parameters:
setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(lang)— sets the target language in the header.- This removes any custom Poedit headers if they exist. gone. deleted. 🫡
- Parameters:
lang(str) – the new target language code
suggestions_in_format = False— indicates if the format can store suggestions and alternative translation for a unit. currentlyFalse, so like… it cannot, bestie 💀translate(source)— returns the translated string for a given source string.- Return type: String or
None
- Return type: String or
unit_iter()— iterator over all the units in this store. loop-de-loop 🔁updatecontributor(name, email=None)— adds contribution comments if necessary. giving credit where it's due 💖updateheader(add=False, **kwargs)— updates the fields in the PO style header.- This will create a header if
add == True. soTrue= new header just dropped ✨
- This will create a header if
updateheaderplural(nplurals, plural)— updates thePlural-FormPO header. plural business, as usual 🌸
🦋 class translate.storage.mo.mounit(source=None, **kwargs)
A class representing a .mo translation message. little .mo baby, so cute 🎀
adderror(errorname, errortext)— adds an error message to this unit.- Parameters:
errorname(string) – A single word toidthe error.errortext(string) – The text describing the error.
- Parameters:
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. respect the format's boundaries okay 😤
addlocations(location)— adds a location or a list of locations.
NOTE: Most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: This method might be removed in future. heads up bestie, don't get too attached 🙈
addnote(text, origin=None, position='append')— adds a note (comment).- Parameters:
text(string) – Usually just a sentence or two.origin(string) – Specifies who/where the comment comes from. Origin can be one of the following text strings: -'translator'-'developer','programmer','source code'(synonyms)
- Parameters:
classmethod buildfromunit(unit)— builds a native unit from a foreign unit.- Preserving as much information as possible. we don't lose data on my watch 💅
getcontext()— gets the message context. the ~context~ ✨static geterrors()— gets all error messages.- Return type: Dictionary
getid()— a unique identifier for this unit.- Return type: string
- Returns: an identifier for this unit that is unique in the store
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. no duplicates allowed, periodt 👑
static getlocations()— a list of source code locations.- Return type: List
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all notes about this unit.- It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see
getlocations()). keep the notes pure pls 🌸
- It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see
gettargetlen()— returns the length of the target string.- Return type: Integer
NOTE: Plural forms might be combined. so the number might not be exactly what you expect 🫠
getunits()— this unit in a list. just the one. a list of one. iconic 💅static hasplural()— tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state. placeholder energy but make it intentional 🎀isblank()— used to see if this unit has no source or target string.
NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. it's giving legacy code 😭
static isfuzzy()— indicates whether this unit is fuzzy. fuzzy vibes check 🦋isheader()— is this a header entry? simple question, simple answer.static isobsolete()— indicate whether a unit is obsolete. old news 💀static isreview()— indicates whether this unit needs review. it's giving "please look at me" 🙈istranslatable()— is this message translateable? (spelling theirs, not mine, love you 💖)istranslated()— indicates whether this unit is translated. the final vibe check ✨
…and this next bit is literally the tail end of a docstring that got detached from its method, so here it is in all its glory:
This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does).
makeobsolete()— flips the unit to obsolete. bye bestie 👋markfuzzy(value=True)— tags the unit as fuzzy, or un-tags it, depending on value. 💅markreviewneeded(needsreview=True, explanation=None)— marks the unit so everyone knows it needs review.- needsreview — defaults to
True. - explanation — optionally drops an explanation in as a note. 🎀
- needsreview — defaults to
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does a basic, format-agnostic merge. no cap, it just merges. 🫶multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees. watch:
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— nukes all the translator's notes. gone, periodt. 💀rich_parsers = []— a list of functions used to parse a string into a rich string tree.rich_source(property) — the rich-string version of the source. ✨ SEE ALSO:rich_to_multistring(),multistring_to_rich().rich_target(property) — the rich-string version of the target. ✨ SEE ALSO:rich_to_multistring(),multistring_to_rich().rich_to_multistring(elem_list)(classmethod) — takes a "rich" string tree and turns it back into a multistring, iconic behaviour:
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setid(value)— sets the unique identified for this unit. (yes, "identified", the docs said what they said, we're leaving it 💅) It's only implemented if the format allows ids independant from other unit properties like source or context.unit_iter()— an iterator that only returns this unit. main character energy, honestly. 👑
🌸 translate.storage.mo.mounpack(filename='messages.mo') 🌸
Helper to unpack Gettext MO files into a Python string. that's it, that's the whole vibe 🫶
🦋 mozilla_lang 🦋
A class to manage Mozilla .lang files.
See https://github.com/mozilla-l10n/langchecker/wiki/.lang-files-format for specifications on the format. 📎
💖 class translate.storage.mozilla_lang.LangStore(inputfile=None, mark_active=False, **kwargs) 💖
We extend TxtFile, since that has a lot of useful stuff for encoding. slay, inheritance. 🧚
Extensions = ['lang']— a list of file extentions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Mozilla .lang'— the human usable name of this store type.UnitClass— alias ofLangUnit.add_unit_to_index(unit)— adds a unit to the source and location idexes.addsourceunit(source)— adds and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— appends the given unit to the object's list of units.- This method should always be used rather than trying to modify the list manually. fr fr, don't touch the list directly.
- unit (
TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, using either thechardetlib or by trying to decode the file. detective era 🔍fallback_detection(text)(static) — simple detection based on BOM in casechardetis not available.findid(id)— finds the unit with matching id by checkingid_index.findunit(source)— finds the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— finds the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
getids()— returns a list of unit ids.getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store.isempty()— returnsTrueif the object doesn't contain any translation units.makeindex()— indexes the items in this store. At least.sourceindexshould be useful.merge_on(property) — the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(lines)— reads in text lines and createstxtunits from the blocks of text.parsefile(storefile)(classmethod) — reads the given file (or opens the given filename) and parses back to an object.parsestring(storestring)(classmethod) — converts the string representation back to an object.remove_unit_from_index(unit)— removes a unit from the source and locaton indexes.removeunit(unit)— removes the given unit from the object's list of units.- This method should always be used rather than trying to modify the list manually.
- unit (
TranslationUnit) — the unit that will be added. (yes, it says "added" on a remove method. we are choosing peace about it 🙈)
require_index()— makes sure the source index exists.save()— saves to the file that data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back usingparsestring(). out should be an open file-like objects to write to.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates if the format can store suggestions and alternative translation for a unit.translate(source)— returns the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— iterator over all the units in this store. 💫
🎀 class translate.storage.mozilla_lang.LangUnit(source=None) 🎀
This is just a normal unit with a weird string output. quirky but valid, we love her 💜
adderror(errorname, errortext)— adds an error message to this unit.- errorname (
string) — a single word to id the error. - errortext (
string) — the text describing the error.
- errorname (
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it.
addlocations(location)— adds a location or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: this method might be removed in future. heads up bestie ⚠️
addnote(text, origin=None, position='append')— adds a note (comment).- text (
string) — usually just a sentence or two. - origin (
string) — specifies who/where the comment comes from. Origin can be one of the following text strings: -'translator'-'developer','programmer','source code'(synonyms).
- text (
buildfromunit(unit)(classmethod) — builds a native unit from a foreign unit. Preserving as much information as possible. 🫶getcontext()(static) — gets the message context.geterrors()(static) — gets all error messages.- Return type:
Dictionary
- Return type:
getid()— a unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store.
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.
- Return type:
getlocations()— a list of source code locations.- Return type:
List
- Return type:
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all notes about this unit.- It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see
getlocations()).
- It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see
gettargetlen()— returns the length of the target string.- Return type:
Integer
- Return type:
NOTE: plural forms might be combined.
getunits()— this unit in a list. that's it. one unit. iconique ✨static hasplural()— tells whether or not this specific unit has plural strings. yes queen, plural check 💅infer_state()— an empty method, deliberately. it's meant to be overridden in sub-classes so they can infer the currentstate(_n)of the unit from its current state.isblank()— used to see if this unit has no source or target string.
⚠️ NOTE (from the page, ngl): this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()— indicates whether this unit is fuzzy 🫠static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete 💀static isreview()— indicates whether this unit needs review 👀istranslatable()— indicates whether this unit can be translated.
This is the one you use to tell real units-for-translation apart from header, obsolete, binary, or other blank units. no cap.
istranslated()— indicates whether this unit is translated.
Use this, not some DIY deduction from .target, so that other classes can implement more functionality (as XLIFF does). trust the method, bestie 🫶
makeobsolete()— makes a unit obsolete.markfuzzy(value=True)— marks the unit as fuzzy, or not fuzzy.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.needsreview— defaults toTrue.explanation— adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does the basic, format-agnostic merging thing.multistring_to_rich(mulstring)— converts a multistring to a list of "rich" string trees ✨
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator's notes. bye notes 👋rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a "rich" string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setid(value)— sets the unique identified for this unit. (their wording, not mine 🫣)
only implemented if the format allows ids independant from other unit properties like source or context.
unit_iter()— an iterator that only returns this unit. it's giving main character energy 🌟
🌈 odf_io
🌈 odf_shared
🐚 omegat
Manage the OmegaT glossary format.
OmegaT glossary format is used by the OmegaT computer aided translation tool.
It's a bilingual base class derived format, with OmegaTFile and OmegaTUnit providing file and unit level access.
💖 Format Implementation
The OmegaT glossary format is a simple Tab Separated Value (TSV) file with the columns: source, target, comment.
The dialect of the TSV files is specified by OmegaTDialect.
🔤 Encoding
The files are either UTF-8 or encoded using the system default. UTF-8 encoded files use the .utf8 extension while system encoded files use the .tab extension.
📋 translate.storage.omegat.OMEGAT_FIELDNAMES = ['source', 'target', 'comment']
Field names for an OmegaT glossary unit.
🧁 class translate.storage.omegat.OmegaTDialect
Describes the properties of an OmegaT generated TAB-delimited glossary file.
🦋 class translate.storage.omegat.OmegaTFile(inputfile=None, **kwargs)
An OmegaT glossary file.
Extensions = ['utf8']— a list of file extensions associated with this store type.Mimetypes = ['application/x-omegat-glossary']— a list of MIME types associated with this store type.Name = 'OmegaT Glossary'— the human usable name of this store type 👑UnitClass— alias ofOmegaTUnit.add_unit_to_index(unit)— adds a unit to the source and location idexes. (again: their spelling, not a typo of mine 🙈)addsourceunit(source)— adds and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— appends the given unit to the object's list of units.
Always use this method rather than trying to modify the list manually, fr fr.
- Parameters:
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, using either the chardet lib or by trying to decode the file.static fallback_detection(text)— simple detection based on BOM, in case chardet is not available.findid(id)— finds the unit with a matching id by checkingid_index.findunit(source)— finds the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— finds the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
getids()— returns a list of unit ids.getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store.isempty()— returnsTrueif the object doesn't contain any translation units.makeindex()— indexes the items in this store. At least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(input)— Parsese the given file or file source string. (yep, "Parsese" 💀)classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— converts the string representation back to an object.remove_unit_from_index(unit)— removes a unit from the source and locaton indexes. 🧹removeunit(unit)— removes the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually.
- Parameters:
unit(TranslationUnit) – the unit that will be added.
require_index()— makes sure the source index exists.save()— saves to the file that data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write to.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates if the format can store suggestions and alternative translation for a unit.translate(source)— returns the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— an iterator over all the units in this store. every single one 🎀
🍓 class translate.storage.omegat.OmegaTFileTab(inputfile=None, **kwargs)
An OmegaT glossary file in the default system encoding.
Extensions = ['tab']— a list of file extensions associated with this store type.Mimetypes = ['application/x-omegat-glossary']— a list of MIME types associated with this store type.Name = 'OmegaT Glossary'— the human usable name of this store type.UnitClass— alias ofOmegaTUnit.add_unit_to_index(unit)— adds a unit to the source and location idexes.addsourceunit(source)— adds and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— appends the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually.
- Parameters:
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— bestie this one just tries to sniff out a file's encoding from the text you hand it, either by vibing with thechardetlib or by literally attempting to decode the thing and seeing what sticks. Returns a tuple. ✨fallback_detection(text)(static) — the emergency backup plan 🎀 — a super simple detection that's all about the BOM, there for whenchardetisn't installed and the main plan ghosts you.findid(id)— hunts down the unit with a matchingidby checkingid_index. iykyk.findunit(source)— finds the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— same energy as above but plural — finds the units with the given source string. ✨- Return type:
TranslationUnitorNone
- Return type:
getids()— hands you back a list of unit ids. that's it, that's the whole function 💅getprojectstyle()— get the project type for this store.getsourcelanguage()— get the source language for this store.gettargetlanguage()— get the target language for this store. 🦋getunits()— returns a list of all the units in this store. the whole squad.isempty()— returnsTrueif the object doesn't contain any translation units. empty store = no thoughts, head empty 💀makeindex()— indexes the items in this store. at least.sourceindexshould be useful. lowkey the main character here 👑merge_on(property) — the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(input)— parses the given file, or the file source string.parsefile(storefile)(classmethod) — reads the given file (or opens the given filename for you, bestie 🫶) and parses it back into an object.parsestring(storestring)(classmethod) — converts the string representation back into an object. round trip, no cap.remove_unit_from_index(unit)— removes a unit from the source and location indexes.removeunit(unit)— removes the given unit from the object's list of units.- fr fr, always use this method instead of trying to modify the list yourself by hand. pls. 🙈
- Parameters:
unit(TranslationUnit) – the unit that will be added. (yes, the docstring says "added" while the method removes it — iconic typo, we don't fix it, we respect it 💅)
require_index()— makes sure the source index actually exists. manifesting it into being ✨save()— saves to the file that the data was originally read from, if it's available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back withparsestring().outshould be an open file-like object you're writing to.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store. 🌈suggestions_in_format = False— indicates whether the format can store suggestions and alternative translations for a unit.translate(source)— returns the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— iterator over all the units in this store. go little iterator go 🧚
🎀 class translate.storage.omegat.OmegaTUnit(source=None)
an OmegaT glossary unit. periodt. 🍓
adderror(errorname, errortext)— slaps an error message onto this unit.- Parameters:
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
- Parameters:
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. like, don't fake it bestie 😭
addlocations(location)— adds a location, or a whole list of locations.
NOTE: most classes shouldn't need to implement this — they should just implement TranslationUnit.addlocation() instead. WARNING: this method might be removed in the future. ⚠️ so maybe don't get too attached.
addnote(text, origin=None, position='append')— adds a note (a comment, basically).- Parameters:
text(string) – usually just a sentence or two.origin(string) – specifies who/where the comment comes from.origincan be one of these text strings:'translator','developer','programmer','source code'(the last three are synonyms, iykyk).
- Parameters:
buildfromunit(unit)(classmethod) — builds a native unit out of a foreign unit, preserving as much info as possible. glow-up conversion ✨dict(property) — gets the dictionary of values for an OmegaT line.getcontext()(static) — gets the message context.getdict()— gets the dictionary of values for an OmegaT line.geterrors()(static) — gets all the error messages.- Return type:
Dictionary
- Return type:
getid()— a unique identifier for this unit 💅- Return type:
string - Returns: an identifier for this unit that's unique in the store.
- derived classes should override this in a way that guarantees a unique identifier for each unit in the store. no duplicates allowed, bestie.
- Return type:
getlocations()(static) — a list of source code locations.- Return type:
List
- Return type:
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all the notes about this unit. it'll probably be freeform text or something reasonable the format can synthesise. it should not include location comments — for those, seegetlocations(). 🎀gettargetlen()— returns the length of the target string.- Return type:
Integer
- Return type:
NOTE: plural forms might be combined. so don't @ me about the number.
getunits()— this unit, in a list. just the one. main character energy 👑hasplural()(static) — tells you whether this specific unit has plural strings.infer_state()— an empty method that subclasses should override to infer the unit's currentstate(_n)from its current state.isblank()— used to check whether this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather than keeping this around. it's giving legacy 🫠
isfuzzy()(static) — indicates whether this unit is fuzzy.isheader()(static) — indicates whether this unit is a header.isobsolete()(static) — indicates whether a unit is obsolete.isreview()(static) — indicates whether this unit needs review.istranslatable()— indicates whether this unit can actually be translated. use this to tell real translation units apart from headers, obsolete units, binary stuff, or other blank units. like a vibe check but for units ✨istranslated()— indicates whether this unit is translated. use this instead of deducing it from.target, so other classes can add more functionality (XLIFF does exactly that, bestie 💅).makeobsolete()— makes a unit obsolete. rip.markfuzzy(value=True)— marks the unit as fuzzy, or un-fuzzy if you passFalse.markreviewneeded(needsreview=True, explanation=None)— marks the unit to show whether it needs review.- Parameters:
needsreview– defaults toTrue.explanation– adds an optional explanation as a note.
- Parameters:
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does a basic, format-agnostic merge. no drama, just merging 🫶multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator's notes. sorry bestie, they're gone 🫠rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.rich_source(property)
SEE ALSO:rich_to_multistring(),multistring_to_rich().
rich_target(property)
SEE ALSO:rich_to_multistring(),multistring_to_rich().
rich_to_multistring(elem_list)(classmethod) — converts a "rich" string tree into a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setdict(newdict)— sets the dictionary of values for an OmegaT line.- Parameters:
newdict(Dict) – a new dictionary with OmegaT line elements. 🦋
- Parameters:
setid(value)— sets the unique identifier for this unit. (the docstring says "identified" — iconic, we keep the energy, we keep the meaning 💀)
…only implemented if the format lets ids be independant from other unit properties like source or context 💅
unit_iter()— an iterator that only ever returns this unit. that's it, that's the whole vibe 🎀
🫧 oo 🫧
Classes that hold units of .oo files (oounit) or entire files (oofile). ✨
These are the specific .oo files for localisation exported by OpenOffice.org — SDF format (previously known as GSI files). 🦋
The behaviour in terms of escaping is explained in detail in the programming comments. iykyk — go read them 📖
translate.storage.oo.escape_help_text(text)— Escapes the help text as it would be in an SDF file. 💖
<, > and " are only escaped in <[[:lower:]]> tags. Some HTML tags make it in in lowercase so those are dealt with. Some OpenOffice.org help tags are not escaped. 🫶
translate.storage.oo.escape_text(text)— Escapes SDF text to be suitable for unit consumption. ✨translate.storage.oo.makekey(ookey, long_keys)— converts an oo key tuple into a unique identifier. 🗝️
Parameters
ookey(tuple) – an oo keylong_keys(Boolean) – Use long keys
Return type — str
Returns — a unique ascii identifier, slay 💅
class translate.storage.oo.normalizechar(normalchars)— the dict-flavoured one, with all the classic dict moves: 🌸clear()— Remove all items from the dict.copy()— Return a shallow copy of the dict.classmethod fromkeys(iterable, value=None, /)— Create a new dictionary with keys from iterable and values set to value.get(key, default=None, /)— Return the value for key if key is in the dictionary, else default.items()— Return a set-like object providing a view on the dict’s items.keys()— Return a set-like object providing a view on the dict’s keys.pop(k[, d]) -> v— remove specified key and return the corresponding value. If the key is not found, return the default if given; otherwise, raise aKeyError. 😭popitem()— Remove and return a(key, value)pair as a 2-tuple. Pairs are returned in LIFO (last-in, first-out) order. RaisesKeyErrorif the dict is empty.setdefault(key, default=None, /)— Insert key with a value of default if key is not in the dictionary. Return the value for key if key is in the dictionary, else default.update([E], **F) -> None.— Update D from mapping/iterable E and F. If E is present and has a.keys()method, then does:for k in E.keys(): D[k] = E[k]If E is present and lacks a.keys()method, then does:for k, v in E: D[k] = vIn either case, this is followed by:for k in F: D[k] = F[k]values()— Return an object providing a view on the dict’s values.
translate.storage.oo.normalizefilename(filename)— Converts any non-alphanumeric (standard roman) characters to_. 🧚class translate.storage.oo.oofile(input=None)— this represents an entire.oofile. 👑UnitClass— alias ofoounitaddline(thisline)— Adds a parsed line to the file.getoutput(skip_source=False, fallback_lang=None)— Converts all the lines back to tab-delimited form.parse(input)— Parses lines and adds them to the file.serialize(out, skip_source=False, fallback_lang=None)— Convert to a string. double check that unicode is handled. 🫶
class translate.storage.oo.ooline(parts=None)— this represents one line, one translation in an.oofile. 📝getkey()— Get the key that identifies the resource.getoutput()— Return a line in tab-delimited form.getparts()— Return a list of parts in this line.gettext()— Obtains the text column and handle escaping.setparts(parts)— Create a line from its tab-delimited parts.settext(text)— Sets the text column and handle escaping.property text— Obtains the text column and handle escaping.
class translate.storage.oo.oomultifile(filename, mode=None, multifilestyle='single')— this takes a huge GSI file and represents it as multiple smaller files… 🔥createsubfileindex()— Reads in all the lines and works out the subfiles.getoofile(subfile)— Returns anoofilebuilt up from the given subfile’s lines.getsubfilename(line)— Looks up the subfile name for the line.getsubfilesrc(subfile)— Returns the list of lines matching the subfile.listsubfiles()— Returns a list of subfiles in the file.openinputfile(subfile)— Returns a pseudo-file object for the given subfile.openoutputfile(subfile)— Returns a pseudo-file object for the given subfile.
class translate.storage.oo.oounit— this represents a number of translations of a resource. 🎀addline(line)— Add a line to theoounit.getoutput(skip_source=False, fallback_lang=None)— Return the lines in tab-delimited form.
translate.storage.oo.unescape_help_text(text)— Unescapes normal text to be suitable for writing to the SDF file. ✨translate.storage.oo.unescape_text(text)— Unescapes SDF text to be suitable for unit consumption. 💖
🦋 placeables 🦋
This module implements basic functionality to support placeables. 🫶
A placeable is used to represent things like:
- Substitutions
For example, in ODF, footnotes appear in the ODF XML where they are defined; so if we extract a paragraph with some footnotes, the translator will have a lot of additional XML to with; so we separate the footnotes out into separate translation units and mark their positions in the original text with placeables. literally so much cleaner 💅
- Hiding of inline formatting data
The translator doesn’t want to have to deal with all the weird formatting conventions of wherever the text came from. 🙈
- Marking variables
This is an old issue — translators translate variable names which should remain untranslated. We can wrap placeables around variable names to avoid this. no cap 🧚
The placeables model follows the XLIFF standard’s list of placeables. Please refer to the XLIFF specification to get a better understanding. 📚
🌈 base 🌈
Contains base placeable classes with names based on XLIFF placeables. See the XLIFF standard for more information about what the names mean. ⭐
💜 class translate.storage.placeables.base.Bpt(sub=None, id=None, rid=None, xid=None, **kwargs)
apply_to_strings(f)— Applyfto all actual strings in the tree.
Parameters — f – Must take one (str or unicode) argument and return a string or unicode.
copy()— Returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. 🫶
delete_range(start_index, end_index)— Delete the text in the range given by the string-indexes start_index and end_index.
Partial nodes will only be removed if they are editable.
Returns — A StringElem representing the removed sub-string, the parent node from which it was deleted as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. 😭 (undo icon behaviour fr)
depth_first(filter=None)— Returns a list of the nodes in the tree in depth-first order. 🌸elem_at_offset(offset)— Get theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— Find the offset of elem in the current tree.
This cannot be reliably used if self.renderer is used and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as replacement for this one. ⚠️
Returns — The string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— More unicode class emulation. ✨find(x)— Find sub-string x in this string tree and return the position at which it starts.find_elems_with(x)— Find all elements in the current sub-tree containing x.flatten(filter=None)— Flatten the tree by returning a depth-first search over the tree’s leaves. 🍓get_index_data(index)— Get info about the specified range in the tree.
Returns — A dictionary with the following items:
elem: The element in which index resides.index: Copy of the index parameteroffset: The offset of index into'elem'.
get_parent_elem(child)— Searches the current sub-tree for and returns the parent of the child element.has_content = True— Whether this string can have sub-elements. 👑insert(offset, text, preferred_parent=None)— Insert the given text at the specified offset of this string-tree’s string (Unicode) representation.insert_between(left, right, text)— Insert the given text between the two parameterStringElems. 🫶iseditable = True— Whether this string should be changable by the user. Not used at the moment. 💅isfragile = False— Whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False. 💀isleaf()— is this instance a leaf node in theStringElemtree? 🌿
A node counts as a leaf node if it's a StringElem (like, the actual class, not a sub-class) and it contains only sub-elements of type str or unicode.
Return type: bool
istranslatable = True— whether this string is translatable into other languages. ✨isvisible = True— whether this string should be visible to the user. Ngl, not used at the moment. 🙈iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first order. 🌀map(f, filter=None)— applyfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— print the tree in an indented manner, starting from the current instance's point in it. 🌳prune()— remove unnecessary nodes to make the tree optimal. Snip snip ✂️remove_type(ptype)— replace nodes with typeptypewith baseStringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this this string. (Their words, not mine 💀)translate()— transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns: the transformed Unicode string representing the sub-tree.
🌸 class translate.storage.placeables.base.Bx(id=None, xid=None, **kwargs)
apply_to_strings(f)— applyfto all actual strings in the tree.
Parameters:
f– must take one (strorunicode) argument and return astringorunicode.
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. 🫶
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they are editable.
Returns: a StringElem representing the removed sub-string, the parent node from which it was deleted as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. Undo era 🔥
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— find the offset ofelemin the current tree.
This cannot be reliably used if self.renderer is used and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as replacement for this one.
Returns: the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— more unicode class emulation.find(x)— find sub-stringxin this string tree and return the position at which it starts.find_elems_with(x)— find all elements in the current sub-tree containingx.flatten(filter=None)— flatten the tree by returning a depth-first search over the tree's leaves. 🍃get_index_data(index)— get info about the specified range in the tree.
Returns: a dictionary with the following items:
elem: the element in whichindexresides.index: copy of theindexparameteroffset: the offset ofindexinto'elem'.
get_parent_elem(child)— searches the current sub-tree for and returns the parent of thechildelement. 👑has_content = False— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— insert the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation. 📥insert_between(left, right, text)— insert the giventextbetween the two parameterStringElems.iseditable = True— whether this string should be changable by the user. Not used at the moment. 🙈isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False. Handle with care 💅isleaf()— whether or not this instance is a leaf node in theStringElemtree.
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool
istranslatable = False— whether this string is translatable into other languages.isvisible = True— whether this string should be visible to the user. Not used at the moment.iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first order. 🌀map(f, filter=None)— applyfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— print the tree in an indented manner, from the current instance's point in it. 🌳prune()— remove unnecessary nodes to make the tree optimal.remove_type(ptype)— replace nodes with typeptypewith baseStringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this this string.translate()— transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns: the transformed Unicode string representing the sub-tree.
🦋 class translate.storage.placeables.base.Ept(sub=None, id=None, rid=None, xid=None, **kwargs)
apply_to_strings(f)— applyfto all actual strings in the tree.
Parameters:
f– must take one (strorunicode) argument and return astringorunicode.
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. 🫶
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they are editable.
Returns: a StringElem representing the removed sub-string, the parent node from which it was deleted as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. 🔥
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— find the offset ofelemin the current tree.
This cannot be reliably used if self.renderer is used and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as replacement for this one.
Returns: the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— more unicode class emulation.find(x)— find sub-stringxin this string tree and return the position at which it starts.find_elems_with(x)— find all elements in the current sub-tree containingx.flatten(filter=None)— flatten the tree by returning a depth-first search over the tree's leaves. 🍃get_index_data(index)— get info about the specified range in the tree.
Returns: a dictionary with the following items:
elem: the element in whichindexresides.index: copy of theindexparameteroffset: the offset ofindexinto'elem'.
get_parent_elem(child)— searches the current sub-tree for and returns the parent of thechildelement.has_content = True— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— dropstextinto this string-tree's string (Unicode) representation at exactlyoffset. That's the whole assignment bestie ✨insert_between(left, right, text)— same vibe, except you're wedgingtextin between the twoStringElems you pass asleftandright🫶iseditable = True— whether the user is allowed to change this string. Lowkey important caveat: not used at the moment, so nobody's actually checking it 🙈isfragile = False— whether this element should be deleted in its entirety when someone partially deletes it 💀 and this one's only checked wheniseditable = False. Periodt.isleaf()— is this instance a leaf node in theStringElemtree or not 🌿
An element counts as a leaf node if it's a StringElem (not one of its sub-classes) and it only holds sub-elements of type str or unicode. That's the whole test, no cap.
- Return type:
bool
istranslatable = True— whether this string is translatable into other languages 🌈 (yes queen, localization rights)isvisible = True— whether this string should be visible to the user. Again: not used at the moment, it's giving decorative 🙈iter_depth_first(filter=None)— iterates through the nodes in the tree in dept-first order (their spelling, not mine, ily 💖).map(f, filter=None)— appliesfto all the nodes wherefilterreturnedTrue. Thefilteris optional, so you can absolutely just map over everything 🔥print_tree(indent=0, verbose=False)— prints the tree all nicely indented, starting from wherever the current instance sits in it 🎀prune()— removes the unnecessary nodes so the tree comes out optimal. Self-care for data structures 💅remove_type(ptype)— swaps out nodes of typeptypefor baseStringElems holding the same sub-elements. Scope check: this only applies to elements below the element tree root node, okay? 👑renderer = None— an optional function that returns the Unicode representation of the string ✨sub = []— the sub-elements that make up this string (yes the original says "this this", we love a typo moment 🫶).translate()— transforms the sub-tree according to whatever class-specific needs it has. It should either be overridden in implementing sub-classes or dynamically replaced by specific applications — pick your fighter 🥊- Returns: the transformed Unicode string representing the sub-tree 💖
💜 class translate.storage.placeables.base.Ex(id=None, xid=None, **kwargs)
apply_to_strings(f)— appliesfto every actual string in the tree 🧚- Parameters:
f– must take one (strorunicode) argument and return a string or unicode.
- Parameters:
copy()— returns a copy of the sub-tree 🎀 but heads up, this should be overridden in sub-classes that carry more data.
NOTE: self.renderer is not copied. Do not come for me, that's just the rules 😭
delete_range(start_index, end_index)— deletes the text in the range given by the string-indexesstart_indexandend_index✂️
Partial nodes only get removed if they're editable, so fragile ones are safe-ish.
- Returns: a
StringElemrepresenting the removed sub-string, the parent node it was deleted from, and the offset it was deleted at. If the root got deleted,Nonecomes back for the parent value. And if the parent and offset values aren'tNone, thenparent.insert(offset, deleted)effectively undoes the delete — an uno reverse card 🃏
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order 📋elem_at_offset(offset)— grabs theStringElemin the tree that contains the string rendered at the givenoffset📍elem_offset(elem)— finds the offset ofelemin the current tree.
Real talk: this can't be reliably used if self.renderer is in play — and even less so if the rendering function renders the string differently on different calls 😬 In Virtaal, the StringElemGUI.index() method is used as a replacement for this one.
- Returns: the string index where element
estarts, or-1ifewasn't found.
encode(encoding='utf-8')— more unicode class emulation, because we simply must 💅find(x)— finds sub-stringxin this string tree and returns the position where it starts 🔍find_elems_with(x)— finds all the elements in the current sub-tree that containx👑flatten(filter=None)— flattens the tree by returning a depth-first search over the tree's leaves 🍃get_index_data(index)— gets info about the specified range in the tree.- Returns: a dictionary with these items in it 📖
elem: the element thatindexlives in.index: a copy of theindexparameter.offset: the offset ofindexintoelem.
- Returns: a dictionary with these items in it 📖
get_parent_elem(child)— searches the current sub-tree and returns the parent of thechildelement 👩👧has_content = False— whether this string can have sub-elements. Respectfully: no 🚫insert(offset, text, preferred_parent=None)— insertstextat the specifiedoffsetof this string-tree's string (Unicode) representation ✨insert_between(left, right, text)— insertstextbetween the two parameterStringElems,leftandright🫶iseditable = True— whether the user should be able to change this string. Not used at the moment, it's giving set dressing 🙈isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False💀isleaf()— whether or not this instance is a leaf node in theStringElemtree 🌿
A node is a leaf node if it's a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
- Return type:
bool
istranslatable = False— whether this string is translatable into other languages. ForExthe answer is a hard no 🚫✨isvisible = True— whether this string should be visible to the user. Not used at the moment, we keep saying it and it keeps being true 😭iter_depth_first(filter=None)— iterates through the nodes in the tree in dept-first order 🌀map(f, filter=None)— appliesfto all nodes for whichfilterreturnedTrue(optional) 🔥print_tree(indent=0, verbose=False)— prints the tree from the current instance's point in an indented manner, very demure, very mindful 🎀prune()— removes unnecessary nodes to make the tree optimal 🌸remove_type(ptype)— replaces nodes with typeptypewith baseStringElems containing the same sub-elements. Again: only applicable to elements below the element tree root node 👑renderer = None— an optional function that returns the Unicode representation of the string ⭐sub = []— the sub-elements that make up this this string (the double "this" is canon, I don't make the rules 🫶).translate()— transforms the sub-tree according to some class-specific needs. Should be either overridden in implementing sub-classes or dynamically replaced by specific applications.- Returns: the transformed Unicode string representing the sub-tree 💖
🦋 class translate.storage.placeables.base.G(sub=None, id=None, rid=None, xid=None, **kwargs)
apply_to_strings(f)— appliesfto all actual strings in the tree 🧚- Parameters:
f– must take one (strorunicode) argument and return a string or unicode.
- Parameters:
copy()— returns a copy of the sub-tree. Should be overridden in sub-classes that hold more data.
NOTE: self.renderer is not copied. Sorry babe 😭
delete_range(start_index, end_index)— deletes the text in the range given by the string-indexesstart_indexandend_index✂️
Partial nodes will only be removed if they are editable — remembered.
- Returns: a
StringElemrepresenting the removed sub-string, the parent node from which it was deleted, and the offset at which it was deleted from.Nonecomes back for the parent value if the root was deleted. If the parent and offset values are notNone,parent.insert(offset, deleted)effectively undoes the delete 🃏
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order 📋elem_at_offset(offset)— gets theStringElemin the tree that contains the string rendered at the givenoffset📍elem_offset(elem)— finds the offset ofelemin the current tree.
Same warning as ever: this cannot be reliably used if self.renderer is used, and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as a replacement for this one 😬
- Returns: the string index where element
estarts, or-1ifewas not found.
encode(encoding='utf-8')— more unicode class emulation 💅find(x)— finds sub-stringxin this string tree and returns the position at which it starts 🔍find_elems_with(x)— finds all elements in the current sub-tree containingx👑flatten(filter=None)— flattens the tree by returning a depth-first search over the tree's leaves 🍃get_index_data(index)— gets info about the specified range in the tree 📖
Like, this one hands you back a dictionary, and inside it you get:
elem— the element thatindexis living in 💖index— a copy of theindexparameter. Yes, literally the same value you passed in.offset— whereindexsits insideelem.
get_parent_elem(child)— searches the current sub-tree, finds the parent of thechildelement, and returns it. It's giving "come get your kid" energy 🫶has_content = True— whether this string can have sub-elements. Spoiler: it can.insert(offset, text, preferred_parent=None)— inserts the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation. Slide it right in ✨insert_between(left, right, text)— inserts the giventextbetween the two parameterStringElems. Sandwich moment 🥪iseditable = True— whether this string should be changable by the user. Not used at the moment. (Iconic flag, zero screen time 💀)isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False. So yeah, handle it carefully 🎀isleaf()— whether or not this instance is a leaf node in theStringElemtree.
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool
istranslatable = True— whether this string is translatable into other languages 🌈isvisible = True— whether this string should be visible to the user. Not used at the moment. (Bestie is on read, lowkey 🙈)iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first order.map(f, filter=None)— applyfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— print the tree, starting from the current instance's point, all indented and pretty 🦋prune()— remove unnecessary nodes to make the tree optimal. Declutter era 💅remove_type(ptype)— replace nodes with typeptypewith baseStringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this string.translate()— transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns: the transformed Unicode string representing the sub-tree.
✨ class translate.storage.placeables.base.It(sub=None, id=None, rid=None, xid=None, **kwargs)
Okay so this is the It class, and it's coming with the whole family of methods, no skips 👑
apply_to_strings(f)— applyfto all actual strings in the tree.
Parameters: f – must take one (str or unicode) argument and return a string or unicode.
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. I know. Devastating. 😭
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they are editable.
Returns: a StringElem representing the removed sub-string, the parent node from which it was deleted, as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. Undo button built in, slay 🔥
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the givenoffset. Point at a spot, get the elem. So satisfying ✨elem_offset(elem)— find the offset ofelemin the current tree.
This cannot be reliably used if self.renderer is used, and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as replacement for this one.
Returns: the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— more unicode class emulation, bestie. It's giving "pretend to be a unicode object" energy 🫶find(x)— find the sub-stringxinside this string tree, and it hands you back the position where it starts. lil detective work 🔍find_elems_with(x)— find all the elements in the current sub-tree that containx. not just one, the whole squad 💅flatten(filter=None)— flattens the tree by giving you a depth-first search over the tree's leaves. so organized of it ✨get_index_data(index)— get the info about the specified range in the tree.
Returns a dictionary with these items in it:
elem: the element thatindexlives insideindex: a copy of theindexparameter (yes, literally a copy, no cap)offset: the offset ofindexintoelem
get_parent_elem(child)— searches the current sub-tree and returns the parent of thechildelement. we love a family reunion 🌸has_content = True— whether this string can have sub-elements. (it can, it's an icon 👑)insert(offset, text, preferred_parent=None)— insert the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation.insert_between(left, right, text)— insert the giventextbetween the two parameter StringElems, i.e.leftandright. sandwich moment 🥪iseditable = True— whether this string should be changable by the user. Not used at the moment, so it's lowkey just vibing there 🙈isfragile = False— whether this element should be deleted in its entirety when it's partially deleted. This one only gets checked wheniseditable = False. handle with care, it's fragile fr fr 💀isleaf()— tells you whether this instance is a leaf node in the StringElem tree.
A node counts as a leaf node if it's a StringElem (not a sub-class) and it contains only sub-elements of type str or unicode.
Return type: bool 🍓
istranslatable = False— whether this string is translatable into other languages.isvisible = True— whether this string should be visible to the user. Not used at the moment — again, just aesthetics, no thoughts 🫠iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first order.map(f, filter=None)— applyfto all the nodes for whichfilterreturnedTrue(the filter is optional).print_tree(indent=0, verbose=False)— print the tree, starting from the current instance's point, all indented and pretty so you can actually read it 🌈prune()— remove the unnecessary nodes so the tree becomes optimal. declutter era 🧹✨remove_type(ptype)— replace nodes of typeptypewith base StringElems that contain the same sub-elements. Heads up: this is only applicable to elements below the element tree root node 👑renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this this string. (yes, this this. the string is that important 💅)translate()— transform the sub-tree according to whatever class-specific needs it has. This method should either be overridden in the implementing sub-classes or dynamically replaced by specific applications.
Returns the transformed Unicode string that represents the sub-tree 🦋
class translate.storage.placeables.base.Sub(sub=None, id=None, rid=None, xid=None, **kwargs)
okay so Sub is basically the same queen as before but with her own little twist — same methods, same energy, one (1) very important flag flip 👀
apply_to_strings(f)— applyfto all the actual strings in the tree.
Parameters: f — must take one (str or unicode) argument and return a string or unicode.
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes that carry more data.
NOTE: self.renderer is not copied. just so you know 💅
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only get removed if they're editable.
Returns a StringElem representing the removed sub-string, the parent node it was deleted from, and the offset it was deleted from. None comes back for the parent value if the root itself was deleted. And if the parent and offset values are not None, then parent.insert(offset, deleted) effectively undoes the delete. undo button unlocked 😭🙏
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the givenoffset.elem_offset(elem)— find the offset ofelemin the current tree.
Heads up: this cannot be reliably used if self.renderer is used — and even less so if the rendering function renders the string differently on different calls. In Virtaal, the StringElemGUI.index() method is used as a replacement for this one.
Returns the string index where element e starts, or -1 if e was not found. bestie, -1 means it ghosted you 👻
encode(encoding='utf-8')— more unicode class emulation. yes again, it's a whole thing ✨find(x)— find sub-stringxin this string tree and return the position where it starts.find_elems_with(x)— find all the elements in the current sub-tree that containx.flatten(filter=None)— flatten the tree by returning a depth-first search over the tree's leaves. 🍃get_index_data(index)— get info about the specified range in the tree.
Returns a dictionary with these items:
elem: the element in whichindexresidesindex: a copy of theindexparameteroffset: the offset ofindexintoelem
get_parent_elem(child)— searches the current sub-tree and returns the parent of thechildelement. 🫶has_content = True— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— insert the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation.insert_between(left, right, text)— insert the giventextbetween the two parameter StringElems.iseditable = True— whether this string should be changable by the user. Not used at the moment.isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False.isleaf()— whether or not this instance is a leaf node in the StringElem tree.
A node is a leaf node if it's a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool ⭐
istranslatable = True— 👀 spot the difference from the base class! this string is translatable into other languages. periodt 💖isvisible = True— whether this string should be visible to the user. Not used at the moment.iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first order.map(f, filter=None)— applyfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— print the tree from the current instance's point in an indented manner. aesthetic AND functional 🌸prune()— remove unnecessary nodes to make the tree optimal.remove_type(ptype)— replace nodes with typeptypewith base StringElems containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this this string. (still this this. iconic behavior 💅)translate()— transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns the transformed Unicode string representing the sub-tree 🦋
class translate.storage.placeables.base.X(id=None, xid=None, **kwargs)
and now for X — she's mysterious, she's minimal, she has exactly one job and she's doing it 🔥
apply_to_strings(f)— applyfto all the actual strings in the tree.
Parameters: f — must take one (str or unicode) argument and return a string or unicode.
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. write that down 📝
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they're editable.
Returns a StringElem representing the removed sub-string, the parent node from which it was deleted, and the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. rewind time, slay ⏪✨
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— finds whereelemsits in the current tree. ⚠️ heads up bestie: this is not reliable ifself.rendereris in play, and even less reliable if that rendering function renders the string differently on different calls 😭. In Virtaal, theStringElemGUI.index()method is used instead of this one, iykyk.- Returns: the string index where element
estarts, or-1ifewas not found.
- Returns: the string index where element
encode(encoding='utf-8')— more unicode class emulation ✨ (yes it's just vibes-compatible with the oldunicodeAPI).find(x)— finds sub-stringxin this string tree and returns the position where it starts 🔍.find_elems_with(x)— finds all the elements in the current sub-tree that containx.flatten(filter=None)— flattens the tree by giving you a depth-first search over the tree's leaves 🍃.get_index_data(index)— gets the info about the specified range in the tree.- Returns: a dictionary with these items:
elem: the element in whichindexresides.index: a copy of theindexparameter.offset: the offset ofindexinto'elem'.
- Returns: a dictionary with these items:
get_parent_elem(child)— searches the current sub-tree and returns the parent of thechildelement 🫶.has_content = False— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— inserts the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation ✍️.insert_between(left, right, text)— inserts the giventextbetween the two parameterStringElems(leftandright).iseditable = False— whether this string should be changeable by the user. Not used at the moment 🙈.isfragile = True— whether this element should be deleted in its entirety when partially deleted 💀. Only checked wheniseditable = False.isleaf()— whether or not this instance is a leaf node in theStringElemtree.- A node is a leaf node if it is a
StringElem(not a sub-class) and contains only sub-elements of typestrorunicode. - Return type:
bool
- A node is a leaf node if it is a
istranslatable = False— whether this string is translatable into other languages 🌈.isvisible = True— whether this string should be visible to the user. Not used at the moment 🙈.iter_depth_first(filter=None)— iterates through the nodes in the tree in dept-first order (yes, "dept" — that's the original's energy, we keep it ✨).map(f, filter=None)— appliesfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— prints the tree, indented and pretty, starting from the current instance's point in the tree 🌸.prune()— removes unnecessary nodes so the tree is optimal 🧹.remove_type(ptype)— replaces nodes with typeptypewith baseStringElemsthat contain the same sub-elements. This only applies to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string 🎀.sub = []— the sub-elements that make up this this string (double "this" is canon, don't @ me).translate()— transforms the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.- Returns: the transformed Unicode string representing the sub-tree ✨.
🌼 general 🌼
Contains general placeable implementations. That is, placeables that does not fit into any other sub-category 👑.
class translate.storage.placeables.general.AltAttrPlaceable(sub=None, id=None, rid=None, xid=None, **kwargs)
The placeable for the “alt=…” attributes inside XML tags 🏷️.
apply_to_strings(f)— appliesfto all the actual strings in the tree.- Parameters:
f– must take one (strorunicode) argument and return a string or unicode.
- Parameters:
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied 🙅♀️
delete_range(start_index, end_index)— deletes the text in the range given by the string-indexesstart_indexandend_index.- Partial nodes will only be removed if they are editable.
- Returns: a
StringElemrepresenting the removed sub-string, the parent node from which it was deleted, as well as the offset at which it was deleted from.Noneis returned for the parent value if the root was deleted. If the parent and offset values are notNone,parent.insert(offset, deleted)effectively undoes the delete ♻️.
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— gets theStringElemin the tree that contains the string rendered at the givenoffset.elem_offset(elem)— finds the offset ofelemin the current tree.- This cannot be reliably used if
self.rendereris used, and even less so if the rendering function renders the string differently upon different calls 😭. In Virtaal theStringElemGUI.index()method is used as replacement for this one. - Returns: the string index where element
estarts, or-1ifewas not found.
- This cannot be reliably used if
encode(encoding='utf-8')— more unicode class emulation ✨.find(x)— finds sub-stringxin this string tree and returns the position at which it starts.find_elems_with(x)— finds all elements in the current sub-tree containingx.flatten(filter=None)— flattens the tree by returning a depth-first search over the tree's leaves 🍃.get_index_data(index)— gets info about the specified range in the tree.- Returns: a dictionary with the following items:
elem: the element in whichindexresides.index: copy of theindexparameter.offset: the offset ofindexinto'elem'.
- Returns: a dictionary with the following items:
get_parent_elem(child)— searches the current sub-tree and returns the parent of thechildelement.has_content = True— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— inserts the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation.insert_between(left, right, text)— inserts the giventextbetween the two parameterStringElems.iseditable = True— whether this string should be changeable by the user. Not used at the moment 🙈.isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False.isleaf()— whether or not this instance is a leaf node in theStringElemtree.- A node is a leaf node if it is a
StringElem(not a sub-class) and contains only sub-elements of typestrorunicode. - Return type:
bool
- A node is a leaf node if it is a
istranslatable = True— whether this string is translatable into other languages 🌈.isvisible = True— whether this string should be visible to the user. Not used at the moment 🙈.iter_depth_first(filter=None)— iterates through the nodes in the tree in dept-first order.map(f, filter=None)— appliesfto all nodes for whichfilterreturnedTrue(optional).classmethod parse(pstr)— a parser method to extract placeables from a string based on a regular expression. Use this function as the@parse()method of a placeable class 🔥.print_tree(indent=0, verbose=False)— prints the tree from the current instance's point in an indented manner.prune()— removes unnecessary nodes to make the tree optimal 🧹.remove_type(ptype)— replaces nodes with typeptypewith baseStringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string 🎀.sub = []— the sub-elements that make up this this string.translate()— transforms the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.- Returns: the transformed Unicode string representing the sub-tree ✨.
class translate.storage.placeables.general.XMLEntityPlaceable(sub=None, id=None, rid=None, xid=None, **kwargs)
The placeable handling XML entities (&xxxxx;-style entities) 🧚.
apply_to_strings(f)— appliesfto all actual strings in the tree.- Parameters:
f– must take one (strorunicode) argument and return a string or unicode.
- Parameters:
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. 💀 like bestie it literally just does not come with you, noted.
delete_range(start_index, end_index)
okay so this one deletes the text sitting in the range you give it — from string-index start_index to string-index end_index, and yes those are string indexes, not vibes 💅
Partial nodes? They only get yeeted if they're editable. if they're not editable, they stay. that's the rule, no cap.
Returns: ngl this return value is doing a lot, so:
- A
StringElemthat represents the removed sub-string ✨ - the parent node it got deleted from
- and the offset it was deleted from 🎀
If the root was the thing that got deleted, then None comes back for the parent value. And listen — if the parent and offset values are not None, then parent.insert(offset, deleted) effectively undoes the delete. it's giving undo button, fr fr 🫶
depth_first(filter=None)
Returns a list of all the nodes in the tree, in depth-first order. that's it that's the whole thing 💖
elem_at_offset(offset)
Grab the StringElem in the tree that contains the string rendered at the offset you pass in. lowkey like "which element is living at this address" 🏠
elem_offset(elem)
Find the offset of elem in the current tree.
⚠️ okay important: this cannot be reliably used if self.renderer is used, and even less so if the rendering function renders the string differently on different calls 😭 so like, don't trust it in that situation, periodt.
In Virtaal, the StringElemGUI.index() method is used as a replacement for this one. iconic workaround, slay.
Returns: the string index where element e starts, or -1 if e was not found. yes, negative one. the classic "she's not here" 💀
encode(encoding='utf-8')
More unicode class emulation. short and sweet, we love a minimal queen ✨
find(x)
Find sub-string x in this string tree and return the position at which it starts. 🔍
find_elems_with(x)
Find all the elements in the current sub-tree that contain x. not just one — all of them, bestie 🌸
flatten(filter=None)
Flattens the tree by returning a depth-first search over the tree's leaves. 🍃
get_index_data(index)
Get info about the specified range in the tree. it's giving detective work 🕵️♀️
Returns: a dictionary with these items in it:
elem: The element in whichindexresides.index: Copy of theindexparameter.offset: The offset ofindexinto'elem'.
get_parent_elem(child)
Searches the current sub-tree and returns the parent of the child element. found family moment 🫶
has_content = True
Whether this string can have sub-elements. yes she can have little sub-element babies 💖
insert(offset, text, preferred_parent=None)
Insert the given text at the specified offset of this string-tree's string (Unicode) representation. the preferred_parent is optional, she's flexible like that 🎀
insert_between(left, right, text)
Insert the given text between the two parameter StringElems — i.e. squish it right in between left and right. 🧚
iseditable = False
Whether this string should be changable by the user. Not used at the moment. (the way that it's just sitting there, unused, doing nothing — relatable honestly 😭)
isfragile = False
Whether this element should be deleted in its entirety when partially deleted. Only checked when iseditable = False. handle with care bestie, she's fragile 💅
isleaf()
Whether or not this instance is a leaf node in the StringElem tree. 🍂
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool
istranslatable = False
Whether this string is translatable into other languages. 🌍
isvisible = True
Whether this string should be visible to the user. Not used at the moment. (again!! unused!! we stay unbothered 😌)
iter_depth_first(filter=None)
Iterate through the nodes in the tree in dept-first order. (that's the manpage's spelling, not mine, i'm just the messenger 🙈)
map(f, filter=None)
Apply f to all nodes for which filter returned True. the filter is optional, so like, you can just map everything if you want 💫
classmethod parse(pstr)
A parser method to extract placeables from a string based on a regular expression. Use this function as the @parse() method of a placeable class. cute decorator energy 🎀
print_tree(indent=0, verbose=False)
Print the tree, starting from the current instance's point, in an indented manner. so satisfying to look at ngl 🖨️✨
prune()
Remove unnecessary nodes to make the tree optimal. self-care for trees 🌿💅
remove_type(ptype)
Replace nodes with type ptype with base StringElems, containing the same sub-elements.
heads up: this is only applicable to elements below the element tree root node. don't go trying it on the root, she's not available for that 🚫
renderer = None
An optional function that returns the Unicode representation of the string. optional!! None by default!! 💖
sub = []
The sub-elements that make up this this string. (yes. the list starts empty. she's an empty list. iconic.)
translate()
Transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications. so like, either subclass it or swap it out at runtime — your call bestie 🦋
Returns: The transformed Unicode string representing the sub-tree. ✨
class translate.storage.placeables.general.XMLTagPlaceable(sub=None, id=None, rid=None, xid=None, **kwargs)
Placeable handling XML tags. 🏷️ okay so this whole class is literally just the previous set of methods again but with XML tag flavor, and we do not skip the rerun, so here's the full rundown, bestie 💖
apply_to_strings(f)
Apply f to all actual strings in the tree. the actual ones. the real ones. 🫶
Parameters: f – Must take one (str or unicode) argument and return a string or unicode.
copy()
Returns a copy of the sub-tree. This should be overridden in sub-classes with more data. 🎀
NOTE: self.renderer is not copied. (still not copied!! still!! 😭)
delete_range(start_index, end_index)
Delete the text in the range given by the string-indexes start_index and end_index.
Partial nodes will only be removed if they are editable.
Returns: A StringElem representing the removed sub-string, the parent node from which it was deleted, as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. 💫
depth_first(filter=None)
Returns a list of the nodes in the tree in depth-first order. 💖
elem_at_offset(offset)
Get the StringElem in the tree that contains the string rendered at the given offset. 🌸
elem_offset(elem)
Find the offset of elem in the current tree.
⚠️ This cannot be reliably used ifself.rendereris used, and even less so if the rendering function renders the string differently upon different calls. In Virtaal theStringElemGUI.index()method is used as replacement for this one.
Returns: The string index where element e starts, or -1 if e was not found. 💀
encode(encoding='utf-8')
More unicode class emulation. same queen, different class 💅
find(x)
Find sub-string x in this string tree and return the position at which it starts. 🔍
find_elems_with(x)
Find all elements in the current sub-tree containing x. 🌈
flatten(filter=None)
Flatten the tree by returning a depth-first search over the tree's leaves. 🍃
get_index_data(index)
Get info about the specified range in the tree.
Returns: A dictionary with the following items:
elem: The element in whichindexresides.index: Copy of theindexparameter.offset: The offset ofindexinto'elem'.
get_parent_elem(child)
Searches the current sub-tree for and returns the parent of the child element. 🫶
has_content = True
Whether this string can have sub-elements. 💖 (still True!!)
insert(offset, text, preferred_parent=None)
Insert the given text at the specified offset of this string-tree's string (Unicode) representation. 🎀
insert_between(left, right, text)
Insert the given text between the two parameter StringElems. 🧚
iseditable = True
Whether this string should be changable by the user. Not used at the moment.
🚨 wait, look at that —Truehere, notFalselike the other class. do not mix them up, i'm serious, that's a different vibe entirely 👑
isfragile = False
Whether this element should be deleted in its entirety when partially deleted. Only checked when iseditable = False. 💅
isleaf()
Whether or not this instance is a leaf node in the StringElem tree.
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool 🍂
istranslatable = False
Whether this string is translatable into other languages. 🌍
isvisible = True
Whether this string should be visible to the user. Not used at the moment. 😌
iter_depth_first(filter=None)
Iterate through the nodes in the tree in dept-first order. 🙈
map(f, filter=None)
Apply f to all nodes for which filter returned True (optional). 💫
classmethod parse(pstr)
A parser method to extract placeables from a string based on a regular expression. Use this function as the @parse() method of a placeable class. 🎀
print_tree(indent=0, verbose=False)
Print the tree from the current instance's point in an indented manner. 🖨️
prune()
Remove unnecessary nodes to make the tree optimal. 🌿
remove_type(ptype)
Replace nodes with type ptype with base StringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node. 🚫
renderer=None— an optional function that hands back the Unicode representation of the string. 🎀sub=[]— the sub-elements that make up this string, aka the little squad it's built from. 💅
translate()
translate()Transforms the sub-tree according to whatever class-specific needs it's got going on. Bestie, this method is meant to be overridden in the sub-classes that implement it, or swapped out dynamically by specific applications. Either way, it's the glow-up step ✨
Returns — the transformed Unicode string representing the sub-tree.
🧚 interfaces
This file contains abstract (semantic) interfaces for placeable implementations. Like, the blueprint energy. 🫶
class translate.storage.placeables.interfaces.BasePlaceable(sub=None, id=None, rid=None, xid=None, **kwargs)
Base class for all placeables. Everything else descends from this one, so show some respect 👑
apply_to_strings(f)
Applies f to all the actual strings in the tree.
Parameters — f must take one (str or unicode) argument and return a string or unicode.
copy()
Returns a copy of the sub-tree. Heads up: this should be overridden in sub-classes that carry more data.
NOTE: self.renderer is not copied. Fr fr, it stays behind. 💀
delete_range(start_index, end_index)
Deletes the text in the range given by the string-indexes start_index and end_index.
Partial nodes only get removed if they're editable.
Returns — a StringElem representing the removed sub-string, the parent node from which it was deleted, and the offset at which it was deleted from. None comes back for the parent value if the root was the thing that got deleted. And if the parent and offset values are not None, then parent.insert(offset, deleted) effectively undoes the delete. Undo button, but make it code 🩹
depth_first(filter=None)
Returns a list of the nodes in the tree in depth-first order.
elem_at_offset(offset)
Gets the StringElem in the tree that contains the string rendered at the given offset.
elem_offset(elem)
Finds the offset of elem in the current tree.
Okay so, this can't be reliably used if self.renderer is in play — and even less so if the rendering function renders the string differently on different calls 😭. In Virtaal the StringElemGUI.index() method is used as a replacement for this one.
Returns — the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')
More unicode class emulation. That's the whole description, bestie. ✨
find(x)
Finds sub-string x in this string tree and returns the position at which it starts.
find_elems_with(x)
Finds all elements in the current sub-tree containing x.
flatten(filter=None)
Flattens the tree by returning a depth-first search over the tree's leaves. 🍃
get_index_data(index)
Gets info about the specified range in the tree.
Returns — a dictionary with the following items:
elem: the element in whichindexresides.index: copy of theindexparameter.offset: the offset ofindexintoelem.
get_parent_elem(child)
Searches the current sub-tree and returns the parent of the child element. Found family vibes 🫶
has_content = True
Whether this string can have sub-elements.
insert(offset, text, preferred_parent=None)
Inserts the given text at the specified offset of this string-tree's string (Unicode) representation.
insert_between(left, right, text)
Inserts the given text between the two parameter StringElems. Literally wedging it in there 💅
iseditable = True
Whether this string should be changeable by the user. Not used at the moment.
isfragile = False
Whether this element should be deleted in its entirety when partially deleted. Only checked when iseditable = False. So like, all-or-nothing energy 🎀
isleaf()
Whether or not this instance is a leaf node in the StringElem tree.
A node is a leaf node if it's a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type — bool
istranslatable = True
Whether this string is translatable into other languages. 🌈
isvisible = True
Whether this string should be visible to the user. Not used at the moment.
iter_depth_first(filter=None)
Iterates through the nodes in the tree in depth-first order.
map(f, filter=None)
Applies f to all nodes for which filter returned True (optional).
print_tree(indent=0, verbose=False)
Prints the tree from the current instance's point, all indented and tidy. 🖨️
prune()
Removes unnecessary nodes to make the tree optimal. Decluttering era ✨
remove_type(ptype)
Replaces nodes with type ptype with base StringElems containing the same sub-elements. This is only applicable to elements below the element tree root node.
renderer = None
An optional function that returns the Unicode representation of the string.
sub = []
The sub-elements that make up this string.
translate()
Transforms the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns — the transformed Unicode string representing the sub-tree.
class translate.storage.placeables.interfaces.InvisiblePlaceable(sub=None, id=None, rid=None, xid=None, **kwargs)
Okay so this one inherits the whole entire lineup from BasePlaceable — same constructor, same sub, id, rid, xid, same **kwargs. Here's the full roll call again so nobody comes for me about missing entries 💅
apply_to_strings(f)
Applies f to all actual strings in the tree.
Parameters — f must take one (str or unicode) argument and return a string or unicode.
copy()
Returns a copy of the sub-tree. Should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. Same rule as before, no cap 🚫
delete_range(start_index, end_index)
Deletes the text in the range given by the string-indexes start_index and end_index.
Partial nodes only get removed if they're editable.
Returns — a StringElem representing the removed sub-string, the parent node from which it was deleted, and the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete.
depth_first(filter=None)
Returns a list of the nodes in the tree in depth-first order.
elem_at_offset(offset)
Gets the StringElem in the tree that contains the string rendered at the given offset.
elem_offset(elem)
Finds the offset of elem in the current tree.
Cannot be reliably used if self.renderer is used, and even less so if the rendering function renders the string differently upon different calls 😭. In Virtaal the StringElemGUI.index() method is used as a replacement for this one.
Returns — the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')
More unicode class emulation.
find(x)
Finds sub-string x in this string tree and returns the position at which it starts.
find_elems_with(x)
Finds all elements in the current sub-tree containing x.
flatten(filter=None)
Flattens the tree by returning a depth-first search over the tree's leaves.
get_index_data(index)
Gets info about the specified range in the tree.
Returns — a dictionary with the following items:
elem: the element in whichindexresides.index: copy of theindexparameter.offset: the offset ofindexintoelem.
get_parent_elem(child)
Searches the current sub-tree and returns the parent of the child element.
has_content = True
Whether this string can have sub-elements.
insert(offset, text, preferred_parent=None)
Inserts the given text at the specified offset of this string-tree's string (Unicode) representation.
insert_between(left, right, text)
Inserts the given text between the two parameter StringElems.
iseditable = True
Whether this string should be changeable by the user. Not used at the moment.
isfragile = False
Whether this element should be deleted in its entirety when partially deleted. Only checked when iseditable = False.
isleaf()
Whether or not this instance is a leaf node in the StringElem tree.
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type — bool
istranslatable = True
Whether this string is translatable into other languages. 🌸
isvisible=True— Whether this string should be visible to the user. Not used at the moment. (iconic attribute, zero job 💅)iter_depth_first(filter=None)— Iterate through the nodes in the tree in depth-first order.map(f, filter=None)— Applyfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— Print the tree from the current instance's point in an indented manner. So pretty 🌸prune()— Remove unnecessary nodes to make the tree optimal. self-care for data structures fr fr ✨remove_type(ptype)— Replace nodes with typeptypewith baseStringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer=None— An optional function that returns the Unicode representation of the string.sub=[]— The sub-elements that make up this this string.translate()— Transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns : The transformed Unicode string representing the sub-tree.
🧩 class translate.storage.placeables.interfaces.ReplacementPlaceable(sub=None, id=None, rid=None, xid=None, **kwargs)
apply_to_strings(f)— Applyfto all actual strings in the tree.
Parameters : f – Must take one (str or unicode) argument and return a string or unicode.
copy()— Returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. 🫶
delete_range(start_index, end_index)— Delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they are editable.
Returns : A StringElem representing the removed sub-string, the parent node from which it was deleted as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. (undo era 💫)
depth_first(filter=None)— Returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— Get theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— Find the offset ofelemin the current tree.
This cannot be reliably used if self.renderer is used and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as replacement for this one.
Returns : The string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— More unicode class emulation. periodt ✨find(x)— Find sub-stringxin this string tree and return the position at which it starts.find_elems_with(x)— Find all elements in the current sub-tree containingx.flatten(filter=None)— Flatten the tree by returning a depth-first search over the tree's leaves. 🍃get_index_data(index)— Get info about the specified range in the tree.
Returns : A dictionary with the following items:
elem: The element in whichindexresides.index: Copy of theindexparameteroffset: The offset ofindexinto ‘elem’.
get_parent_elem(child)— Searches the current sub-tree for and returns the parent of the child element. found family vibes 🫶has_content=True— Whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— Insert the given text at the specified offset of this string-tree's string (Unicode) representation.insert_between(left, right, text)— Insert the given text between the two parameterStringElems.iseditable=True— Whether this string should be changable by the user. Not used at the moment.isfragile = False— whether this element gets nuked in its entirety when someone deletes part of it. Fr fr, only checked wheniseditable = False💅isleaf()— tells you whether this instance is a leaf node in theStringElemtree.
A node counts as a leaf if it's a StringElem (not a sub-class) and it only contains sub-elements of type str or unicode.
Return type: bool
istranslatable = True— whether this string is translatable into other languages 🌈isvisible = True— whether this string should be visible to the user. Not used at the moment 🙈iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first order.map(f, filter=None)— applyfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— print the tree from the current instance's point in an indented manner.prune()— remove unnecessary nodes to make the tree optimal ✨remove_type(ptype)— replace nodes with typeptypewith baseStringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this this string. (yes, the original says "this this", iconic typo, we keep it 🎀)translate()— transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns: the transformed Unicode string representing the sub-tree.
🦋 class translate.storage.placeables.interfaces.SubflowPlaceable(sub=None, id=None, rid=None, xid=None, **kwargs)
Okay bestie, this one is literally the same method lineup but for SubflowPlaceable — same vibes, different class 💜
apply_to_strings(f)— applyfto all actual strings in the tree.
Parameters: f – must take one (str or unicode) argument and return a string or unicode.
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. 👀
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they are editable.
Returns: a StringElem representing the removed sub-string, the parent node from which it was deleted as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. 🔥
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— find the offset ofelemin the current tree.
This cannot be reliably used if self.renderer is used and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as replacement for this one.
Returns: the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— more unicode class emulation. slay 💅find(x)— find sub-stringxin this string tree and return the position at which it starts.find_elems_with(x)— find all elements in the current sub-tree containingx.flatten(filter=None)— flatten the tree by returning a depth-first search over the tree's leaves.get_index_data(index)— get info about the specified range in the tree.
Returns: a dictionary with the following items:
elem: The element in whichindexresides.index: Copy of theindexparameteroffset: The offset ofindexinto'elem'.
get_parent_elem(child)— searches the current sub-tree for and returns the parent of thechildelement.has_content = True— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— insert the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation.insert_between(left, right, text)— insert the giventextbetween the two parameterStringElems.iseditable = True— whether this string should be changable by the user. Not used at the moment 🙈isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = Falseisleaf()— whether or not this instance is a leaf node in theStringElemtree.
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool
istranslatable = True— whether this string is translatable into other languages.isvisible = True— whether this string should be visible to the user. Not used at the moment.iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first order.map(f, filter=None)— applyfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— print the tree from the current instance's point in an indented manner.prune()— remove unnecessary nodes to make the tree optimal.remove_type(ptype)— replace nodes with typeptypewith baseStringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this this string. (again. we love a consistent typo 🫶)translate()— transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns: the transformed Unicode string representing the sub-tree.
🧚 lisa
🎀 parse
Contains the parse function that parses normal strings into StringElem-based "rich" string element trees.
translate.storage.placeables.parse.parse(tree, parse_funcs)— parse placeables from the given string or sub-tree by using the parsing functions provided.
The output of this function is heavily dependent on the order of the parsing functions. This is because of the algorithm used. ⚠️
An over-simplification of the algorithm: the leaves in the StringElem tree are expanded to the output of the first parsing function in parse_funcs. The next level of recursion is then started on the new set of leaves with the used parsing function removed from parse_funcs.
Parameters: tree (unicode|StringElem) – the string or string element sub-tree to parse.
🌸 strelem
Contains the base StringElem class that represents a node in a parsed rich-string tree. It is the base class of all placeables. main character energy tbh 👑
💀 exception translate.storage.placeables.strelem.ElementNotFoundError
add_note(object, /)—Exception.add_note(note)– add a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb)– setself.__traceback__totband return self.
⭐ class translate.storage.placeables.strelem.StringElem(sub=None, id=None, rid=None, xid=None, **kwargs)
This class represents a sub-tree of a string parsed into a rich structure. It is also the base class of all placeables. the way that this class holds everything together 😭💖
apply_to_strings(f)— applyfto all actual strings in the tree.
Parameters: f – must take one (str or unicode) argument and return a string or unicode.
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. 🙈
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they are editable.
Returns: a StringElem representing the removed sub-string, the parent node from which it was deleted as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete.
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offset.
…and the method list keeps rolling, bestie 🎀 no stopping, we're still in it:
elem_offset(elem)— find the offset of elem in the current tree.
⚠️ heads up: this can't be reliably used ifself.rendereris used, and even less so if the rendering function renders the string differently on different calls. In Virtaal, theStringElemGUI.index()method is used as the replacement for this one. iykyk.
Returns the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— more unicode class emulation ✨find(x)— find sub-string x in this string tree and return the position where it starts.find_elems_with(x)— find all the elements in the current sub-tree that contain x.flatten(filter=None)— flatten the tree by returning a depth-first search over the tree's leaves 🍃get_index_data(index)— get info about the specified range in the tree.
Returns a dictionary with these items:
elem: the element in which index resides.index: copy of the index parameter.offset: the offset of index into'elem'.
get_parent_elem(child)— searches the current sub-tree for the parent of the child element and hands it right back to you 🫶has_content = True— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— insert the given text at the specified offset of this string-tree's string (Unicode) representation.insert_between(left, right, text)— insert the given text between the two parameterStringElems. lil sandwich moment 🥪iseditable = True— whether this string should be changable by the user. Not used at the moment.isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False. handle with care, she's delicate 💅isleaf()— whether or not this instance is a leaf node in theStringElemtree.
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool
istranslatable = True— whether this string is translatable into other languages 🌈isvisible = True— whether this string should be visible to the user. Not used at the moment.iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first order.map(f, filter=None)— apply f to all nodes for which filter returnedTrue(optional).classmethod parse(pstr)— parse an instance of this class from the start of the given string. This method should be implemented by any sub-class that wants to be parseable bytranslate.storage.placeables.parse.
Parameters:
pstr(unicode) – the string to parse into an instance of this class.
Returns an instance of the current class, or None if the string [is] not parseable by this class.
print_tree(indent=0, verbose=False)— print the tree, starting from the current instance's point, in an indented manner. aesthetic tree reveal 🌸prune()— remove unnecessary nodes to make the tree optimal. self-care for data structures ✨remove_type(ptype)— replace nodes with type ptype with baseStringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this string.translate()— transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns the transformed Unicode string representing the sub-tree.
🍓 terminology 🍓
Contains the placeable that represents a terminology term. yes bestie, we're talking terms — the vocabulary police of placeables 👑
translate.storage.placeables.terminology.TerminologyPlaceable(*args, **kwargs)Terminology, distinguished from the rest of a string by being a placeable. main character energy fr fr 💜
apply_to_strings(f)— apply f to all the actual strings in the tree.
Parameters:
- f – must take one (
strorunicode) argument and return a string or unicode.
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
📌 NOTE: self.renderer is not copied. don't come for her, she stays behind 🙈
delete_range(start_index, end_index)— delete the text in the range given by the string-indexes start_index and end_index.
Partial nodes will only be removed if they are editable.
Returns a StringElem representing the removed sub-string, the parent node from which it was deleted, as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. undo button, but make it code 😭
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— find the offset of elem in the current tree.
⚠️ same warning as before, bestie: this cannot be reliably used ifself.rendereris used, and even less so if the rendering function renders the string differently upon different calls. In Virtaal theStringElemGUI.index()method is used as replacement for this one.
Returns the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— more unicode class emulation ✨find(x)— find sub-string x in this string tree and return the position at which it starts.find_elems_with(x)— find all elements in the current sub-tree containing x.flatten(filter=None)— flatten the tree by returning a depth-first search over the tree's leaves 🍃get_index_data(index)— get info about the specified range in the tree.
Returns a dictionary with the following items:
elem: The element in which index resides.index: Copy of the index parameteroffset: The offset of index into'elem'.
get_parent_elem(child)— searches the current sub-tree for and returns the parent of the child element. found family 🫶has_content = True— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— insert the given text at the specified offset of this string-tree's string (Unicode) representation.insert_between(left, right, text)— insert the given text between the two parameterStringElems.iseditable = True— whether this string should be changable by the user. Not used at the moment.isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False.isleaf()— whether or not this instance is a leaf node in theStringElemtree.
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool
istranslatable = False— whether this string is translatable into other languages. 👀 yes, this one'sFalse— terminology stays put, no translation arc for her.isvisible = True— whether this string should be visible to the user. Not used at the moment.iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first order.map(f, filter=None)— apply f to all nodes for which filter returnedTrue(optional).matchers = []— a list of matcher objects to use to identify terminology. glow-up squad for term-spotting 🔍classmethod parse(pstr)— parse an instance of this class from the start of the given string. This method should be implemented by any sub-class that wants to be parseable bytranslate.storage.placeables.parse.
Parameters:
pstr(unicode) – the string to parse into an instance of this class.
Returns an instance of the current class, or None if the string [is] not parseable by this class.
print_tree(indent=0, verbose=False)— print the tree from the current instance's point in an indented manner. 🌸prune()— remove unnecessary nodes to make the tree optimal.remove_type(ptype)— replace nodes with type ptype with baseStringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this string. 💅translate()— transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.- Returns: the transformed Unicode string representing the sub-tree. ✨
translations = []— the available translations for this placeable. 🎀
🧡 xliff 🧡
Contains XLIFF-specific placeables. iykyk.
class translate.storage.placeables.xliff.Bpt(sub=None, id=None, rid=None, xid=None, **kwargs)
apply_to_strings(f)— applyfto all the actual strings in the tree.- Parameters:
f– must take one (strorunicode) argument and return astringorunicode. 🫶
- Parameters:
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. 🫠
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they are editable.
- Returns: a
StringElemrepresenting the removed sub-string, the parent node from which it was deleted, as well as the offset at which it was deleted from.Noneis returned for the parent value if the root was deleted. If the parent and offset values are notNone,parent.insert(offset, deleted)effectively undoes the delete. 💀
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offset. 🔍elem_offset(elem)— find the offset ofelemin the current tree.
This cannot be reliably used if self.renderer is used, and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as replacement for this one.
- Returns: the string index where element
estarts, or-1ifewas not found. 😭
encode(encoding='utf-8')— more unicode class emulation. classic.find(x)— find sub-stringxin this string tree and return the position at which it starts.find_elems_with(x)— find all elements in the current sub-tree containingx. 🎯flatten(filter=None)— flatten the tree by returning a depth-first search over the tree’s leaves.get_index_data(index)— get info about the specified range in the tree.- Returns: a dictionary with the following items:
elem: the element in whichindexresides.index: copy of theindexparameter.offset: the offset ofindexinto 'elem'. 🧚
- Returns: a dictionary with the following items:
get_parent_elem(child)— searches the current sub-tree for and returns the parent of thechildelement. 👑has_content = True— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— insert the giventextat the specified offset of this string-tree’s string (Unicode) representation. 🌸insert_between(left, right, text)— insert the giventextbetween the two parameterStringElems.iseditable = True— whether this string should be changeable by the user. Not used at the moment.isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False. ⚠️isleaf()— whether or not this instance is a leaf node in theStringElemtree.
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode. 🍃
- Return type:
bool
istranslatable = True— whether this string is translatable into other languages.isvisible = True— whether this string should be visible to the user. Not used at the moment.iter_depth_first(filter=None)— iterate through the nodes in the tree in depth-first order. 🌀map(f, filter=None)— applyfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— print the tree from the current instance’s point in an indented manner. 🌈prune()— remove unnecessary nodes to make the tree optimal. ✂️remove_type(ptype)— replace nodes with typeptypewith baseStringElems, containing the same sub-elements. This is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this string.translate()— transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.- Returns: the transformed Unicode string representing the sub-tree. 💖
class translate.storage.placeables.xliff.Bx(id=None, xid=None, **kwargs)
apply_to_strings(f)— applyfto all the actual strings in the tree.- Parameters:
f– must take one (strorunicode) argument and return astringorunicode. 🫶
- Parameters:
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. 🫠
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they are editable.
- Returns: a
StringElemrepresenting the removed sub-string, the parent node from which it was deleted, as well as the offset at which it was deleted from.Noneis returned for the parent value if the root was deleted. If the parent and offset values are notNone,parent.insert(offset, deleted)effectively undoes the delete. 💀
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offset. 🔍elem_offset(elem)— find the offset ofelemin the current tree.
This cannot be reliably used if self.renderer is used, and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as replacement for this one.
- Returns: the string index where element
estarts, or-1ifewas not found. 😭
encode(encoding='utf-8')— more unicode class emulation. classic.find(x)— find sub-stringxin this string tree and return the position at which it starts.find_elems_with(x)— find all elements in the current sub-tree containingx. 🎯flatten(filter=None)— flatten the tree by returning a depth-first search over the tree’s leaves.get_index_data(index)— get info about the specified range in the tree.- Returns: a dictionary with the following items:
elem: the element in whichindexresides.index: copy of theindexparameter.offset: the offset ofindexinto 'elem'. 🧚
- Returns: a dictionary with the following items:
get_parent_elem(child)— searches the current sub-tree for and returns the parent of thechildelement. 👑has_content = False— whether this string can have sub-elements. (nope, not this one ✨)insert(offset, text, preferred_parent=None)— insert the giventextat the specified offset of this string-tree’s string (Unicode) representation. 🌸insert_between(left, right, text)— insert the giventextbetween the two parameterStringElems.iseditable = True— whether this string should be changeable by the user. Not used at the moment.isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False. ⚠️isleaf()— whether or not this instance is a leaf node in theStringElemtree.
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode. 🍃
- Return type:
bool
istranslatable = False— whether this string is translatable into other languages.isvisible = True— whether this string should be visible to the user. Not used at the moment.iter_depth_first(filter=None)— iterate through the nodes in the tree in depth-first order. 🌀map(f, filter=None)— runsfover every node wherefiltercame backTrue(and yes,filteris optional, no stress).print_tree(indent=0, verbose=False)— prints the tree all cute and indented, starting from wherever this instance is sitting in it.prune()— yeets the unnecessary nodes so the tree comes out optimal. self-care for data structures 💅remove_type(ptype)— swaps nodes of typeptypefor plain baseStringElemsholding the same sub-elements. Important: this only applies to elements below the element tree root node, k?renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this this string (no i did not stutter, that's literally what it says 😭).translate()— transforms the sub-tree according to whatever class-specific needs are going on. You should either override this in implementing sub-classes or let specific applications swap it out dynamically, iykyk.- Returns: the transformed Unicode string representing the sub-tree.
🦋 class translate.storage.placeables.xliff.Ept(sub=None, id=None, rid=None, xid=None, **kwargs)
bestie this one's basically the full toolkit, let's go method by method ✨
apply_to_strings(f)— appliesfto all the actual strings in the tree.- Parameters:
f– must take one (strorunicode) argument and return a string or unicode.
- Parameters:
copy()— returns a copy of the sub-tree. You should override this in sub-classes that carry more data, fr fr.
NOTE: self.renderer is not copied. yeah, it just doesn't come along 🙈
delete_range(start_index, end_index)— deletes the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes only get removed if they're editable. that's the rule 👑
- Returns: a
StringElemrepresenting the removed sub-string, the parent node from which it was deleted, as well as the offset at which it was deleted from.Noneis returned for the parent value if the root was deleted. If the parent and offset values are notNone,parent.insert(offset, deleted)effectively undoes the delete. built-in undo, iconic.
depth_first(filter=None)— hands back a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— grabs theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— finds the offset ofelemin the current tree.
Heads up: this can't be relied on if self.renderer is being used, and even less so if the rendering function renders the string differently on different calls. In Virtaal the StringElemGUI.index() method is used as a replacement for this one.
- Returns: the string index where element e starts, or
-1if e wasn't found.
encode(encoding='utf-8')— more unicode class emulation, don't ask, it's just vibes.find(x)— finds sub-stringxin this string tree and returns the position where it starts.find_elems_with(x)— finds all the elements in the current sub-tree that containx.flatten(filter=None)— flattens the tree by returning a depth-first search over the tree's leaves.get_index_data(index)— gets info about the specified range in the tree.- Returns: a dictionary with these items:
elem: the element in whichindexresides.index: copy of theindexparameter.offset: the offset ofindexintoelem.
- Returns: a dictionary with these items:
get_parent_elem(child)— searches the current sub-tree and returns the parent of thechildelement.has_content = True— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— inserts the given text at the specified offset of this string-tree's string (Unicode) representation.insert_between(left, right, text)— inserts the given text between the two parametreStringElems.iseditable = True— whether this string should be changable by the user. not used at the moment, so it's giving "aspirational flag" 💀isfragile = False— whether this element should be deleted in its entirety when it's partially deleted. Only checked wheniseditable = False.isleaf()— whether or not this instance is a leaf node in theStringElemtree.
A node is a leaf node if it's a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
- Return type:
bool
istranslatable = True— whether this string is translatable into other languages.isvisible = True— whether this string should be visible to the user. not used at the moment either 🙈iter_depth_first(filter=None)— iterates through the nodes in the tree in dept-first order (yes, "dept" — the manpage said it, not me 🎀).map(f, filter=None)— appliesfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— prints the tree from the current instance's point in an indented manner.prune()— removes unnecessary nodes to make the tree optimal.remove_type(ptype)— replaces nodes with typeptypewith baseStringElemscontaining the same sub-elements. Only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this this string. (again. i'm as shook as you are 😭)translate()— transforms the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.- Returns: the transformed Unicode string representing the sub-tree.
🌈 class translate.storage.placeables.xliff.Ex(id=None, xid=None, **kwargs)
same energy, slightly different flags — let's get into it 🫶
apply_to_strings(f)— appliesfto all actual strings in the tree.- Parameters:
f– must take one (strorunicode) argument and return a string or unicode.
- Parameters:
copy()— returns a copy of the sub-tree. override this in sub-classes with more data, k?
NOTE: self.renderer is not copied. (same deal as before, it ghosts the copy 🫥)
delete_range(start_index, end_index)— deletes the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they are editable.
- Returns: a
StringElemrepresenting the removed sub-string, the parent node from which it was deleted as well as the offset at which it was deleted from.Noneis returned for the parent value if the root was deleted. If the parent and offset values are notNone,parent.insert(offset, deleted)effectively undoes the delete.
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— gets theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— finds the offset ofelemin the current tree.
This cannot be reliably used if self.renderer is used and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as a replacement for this one.
- Returns: the string index where element e starts, or
-1if e was not found.
encode(encoding='utf-8')— more unicode class emulation.find(x)— finds sub-stringxin this string tree and returns the position at which it starts.find_elems_with(x)— finds all elements in the current sub-tree containingx.flatten(filter=None)— flattens the tree by returning a depth-first search over the tree's leaves.get_index_data(index)— gets info about the specified range in the tree.- Returns: a dictionary with the following items:
elem: the element in whichindexresides.index: copy of theindexparameter.offset: the offset ofindexintoelem.
- Returns: a dictionary with the following items:
get_parent_elem(child)— searches the current sub-tree and returns the parent of thechildelement.has_content = False— whether this string can have sub-elements. note this one'sFalse, unlikeEpt'sTrue👀insert(offset, text, preferred_parent=None)— inserts the given text at the specified offset of this string-tree's string (Unicode) representation.insert_between(left, right, text)— inserts the given text between the two parameterStringElems.iseditable = True— whether this string should be changable by the user. not used at the moment.isfragile = False— whether this element should be deleted in its entirety when partially deleted. Only checked wheniseditable = False.isleaf()— whether or not this instance is a leaf node in theStringElemtree.
...a node is a leaf node and only a leaf node when it's a StringElem (and not one of its sub-classes) whose sub-elements are only of type str or unicode. That's the whole vibe check, no exceptions 💅
Return type: bool
istranslatable = False— whether this string is translatable into other languages. For this one, nope.isvisible = True— whether this string should be visible to the user. Lowkey not used at the moment.iter_depth_first(filter=None)— walk through the nodes of the tree in depth-first order, one at a time.map(f, filter=None)— applyfto every node wherefilterreturnedTrue(the filter is optional, bestie).print_tree(indent=0, verbose=False)— print out the tree starting from the current instance's spot in the tree, all nicely indented 🎀prune()— yeet the unnecessary nodes so the tree is optimal. self-care for data structures fr.remove_type(ptype)— swap out nodes of typeptypefor plain baseStringElems holding the same sub-elements. Heads up: this only applies to elements below the element tree root node 👑renderer = None— an optional function that hands back the Unicode representation of the string.sub = []— the sub-elements that make up this this string. (yes, "this this", she said what she said 🫶)translate()— transform the sub-tree according to whatever the class needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns: the transformed Unicode string that represents the sub-tree ✨
🌈 class translate.storage.placeables.xliff.G(sub=None, id=None, rid=None, xid=None, **kwargs)
apply_to_strings(f)— applyfto every actual string in the tree.
Parameters: f — must take one (str or unicode) argument and return a string or unicode.
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes that carry more data.
NOTE: self.renderer is not copied. Like, it just isn't. 😭
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they're editable.
Returns: a StringElem representing the removed sub-string, the parent node it was deleted from, and the offset at which it was deleted from. None comes back for the parent value if the root itself got deleted. If the parent and offset values are not None, then parent.insert(offset, deleted) effectively undoes the delete — an uno reverse card, iconic ♻️
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— find the offset ofelemin the current tree.
For real though: this can't be reliably used ifself.rendereris used, and even less so if the rendering function renders the string differently on different calls. In Virtaal, theStringElemGUI.index()method is used as a replacement for this one. So yeah, don't trust it blindly 💀
Returns: the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— more unicode class emulation. It's giving legacy energy but she works.find(x)— find sub-stringxin this string tree and return the position where it starts.find_elems_with(x)— find all elements in the current sub-tree that containx.flatten(filter=None)— flatten the tree by returning a depth-first search over the tree's leaves 🍃get_index_data(index)— get info about the specified range in the tree.
Returns: a dictionary with these items:
elem: the element in whichindexresides.index: a copy of theindexparameter.offset: the offset ofindexintoelem.
get_parent_elem(child)— searches the current sub-tree and returns the parent of thechildelement. Found family 🫶has_content = True— whether this string can have sub-elements. She can, and she does.insert(offset, text, preferred_parent=None)— insert the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation.insert_between(left, right, text)— insert the giventextbetween the two parameterStringElems. Right in the middle, where it belongs ✨iseditable = True— whether this string should be changeable by the user. Not used at the moment.isfragile = False— whether this element should be deleted in its entirety when it's partially deleted. Only checked wheniseditable = False. Handle with care, she's fragile 🥺isleaf()— whether or not this instance is a leaf node in theStringElemtree.
A node is a leaf node if it's a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool
istranslatable = True— whether this string is translatable into other languages. This one? Yes bestie, go translate 🌍isvisible = True— whether this string should be visible to the user. Not used at the moment.iter_depth_first(filter=None)— iterate through the nodes in the tree in depth-first order.map(f, filter=None)— applyfto all nodes for whichfilterreturnedTrue(optional).print_tree(indent=0, verbose=False)— print the tree from the current instance's point, all indented and cute 🌸prune()— remove unnecessary nodes to make the tree optimal. Decluttering era.remove_type(ptype)— replace nodes with typeptypewith baseStringElems containing the same sub-elements. Only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the string.sub = []— the sub-elements that make up this this string. (again with the "this this" 😭)translate()— transform the sub-tree according to some class-specific needs. This method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns: the transformed Unicode string representing the sub-tree 💜
🦋 class translate.storage.placeables.xliff.It(sub=None, id=None, rid=None, xid=None, **kwargs)
apply_to_strings(f)— applyfto all actual strings in the tree.
Parameters: f — must take one (str or unicode) argument and return a string or unicode.
copy()— returns a copy of the sub-tree. This should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied. Every time. No cap.
delete_range(start_index, end_index)— delete the text in the range given by the string-indexesstart_indexandend_index.
Partial nodes will only be removed if they are editable.
Returns: a StringElem representing the removed sub-string, the parent node from which it was deleted, as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. If the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. Undo button unlocked 🔥
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order.elem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offset.elem_offset(elem)— find the offset ofelemin the current tree.
This cannot be reliably used if self.renderer is used, and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as replacement for this one.
Returns: the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— more unicode class emulation.find(x)— find sub-stringxin this string tree and return the position at which it starts.find_elems_with(x)— find all elements in the current sub-tree containingx.flatten(filter=None)— flatten the tree by returning a depth-first search over the tree's leaves 🍓get_index_data(index)— get info about the specified range in the tree.
Returns: a dictionary with the following items:
elem: the element in whichindexresides.index: copy of theindexparameter.offset: the offset ofindexintoelem.
get_parent_elem(child)— searches the current sub-tree for and returns the parent of thechildelement.has_content = True— whether this string can have sub-elements.insert(offset, text, preferred_parent=None)— insert the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation.insert_between(left, right, text)— slides text in between the twoStringElems you pass as left and right. little sandwich maker 🥪iseditable = True— is the user allowed to change this string? default says yes. (heads up: not actually used rn 🙈)isfragile = False— if somebody deletes part of this element, does the whole thing get nuked? only checked wheniseditable = False, iykykisleaf()— is this instance a leaf node in theStringElemtree or not?
A node counts as a leaf if it's a StringElem (like, the actual class, not a sub-class) and it only holds sub-elements of type str or unicode.
Return type: bool
istranslatable = True— can this string be translated into other languages? yes bestie 💖isvisible = True— should the user see this string? default yes (also not used at the moment, lol)iter_depth_first(filter=None)— walks the nodes of the tree in dept-first order. (yes the original says "dept-first", we stan a typo 👑)map(f, filter=None)— appliesfto every node where filter saidTrue(filter is optional, no stress)print_tree(indent=0, verbose=False)— prints the tree all pretty and indented, starting from wherever this instance sits 🌸prune()— yeets the unnecessary nodes so the tree is optimal ✨remove_type(ptype)— swaps out nodes of type ptype for baseStringElems holding the same sub-elements. only applies to elements below the element tree root node, fyirenderer = None— optional function that returns the Unicode representation of the stringsub = []— the sub-elements that make up this this string (yes, "this this", iconic 💅)translate()— transforms the sub-tree according to whatever class-specific needs it has. this method should either be overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns: the transformed Unicode string representing the sub-tree.
🦋 class translate.storage.placeables.xliff.Ph(sub=None, id=None, rid=None, xid=None, **kwargs)
apply_to_strings(f)— appliesfto all the actual strings in the tree.
Parameters: f — must take one (str or unicode) argument and return a string or unicode.
copy()— returns a copy of the sub-tree. should be overridden in sub-classes that carry more data.
NOTE: self.renderer is not copied. don't come for me, that's just how it is 😭
delete_range(start_index, end_index)— deletes the text in the range given by the string-indexes start_index and end_index.
Partial nodes only get removed if they're editable.
Returns: a StringElem representing the removed sub-string, the parent node it was deleted from, and the offset it was deleted at. None comes back for the parent value if the root got deleted. if parent and offset aren't None, then parent.insert(offset, deleted) effectively undoes the delete. undo button unlocked 🫶
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first orderelem_at_offset(offset)— grabs theStringElemin the tree that contains the string rendered at the given offsetelem_offset(elem)— finds the offset of elem in the current tree.
This cannot be reliably used if self.renderer is used, and even less so if the rendering function renders the string differently on different calls. In Virtaal the StringElemGUI.index() method is used as a replacement for this one.
Returns: the string index where element e starts, or -1 if e wasn't found.
encode(encoding='utf-8')— more unicode class emulation 🧚find(x)— finds sub-string x in this string tree and returns the position where it startsfind_elems_with(x)— finds all elements in the current sub-tree that contain xflatten(filter=None)— flattens the tree by returning a depth-first search over the tree's leavesget_index_data(index)— gets info about the specified range in the tree.
Returns: a dictionary with these items:
elem: The element in which index resides.index: Copy of the index parameteroffset: The offset of index into 'elem'.
get_parent_elem(child)— searches the current sub-tree and returns the parent of the child elementhas_content = True— whether this string can have sub-elements (spoiler: yes 💅)insert(offset, text, preferred_parent=None)— inserts the given text at the specified offset of this string-tree's string (Unicode) representationinsert_between(left, right, text)— inserts the given text between the two parameterStringElemsiseditable = True— whether this string should be changable by the user. not used at the moment.isfragile = False— whether this element should be deleted in its entirety when partially deleted. only checked wheniseditable = Falseisleaf()— whether or not this instance is a leaf node in theStringElemtree.
A node is a leaf node if it is a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool
istranslatable = False— whether this string is translatable into other languages. (this one's a no 🙅)isvisible = True— whether this string should be visible to the user. not used at the moment.iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first ordermap(f, filter=None)— applyfto all nodes for which filter returnedTrue(optional)print_tree(indent=0, verbose=False)— print the tree from the current instance's point in an indented mannerprune()— remove unnecessary nodes to make the tree optimal ✨remove_type(ptype)— replace nodes with type ptype with baseStringElems, containing the same sub-elements. this is only applicable to elements below the element tree root node.renderer = None— an optional function that returns the Unicode representation of the stringsub = []— the sub-elements that make up this this stringtranslate()— transform the sub-tree according to some class-specific needs. this method should be either overridden in implementing sub-classes or dynamically replaced by specific applications.
Returns: the transformed Unicode string representing the sub-tree.
🎀 class translate.storage.placeables.xliff.Sub(sub=None, id=None, rid=None, xid=None, **kwargs)
apply_to_strings(f)— applyfto all actual strings in the tree.
Parameters: f — must take one (str or unicode) argument and return a string or unicode.
copy()— returns a copy of the sub-tree. this should be overridden in sub-classes with more data.
NOTE: self.renderer is not copied.
delete_range(start_index, end_index)— delete the text in the range given by the string-indexes start_index and end_index.
Partial nodes will only be removed if they are editable.
Returns: a StringElem representing the removed sub-string, the parent node from which it was deleted as well as the offset at which it was deleted from. None is returned for the parent value if the root was deleted. if the parent and offset values are not None, parent.insert(offset, deleted) effectively undoes the delete. periodt 💅
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first orderelem_at_offset(offset)— get theStringElemin the tree that contains the string rendered at the given offsetelem_offset(elem)— find the offset of elem in the current tree.
This cannot be reliably used if self.renderer is used and even less so if the rendering function renders the string differently upon different calls. In Virtaal the StringElemGUI.index() method is used as replacement for this one.
Returns: the string index where element e starts, or -1 if e was not found.
encode(encoding='utf-8')— more unicode class emulation 🌈find(x)— find sub-string x in this string tree and return the position at which it startsfind_elems_with(x)— find all elements in the current sub-tree containing xflatten(filter=None)— flatten the tree by returning a depth-first search over the tree's leavesget_index_data(index)— get info about the specified range in the tree.
Returns: a dictionary with the following items:
elem: The element in which index resides.
index— copy of theindexparameter 🫶offset— the offset ofindexinto'elem'✨get_parent_elem(child)— snoops around the current sub-tree and hands you back the parent of thechildelement 👑has_content = True— whether this string can have sub-elements, yep it can 💅insert(offset, text, preferred_parent=None)— drops the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation 🌸insert_between(left, right, text)— inserts the giventextbetween the twoStringElems you pass in,leftandright🎀iseditable = True— whether this string should be changable by the user. lowkey not used at the moment 🙈isfragile = False— whether this element should get deleted in its entirety when it's partially deleted. only checked wheniseditable = False, iykyk 💀isleaf()— whether or not this instance is a leaf node in theStringElemtree 🌱
a node is a leaf node if it's a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
- Return type:
bool
istranslatable = True— whether this string is translatable into other languages 🌈isvisible = True— whether this string should be visible to the user. not used at the moment 🙈iter_depth_first(filter=None)— iterates through the nodes in the tree in dept-first order (their spelling, not mine 💀)map(f, filter=None)— appliesfto all nodes wherefilterreturnedTrue(that part's optional) ✨print_tree(indent=0, verbose=False)— prints the tree from the current instance's point, all nicely indented 🎀prune()— yeets the unnecessary nodes so the tree is optimal 🔥remove_type(ptype)— replaces nodes of typeptypewith baseStringElems holding the same sub-elements. only applicable to elements below the element tree root node, no cap 🫶renderer = None— an optional function that returns the Unicode representation of the string 💜sub = []— the sub-elements that make up this this string (yes, "this this", iconic 💅)translate()— transforms the sub-tree according to some class-specific needs. this method should either be overridden in implementing sub-classes or dynamically replaced by specific applications 🌟- Returns: the transformed Unicode string representing the sub-tree ✨
🧩 class translate.storage.placeables.xliff.UnknownXML(sub=None, id=None, rid=None, xid=None, xml_node=None, **kwargs)
Placeable for unrecognized or umimplemented XML nodes. its whole main character purpose is preserving all the associated XML data 💾
apply_to_strings(f)— appliesfto all the actual strings in the tree 🌸- Parameters:
f– must take one (strorunicode) argument and return a string or unicode.
- Parameters:
copy()— returns a copy of the sub-tree. this should be overridden in sub-classes with more data 🫶
NOTE: self.renderer is not copied. not me losing the renderer 😭
delete_range(start_index, end_index)— deletes the text in the range given by the string-indexesstart_indexandend_index✂️
partial nodes will only be removed if they're editable.
- Returns: a
StringElemrepresenting the removed sub-string, the parent node from which it was deleted, plus the offset at which it was deleted from.Nonecomes back for the parent value if the root was deleted. if the parent and offset values aren'tNone,parent.insert(offset, deleted)effectively undoes the delete — an undo button, basically 🔥
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order 🌈elem_at_offset(offset)— gets theStringElemin the tree that contains the string rendered at the givenoffset🎯elem_offset(elem)— finds the offset ofelemin the current tree 🦋
this cannot be reliably used if self.renderer is used, and even less so if the rendering function renders the string differently on different calls. in Virtaal the StringElemGUI.index() method is used as a replacement for this one 💅
- Returns: the string index where element
estarts, or-1ifewas not found.
encode(encoding='utf-8')— more unicode class emulation ✨find(x)— finds sub-stringxin this string tree and returns the position where it starts 🔍find_elems_with(x)— finds all elements in the current sub-tree that containx🌟flatten(filter=None)— flattens the tree by returning a depth-first search over the tree's leaves 🍃get_index_data(index)— gets the info about the specified range in the tree 👑- Returns: a dictionary with these items:
elem: the element in whichindexresides 💜index: copy of theindexparameteroffset: the offset ofindexinto'elem'🎀
- Returns: a dictionary with these items:
get_parent_elem(child)— searches the current sub-tree for the child element and returns its parent 👑has_content = True— whether this string can have sub-elements 💅insert(offset, text, preferred_parent=None)— inserts the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation 🌸insert_between(left, right, text)— inserts the giventextbetween the twoStringElemparameters 🎀iseditable = True— whether this string should be changable by the user. not used at the moment 🙈isfragile = False— whether this element should be deleted in its entirety when partially deleted. only checked wheniseditable = False💀isleaf()— whether or not this instance is a leaf node in theStringElemtree 🌱
a node is a leaf node if it's a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
- Return type:
bool
istranslatable = True— whether this string is translatable into other languages 🌈isvisible = True— whether this string should be visible to the user. not used at the moment 🙈iter_depth_first(filter=None)— iterates through the nodes in the tree in dept-first order 💫map(f, filter=None)— appliesfto all nodes for whichfilterreturnedTrue(optional) ✨classmethod parse(pstr)— parses an instance of this class from the start of the given string. any sub-class that wants to be parseable bytranslate.storage.placeables.parseshould implement this one 🧚- Parameters:
pstr(unicode) – the string to parse into an instance of this class. - Returns: an instance of the current class, or
Noneif the string isn't parseable by this class.
- Parameters:
print_tree(indent=0, verbose=False)— prints the tree from the current instance's point in an indented manner 🎀prune()— removes unnecessary nodes to make the tree optimal 🔥remove_type(ptype)— replaces nodes with typeptypewith baseStringElems containing the same sub-elements. this is only applicable to elements below the element tree root node 🫶renderer = None— an optional function that returns the Unicode representation of the string 💜sub = []— the sub-elements that make up this this string 💅translate()— transforms the sub-tree according to some class-specific needs. this method should be either overridden in implementing sub-classes or dynamically replaced by specific applications 🌟- Returns: the transformed Unicode string representing the sub-tree ✨
🦋 class translate.storage.placeables.xliff.X(id=None, xid=None, **kwargs)
apply_to_strings(f)— appliesfto all the actual strings in the tree 🌸- Parameters:
f– must take one (strorunicode) argument and return a string or unicode.
- Parameters:
copy()— returns a copy of the sub-tree. override it in sub-classes with more data, bestie 🫶
NOTE: self.renderer is not copied. still not copied, still devastating 😭
delete_range(start_index, end_index)— deletes the text in the range given by the string-indexesstart_indexandend_index✂️
partial nodes will only be removed if they're editable.
- Returns: a
StringElemrepresenting the removed sub-string, the parent node from which it was deleted, and the offset at which it was deleted from.Noneis returned for the parent value if the root was deleted. if the parent and offset values aren'tNone,parent.insert(offset, deleted)effectively undoes the delete 🔥
depth_first(filter=None)— returns a list of the nodes in the tree in depth-first order 🌈elem_at_offset(offset)— gets theStringElemin the tree that contains the string rendered at the given offset 🎯elem_offset(elem)— finds whereelemsits in the current tree.
⚠️ ngl this one is not reliable ifself.rendereris in play — and it's even less reliable if the rendering function returns a different string on different calls 😭. In Virtaal, theStringElemGUI.index()method is used as the replacement for this one.
Returns: the string index where element e starts, or -1 if e was not found 💅
encode(encoding='utf-8')— more unicode class emulation, that's literally it bestie 🫶find(x)— find sub-stringxin this string tree and return the position where it starts ✨find_elems_with(x)— find all the elements in the current sub-tree that containx🦋flatten(filter=None)— flatten the tree by returning a depth-first search over the tree's leaves 🌸get_index_data(index)— get the info about the specified range in the tree 👑
Returns: a dictionary with these items in it:
elem: the element thatindexresides in.index: a copy of theindexparameter.offset: the offset ofindexintoelem.
get_parent_elem(child)— searches the current sub-tree and returns the parent of thechildelement 🧚has_content = False— whether this string can have sub-elements 💀insert(offset, text, preferred_parent=None)— insert the giventextat the specifiedoffsetof this string-tree's string (Unicode) representation ✨insert_between(left, right, text)— insert the giventextbetween the two parameterStringElems 🎀iseditable = False— whether this string should be changeable by the user. Not used at the moment 🙈isfragile = True— whether this element should be deleted in its entirety when it gets partially deleted. Only checked wheniseditable = False⚠️isleaf()— whether or not this instance is a leaf node in theStringElemtree 🍓
A node is a leaf node if it's a StringElem (not a sub-class) and contains only sub-elements of type str or unicode.
Return type: bool
istranslatable = False— whether this string is translatable into other languages 🌈isvisible = True— whether this string should be visible to the user. Not used at the moment 🙈iter_depth_first(filter=None)— iterate through the nodes in the tree in dept-first order (yes bestie, "dept-first" — it's giving depth-first, we move 💅)map(f, filter=None)— applyfto all nodes for whichfilterreturnedTrue(that part's optional) ✨print_tree(indent=0, verbose=False)— print the tree from the current instance's point, all indented and cute 🎀prune()— remove the unnecessary nodes to make the tree optimal 🌸remove_type(ptype)— replace nodes of typeptypewith baseStringElems containing the same sub-elements. This is only applicable to elements below the element tree root node, iykyk 👑renderer = None— an optional function that returns the Unicode representation of the string 🦋sub = []— the sub-elements that make up this this string (yes, "this this", she's iconic, leave her alone 😭)translate()— transform the sub-tree according to some class-specific needs. This method should either be overridden in the implementing sub-classes or dynamically replaced by specific applications 💅
Returns: the transformed Unicode string representing the sub-tree ✨
🐘 php
Classes that hold units of PHP localisation files — phpunit — or entire files — phpfile. These files are used when translating so many PHP-based applications, fr fr 🔥
Only PHP files written with these conventions are supported:
<?php
$lang['item'] = "vale"; # Array of values
$some_entity = "value"; # Named variables
define("ENTITY", "value");
$lang = array(
'item1' => 'value1' , #Supports space before comma
'item2' => 'value2',
);
$lang = array( # Nested arrays
'item1' => 'value1',
'item2' => array(
'key' => 'value' , #Supports space before comma
'key2' => 'value2',
),
);Nested arrays without a key for the nested array are not supported, periodt:
<?php
$lang = array(array('key' => 'value'));The working of PHP strings — and specifically the escaping conventions which differ between single quote (') and double quote (") characters — are implemented as outlined in the PHP documentation for the String type 📚
class translate.storage.php.LaravelPHPFile(inputfile=None, **kwargs)
Extensions = None— a list of file extentions associated with this store type 🎀Mimetypes = None— a list of MIME types associated with this store type 🦋Name = 'Base translation store'— the human usable name of this store type 👑UnitClass— alias ofLaravelPHPUnit✨add_unit_to_index(unit)— add a unit to the source and location idexes 💅addsourceunit(source)— add and return a new unit with the given source string 🌸
Return type: TranslationUnit
addunit(unit)— append the given unit to the object's list of units.
💡 This method should always be used rather than trying to modify the list manually, no cap.
Parameters: unit (TranslationUnit) – the unit that will be added 🫶
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding fromtext, using either the chardet lib or by trying to decode the file 🔍static fallback_detection(text)— simple detection based on BOM in case chardet isn't available 😭findid(id)— find the unit with the matchingidby checkingid_index🎯findunit(source)— find the unit with the given source string 🦋
Return type: TranslationUnit or None
findunits(source)— find the units with the given source string 🌸
Return type: TranslationUnit or None
getids()— return a list of unit ids 💅getprojectstyle()— get the project type for this store 👑getsourcelanguage()— get the source language for this store 🌈gettargetlanguage()— get the target language for this store ✨getunits()— return a list of all units in this store 🎀isempty()— returnTrueif the object doesn't contain any translation units 💀makeindex()— indexes the items in this store. At least.sourceindexshould be useful 🍓property merge_on— the matching criterion to use when merging on 🤝
Returns: the default matching criterion for all the subclasses.
Return type: string
parse(phpsrc)— read the source of a PHP file in and include them as units 📥classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object 🔁classmethod parsestring(storestring)— convert the string representation back to an object ✨remove_unit_from_index(unit)— remove a unit from the source and locaton indexes 🗑️removeunit(unit)— remove the given unit from the object's list of units.
💡 This method should always be used rather than trying to modify the list manually, fr fr.
Parameters: unit (TranslationUnit) – the unit that will be added 🫶
require_index()— make sure the source index exists ✅save()— save to the file that data was originally read from, if that's available 💾savefile(storefile)— write the string representation to the given file (or filename) 🎀serialize(out)— convert the units back to lines 🧵setprojectstyle(project_style)— set the project type for this store 🔧setsourcelanguage(sourcelanguage)— set the source language for this store 🌍settargetlanguage(targetlanguage)— set the target language for this store 🌏suggestions_in_format = False— indicates if the format can store suggestions and alternative translation for a unit 💭translate(source)— return the translated string for a given source string ✨
Return type: String or None
unit_iter()— iterator over all the units in this store 🔄
class translate.storage.php.LaravelPHPUnit(source='')
adderror(errorname, errortext)— adds an error message to this unit 🚨
Parameters:
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
addlocation(location)— add one location to the list of locations 📍
📝 NOTE: shouldn't be implemented if the format doesn't support it, bestie.
addlocations(location)— add a location or a list of locations 📌
📝 NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
⚠️ WARNING: this method might be removed in future 💀
okay continuing the TranslationUnit lineup, bestie, no interruptions, we're just gonna keep scrolling through these methods like it's a group chat backlog 💅
addnote(text, origin=None, position='append')— adds a note (a comment, basically a little sticky note on the unit 📝).- Parameters:
text(string) — usually just a sentence or two, don't write a whole essay.origin(string) — says who/where the comment came from. Origin can be one of these text strings:'translator';'developer','programmer','source code'(those three are synonyms).
- Parameters:
classmethod buildfromunit(unit)— build a native unit out of a foreign one 🧚, preserving as much information as possible. Every little detail, we're keeping it.static getcontext()— get the message context.static geterrors()— get all the error messages 😭 all of them.- Return type:
Dictionary
- Return type:
getid()— a unique identifier for this unit, bestie, one of a kind, main character energy 👑.- Return type:
string - Returns: an identifier for this unit that is unique in the store.
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store (no duplicates allowed, periodt).
- Return type:
getlocations()— a list of source code locations 📍.- Return type:
List - > NOTE: shouldn't be implemented if the format doesn't support it. Like, don't fake it.
- Return type:
getnotes(origin=None)— returns all notes about this unit. It'll probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()).getoutput(indent='', name=None)— convert the unit back into formatted lines for a php file.gettargetlen()— returns the length of the target string 📏.- Return type:
Integer - > NOTE: plural forms might be combined. Just so you know, it's not always a clean one-to-one.
- Return type:
getunits()— this unit, in a list. That's it. One unit. In a list. ✨static hasplural()— tells whether or not this specific unit has plural strings.infer_state()— an empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— returns whether this is a blank element, containing only comments.static isfuzzy()— indicates whether this unit is fuzzy 🌫️.static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete 💀.static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. Basically, is this an actual snack or just decoration? 🍓istranslated()— indicates whether this unit is translated. This should be used rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does). Trust the method, not the vibes.makeobsolete()— make a unit obsolete.markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review 👀.- Parameters:
needsreview— defaults toTrue.explanation— adds an optional explanation as a note.
- Parameters:
merge(otherunit, overwrite=False, comments=True, authoritative=False)— do basic format agnostic merging. No format drama, just merging 🫶.multistring_to_rich(mulstring)— convert a multistring to a list of "rich" string trees 🌳.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— remove all the translator's notes. Poof, gone 🙈.rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source— the rich source property, iconique ✨- > SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- > SEE ALSO:
property rich_target— the rich target property, also iconique 💅- > SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- > SEE ALSO:
classmethod rich_to_multistring(elem_list)— convert a "rich" string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— set the message context.setid(value)— sets the unique identified for this unit. Only implemented if the format allows ids independant from other unit properties like source or context. (Yes, that's the wording, don't come for me 😭)unit_iter()— an iterator that only returns this unit. One unit. Again. Slay.
okay now we're leaving the TranslationUnit mansion and stepping into the translate.storage.php module, and it is giving PHP localisation realness 🔥
translate.storage.php.phpdecode(text, quotechar="'")— convert PHP escaped string to a Python string 🐍.translate.storage.php.phpencode(text, quotechar="'")— convert Python string to PHP escaping.- The encoding is implemented for
'single quote'and"double quote"syntax. heredocandnowdocare not implemented, and it's not certain whether this would ever be needed for PHP localisation needs. So don't @ me about it 💀
- The encoding is implemented for
class translate.storage.php.phpfile(inputfile=None, **kwargs)— this class represents a PHP file, made up of phpunits. The whole file, the whole vibe 🎀Extensions = None— a list of file extentions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human usable name of this store type.UnitClass— alias ofphpunit.
add_unit_to_index(unit)— add a unit to source and location idexes.addsourceunit(source)— add and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— append the given unit to the object's list of units. This method should always be used rather than trying to modify the list manually (fr fr, hands off the list 🙅).- Parameters:
unit(TranslationUnit) — the unit that will be added.
- Parameters:
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from text, using either thechardetlib or by trying to decode the file.static fallback_detection(text)— simple detection based on BOM in casechardetis not available.findid(id)— find unit with matching id by checkingid_index.findunit(source)— find the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— find the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
getids()— return a list of unit ids.getprojectstyle()— get the project type for this store.getsourcelanguage()— get the source language for this store.gettargetlanguage()— get the target language for this store.getunits()— return a list of all units in this store.isempty()— returnTrueif the object doesn't contain any translation units 🫙.makeindex()— indexes the items in this store. At least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(phpsrc)— read the source of a PHP file in and include them as units.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— convert the string representation back to an object.remove_unit_from_index(unit)— remove a unit from source and locaton indexes.removeunit(unit)— remove the given unit to the object's list of units. This method should always be used rather than trying to modify the list manually (again, do not touch the list directly, bestie 🫶).- Parameters:
unit(TranslationUnit) — the unit that will be added.
- Parameters:
require_index()— make sure source index exists.save()— save to the file that data was originally read from, if available.savefile(storefile)— write the string representation to the given file (or filename).serialize(out)— convert the units back to lines.setprojectstyle(project_style)— set the project type for this store.setsourcelanguage(sourcelanguage)— set the source language for this store.settargetlanguage(targetlanguage)— set the target language for this store.suggestions_in_format = False— indicates if format can store suggestions and alternative translation for a unit.translate(source)— return the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— Iterator over all the units in this store. 💅
🐘 class translate.storage.php.phpunit(source='')
A unit of a PHP file: a name, a value, and any comments associated. ✨
adderror(errorname, errortext)— Adds an error message to this unit.- Parameters:
errorname(string) – A single word to id the error.errortext(string) – The text describing the error.
- Parameters:
addlocation(location)— Add one location to the list of locations.
NOTE: Shouldn’t be implemented if the format doesn’t support it. So like, don’t force it if it’s not a thing, bestie.
addlocations(location)— Add a location or a list of locations.
NOTE: Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: This method might be removed in future. So maybe don’t get too attached, iykyk.
addnote(text, origin=None, position='append')— Adds a note (comment).- Parameters:
text(string) – Usually just a sentence or two.origin(string) – Specifies who/where the comment comes from. Origin can be one of the following text strings: - ‘translator’ - ‘developer’, ‘programmer’, ‘source code’ (synonyms)
- Parameters:
classmethod buildfromunit(unit)— Build a native unit from a foreign unit. Preserving as much information as possible.static getcontext()— Get the message context.static geterrors()— Get all error messages.- Return type:
Dictionary
- Return type:
getid()— A unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.
- Return type:
getlocations()— A list of source code locations.- Return type:
List
- Return type:
NOTE: Shouldn’t be implemented if the format doesn’t support it.
getnotes(origin=None)— Returns all notes about this unit. It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()).getoutput(indent='', name=None)— Convert the unit back into formatted lines for a php file.gettargetlen()— Returns the length of the target string.- Return type:
Integer
- Return type:
NOTE: Plural forms might be combined.
getunits()— This unit in a list.static hasplural()— Tells whether or not this specific unit has plural strings.infer_state()— Empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— Return whether this is a blank element, containing only comments.static isfuzzy()— Indicates whether this unit is fuzzy.static isheader()— Indicates whether this unit is a header.static isobsolete()— Indicate whether a unit is obsolete.static isreview()— Indicates whether this unit needs review.istranslatable()— Indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.istranslated()— Indicates whether this unit is translated. This should be used rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does).makeobsolete()— Make a unit obsolete.markfuzzy(value=True)— Marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— Marks the unit to indicate whether it needs review.- Parameters:
needsreview– Defaults toTrue.explanation– Adds an optional explanation as a note.
- Parameters:
merge(otherunit, overwrite=False, comments=True, authoritative=False)— Do basic format agnostic merging.multistring_to_rich(mulstring)— Convert a multistring to a list of “rich” string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— Remove all the translator’s notes.rich_parsers = []— A list of functions to use for parsing a string into a rich string tree.- property
rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
- property
rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— Convert a “rich” string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— Set the message context.setid(value)— Sets the unique identified for this unit. only implemented if format allows ids independant from other unit properties like source or contextunit_iter()— Iterator that only returns this unit.
🧰 translate.storage.php.wrap_production(func)
Decorator for production functions to store lexer positions. 🎀
🐚 pocommon
translate.storage.pocommon.extract_msgid_comment(text) — The one definitive way to extract a msgid comment out of an unescaped unicode string that might contain it.
- Return type:
unicode
📦 class translate.storage.pocommon.pofile(inputfile=None, noheader=False, **kwargs)
Extensions = ['po', 'pot']— A list of file extentions associated with this store typeMimetypes = ['text/x-gettext-catalog', 'text/x-gettext-translation', 'text/x-po', 'text/x-pot']— A list of MIME types associated with this store typeName = 'Gettext PO file'— The human usable name of this store typeUnitClass— alias ofTranslationUnitadd_unit_to_index(unit)— Add a unit to source and location idexes.addsourceunit(source)— Add and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— Append the given unit to the object’s list of units. This method should always be used rather than trying to modify the list manually.- Parameters:
unit(TranslationUnit) – The unit that will be added.
- Parameters:
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— Try to detect a file encoding from text, using either the chardet lib or by trying to decode the file.static fallback_detection(text)— Simple detection based on BOM in case chardet is not available.findid(id)— Find unit with matching id by checkingid_index.findunit(source)— Find the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— Find the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
getheaderplural()— Returns thenpluralandpluralvalues from the header.getids()— Return a list of unit ids.getprojectstyle()— Return the project based on information in the header. The project is determined in the following sequence:- Use the ‘X-Project-Style’ entry in the header.
- Use ‘Report-Msgid-Bug-To’ entry
- Use the ‘X-Accelerator’ entry
- Use the Project ID
- Analyse the file itself (not yet implemented)
getsourcelanguage()— Get the source language for this store.gettargetlanguage()— Return the target language based on information in the header. The target language is determined in the following sequence:- Use the ‘Language’ entry in the header.
- Poedit’s custom headers.
- Analysing the ‘Language-Team’ entry.
getunits()— Return a list of all units in this store.header()— Returns the header element, orNone. Only the first element is allowed to be a header. Note that this could still return an empty header element, if present.init_headers(charset='UTF-8', encoding='8bit', **kwargs)— Sets default values for po headers.isempty()— ReturnTrueif the object doesn’t contain any translation units.makeheader(**kwargs)— Creates a header for the filename you pass in. Peek at.makeheaderdict()if you need the parameter tea ☕.makeheaderdict(charset='CHARSET', encoding='ENCODING', project_id_version=None, pot_creation_date=None, po_revision_date=None, last_translator=None, language_team=None, mime_version=None, plural_forms=None, report_msgid_bugs_to=None, **kwargs)— Creates a header dictionary with useful defaults. 💅pot_creation_datecan beNone(current date) or a value (datetime or string).po_revision_datecan beNone(form),False(=pot_creation_date),True(=now), or a value (datetime or string).- Returns: Dictionary with the header items.
- Return type:
dict of strings.
makeindex()— Indexes the items in this store. At least.sourceindexshould be useful. ✨- property
merge_on— The matching criterion to use when merging on. mergeheaders(otherstore)— Merges another header with this header. This header is assumed to be the template.parse(data)— Parser to process the given source string.- classmethod
parsefile(storefile)— Reads the given file (or opens the given filename) and parses back to an object. parseheader()— Parses the PO header and returns the interpreted values as a dictionary.- classmethod
parsestring(storestring)— Convert the string representation back to an object. remove_unit_from_index(unit)— Remove a unit from source and locaton indexes. 🫶removeunit(unit)— Remove the given unit to the object’s list of units.- This method should always be used rather than trying to modify the list manually.
- Parameters:
unit(TranslationUnit) – The unit that will be added.
require_index()— Make sure source index exists.save()— Save to the file that data was originally read from, if available.savefile(storefile)— Write the string representation to the given file (or filename).serialize(out)— Converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write to.setprojectstyle(project_style)— Set the project in the header.- Parameters:
project_style(str) – the new project.
- Parameters:
setsourcelanguage(sourcelanguage)— Set the source language for this store.settargetlanguage(lang)— Set the target language in the header.- This removes any custom Poedit headers if they exist.
- Parameters:
lang(str) – the new target language code.
suggestions_in_format = False— Indicates if format can store suggestions and alternative translation for a unit.translate(source)— Return the translated string for a given source string.- Return type: String or
None.
- Return type: String or
unit_iter()— Iterator over all the units in this store.updatecontributor(name, email=None)— Add contribution comments if necessary.updateheader(add=False, **kwargs)— Updates the fields in the PO style header.- This will create a header if
add == True.
- This will create a header if
updateheaderplural(nplurals, plural)— Update thePlural-FormPO header.
💅 class translate.storage.pocommon.pounit(source=None) 💅
adderror(errorname, errortext)— Adds an error message to this unit.addlocation(location)— Add one location to the list of locations.
NOTE: Shouldn’t be implemented if the format doesn’t support it.
addlocations(location)— Add a location or a list of locations.
NOTE: Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: This method might be removed in future.
addnote(text, origin=None, position='append')— Adds a note (comment). 🎀- Parameters:
text(string) – Usually just a sentence or two.origin(string) – Specifies who/where the comment comes from. Origin can be one of the following text strings:- ‘translator’
- ‘developer’, ‘programmer’, ‘source code’ (synonyms)
- Parameters:
- classmethod
buildfromunit(unit)— Build a native unit from a foreign unit.- Preserving as much information as possible.
- static
getcontext()— Get the message context. geterrors()— Get all error messages.getid()— A unique identifier for this unit.- Return type: string
- Returns: an identifier for this unit that is unique in the store.
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.
- static
getlocations()— A list of source code locations.- Return type: List
NOTE: Shouldn’t be implemented if the format doesn’t support it.
getnotes(origin=None)— Returns all notes about this unit.- It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see
getlocations()).
- It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see
gettargetlen()— Returns the length of the target string.- Return type: Integer
NOTE: Plural forms might be combined.
getunits()— This unit in a list.- static
hasplural()— Tells whether or not this specific unit has plural strings. infer_state()— Empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state.isblank()— Used to see if this unit has no source or target string.
NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
isfuzzy()— Indicates whether this unit is fuzzy.- static
isheader()— Indicates whether this unit is a header. isobsolete()— Indicate whether a unit is obsolete.isreview()— Indicates whether this unit needs review.istranslatable()— Indicates whether this unit can be translated.- This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.
istranslated()— Indicates whether this unit is translated.- This should be used rather than deducing it from
.target, to ensure that other classes can implement more functionality (as XLIFF does).
- This should be used rather than deducing it from
makeobsolete()— Make a unit obsolete.markfuzzy(present=True)— Marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— Marks the unit to indicate whether it needs review. Adds an optional explanation as a note.merge(otherunit, overwrite=False, comments=True, authoritative=False)— Do basic format agnostic merging.multistring_to_rich(mulstring)— Convert a multistring to a list of “rich” string trees. 🌈
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— Remove all the translator’s notes.rich_parsers = []— A list of functions to use for parsing a string into a rich string tree.- property
rich_source— SEE ALSO:rich_to_multistring(),multistring_to_rich(). - property
rich_target— SEE ALSO:rich_to_multistring(),multistring_to_rich(). - classmethod
rich_to_multistring(elem_list)— Convert a “rich” string tree to a multistring. ✨
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— Set the message context.setid(value)— Sets the unique identified for this unit.- only implemented if format allows ids independant from other unit properties like source or context.
unit_iter()— Iterator that only returns this unit.translate.storage.pocommon.quote_plus(text)— Quote the query fragment of a URL; replacing ‘ ‘ with ‘+’. 🔥translate.storage.pocommon.unquote_plus(text)—unquote(‘%7e/abc+def’) -> ‘~/abc def’.
🌸 poheader 🌸
class that handles all header functions for a header in a po file.
translate.storage.poheader.parseheaderstring(input)— Parses an input string with the definition of a PO header and returns the interpreted values as a dictionary.
🧾 class translate.storage.poheader.poheader
okay so this class is the little header-handling gremlin for PO files — it implements functionality for manipulation of po file headers. It is a mix-in class and useless on its own, bestie. It must be used from all classes which represent a po file. Don’t try to make it a solo main character. 💅
getheaderplural()— Returns thenpluralandpluralvalues from the header.getprojectstyle()— Return the project based on information in the header. The project is determined in this sequence:- Use the ‘
X-Project-Style’ entry in the header. - Use ‘
Report-Msgid-Bug-To’ entry. - Use the ‘
X-Accelerator’ entry. - Use the Project ID.
- Analyse the file itself (not yet implemented).
- Use the ‘
gettargetlanguage()— Return the target language based on information in the header. The target language is determined in this sequence:- Use the ‘
Language’ entry in the header. - Poedit’s custom headers.
- Analysing the ‘
Language-Team’ entry.
- Use the ‘
header()— Returns the header element, orNone. Only the first element is allowed to be a header. Note that this could still return an empty header element, if present.init_headers(charset='UTF-8', encoding='8bit', **kwargs)— Sets default values for po headers.makeheader(**kwargs)— Create a header for the given filename. Check.makeheaderdict()for information on parameters.makeheaderdict(charset='CHARSET', encoding='ENCODING', project_id_version=None, pot_creation_date=None, po_revision_date=None, last_translator=None, language_team=None, mime_version=None, plural_forms=None, report_msgid_bugs_to=None, **kwargs)— Create a header dictionary with useful defaults.pot_creation_datecan beNone(current date) or a value (datetime or string).po_revision_datecan beNone(form),False(=pot_creation_date),True(=now), or a value (datetime or string).- Returns: Dictionary with the header items.
- Return type:
dict of strings.
mergeheaders(otherstore)— Merges another header with this header. This header is assumed to be the template.parseheader()— Parses the PO header and returns the interpreted values as a dictionary.setprojectstyle(project_style)— Set the project in the header.- Parameters:
project_style(str) – the new project.
- Parameters:
settargetlanguage(lang)— Set the target language in the header. This removes any custom Poedit headers if they exist.- Parameters:
lang(str) – the new target language code.
- Parameters:
updatecontributor(name, email=None)— Add contribution comments if necessary.updateheader(add=False, **kwargs)— Updates the fields in the PO style header. This will create a header ifadd == True.updateheaderplural(nplurals, plural)— Update thePlural-FormPO header.
🕰️ translate.storage.poheader.tzstring()
Returns the timezone as a string in the format [+-]0000, eg +0200.
- Return type:
str
🧩 translate.storage.poheader.update(existing, add=False, **kwargs)
Update an existing header dictionary with the values in kwargs, adding new values only if add is true.
- Returns: Updated dictionary of header entries.
- Return type:
dict of strings
🐍 poparser
From the GNU gettext manual:
WHITE-SPACE # TRANSLATOR-COMMENTS #. AUTOMATIC-COMMENTS #| PREVIOUS MSGID (Gettext 0.16 - check if this is the correct position - not yet implemented) #: REFERENCE… #, FLAG… msgctxt CONTEXT (Gettext 0.15) msgid UNTRANSLATED-STRING msgstr TRANSLATED-STRING.💀 exception translate.storage.poparser.PoParseError(parse_state, message=None)
add_note(object, /)—Exception.add_note(note) – add a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.translate.storage.poparser.append(self, object, /)— Append object to the end of the list.translate.storage.poparser.decode(self, /, encoding='utf-8', errors='strict')— Decode the bytes using the codec registered for encoding.encoding— The encoding with which to decode the bytes.errors— The error handling scheme to use for the handling of decoding errors. The default is ‘strict’ meaning that decoding errors raise aUnicodeDecodeError. Other possible values are ‘ignore’ and ‘replace’ as well as any other name registered withcodecs.register_errorthat can handleUnicodeDecodeErrors.
translate.storage.poparser.decode_header(unit, decode)— The header has been arbitrarily decoded with a single-byte encoding. We re-encode it to decode values with the proper encoding defined in the header (usingdecode_listabove).translate.storage.poparser.find()— Return the lowest index inSwhere substringsubis found, such thatsubis contained withinS[start:end]. Optional argumentsstartandendare interpreted as in slice notation. Return-1on failure.translate.storage.poparser.isspace(self, /)— ReturnTrueif the string is a whitespace string,Falseotherwise. A string is whitespace if all characters in the string are whitespace and there is at least one character in the string.translate.storage.poparser.read_obsolete_lines(parse_state)— Read all the lines belonging to the current unit if obsolete.translate.storage.poparser.read_prevmsgid_lines(parse_state)— Read all the lines that start with#|. These lines contain the previousmsgidandmsgctxtinfo. We strip away the leading#|and read until we stop seeing#|.translate.storage.poparser.rfind()— Return the highest index inSwhere substringsubis found, such thatsubis contained withinS[start:end]. Optional argumentsstartandendare interpreted as in slice notation. Return-1on failure.translate.storage.poparser.startswith()— ReturnTrueif the string starts with the specified prefix,Falseotherwise.prefix— A string or a tuple of strings to try.start— Optional start position. Default: start of the string.end— Optional stop position. Default: end of the string.
💾 po
A class loader that will load C or Python implementations of the PO class depending on the USECPO variable.
Use the environment variable USECPO=2 (or USECPO=1) to choose the C implementation which uses Gettext’s libgettextpo for high parsing speed. Otherwise the local Python based parser is used (slower but very well tested).
🦋 poxliff
XLIFF classes specifically suited for handling the PO representation in XLIFF. This way the API supports plurals as if it was a PO file, for example.
🧚 class translate.storage.poxliff.PoXliffFile(*args, **kwargs)
a file for the po variant of Xliff files. 🎀
Extensions = ['xlf', 'xliff', 'sdlxliff']— A list of file extentions associated with this store type.Mimetypes = ['application/x-xliff', 'application/x-xliff+xml']— A list of MIME types associated with this store type.Name = 'XLIFF Translation File'— The human usable name of this store type.UnitClass— alias ofPoXliffUnit.add_unit_to_index(unit)— Add a unit to source and location idexes.addheader()— Initialise the file header.addsourceunit(source, filename='NoName', createifmissing=False)— adds the given trans-unit to the last used body node. If the filename has changed it uses the slow method instead (will create the nodes required if asked). Returns success.addunit(unit, new=True)— Append the given unit to the object’s list of units. This method should always be used rather than trying to modify the list manually.- Parameters:
unit(TranslationUnit) – The unit that will be added.
- Parameters:
createfilenode(filename, sourcelanguage='en-US', datatype='po')— creates a filenode with the given filename. All parameters are needed for XLIFF compliance.creategroup(filename='NoName', createifmissing=False, restype=None)— Adds a group tag into the specified file.detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— Try to detect a file encoding from text, using either the chardet lib or by trying to decode the file.static fallback_detection(text)— Simple detection based on BOM in case chardet is not available.findid(id)— Find unit with matching id by checkingid_index.findunit(source)— Find the unit with the given source string.- Return type:
TranslationUnit or None
- Return type:
findunits(source)— Find the units with the given source string.- Return type:
TranslationUnit or None
- Return type:
getbodynode(filenode, createifmissing=False)— Finds the body node for the given filenode.getdatatype(filename=None)— Returns the datatype of the stored file. If no filename is given, the datatype of the first file is given.getdate(filename=None)— Returns the date attribute for the file. If no filename is given, the date of the first file is given. If the date attribute is not specified,Noneis returned.
Returns — the date attribute of that file 🗓️
Return type — Date or None
static getfilename(filenode)
Gives you back the name of whatever file you hand it. literally just the name, that's it 💅
getfilenames()
Returns all the filenames chillin' in this XLIFF file. the whole squad 🫶
getfilenode(filename, createifmissing=False)
Hunts down the filenode with the name you gave it. if it's not there and you set createifmissing=False… well, don't expect a miracle ✨
getheadernode(filenode, createifmissing=False)
Same vibe, but for the header node of that filenode. 🎀
getheaderplural()
Pulls the nplural and plural values out of the header. plural realness 👑
getids()
Returns a list of unit ids. a lil roster of everybody in the store 📋
getprojectstyle()
Gets the project type for this store.
getsourcelanguage()
Gets the source language for this store. where it's coming from 🌍
gettargetlanguage()
Gets the target language for this store. where it's going to 🌈
getunits()
Returns a list of all the units in this store. everyone's invited fr fr 🎉
header()
Returns the header element, or None. Only the first element is allowed to be a header. ngl though, this could still hand you back an empty header element if one's present — so check before you celebrate 🙈
init_headers(charset='UTF-8', encoding='8bit', **kwargs)
Sets up the default values for po headers. cozy lil defaults: charset='UTF-8', encoding='8bit' 😌
initbody()
Initialises self.body so you never have to go back and re-fetch it from the XML again. efficiency slay ⚡
isempty()
Returns True if the object doesn't contain any translation units. empty = no units, periodt 💀
makeheader(**kwargs)
Creates a header for the given filename. and for all the deets on what params you can pass, check .makeheaderdict() — that's the one with the lore 📖
makeheaderdict(charset='CHARSET', encoding='ENCODING', project_id_version=None, pot_creation_date=None, po_revision_date=None, last_translator=None, language_team=None, mime_version=None, plural_forms=None, report_msgid_bugs_to=None, **kwargs)
Builds you a header dictionary with useful defaults baked in 🎂
okay so about those dates, read carefully bestie:
pot_creation_datecan beNone(which means current date) or an actual value (adatetimeor a string)po_revision_datecan beNone(form),False(which equalspot_creation_date),True(which equals now), or an actual value (adatetimeor a string)
Returns — a dictionary with the header items
Return type — dict of strings
makeindex()
Indexes the items in this store. at minimum .sourceindex should be useful to you 🔍
property merge_on
The matching criterion to use when merging on.
Returns — the default matching criterion for all the subclasses
Return type — string
mergeheaders(otherstore)
Merges another header into this header. and heads up: this header is assumed to be the template 👑
namespaced(name)
Returns name in Clark notation. like, for example, namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourcethis is needed all over the place throughout lxml. iykyk 🧚
parse(xml)
Populates this object from the xml string you give it. in goes string, out goes object ✨
classmethod parsefile(storefile)
Reads the given file (or opens the given filename) and parses it back into an object. feed it a file, get an object 🍓
parseheader()
Parses the PO header and returns the interpreted values as a dictionary.
classmethod parsestring(storestring)
Parses the string and returns the correct file object for it. 🎀
remove_unit_from_index(unit)
Removes a unit from the source and locaton indexes. (yes, "locaton" — that's how it's spelled here, don't come for me 💀)
removedefaultfile()
We want to nuke the default file-tag as early as possible, if we know it's still present and empty. bye bye default 🫡
removeunit(unit)
Removes the given unit from the object's list of units.
bestie, always use this method instead of trying to mess with the list manually. seriously. hand-editing that list is not the move 😭
Parameters — unit (TranslationUnit) — the unit that will be added.
require_index()
Makes sure the source index actually exists. no index? now there's an index 💅
save()
Saves to the file that the data was originally read from — if that's available.
savefile(storefile)
Writes the string representation to the given file (or filename).
serialize(out)
Converts to a string containing the file's XML. 🧚
static setfilename(filenode, filename)
Sets the name of the given file.
setprojectstyle(project_style)
Sets the project type for this store.
setsourcelanguage(language)
Sets the source language for this store. 🌍
settargetlanguage(language)
Sets the target language for this store. 🌈
property sourcelanguage
The type of the None singleton. (yes that's genuinely what it says 🫠)
suggestions_in_format = True
xliff units have alttrans tags which can be used to store suggestions. so true bestie 💖
switchfile(filename, createifmissing=False)
Adds the given trans-unit (and it will create the nodes required, if you ask it to).
Returns — success
Return type — Boolean
property targetlanguage
The type of the None singleton. (again. yes. 🫠)
translate(source)
Returns the translated string for a given source string. in goes source, out comes the translation ✨
Return type — String or None
unit_iter()
Iterator over all the units in this store. loop girlies rise 🌀
updatecontributor(name, email=None)
Adds contribution comments if necessary.
updateheader(add=False, **kwargs)
Updates the fields in the PO style header. and it will create a header if add == True 👑
updateheaderplural(nplurals, plural)
Updates the Plural-Form PO header.
🦋 class translate.storage.poxliff.PoXliffUnit(source=None, empty=False, **kwargs)
A class made specifically to handle the plural units created from a po file. plural unit specialists, if you will 💅
PoXliffUnit(source=None, empty=False, **kwargs)addalttrans(txt, origin=None, lang=None, sourcetxt=None, matchquality=None)
Adds an alt-trans tag and all the alt-trans components to the unit.
Parameters — txt (String) — alternative translation of the source text.
adderror(errorname, errortext)
Adds an error message to this unit. 🚨
addlocation(location)
Adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. don't force it bestie 🙈
addlocations(location)
Adds a location, or a whole list of locations.
NOTE: most classes shouldn't need to implement this — they should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. ⚠️ don't get too attached
addnote(text, origin=None, position='append')
Adds a note, specifically inside a "note" tag. default position='append' 🎀
classmethod buildfromunit(unit)
Builds a native unit out of a foreign unit — preserving as much information as possible. maximal preservation energy 🫶
static correctorigin(node, origin)
Checks against the node tag's origin (e.g. note or alt-trans).
createcontextgroup(name, contexts=None, purpose=None)
Adds the context group to the trans-unit, with contexts being a list of (type, text) tuples describing each context. 🗂️
createlanguageNode(lang, text, purpose)
Returns an xml Element set up with the given parameters.
delalttrans(alternative)
Removes the alternative you supplied from the list of alt-trans tags. yeet 🗑️
getNodeText(languageNode, xml_space='preserve')
Retrieves the term from the given languageNode. default xml_space='preserve' 💅
get_rich_target(lang=None)
Retrieves the "target" text (the second entry) — or the entry in the specified language, if it exists. 🎯
getalttrans(origin=None)
Returns <alt-trans> for the given origin as a list of units. and no origin means all the alternatives. all of them. the whole buffet 🍓
getautomaticcomments()
Returns the automatic comments (x-po-autocomment) — which corresponds to the #. style po comments. ✨
static getcontext()
Gets the message context.
getcontextgroups(name)
Returns the contexts inside the context groups with the specified name.
getcontextgroupsbyattribute(attributeName, attributeValue)
Returns the contexts inside the context groups matching the specified attributeName and attributeValue. 🔍
geterrors()
Gets all the error messages. every single one 🚨
getid()
A unique identifier for this unit.
Return type — string
Returns — an identifier for this unit that's unique in the store
derived classes should override this so it guarantees a unique identifier for each unit in the store. no duplicates allowed, that's the rule 👑
getlanguageNode(lang=None, index=None)
Retrieves a languageNode either by language or by index. pick your fighter 🥊
getlanguageNodes()
We override this one to get the source and target nodes. both of them 🫶
getlocations()
Returns all the references (source locations). 📍
getnotes(origin=None)
Returns all the notes about this unit.
it'll probably be freeform text or something reasonable that can be synthesised by the format. and just so we're clear — it should not include location comments (see getlocations() for those). keep 'em separated bestie 💅
getrestype()
Returns the restype attribute in the trans-unit tag.
gettarget(lang=None)
Retrieves the "target" text (the second entry) — or the entry in the specified language, if it exists. 🎯
gettargetlen()
Returns the length of the target string. count 'em 📏
- Return type — Integer
NOTE: plural forms might get combined, bestie. don't @ me.
🌸 methods that keep going 🌸
gettranslatorcomments()— hands you the translator comments (that's thex-po-trancommentfield), which lines up with the#style po comments. iykyk.getunits()— literally just this unit, in a list. one-unit list. iconic behavior.hasplural()— tells you whether this specific unit has plural strings.infer_state()— an empty method 💀 that you're supposed to override in sub-classes so it can infer the currentstate(_n)of the unit from its current state. it's giving "template method" energy.isapproved()— states whether this unit is approved ✅isblank()— used to see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and the vibes suggest we might want to move in that direction rather and get rid of this. so it's a whole thing.
isfuzzy()— indicates whether this unit is fuzzy 🫠isheader()— indicates whether this unit is a header 👑- static
isobsolete()— indicate whether a unit is obsolete 💀 isreview()— states whether this unit needs to be reviewed 👀istranslatable()— indicates whether this unit can be translated.
this should be used to distinguish real units for translation from header, obsolete, binary or other blank units. so like, the actual main characters vs the npcs.
istranslated()— indicates whether this unit is translated.
this should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does). don't be out here guessing from .target, bestie, that's not the move.
makeobsolete()— make a unit obsolete 🪦markapproved(value=True)— mark this unit as approved ✅markfuzzy(value=True)— marks the unit as fuzzy or not 🫠markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review. adds an optional explanation as a note 📝merge(otherunit, overwrite=False, comments=True, authoritative=False)— do basic format agnostic merging. no format drama, just vibes.multistring_to_rich(mulstring)— convert a multistring to a list of "rich" string trees 🌳
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]namespaced(name)— returns name in Clark notation.
for example namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourcethis is needed throughout lxml. fr fr, it's everywhere.
removenotes(origin=None)— remove all the translator notes 🧹rich_parsers = [...]— a list of functions to use for parsing a string into a rich string tree:
rich_parsers = [<bound method regex_parse of <class 'translate.storage.placeables.general.NewlinePlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.XMLTagPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.AltAttrPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.XMLEntityPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.PythonFormattingPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.JavaMessageFormatPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.FormattingPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.QtFormattingPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.UrlPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.FilePlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.EmailPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.CapsPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.CamelCasePlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.OptionPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.PunctuationPlaceable'>>, <bound method regex_parse of <class 'translate.storage.placeables.general.NumberPlaceable'>>]- property
rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
- property
rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
- classmethod
rich_to_multistring(elem_list)— convert a "rich" string tree to a multistring ✨
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— set the message context 🎯setid(id)— sets the unique identified for this unit. only implemented if the format allows ids that are independent from other unit properties like source or context.settarget(target, lang='xx', append=False)— sets the target string to the given value.unit_iter()— iterator that only returns this unit. one and done 🫶
🧚 project 🧚
class translate.storage.project.Project(projstore=None)— manages a project store as well as the processes involved in a project workflow. she's the project manager, literally.add_source(srcfile, src_fname=None)— proxy forself.store.append_sourcefile().add_source_convert(srcfile, src_fname=None, convert_options=None, extension=None)— convenience method that callsadd_source()andconvert_forward()and returns the results from both. two-for-one, we love efficiency 💅close()— close underlying store. proxy forself.store.close().convert_forward(input_fname, template=None, output_fname=None, **options)— convert the given input file to the next type in the process.
source document (eg. ODT) → translation file (eg. XLIFF) → translated document (eg. ODT). the whole glow-up pipeline 🌈
Parameters
input_fname(basestring) – the project name of the file to convertconvert_options(Dictionary (optional)) – passed as-is totranslate.convert.factory.convert().
Returns 2-tuple — the converted file object and its project name.
export_file(fname, destfname)— export the file with the specified filename to the given destination. this method will raiseFileNotInProjectErrorvia the call toget_file()iffnameis not found in the project. so like, know your files or get an error bestie 😭get_file(fname)— return store filename. proxy forself.store.get_file().get_proj_filename(realfname)— return project filename. proxy forself.store.get_proj_filename().get_real_filename(projfname)— try and find a real file name for the given project file name 🔍remove_file(projfname, ftype=None)— remove a file. proxy forself.store.remove_file().save(filename=None)— save a store. proxy forself.store.save().update_file(proj_fname, infile)— update a file. proxy forself.store.update_file().
🦋 projstore 🦋
- exception
translate.storage.projstore.FileExistsInProjectErroradd_note(object, /)—Exception.add_note(note)– add a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb)– setself.__traceback__to tb and returnself.
- exception
translate.storage.projstore.FileNotInProjectErroradd_note(object, /)—Exception.add_note(note)– add a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb)– setself.__traceback__to tb and returnself.
class translate.storage.projstore.ProjectStore— basic project file container 📦append_file(afile, fname, ftype='trans', delete_orig=False)— append the given file to the project with the given filename, marked to be of type ftype ('src','trans','tgt').
Parameters
delete_orig(bool) – whether or not the original (given) file should be deleted after being appended. this is set toTruebyconvert_forward(). not used in this class.
get_file(fname, mode='rb')— retrieve the file with the given name from the project store.
the file is looked up in the self._files dictionary. the values in this dictionary may be None, to indicate that the file is not cacheable and needs to be retrieved in a special way. this special way must be defined in this method of sub-classes. the value may also be a string, which indicates that it is a real file accessible via open. so like, it's either a real file, a None situation, or a subclass's whole custom thing. 🫠
Parameters
mode(str) – the mode in which to re-open the file (if it is closed).
get_filename_type(fname)— get the type of file ('src','trans','tgt') with the given name.get_proj_filename(realfname)— try and find a project file name for the given real file name.
load(*args, **kwargs)— Loads the project up in some way. Which way? Unspecified bestie, this is the base class 💅 it's literally undefined here.remove_file(fname, ftype=None)— Yeets the file with the given project name out of the project. If you don't hand it the file type ('src','trans'or'tgt'), it'll just guess for you 🫶 lowkey smart of it.save(filename=None, *args, **kwargs)— Saves the project. How? Again: undefined for this base class 😭 it's giving "figure it out yourself."- property
sourcefiles— Read-only access toself._sourcefiles✨ look, don't touch. - property
targetfiles— Read-only access toself._targetfiles✨ same energy. - property
transfiles— Read-only access toself._transfiles✨ no cap, hands off. update_file(pfname, infile)— Removes the project file namedpfnameand drops the contents ofinfileinto the project under that same file name 🔁 out with the old, in with the new.- Returns — whatever
ProjectStore.append_file()hands back.
- Returns — whatever
🎀 properties 🎀
These are the classes that hold units of .properties — and similar — files, the ones used when you're translating Java, Mozilla, MacOS and other software 🌸
The propfile class is a monolingual class, and propunit is the one giving you unit-level access.
The .properties store has kinda glowened up into a general key/value pair class, and Dialect is the icon that lets you change how parsing and handling behaves across the different dialects 👑
Currently we support:
- Java
.properties - Mozilla
.properties - Adobe Flex files
- MacOS X
.stringsfiles - Skype
.langfiles - XWiki
.properties
Here's the reference material and the descriptions of each dialect we've got 💖
Java — Java .properties are supported completely, except for the ability to drop pairs that aren't translated. The .properties file description linked here is a genuinely good reference for the .properties spec. .properties files may also hold Java MessageFormat messages. There's no special handling for MessageFormat in this storage class, but that could land in future 👀 All delimiter types, comments, line continuations and space handling in delimiters are supported ✅
Mozilla — Mozilla files use '=' as the delimiter, are UTF-8 encoded, and thus don't need \u escaping. Any \U values get converted into correct Unicode characters 🦋
Strings — Mac OS X .strings files are implemented using these two articles as references.
Flex — Adobe Flex files seem to be normal .properties files but in UTF-8, just like Mozilla files. This page is the info used to implement the dialect.
Skype — Skype .lang files seem to be UTF-16 encoded .properties files 🫶
XWiki — XWiki translation files are standard Java .properties but with specific escaping support for simple quotes, plus support for missing translations. This XWiki document is the info used to implement the dialect.
Here's the simple summary of what's permissible 👇
Comments supported:
# a comment
// a comment (only at the beginning of a line)
# The following are # escaped to render in docs
# ! is standard but not widely supported
#! a comment
# /* is non-standard but used on some implementations
#/* a comment (not across multiple lines) */Name and value pairs:
# Delimiters
key = value
key : value
# Whitespace delimiter
# key[sp]value
# Space in key and around value
\ key\ = \ value
# Note that the b and c are escaped for reST rendering
b = a string with escape sequences \\t \\n \\r \\\\ \\" \\' \\ (space) \u0123
c = a string with a continuation line \\
continuation line
# Special cases
# key with no value
//key (escaped; doesn't render in docs)
# value no key (extractable in prop2po but not mergeable in po2prop)
=value
# .strings specific
"key" = "value";✨ class translate.storage.properties.Dialect ✨
The settings for all the different behaviours in key=value files 🎀
static encode(string, encoding=None)— Encodes the string.classmethod find_delimiter(line)— Finds the type and position of the delimiter in a property line. Property files can be delimited by“=”,“:”or whitespace (space for now). We find the position of each delimiter, then find the one that appears first 👀- Parameters
line(str) — A properties linedelimiters(list) — valid delimiters
- Returns — delimiter character and offset within line
- Return type —
Tuple (delimiter char, Offset Integer)
- Parameters
static key_strip(key)— Strips the unneeded characters off the key ✂️static value_strip(value)— Strips the unneeded characters off the value ✂️
🌈 class translate.storage.properties.DialectFlex 🌈
static encode(string, encoding=None)— Encodes the string.classmethod find_delimiter(line)— Finds the type and position of the delimiter in a property line. Property files can be delimited by“=”,“:”or whitespace (space for now). We find the position of each delimiter, then find the one that appears first 👀- Parameters
line(str) — A properties linedelimiters(list) — valid delimiters
- Returns — delimiter character and offset within line
- Return type —
Tuple (delimiter char, Offset Integer)
- Parameters
static key_strip(key)— Strips the unneeded characters off the key ✂️static value_strip(value)— Strips the unneeded characters off the value ✂️
🦋 class translate.storage.properties.DialectGaia 🦋
static encode(string, encoding=None)— Encodes the string.classmethod find_delimiter(line)— Finds the type and position of the delimiter in a property line. Property files can be delimited by“=”,“:”or whitespace (space for now). We find the position of each delimiter, then find the one that appears first 👀- Parameters
line(str) — A properties linedelimiters(list) — valid delimiters
- Returns — delimiter character and offset within line
- Return type —
Tuple (delimiter char, Offset Integer)
- Parameters
static key_strip(key)— Strips the unneeded characters off the key ✂️static value_strip(value)— Strips the unneeded characters off the value ✂️
💜 class translate.storage.properties.DialectGwt 💜
Heads up bestie — this one'sencodeis aclassmethod, not astaticone. Different flavour, same name 🫶
classmethod encode(string, encoding=None)— Encodes the string.classmethod find_delimiter(line)— Finds the type and position of the delimiter in a property line. Property files can be delimited by“=”,“:”or whitespace (space for now). We find the position of each delimiter, then find the one that appears first 👀- Parameters
line(str) — A properties linedelimiters(list) — valid delimiters
- Returns — delimiter character and offset within line
- Return type —
Tuple (delimiter char, Offset Integer)
- Parameters
static key_strip(key)— Strips the unneeded characters off the key ✂️static value_strip(value)— Strips the unneeded characters off the value ✂️
🍓 class translate.storage.properties.DialectJava 🍓
static encode(string, encoding=None)— Encodes the string.classmethod find_delimiter(line)— Finds the type and position of the delimiter in a property line. Property files can be delimited by“=”,“:”or whitespace (space for now). We find the position of each delimiter, then find the one that appears first 👀- Parameters
line(str) — A properties linedelimiters(list) — valid delimiters
- Returns — delimiter character and offset within line
- Return type —
Tuple (delimiter char, Offset Integer)
- Parameters
static key_strip(key)— Strips the unneeded characters off the key ✂️static value_strip(value)— Strips the unneeded characters off the value ✂️
🧚 class translate.storage.properties.DialectJavaUtf16 🧚
static encode(string, encoding=None)— Encodes the string.classmethod find_delimiter(line)— Finds the type and position of the delimiter in a property line. Property files can be delimited by“=”,“:”or whitespace (space for now). We find the position of each delimiter, then find the one that appears first 👀- Parameters
line(str) — A properties linedelimiters(list) — valid delimiters
- Returns — delimiter character and offset within line
- Return type —
Tuple (delimiter char, Offset Integer)
- Parameters
static key_strip(key)— Strips the unneeded characters off the key ✂️static value_strip(value)— Strips the unneeded characters off the value ✂️
⭐ class translate.storage.properties.DialectJavaUtf8 ⭐
static encode(string, encoding=None)— Encodes the string.classmethod find_delimiter(line)— Finds the type and position of the delimiter in a property line. Property files can be delimited by“=”,“:”or whitespace (space for now). We find the position of each delimiter, then find the one that appears first 👀- Parameters
line(str) — A properties linedelimiters(list) — valid delimiters
- Parameters
- Returns — literally the delimiter character plus where it sits in the line 💫
- Return type —
Tuple (delimiter char, Offset Integer) static key_strip(key)— strips the characters you don't need off the key. that's it, that's the job 🧹static value_strip(value)— same energy but for the value: strip the unneeded chars, move on ✂️
🦋 class translate.storage.properties.DialectJoomla
okay so Joomla's dialect walked in and it's serving the exact same four moves as the others, and honestly? consistency is iconic.
static encode(string, encoding=None)— encodes the string for you 💅classmethod find_delimiter(line)— finds the type AND the position of the delimiter in a property line.
so here's the tea ☕ property files can be delimited by "=", ":" or whitespace (space for now, bestie). we find where each delimiter sits, then we pick whichever one shows up first. first come, first served, no line-cutting.
Parameters
- line (
str) – a properties line - delimiters (
list) – valid delimiters
Returns — delimiter character and offset within line
Return type — Tuple (delimiter char, Offset Integer)
static key_strip(key)— Strip unneeded characters from the key 🎀static value_strip(value)— Strip unneeded characters from the value 🎀
🌸 class translate.storage.properties.DialectMozilla
Mozilla said "me too" and honestly we respect the consistency.
static encode(string, encoding=None)— Encode the string. no notes, slay ✨classmethod find_delimiter(line)— Find the type and position of the delimiter in a property line.
Property files can be delimited by "=", ":" or whitespace (space for now). We find the position of each delimiter, then find the one that appears first. it's giving "first one wins" and we love a clear rule 🔥
Parameters
- line (
str) – A properties line - delimiters (
list) – valid delimiters
Returns — delimiter character and offset within line
Return type — Tuple (delimiter char, Offset Integer)
static key_strip(key)— Strip unneeded characters from the key.static value_strip(value)— Strip unneeded characters from the value.
💜 class translate.storage.properties.DialectSkype
Skype dialect... the way this one also has the same four methods. main character energy but make it predictable 🫶
static encode(string, encoding=None)— Encode the string 💫classmethod find_delimiter(line)— Find the type and position of the delimiter in a property line.
Property files can be delimited by "=", ":" or whitespace (space for now). We find the position of each delimiter, then find the one that appears first. yeah, same detective work, different dialect 🕵️♀️
Parameters
- line (
str) – A properties line - delimiters (
list) – valid delimiters
Returns — delimiter character and offset within line
Return type — Tuple (delimiter char, Offset Integer)
static key_strip(key)— Strip unneeded characters from the key.static value_strip(value)— Strip unneeded characters from the value.
🎀 class translate.storage.properties.DialectStrings
the DialectStrings girlies are here and they're bringing the exact same toolkit:
static encode(string, encoding=None)— Encode the string ⭐classmethod find_delimiter(line)— Find the type and position of the delimiter in a property line.
Property files can be delimited by "=", ":" or whitespace (space for now). We find the position of each delimiter, then find the one that appears first. it's giving algorithm but make it cute 💅
Parameters
- line (
str) – A properties line - delimiters (
list) – valid delimiters
Returns — delimiter character and offset within line
Return type — Tuple (delimiter char, Offset Integer)
static key_strip(key)— Strip unneeded characters from the key.static value_strip(value)— Strip unneeded characters from the value.
🌈 class translate.storage.properties.DialectStringsUtf8
UTF-8 variant, same vibes, more bytes bestie 😌
static encode(string, encoding=None)— Encode the string 🔥classmethod find_delimiter(line)— Find the type and position of the delimiter in a property line.
Property files can be delimited by "=", ":" or whitespace (space for now). We find the position of each delimiter, then find the one that appears first. (yes the description is copy-paste energy across dialects but that's literally how the source describes it, iykyk 🧚)
Parameters
- line (
str) – A properties line - delimiters (
list) – valid delimiters
Returns — delimiter character and offset within line
Return type — Tuple (delimiter char, Offset Integer)
static key_strip(key)— Strip unneeded characters from the key.static value_strip(value)— Strip unneeded characters from the value.
👑 class translate.storage.properties.DialectXWiki
okay THIS one has lore 🍓 the XWiki dialect is mainly a Java properties behaviour, BUT with special handling of simple quotes: they get escaped by doubling them when an argument on the form "{X}" is provided, where X is a number. that's the whole twist and it ate, no cap.
static encode(string, encoding=None)— Encode the string ✨classmethod find_delimiter(line)— Find the type and position of the delimiter in a property line.
Property files can be delimited by "=", ":" or whitespace (space for now). We find the position of each delimiter, then find the one that appears first.
Parameters
- line (
str) – A properties line - delimiters (
list) – valid delimiters
Returns — delimiter character and offset within line
Return type — Tuple (delimiter char, Offset Integer)
static key_strip(key)— Strip unneeded characters from the key.static value_strip(value)— Strip unneeded characters from the value.
🧚 class translate.storage.properties.XWikiFullPage(*args, **kwargs)
this class represents a full XWiki Page translation 🫶 like, this file does NOT contain properties — its whole content needs to be translated. the entire thing. no keys, no values, just vibes and content. more info lives over at https://dev.xwiki.org/xwiki/bin/view/Community/XWiki%20Translations%20Formats/#HXWikiFullContentTranslation 💖
Extensions=['xml']— a list of file extentions associated with this store type (yes, "extentions", that's how it's spelled here, we're keeping it 💀)Mimetypes=None— a list of MIME types associated with this store typeName='XWiki Full Page'— the human usable name of this store typeUnitClass— alias ofxwikiunit
add_unit_to_index(unit)— add a unit to source and location idexes 📇addsourceunit(source)— add and returns a new unit with the given source string.- Return type —
TranslationUnit
- Return type —
addunit(unit)— append the given unit to the object's list of units.
this method should ALWAYS be used rather than trying to modify the list manually. bestie do not touch the list yourself, i'm begging 🙈
Parameters
- unit (
TranslationUnit) – The unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from text, using either the chardet lib or by trying to decode the file. detective work 🔍static fallback_detection(text)— simple detection based on BOM in case chardet is not available. the backup bestie 🧍♀️findid(id)— find unit with matching id by checkingid_index.findunit(source)— find the unit with the given source string.- Return type —
TranslationUnitorNone
- Return type —
findunits(source)— find the units with the given source string.- Return type —
TranslationUnitorNone
- Return type —
getids()— return a list of unit ids 🔢getprojectstyle()— get the project type for this store.getsourcelanguage()— get the source language for this store.gettargetlanguage()— get the target language for this store.getunits()— return a list of all units in this store 💖isempty()— returnTrueif the object doesn't contain any translation units.makeindex()— indexes the items in this store. at least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.- Returns — the default matching criterion for all the subclasses.
- Return type —
string
parse(propsrc)— read the source of a properties file in and include them as units.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— convert the string representation back to an object.remove_unit_from_index(unit)— remove a unit from source and locaton indexes (again with the spelling, keeping it exactly as-is 🎀).removeunit(unit)— remove the given unit to the object's list of units.
same rule as before: this method should always be used rather than trying to modify the list manually. hands off the list, bestie ✋
Parameters
- unit (
TranslationUnit) – The unit that will be added.
require_index()— make sure source index exists.save()— save to the file that data was originally read from, if available.savefile(storefile)— write the string representation to the given file (or filename).serialize(out)— write the units back to file 📝setprojectstyle(project_style)— sets the project type for this store, no biggie 💜setsourcelanguage(sourcelanguage)— sets the source language for this store 🌸settargetlanguage(targetlanguage)— and this one sets the target language for this store 🎀suggestions_in_format = False— a lil flag saying whether this format can actually store suggestions and alternative translations for a unit. It'sFalse, bestie, so… it can't 🙈translate(source)— hands you back the translated string for a given source string. Return type isStringorNone, so handle that vibe check responsibly ✨unit_iter()— iterator over all the units in this store. every. single. one. 🦋
class translate.storage.properties.XWikiPageProperties(*args, **kwargs)
Okay so this one represents an XWiki Page containing translation properties, exactly as described over at:
https://dev.xwiki.org/xwiki/bin/view/Community/XWiki%20Translations%20Formats/#HXWikiPageProperties(iykyk — that's the spec, go read it 💅)
Extensions = ['xml']— a list of file extensions associated with this store type ⭐Mimetypes = None— a list of MIME types associated with this store type. it'sNone. that's the whole tea ☕Name = 'XWiki Page Properties'— the human-usable name of this store type, iconic 👑UnitClass— alias ofxwikiunit🧚add_unit_to_index(unit)— adds a unit to the source and location indexes 🔥addsourceunit(source)— adds and returns a shiny new unit with the given source string. Return type:TranslationUnit🌈addunit(unit)— appends the given unit to the object's list of units.- Parameters:
unit(TranslationUnit) – the unit that will be added.
- Parameters:
fr fr, always use this method instead of trying to mess with the list manually. do not touch the list. hands off the list 🙈
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to sniff out a file encoding fromtext, either by using thechardetlib or by just… trying to decode the file. lowkey brave of it 💀static fallback_detection(text)— simple detection based on the BOM, for whenchardetisn't available. the backup bestie 🫶findid(id)— finds the unit with a matchingidby checkingid_index🔍findunit(source)— finds the unit with the given source string. Return type:TranslationUnitorNone✨findunits(source)— finds the units (plural!) with the given source string. Return type:TranslationUnitorNone🌸getids()— returns a list of unit ids 📋getprojectstyle()— gets the project type for this store 💜getsourcelanguage()— gets the source language for this store 🌱gettargetlanguage()— gets the target language for this store 🎯getunits()— returns a list of all the units in this store. the whole squad 👯isempty()— returnsTrueif the object doesn't contain any translation units. empty-core 🫥makeindex()— indexes the items in this store. at minimum,.sourceindexshould be useful 💅property merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string🧵
parse(propsrc)— reads the source of a properties file in and includes them as units 📥classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back into an object 🗂️classmethod parsestring(storestring)— converts the string representation back into an object 🔁remove_unit_from_index(unit)— removes a unit from the source and location indexes 🧹removeunit(unit)— removes the given unit from the object's list of units.- Parameters:
unit(TranslationUnit) – the unit that will be added.
- Parameters:
same rule as before: use this method, don't go editing the list with your own two hands 😭
require_index()— makes sure the source index exists. if not, it simply manifests one ✨save()— saves to the file that the data was originally read from, if that file is available 💾savefile(storefile)— writes the string representation to the given file (or filename) 📝serialize(out)— writes the units back to file 🖋️setprojectstyle(project_style)— sets the project type for this store 💜setsourcelanguage(sourcelanguage)— sets the source language for this store 🌸settargetlanguage(targetlanguage)— sets the target language for this store 🎀suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit. stillFalse🙈translate(source)— returns the translated string for a given source string. Return type:StringorNone✨unit_iter()— iterator over all the units in this store. round two, same energy 🦋
translate.storage.properties — module-level goodies
accesskeysuffixes = ('.accesskey', '.accessKey', '.akey')— accesskey suffixes: entries with this suffix may be combined with labels ending inlabelsuffixesinto accelerator notation. weird but iconic notation hack, ngl ⌨️get_comment_end(line)— determines whether a line ends a new multi-line comment.- Parameters:
line(unicode) – a properties line - Returns:
Trueif the line ends a new multi-line comment - Return type:
bool🏁
- Parameters:
get_comment_one_line(line)— determines whether a line is a one-line comment.- Parameters:
line(unicode) – a properties line - Returns:
Trueif the line is a one-line comment - Return type:
bool☝️
- Parameters:
get_comment_start(line)— determines whether a line starts a new multi-line comment.- Parameters:
line(unicode) – a properties line - Returns:
Trueif the line starts a new multi-line comment - Return type:
bool🚩
- Parameters:
class translate.storage.properties.gwtfile(*args, **kwargs)
Extensions = ['properties']— a list of file extensions associated with this store type ⭐Mimetypes = None— a list of MIME types associated with this store type.Noneagain, she's consistent 💅Name = 'Gwt Properties'— the human-usable name of this store type 👑UnitClass— alias ofpropunit🧚add_unit_to_index(unit)— adds a unit to the source and location indexes 🔥addsourceunit(source)— adds and returns a new unit with the given source string. Return type:TranslationUnit🌈addunit(unit)— appends the given unit to the object's list of units.- Parameters:
unit(TranslationUnit) – the unit that will be added.
- Parameters:
yes, again: use this method, not your own gremlin hands on the list 🙈
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, either via thechardetlib or by trying to decode the file 💀static fallback_detection(text)— simple detection based on BOM in casechardetis not available 🫶findid(id)— finds the unit with matchingidby checkingid_index🔍findunit(source)— finds the unit with the given source string. Return type:TranslationUnitorNone✨findunits(source)— finds the units with the given source string. Return type:TranslationUnitorNone🌸getids()— returns a list of unit ids 📋getprojectstyle()— gets the project type for this store 💜getsourcelanguage()— gets the source language for this store 🌱gettargetlanguage()— gets the target language for this store 🎯getunits()— returns a list of all units in this store 👯isempty()— returnsTrueif the object doesn't contain any translation units 🫥makeindex()— indexes the items in this store. at least.sourceindexshould be useful 💅property merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string🧵
parse(propsrc)— reads the source of a properties file in and includes them as units 📥classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back to an object 🗂️classmethod parsestring(storestring)— converts the string representation back to an object 🔁remove_unit_from_index(unit)— removes a unit from the source and location indexes 🧹removeunit(unit)— removes the given unit from the object's list of units.- Parameters:
unit(TranslationUnit) – the unit that will be added.
- Parameters:
bestie I'm begging you, use the method 😭
require_index()— makes sure the source index exists ✨save()— saves to the file that data was originally read from, if available 💾savefile(storefile)— writes the string representation to the given file (or filename) 📝serialize(out)— writes the units back to file 🖋️setprojectstyle(project_style)— sets the project type for this store 💜setsourcelanguage(sourcelanguage)— sets the source language for this store 💅settargetlanguage(targetlanguage)— sets the target language for this store, no capsuggestions_in_format = False— tells you whether this format can actually store suggestions and alternative translations for a unit (spoiler: this one can't 🙈)translate(source)— hands you back the translated string for a given source string. Return type:StringorNoneunit_iter()— iterator over every unit in this store, go off 🦋
🧵 translate.storage.properties.is_line_continuation(line)
Bestie, this one figures out whether a line has a line continuation marker on it.
So <pink>.properties</pink> files can be terminated with a backslash (\), which is basically the file saying "the 'value' continues on the next line, keep reading". Important caveat though: continuation is only valid if there's an odd number of backslashes — an even number would just give you a set of N/2 slashes, not an escape. Math is math 💀
Parameters
line(str) — a properties line
Returns
Does line end with a line continuation
Return type
Boolean
☕ class translate.storage.properties.javafile(*args, **kwargs)
Extensions = ['properties']— the list of file extensions associated with this store typeMimetypes = None— the list of MIME types associated with this store type (it'sNone, don't come for me)Name = 'Java Properties'— the human-usable name of this store typeUnitClass— alias ofpropunit, iconicadd_unit_to_index(unit)— adds a unit to the source and location indexesaddsourceunit(source)— adds and returns a new unit with the given source string. Return type:TranslationUnitaddunit(unit)— appends the given unit to the object's list of units. This method should always be used rather than trying to modify the list manually. Parameters:unit(TranslationUnit) — the unit that will be addeddetect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding fromtext, either by using thechardetlib or by just trying to decode the file itselfstatic fallback_detection(text)— simple detection based on BOM in casechardetisn't available, the backup bestie 🫶findid(id)— finds the unit with the matchingidby checkingid_indexfindunit(source)— finds the unit with the given source string. Return type:TranslationUnitorNonefindunits(source)— finds the units with the given source string. Return type:TranslationUnitorNonegetids()— returns a list of unit idsgetprojectstyle()— gets the project type for this storegetsourcelanguage()— gets the source language for this storegettargetlanguage()— gets the target language for this storegetunits()— returns a list of all units in this storeisempty()— returnsTrueif the object doesn't contain any translation unitsmakeindex()— indexes the items in this store. At least.sourceindexshould be useful ✨property merge_on— the matching criterion to use when merging on. Returns the default matching criterion for all the subclasses. Return type:stringparse(propsrc)— reads the source of a properties file in and includes them as unitsclassmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an objectclassmethod parsestring(storestring)— converts the string representation back to an objectremove_unit_from_index(unit)— removes a unit from the source and location indexesremoveunit(unit)— removes the given unit from the object's list of units. Again, this method should always be used rather than trying to modify the list manually. Parameters:unit(TranslationUnit) — the unit that will be addedrequire_index()— makes sure the source index existssave()— saves to the file that data was originally read from, if availablesavefile(storefile)— writes the string representation to the given file (or filename)serialize(out)— writes the units back to filesetprojectstyle(project_style)— sets the project type for this storesetsourcelanguage(sourcelanguage)— sets the source language for this storesettargetlanguage(targetlanguage)— sets the target language for this storesuggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unittranslate(source)— returns the translated string for a given source string. Return type:StringorNoneunit_iter()— iterator over all the units in this store
☕ class translate.storage.properties.javautf16file(*args, **kwargs)
Extensions = ['properties']— a list of file extensions associated with this store typeMimetypes = None— a list of MIME types associated with this store typeName = 'Java Properties (UTF-16)'— the human-usable name of this store type, and yes she's the UTF-16 one 💜UnitClass— alias ofpropunitadd_unit_to_index(unit)— adds a unit to the source and location indexesaddsourceunit(source)— adds and returns a new unit with the given source string. Return type:TranslationUnitaddunit(unit)— appends the given unit to the object's list of units. Always use this method rather than trying to modify the list manually, fr fr. Parameters:unit(TranslationUnit) — the unit that will be addeddetect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding fromtext, using either thechardetlib or by trying to decode the filestatic fallback_detection(text)— simple detection based on BOM in casechardetisn't availablefindid(id)— finds the unit with the matchingidby checkingid_indexfindunit(source)— finds the unit with the given source string. Return type:TranslationUnitorNonefindunits(source)— finds the units with the given source string. Return type:TranslationUnitorNonegetids()— returns a list of unit idsgetprojectstyle()— gets the project type for this storegetsourcelanguage()— gets the source language for this storegettargetlanguage()— gets the target language for this storegetunits()— returns a list of all units in this storeisempty()— returnsTrueif the object doesn't contain any translation unitsmakeindex()— indexes the items in this store, with at least.sourceindexbeing useful 🌈property merge_on— the matching criterion to use when merging on. Returns the default matching criterion for all the subclasses. Return type:stringparse(propsrc)— reads the source of a properties file in and includes them as unitsclassmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an objectclassmethod parsestring(storestring)— converts the string representation back to an objectremove_unit_from_index(unit)— removes a unit from the source and location indexesremoveunit(unit)— removes the given unit from the object's list of units. Use this instead of editing the list by hand, bestie. Parameters:unit(TranslationUnit) — the unit that will be addedrequire_index()— makes sure the source index existssave()— saves to the file that data was originally read from, if availablesavefile(storefile)— writes the string representation to the given file (or filename)serialize(out)— writes the units back to filesetprojectstyle(project_style)— sets the project type for this storesetsourcelanguage(sourcelanguage)— sets the source language for this storesettargetlanguage(targetlanguage)— sets the target language for this storesuggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unittranslate(source)— returns the translated string for a given source string. Return type:StringorNoneunit_iter()— iterator over all the units in this store
☕ class translate.storage.properties.javautf8file(*args, **kwargs)
Extensions = ['properties']— the list of file extensions associated with this store typeMimetypes = None— the list of MIME types associated with this store typeName = 'Java Properties (UTF-8)'— the human-usable, actual-people-can-read-it name of this store type 💖UnitClass— alias ofpropunit(that's the unit flavor this store rolls with)add_unit_to_index(unit)— drops a unit into the source index and the location index ✨addsourceunit(source)— makes a brand-new unit from the source string you hand it, and gives it right back to you- Return type:
TranslationUnit
- Return type:
addunit(unit)— appends the unit you pass to this object's list of units- Lowkey a rule, bestie: always use this instead of poking at the list yourself 🙈
- Parameters:
unit(TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to sniff out a file encoding from the text, either by leaning on thechardetlib or by just… attempting to decode the thing 🕵️♀️static fallback_detection(text)— the simple, no-frills detection based on the BOM, for whenchardetisn't installed 💅findid(id)— hunts down the unit with the matching id by checkingid_indexfindunit(source)— finds the unit with the given source string- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— finds the units (plural, babes) with the given source string- Return type:
TranslationUnitorNone
- Return type:
getids()— returns a list of unit ids 📋getprojectstyle()— gets the project type for this storegetsourcelanguage()— gets the source language for this store 🌍gettargetlanguage()— gets the target language for this store 🎯getunits()— returns a list of all units in this storeisempty()— returnsTrueif the object doesn't contain any translation units (so, an empty vibe check fr fr)makeindex()— indexes the items in this store; at minimum.sourceindexshould be useful 🗂️property merge_on— the matching criterion used when merging on- Returns: the default matching criterion for all the subclasses
- Return type:
string
parse(propsrc)— reads the source of a properties file in and includes it as unitsclassmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back into an objectclassmethod parsestring(storestring)— converts the string representation back into an objectremove_unit_from_index(unit)— removes a unit from the source and location indexes 🧹removeunit(unit)— removes the given unit from the object's list of units- Same energy as
addunit: this is the method to use, don't be out here editing the list manually 😭 - Parameters:
unit(TranslationUnit) — the unit that will be added.
- Same energy as
require_index()— makes sure the source index actually exists 🫶save()— saves to the file that the data was originally read from, if that's availablesavefile(storefile)— writes the string representation to the given file (or filename)serialize(out)— writes the units back to filesetprojectstyle(project_style)— sets the project type for this storesetsourcelanguage(sourcelanguage)— sets the source language for this storesettargetlanguage(targetlanguage)— sets the target language for this storesuggestions_in_format = False— tells you whether this format can store suggestions and alternative translations for a unit (spoiler: it can't 💀)translate(source)— returns the translated string for a given source string- Return type:
StringorNone
- Return type:
unit_iter()— an iterator over all the units in this store 🔁
🦋 class translate.storage.properties.joomlafile(*args, **kwargs)
Extensions = ['ini']— a list of file extensions associated with this store typeMimetypes = None— a list of MIME types associated with this store typeName = 'Joomla Translations'— the human-usable name of this store type 👑UnitClass— alias ofpropunitadd_unit_to_index(unit)— add a unit to the source and location indexesaddsourceunit(source)— add and return a new unit with the given source string- Return type:
TranslationUnit
- Return type:
addunit(unit)— append the given unit to the object's list of units- Again for the people in the back: use this method, don't touch the list manually 🙏
- Parameters:
unit(TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from the text, using either thechardetlib or by trying to decode the filestatic fallback_detection(text)— simple detection based on the BOM, in casechardetisn't availablefindid(id)— find the unit with the matching id by checkingid_indexfindunit(source)— find the unit with the given source string- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— find the units with the given source string- Return type:
TranslationUnitorNone
- Return type:
getids()— return a list of unit idsgetprojectstyle()— get the project type for this storegetsourcelanguage()— get the source language for this storegettargetlanguage()— get the target language for this storegetunits()— return a list of all units in this storeisempty()— returnTrueif the object doesn't contain any translation unitsmakeindex()— indexes the items in this store; at least.sourceindexshould be usefulproperty merge_on— the matching criterion to use when merging on- Returns: the default matching criterion for all the subclasses
- Return type:
string
parse(propsrc)— read the source of a properties file in and include them as unitsclassmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an objectclassmethod parsestring(storestring)— convert the string representation back to an objectremove_unit_from_index(unit)— remove a unit from the source and location indexesremoveunit(unit)— remove the given unit from the object's list of units- Parameters:
unit(TranslationUnit) — the unit that will be added.
- Parameters:
require_index()— make sure the source index existssave()— save to the file that data was originally read from, if availablesavefile(storefile)— write the string representation to the given file (or filename)serialize(out)— write the units back to filesetprojectstyle(project_style)— set the project type for this storesetsourcelanguage(sourcelanguage)— set the source language for this storesettargetlanguage(targetlanguage)— set the target language for this storesuggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unittranslate(source)— return the translated string for a given source string- Return type:
StringorNone
- Return type:
unit_iter()— iterator over all the units in this store
translate.storage.properties.labelsuffixes = ('.label', '.title')— label suffixes: entries rocking this suffix are able to be combined with accesskeys found in entries ending withaccesskeysuffixes🏷️
🌸 class translate.storage.properties.propfile(inputfile=None, personality='java', encoding=None)
This class represents a .properties file, made up of propunits ✨
Extensions = None— a list of file extensions associated with this store typeMimetypes = None— a list of MIME types associated with this store typeName = 'Base translation store'— the human-usable name of this store typeUnitClass— alias ofpropunitadd_unit_to_index(unit)— add a unit to the source and location indexesaddsourceunit(source)— add and return a new unit with the given source string- Return type:
TranslationUnit
- Return type:
addunit(unit)— append the given unit to the object's list of units- You already know the drill: this method only, no manual list surgery 💅
- Parameters:
unit(TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from the text, using either thechardetlib or by trying to decode the filestatic fallback_detection(text)— simple detection based on the BOM, in casechardetisn't availablefindid(id)— finds the unit with the matching id by checkingid_index🕵️♀️findunit(source)— finds the unit with the given source string.- Return type:
TranslationUnit or None
- Return type:
findunits(source)— finds the units with the given source string.- Return type:
TranslationUnit or None
- Return type:
getids()— returns a list of unit ids. 🎀getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store.isempty()— returnsTrueif the object doesn’t contain any translation units. 🌸makeindex()— indexes the items in this store. At least.sourceindexshould be useful.- property
merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(propsrc)— reads the source of a properties file in and includes them as units. 💅- classmethod
parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object. - classmethod
parsestring(storestring)— converts the string representation back to an object. remove_unit_from_index(unit)— removes a unit from source and locaton indexes. 🫶removeunit(unit)— removes the given unit to the object’s list of units.- This method should always be used rather than trying to modify the list manually.
- Parameters:
unit(TranslationUnit) – the unit that will be added.
require_index()— makes sure source index exists.save()— saves to the file that data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— writes the units back to file.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates if format can store suggestions and alternative translation for a unit.translate(source)— returns the translated string for a given source string.- Return type:
String or None
- Return type:
unit_iter()— iterator over all the units in this store.
🧚 class translate.storage.properties.proppluralunit(source='', personality='java')
Here’s the whole method lineup for this cutie:
adderror(errorname, errortext)— adds an error message to this unit.- Parameters:
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
- Parameters:
addlocation(location)— adds one location to the list of locations.
NOTE: Shouldn’t be implemented if the format doesn’t support it. 🙈
addlocations(location)— adds a location or a list of locations.
NOTE: Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: This method might be removed in future. 💀
addnote(text, origin=None, position='append')— adds a note (comment).- Parameters:
text(string) – usually just a sentence or two.origin(string) – specifies who/where the comment comes from. Origin can be one of the following text strings:'translator','developer','programmer','source code'(synonyms).
- Parameters:
- classmethod
buildfromunit(unit)— builds a native unit from a foreign unit. Preserving as much information as possible. 🦋 - static
getcontext()— gets the message context. - static
geterrors()— gets all error messages.- Return type:
Dictionary
- Return type:
getid()— a unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store.
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.
- Return type:
getlocations()— a list of source code locations.- Return type:
List
- Return type:
NOTE: Shouldn’t be implemented if the format doesn’t support it.
getnotes(origin=None)— returns all notes about this unit. It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()). 💖gettargetlen()— returns the length of the target string.- Return type:
Integer
- Return type:
NOTE: Plural forms might be combined.
getunits()— this unit in a list.hasplural(key=None)— tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— returns whether this is a blank element, containing only comments.- static
isfuzzy()— indicates whether this unit is fuzzy. - static
isheader()— indicates whether this unit is a header. - static
isobsolete()— indicate whether a unit is obsolete. - static
isreview()— indicates whether this unit needs review. 👑 istranslatable()— indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.istranslated()— indicates whether this unit is translated. This should be used rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does).makeobsolete()— makes a unit obsolete.markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.- Parameters:
needsreview– defaults toTrue.explanation– adds an optional explanation as a note.
- Parameters:
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic format agnostic merging. 🌈multistring_to_rich(mulstring)— converts a multistring to a list of “rich” string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator’s notes.rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.- property
rich_source- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
- property
rich_target- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
- classmethod
rich_to_multistring(elem_list)— converts a “rich” string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setid(value)— sets the unique identified for this unit. Only implemented if format allows ids independant from other unit properties like source or context.unit_iter()— iterator that only returns this unit.
🦋 class translate.storage.properties.propunit(source='', personality='java')
An element of a properties file i.e. a name and value, and any comments associated.
adderror(errorname, errortext)— adds an error message to this unit.- Parameters:
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
- Parameters:
addlocation(location)— adds one location to the list of locations.
NOTE: Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: This method might be removed in future. 💀
addnote(text, origin=None, position='append')— Adds a note (comment). 📝text(string) — Usually just a sentence or two.origin(string) — Specifies who/where the comment comes from. Origin can be one of the following text strings:'translator''developer','programmer','source code'(synonyms)
classmethod buildfromunit(unit)— Build a native unit from a foreign unit. Preserving as much information as possible. 🧚static get_missing_part()— Return the string representing a missing translation.static getcontext()— Get the message context.static geterrors()— Get all error messages.- Return type:
Dictionary
- Return type:
getid()— A unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.
- Return type:
getlocations()— A list of source code locations.- Return type:
List - NOTE: Shouldn’t be implemented if the format doesn’t support it. 🙈
- Return type:
getnotes(origin=None)— Returns all notes about this unit. It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()).getoutput()— Convert the element back into formatted lines for a.propertiesfile.gettargetlen()— Returns the length of the target string.- Return type:
Integer - NOTE: Plural forms might be combined. 🧩
- Return type:
getunits()— This unit in a list.static hasplural()— Tells whether or not this specific unit has plural strings.infer_state()— Empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— Returns whether this is a blank element, containing only comments.static isfuzzy()— Indicates whether this unit is fuzzy.static isheader()— Indicates whether this unit is a header.static isobsolete()— Indicate whether a unit is obsolete.static isreview()— Indicates whether this unit needs review.istranslatable()— Indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.istranslated()— Indicates whether this unit is translated. This should be used rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does).makeobsolete()— Make a unit obsolete.markfuzzy(value=True)— Marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— Marks the unit to indicate whether it needs review.needsreview— Defaults toTrue.explanation— Adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— Do basic format agnostic merging.multistring_to_rich(mulstring)— Convert a multistring to a list of “rich” string trees. 🌳
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— Remove all the translator’s notes.static represents_missing(line)— The line represents a missing translation.rich_parsers = []— A list of functions to use for parsing a string into a rich string tree.property rich_source- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
property rich_target- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
classmethod rich_to_multistring(elem_list)— Convert a “rich” string tree to a multistring. ✨
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— Set the message context.setid(value)— Sets the unique identified for this unit. Only implemented if format allows ids independant from other unit properties like source or context.static strip_missing_part(line)— Remove the missing prefix from the line.unit_iter()— Iterator that only returns this unit.
translate.storage.properties.register_dialect(dialect)— Decorator that registers the dialect.
class translate.storage.properties.stringsfile(*args, **kwargs)
Extensions = ['strings']— A list of file extentions associated with this store type.Mimetypes = None— A list of MIME types associated with this store type.Name = 'OS X Strings'— The human usable name of this store type.UnitClass— alias ofpropunit.add_unit_to_index(unit)— Add a unit to source and location idexes.addsourceunit(source)— Add and returns a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— Append the given unit to the object’s list of units. This method should always be used rather than trying to modify the list manually.unit(TranslationUnit) — The unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— Try to detect a file encoding from text, using either the chardet lib or by trying to decode the file.static fallback_detection(text)— Simple detection based on BOM in case chardet is not available.findid(id)— Find unit with matching id by checking id_index.findunit(source)— Find the unit with the given source string.- Return type:
TranslationUnit or None
- Return type:
findunits(source)— Find the units with the given source string.- Return type:
TranslationUnit or None
- Return type:
getids()— Return a list of unit ids.getprojectstyle()— Get the project type for this store.getsourcelanguage()— Get the source language for this store.gettargetlanguage()— Get the target language for this store.getunits()— Return a list of all units in this store.isempty()— ReturnTrueif the object doesn’t contain any translation units.makeindex()— Indexes the items in this store. At least.sourceindexshould be useful.property merge_on— The matching criterion to use when merging on.- Returns: The default matching criterion for all the subclasses.
- Return type:
string
parse(propsrc)— Read the source of a properties file in and include them as units.classmethod parsefile(storefile)— Reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— Convert the string representation back to an object.remove_unit_from_index(unit)— Remove a unit from source and locaton indexes.removeunit(unit)— Remove the given unit to the object’s list of units. This method should always be used rather than trying to modify the list manually.unit(TranslationUnit) — The unit that will be added.
require_index()— Make sure source index exists.save()— Save to the file that data was originally read from, if available.savefile(storefile)— Write the string representation to the given file (or filename).serialize(out)— Write the units back to file.setprojectstyle(project_style)— Set the project type for this store.setsourcelanguage(sourcelanguage)— Set the source language for this store.settargetlanguage(targetlanguage)— Set the target language for this store.suggestions_in_format = False— Indicates if format can store suggestions and alternative translation for a unit.translate(source)— hands you back the translated string for a source string you give it. Return type:StringorNone. ✨unit_iter()— an iterator that walks over every single unit in this store. all of them. yes bestie, all of them 💅
🦋 class translate.storage.properties.stringsutf8file(*args, **kwargs)
Okay so this one is the OS X Strings (UTF-8) flavour of things — here's the whole vibe check:
Extensions = ['strings']— the list of file extensions that get associated with this store type.Mimetypes = None— the list of MIME types associated with this store type. and it's literallyNone, no cap.Name = 'OS X Strings (UTF-8)'— the human-usable name of this store type, i.e. what you'd actually read.UnitClass— alias ofpropunit.add_unit_to_index(unit)— drops a unit into the source and location indexes.addsourceunit(source)— adds and returns a brand-new unit holding the source string you passed in. Return type:TranslationUnit.addunit(unit)— appends the given unit to the object's list of units. 🫶
This method should always be used rather than trying to modify the list manually. like, fr fr, don't touch the list yourself, bestie.
Parameters: unit (TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, either using the chardet lib or by just trying to decode the file. icon behaviour honestly.static fallback_detection(text)— simple detection based on BOM in case chardet isn't available. the backup bestie 🎀findid(id)— finds a unit with a matching id by checkingid_index.findunit(source)— finds the unit with the given source string. Return type:TranslationUnitorNone.findunits(source)— finds the units with the given source string. (yes, plural, it's giving "all the matches"). Return type:TranslationUnitorNone.getids()— returns a list of unit ids.getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store. the whole roster 💜isempty()— returnsTrueif the object doesn't contain any translation units. sad little empty store vibe.makeindex()— indexes the items in this store. at least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on. Returns the default matching criterion for all the subclasses. Return type:string.parse(propsrc)— reads the source of a properties file in and includes them as units.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— converts the string representation back to an object.remove_unit_from_index(unit)— removes a unit from the source and location indexes.removeunit(unit)— removes the given unit from the object's list of units.
Same rule as before: always use this method instead of poking the list manually. 🙈
Parameters: unit (TranslationUnit) — the unit that will be added.
require_index()— makes sure the source index exists.save()— saves to the file that data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— writes the units back to file.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit. spoiler: this one cannot. 💀translate(source)— returns the translated string for a given source string. Return type:StringorNone.unit_iter()— iterator over all the units in this store.
🌈 class translate.storage.properties.xwikifile(*args, **kwargs)
And now the XWiki Properties store — very similar energy, different extension:
Extensions = ['properties']— a list of file extensions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'XWiki Properties'— the human-usable name of this store type.UnitClass— alias ofxwikiunit.add_unit_to_index(unit)— adds a unit to the source and location indexes.addsourceunit(source)— adds and returns a new unit with the given source string. Return type:TranslationUnit.addunit(unit)— appends the given unit to the object's list of units.
Again: always use this method rather than trying to modify the list manually. consistency is key, bestie 🔥
Parameters: unit (TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, using either the chardet lib or by trying to decode the file.static fallback_detection(text)— simple detection based on BOM in case chardet is not available.findid(id)— finds a unit with matching id by checkingid_index.findunit(source)— finds the unit with the given source string. Return type:TranslationUnitorNone.findunits(source)— finds the units with the given source string. Return type:TranslationUnitorNone.getids()— returns a list of unit ids.getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store.isempty()— returnsTrueif the object doesn't contain any translation units.makeindex()— indexes the items in this store. at least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on. Returns the default matching criterion for all the subclasses. Return type:string.parse(propsrc)— reads the source of a properties file in and includes them as units.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object.classmethod parsestring(storestring)— converts the string representation back to an object.remove_unit_from_index(unit)— removes a unit from the source and location indexes.removeunit(unit)— removes the given unit from the object's list of units.
Use this method, don't hand-edit the list. okay? okay. 🫶
Parameters: unit (TranslationUnit) — the unit that will be added.
require_index()— makes sure the source index exists.save()— saves to the file that data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— writes the units back to file.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit. nope, not this one either 💀translate(source)— returns the translated string for a given source string. Return type:StringorNone.unit_iter()— iterator over all the units in this store.
👑 class translate.storage.properties.xwikiunit(source='', personality='xwiki')
This one represents an XWiki translation unit, and the way it differs from a propunit is literally twofold — bestie wrote it as a numbered list so I'm keeping it as one:
- the dialect used is
xwikifor simple quote escape handling - missing translations are output with a dedicated
"### Missing: "prefix.
Obsessed with a class that tells you when stuff is missing, honestly.
adderror(errorname, errortext)— adds an error message to this unit. Parameters:errorname(string) — a single word to id the error.errortext(string) — the text describing the error.
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. periodt. 💅
addlocations(location)— adds a location or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation(). iykyk 🧚
⚠️ WARNING: This method might be removed in future.
addnote(text, origin=None, position='append') — Adds a note (comment). Lowkey useful for leaving lil messages 💬
text(string) – Usually just a sentence or two.origin(string) – Specifies who/where the comment comes from. Origin can be one of the following text strings:'translator','developer','programmer','source code'(synonyms). Slay, take ur pick ✨
buildfromunit(unit) (classmethod) — Build a native unit from a foreign unit. Preserving as much information as possible. No info left behind, periodt 💅
get_missing_part() (static) — Return the string representing a missing translation. It's giving "where did she go" energy 😭
getcontext() (static) — Get the message context.
geterrors() (static) — Get all error messages.
Return type: Dictionary
getid() — A unique identifier for this unit.
Return type: string
Returns: an identifier for this unit that is unique in the store. Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. Like a fingerprint but for translation units, iykyk 🔍
getlocations() — A list of source code locations.
Return type: List
NOTE: Shouldn't be implemented if the format doesn't support it. Don't force it bestie 🙈
getnotes(origin=None) — Returns all notes about this unit. It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see getlocations()). Keep it cute and separate ✨
getoutput() — Convert the element back into formatted lines for a .properties file.
gettargetlen() — Returns the length of the target string.
Return type: Integer
NOTE: Plural forms might be combined. So the length might be doing the most 💀
getunits() — This unit in a list. Just the one, main character energy 👑
hasplural() (static) — Tells whether or not this specific unit has plural strings.
infer_state() — Empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state. It's giving "figure it out from context" vibes 🧚
isblank() — Returns whether this is a blank element, containing only comments. No actual content, just notes 🎀
isfuzzy() (static) — Indicates whether this unit is fuzzy. Not quite sure of itself, relatable 🫶
isheader() (static) — Indicates whether this unit is a header.
isobsolete() (static) — Indicate whether a unit is obsolete.
isreview() (static) — Indicates whether this unit needs review. Needs a second pair of eyes 👀
istranslatable() — Indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. Like sorting the real ones from the extras 💅
istranslated() — Indicates whether this unit is translated. This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does). Trust the method, not the vibe check 🔥
makeobsolete() — Make a unit obsolete. It's giving "retired, don't @ me" 💀
markfuzzy(value=True) — Marks the unit as fuzzy or not.
markreviewneeded(needsreview=True, explanation=None) — Marks the unit to indicate whether it needs review.
Parameters:
needsreview– Defaults toTrue.explanation– Adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False) — Do basic format agnostic merging. Blend them together smoothly 🧚
multistring_to_rich(mulstring) — Convert a multistring to a list of "rich" string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None) — Remove all the translator's notes. Bye bye notes 👋
represents_missing(line) (classmethod) — Return true if the line represents a missing translation.
rich_parsers = [] — A list of functions to use for parsing a string into a rich string tree.
property rich_source — SEE ALSO: rich_to_multistring(), multistring_to_rich().
property rich_target — SEE ALSO: rich_to_multistring(), multistring_to_rich().
rich_to_multistring(elem_list) (classmethod) — Convert a "rich" string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context) — Set the message context.
setid(value) — Sets the unique identified for this unit. only implemented if format allows ids independant from other unit properties like source or context
strip_missing_part(line) (classmethod) — Remove the missing prefix from the line.
unit_iter() — Iterator that only returns this unit. Just this one, no extras 💖
🐍 pypo
Classes that hold units of Gettext .po files (pounit) or entire files (pofile). Get into it bestie, this is the PO file zone ✨
class translate.storage.pypo.PoWrapper(width=77)
Customized TextWrapper.
- custom word separator regexp
- full width chars accounting, based on ‐ https://bugs.python.org/issue24665
- dropped support for unused features (for example
max_linesordrop_whitespace)
fill(text: string) -> string — Reformat the single paragraph in 'text' to fit in lines of no more than 'self.width' columns, and return a new string containing the entire wrapped paragraph. Wrapping it up cute 🎀
wrap(text: string) -> [string] — Reformat the single paragraph in 'text' so it fits in lines of no more than 'self.width' columns, and return a list of wrapped lines. Tabs in 'text' are expanded with string.expandtabs(), and all other whitespace characters (including newline) are converted to space. Slay formatting 💅
translate.storage.pypo.cjklen(text: str) -> int — Return the real width of an unicode text, the len of any other type. Fullwidth and Wide CJK chars are double-width. Math but make it text 📏
translate.storage.pypo.cjkslices(text: str, index: int) -> tuple[str, str] — Return the two slices of a text cut to the index.
translate.storage.pypo.escapeforpo(line) — Escapes a line for po format. assumes no n occurs in the line.
Parameters:
line– unescaped text
translate.storage.pypo.lsep = '\n#: ' — Separator for #: entries
class translate.storage.pypo.pofile(inputfile=None, width=None, **kwargs)
A .po file containing various units. The whole package, no cap 💖
Extensions = ['po', 'pot']— A list of file extentions associated with this store typeMimetypes = ['text/x-gettext-catalog', 'text/x-gettext-translation', 'text/x-po', 'text/x-pot']— A list of MIME types associated with this store typeName = 'Gettext PO file'— The human usable name of this store typeUnitClass— alias ofpounit
add_unit_to_index(unit) — Add a unit to source and location idexes. Keeping track of everything, organized queen 👑
addsourceunit(source) — Add and returns a new unit with the given source string.
Return type: TranslationUnit
addunit(unit) — Append the given unit to the object's list of units. This method should always be used rather than trying to modify the list manually. Don't touch the list manually bestie, use the method 🙈
Parameters:
unit(TranslationUnit) – The unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str] — Try to detect a file encoding from text, using either the chardet lib or by trying to decode the file. Detective vibes 🔍
fallback_detection(text) (static) — Simple detection based on BOM in case chardet is not available. Backup plan, we love a safety net 🫶
findid(id) — Find unit with matching id by checking id_index. Searching for the one 💅
findunit(source) — Find the unit with the given source string.
Return type: TranslationUnit or None
findunits(source) — Find the units with the given source string.
Return type: TranslationUnit or None
getheaderplural() — Returns the nplural and plural values from the header.
getids() — Return a list of unit ids.
getprojectstyle() — Return the project based on information in the header. The project is determined in the following sequence:
- Use the
'X-Project-Style'entry in the header. - Use
'Report-Msgid-Bug-To'entry - Use the
'X-Accelerator'entry - Use the Project ID
- Analyse the file itself (not yet implemented)
getsourcelanguage() — Get the source language for this store.
gettargetlanguage() — Return the target language based on information in the header. The target language is determined in the following sequence:
- Use the
'Language'entry in the header. - Poedit's custom headers.
- Analysing the ‘Language-Team’ entry.
getunits()
Hands you a list of every single unit chilling in this store. All of them. The whole squad. 💅
header()
Returns the header element — or None if there isn't one. Heads up: only the first element is allowed to be a header (main character rules, sorry). Also note that this can totally still hand you back an empty header element if one happens to be present. It exists. It's just not doing much. 🙈
init_headers(charset='UTF-8', encoding='8bit', **kwargs)
Sets up default values for your po headers. Fresh start energy. ✨
isempty()
Returns True if the object doesn't contain any translation units. That's it. Empty. Void. Nothing. 🫙
makeheader(**kwargs)
Creates a header for the given filename. Wanna know what the parameters do? Go peek at .makeheaderdict() — it's all documented there, bestie. 🎀
makeheaderdict(charset='CHARSET', encoding='ENCODING', project_id_version=None, pot_creation_date=None, po_revision_date=None, last_translator=None, language_team=None, mime_version=None, plural_forms=None, report_msgid_bugs_to=None, **kwargs)
Builds you a header dictionary with genuinely useful defaults. Iconic behaviour, honestly. 👑
Now the date deets, listen up:
pot_creation_datecan beNone(which means current date) or a value (adatetimeor a string).po_revision_datecan beNone(which means form),False(which means=pot_creation_date),True(which means=now), or a value (adatetimeor a string).
Returns
- A dictionary with the header items.
Return type
dictof strings.
makeindex()
Indexes all the items in this store. At minimum, .sourceindex should be useful. Go off, index. 🔥
property merge_on
The matching criterion to use when merging on. That's the vibe check for merges. ⭐
mergeheaders(otherstore)
Merges another header together with this header. This header is assumed to be the template — so it's the blueprint, the moment, the main character. 🦋
parse(input)
Parses the given file, or a file source string. Feed it either, it eats. 🍓
classmethod parsefile(storefile)
Reads the given file (or opens the given filename for you) and parses it back into an object. Like a glow-up in reverse: file in, object out. ✨
parseheader()
Parses the PO header and returns the interpreted values as a dictionary. It's giving translator. 🧚
classmethod parsestring(storestring)
Converts the string representation back into an object. Strings can be objects too, periodt. 💜
remove_unit_from_index(unit)
Removes a unit from the source and location indexes. Bye babe. 👋
removeduplicates(duplicatestyle='merge')
Makes sure every msgid is unique; merges comments and so on from the duplicates into the original. One of each, no clones. 🎀
removeunit(unit)
Removes the given unit to the object's list of units.
This method should always be used rather than trying to mess with the list manually. Do not touch the list by hand, I'm literally begging. 😭
Parameters
- unit (
TranslationUnit) — the unit that will be added.
require_index()
Makes sure the source index exists. If it's not there, it gets made. Simple as. 💅
save()
Saves to the file that the data was originally read from — if that file is available. Memory-based glow-up. 🌸
savefile(storefile)
Writes the string representation to the given file (or filename). 💾
serialize(out)
Writes to file. That's the whole bit. ✍️
setprojectstyle(project_style)
Sets the project in the header.
Parameters
- project_style (
str) — the new project.
setsourcelanguage(sourcelanguage)
Sets the source language for this store. Where it's coming from, linguistically. 🌈
settargetlanguage(lang)
Sets the target language in the header.
Careful bestie — this removes any custom Poedit headers if they exist. They're gone. Poof. 💀
Parameters
- lang (
str) — the new target language code.
suggestions_in_format = False
Tells you whether the format can store suggestions and alternative translations for a unit. False means nope, it cannot. 🙅♀️
translate(source)
Returns the translated string for a given source string.
Return type
StringorNone.
unit_iter()
An iterator over all the units in this store. One at a time, no skipping. 🔁
updatecontributor(name, email=None)
Adds contribution comments if necessary. Give credit where credit is due, always. 🫶
updateheader(add=False, **kwargs)
Updates the fields in the PO style header.
It will create a header if add == True. Otherwise it just updates what's already there. No gatekeeping. ⭐
updateheaderplural(nplurals, plural)
Updates the Plural-Form PO header. Plural girlies rise. 🦋
class translate.storage.pypo.pounit(source=None, wrapper=None, **kwargs)
adderror(errorname, errortext)
Adds an error message to this unit. Logging the drama, but productively. 📝
addlocation(location)
Adds a location to sourcecomments in the PO unit.
Parameters
- location (
String) — text location, e.g.'file.c:23'. Does not include#:.
addlocations(location)
Adds a location, or a list of locations. Whichever you're holding. 📍
NOTE: Most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: This method might be removed in future. So don't get too attached. 😬
addnote(text, origin=None, position='append')
This is modeled on the XLIFF method.
See translate.storage.xliff.xliffunit.addnote(). 💬
classmethod buildfromunit(unit)
Builds a native unit from a foreign unit, preserving as much information as possible. Adopting it into the family. 🫶
getalttrans()
Returns a list of alternate units.
The previous msgid and the current msgstr get combined to form a single alternative unit. Two become one. 💫
getcontext()
Gets the message context. Context is everything, honestly. 🔍
geterrors()
Gets all error messages. All of them. The full receipts. 🧾
getid()
Returns a unique identifier for this unit. No two units share an id, that's the whole point. 👑
getlocations()
Gets a list of locations from sourcecomments in the PO unit.
rtype: List return: a list of the locations with '#: ' stripped off.
getnotes(origin=None)
Returns comments based on the origin value.
Parameters
- origin —
programmer,developer,source code,translatororNone.
gettargetlen()
Returns the length of the target string.
Return type
Integer
NOTE: Plural forms might be combined. So the number might be doing a little bit extra. 🎀
getunits()
This unit — in a list. Just the one. It's a party of one. 🎉
hasmarkedcomment(commentmarker)
Checks whether the given comment marker is present.
These should appear as:
# (commentmarker) ...hasplural()
Returns whether this pounit contains plural strings… 👀
hastypecomment(typecomment, parsed=None)
Checks whether the given type comment is present. 🔎
infer_state()
An empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state. It's a placeholder doing absolutely nothing, and it's slaying at it. 💅
isblank()
Used to see if this unit has no source or target string.
NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. Lowkey a legacy situation. 🙈
isfuzzy()
Indicates whether this unit is fuzzy. Blurry vibes. 🌀
isheader()
Indicates whether this unit is a header. Is it the header? Inquiring minds want to know. 🎀
isobsolete()
Indicates whether a unit is obsolete. Old news. Archived. 💀
isreview()
Indicates whether this unit needs review. The vibe check hasn't been passed yet. 👀
istranslatable()
Indicates whether this unit can be translated.
This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. Basically: is this an actual thing to translate, or is it background noise? ✨
istranslated()
Indicates whether this unit is translated.
This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does). Use the method, bestie, not the vibes. 🫶
makeobsolete()
Makes this unit obsolete. Retirement arc. 🌅
markfuzzy(present=True)
Marks the unit as fuzzy — or not, depending on present. 🌀
markreviewneeded(needsreview=True, explanation=None)
Marks the unit to indicate whether it needs review. Also adds an optional explanation as a note. Receipts included. 📝
merge(otherpo, overwrite=False, comments=True, authoritative=False)
Merges otherpo (with the same msgid) into this one.
Overwrite non-blank self.msgstr only if overwrite is True. Merge comments only if comments is True. Respect the flags, she's specific. 🔥
property msgidcomment
Extracts KDE style msgid comments from the unit.
Return type
String
Returns
- The extracted msgidcomments found in this unit's
msgid.
multistring_to_rich(mulstring)
Converts a multistring into a list of "rich" string trees. Glow-up for strings. 🌈
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]property prev_source
Returns the unescaped msgid. Cleaned up and ready. ✨
removenotes(origin=None)
Removes all the translator's notes (the "other comments"). Sorry notes, you're out. 🧹
resurrect()
Makes an obsolete unit normal again. Comeback era!! 🔥👑
rich_parsers = []
A list of functions to use for parsing a string into a rich string tree. Empty by default, filling it is your moment. 🎀
property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich(). 💜
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich(). 💜
classmethod rich_to_multistring(elem_list)
Converts a "rich" string tree to a multistring. The reverse glow-up. 🦋
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context, bestie. That's it, that's the whole vibe ✨setid(value)— sets the unique identifier for this unit. Lowkey only actually works if the format lets ids be independent from other unit properties like source or context 🫶settypecomment(typecomment, present=True)— flips whether a given typecomment is hanging out in there or not 🎀- property
source— hands you the unescaped msgid, no cap - property
target— hands you the unescaped msgstr 💖 unit_iter()— an iterator that only ever returns this unit. main character energy, honestly 👑
🦋 translate.storage.pypo.quoteforpo(text, wrapper_obj=None)
Quotes the text you give it for a PO file, and gives you back the quoted and escaped lines. Iconic little helper, fr fr 💅
🦋 translate.storage.pypo.splitlines(text)
Splits lines based on the first newline char it finds.
Okay so real talk — it can not use universal newlines, because those match any newline-like character inside the text, and that breaks on files that have unix newlines plus LF chars inside comments 😭 The code looks for the first msgid and then checks what newline was used after it. That should safely cover weird newlines lurking in comments or filenames, while still properly parsing po files with any newlines. slay 🌸
🦋 translate.storage.pypo.unescape(line)
Unescapes the line you pass in. Heads up: the quotes on either side should already have been removed before you call this, bestie 🎀
👑 qm
Module for parsing Qt .qm files. We love a niche format, don't we 💜
NOTE: based on documentation from Gettext's.qmimplementation (seewrite-qt.c) and on observation of the output oflrelease. iykyk 🔥
NOTE: certain deprecated section tags are not implemented. These will break and print out the missing tag. They're easy to implement and should follow the structure in 03 (Translation). We could find no examples that use these, so we'd rather leave them unimplemented until we actually have test data 🙈
NOTE: many.qmfiles are unable to be parsed as they do not have the source text. We assume that since they use a hash table to look up the data there's actually no need for the source text. It seems however that in Qt4'slreleaseall data is included in the resultant.qmfile 💀
NOTE: we can only parse, not create, a .qm file. The main issue is that we need to implement the hashing algorithm (which seems identical to the Gettext hash algorithm). Unlike Gettext it seems the hash is required, but that has not been validated 😭
NOTE: the code can parse files correctly. But it could be cleaned up to be more readable, especially the part that breaks the file into sections 🫶
http://qt.gitorious.org/+kde-developers/qt/kde-qt/blobs/master/tools/linguist/shared/qm.cpp
Plural information QLocale languages 🌈
🌟 class translate.storage.qm.qmfile(inputfile=None, **kwargs)
A class representing a .qm file. She's the whole store, basically 💖
Extensions = ['qm']— a list of file extensions associated with this store typeMimetypes = ['application/x-qm']— a list of MIME types associated with this store typeName = 'Qt .qm file'— the human-usable name of this store type, for when you forget what you're looking at 🎀UnitClass— alias ofqmunitadd_unit_to_index(unit)— add a unit to the source and location indexesaddsourceunit(source)— add and return a new unit with the given source string. Return type:TranslationUnit✨addunit(unit)— append the given unit to the object's list of units. Always use this method instead of trying to mess with the list manually, okay?- Parameters:
unit(TranslationUnit) – the unit that will be added.
- Parameters:
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from the text, either using thechardetlib or by trying to decode the file 🔍static fallback_detection(text)— simple detection based on the BOM, for whenchardetisn't availablefindid(id)— finds the unit with the matching id by checkingid_indexfindunit(source)— finds the unit with the given source string. Return type:TranslationUnitorNonefindunits(source)— finds the units with the given source string. Return type:TranslationUnitorNonegetids()— returns a list of unit idsgetprojectstyle()— gets the project type for this storegetsourcelanguage()— gets the source language for this storegettargetlanguage()— gets the target language for this storegetunits()— returns a list of all units in this storeisempty()— returnsTrueif the object doesn't contain any translation unitsmakeindex()— indexes the items in this store. At least.sourceindexshould be useful 💅- property
merge_on— the matching criterion to use when merging on- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(input)— parses the given file or file source stringclassmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back to an objectclassmethod parsestring(storestring)— converts the string representation back to an objectremove_unit_from_index(unit)— removes a unit from the source and location indexesremoveunit(unit)— removes the given unit from the object's list of units. Again: use this method, don't touch the list yourself 🙈- Parameters:
unit(TranslationUnit) – the unit that will be added.
- Parameters:
require_index()— make sure the source index existssave()— save to the file that data was originally read from, if availablesavefile(storefile)— writes the string representation to the given file (or filename)serialize(out)— outputs a string representation of the.qmdata filesetprojectstyle(project_style)— sets the project type for this storesetsourcelanguage(sourcelanguage)— sets the source language for this storesettargetlanguage(targetlanguage)— sets the target language for this storesuggestions_in_format = False— indicates whether the format can store suggestions and alternative translations for a unittranslate(source)— returns the translated string for a given source string. Return type:StringorNoneunit_iter()— iterator over all the units in this store ⭐
🌟 class translate.storage.qm.qmunit(source=None)
A class representing a .qm translation message. The little icon she is 💖
adderror(errorname, errortext)— adds an error message to this unit- Parameters:
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
- Parameters:
addlocation(location)— adds one location to the list of locations
NOTE: shouldn't be implemented if the format doesn't support it 🫶
addlocations(location)— adds a location, or a list of locations
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future 😬
addnote(text, origin=None, position='append')— adds a note (comment)- Parameters:
text(string) – usually just a sentence or two.origin(string) – specifies who/where the comment comes from. Origin can be one of the following text strings:- 'translator' - 'developer', 'programmer', 'source code'(synonyms)
- Parameters:
classmethod buildfromunit(unit)— builds a native unit from a foreign unit, preserving as much information as possible 🧚static getcontext()— gets the message contextstatic geterrors()— gets all error messages. Return type:Dictionarygetid()— a unique identifier for this unit- Return type:
string - Returns: an identifier for this unit that is unique in the store.
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store 👑
- Return type:
static getlocations()— a list of source code locations- Return type:
List
- Return type:
NOTE: shouldn't be implemented if the format doesn't support it 💅
getnotes(origin=None)— returns all notes about this unit. It'll probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()) 🎀gettargetlen()— returns the length of the target string- Return type:
Integer
- Return type:
NOTE: plural forms might be combined 🔥
getunits()— this unit in a list 🌸static hasplural()— tells you whether this specific unit has plural strings. yep, unit by unit, no shortcuts ✨infer_state()— an empty method you're supposed to override in your subclasses so the unit can figure out its currentstate(_n)from its currentstate. it's giving "build-your-own" energy 🎀isblank()— checks if this unit has no source or target string at all.
NOTE: this is probably used more to find translatable units, and honestly we might want to move in that direction instead and just get rid of this. 🙈 real "we'll deal with it later" behaviour.
static isfuzzy()— says whether this unit is fuzzy.static isheader()— says whether this unit is a header.static isobsolete()— says whether a unit is obsolete (rip bozo 💀).static isreview()— says whether this unit needs review. 👀istranslatable()— says whether this unit can be translated.
use this to tell real, translatable units apart from headers, obsolete units, binary stuff, or any other blank units. it's the vibe check that keeps the fakes out 💅
istranslated()— says whether this unit is translated.
fr use this instead of deducing it from .target, because that way other classes can implement more functionality (XLIFF does exactly this 🫶).
makeobsolete()— makes a unit obsolete.markfuzzy(value=True)— marks the unit as fuzzy, or un-fuzzy.markreviewneeded(needsreview=True, explanation=None)— marks the unit to show whether it needs review.
Parameters
- needsreview — defaults to
True. - explanation — adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does a basic, format-agnostic merge. lowkey the peacemaker of the class 🤝multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator's notes. all of them. bye 👋rich_parsers = []— a list of functions used for parsing a string into a rich string tree.property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a "rich" string tree into a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setid(value)— sets the unique identifier for this unit.
only implemented if the format allows ids that are independant from other unit properties like source or context.
unit_iter()— an iterator that only returns this one unit. main character energy, honestly 👑
🧰 translate.storage.qm.qmunpack(file_='messages.qm')
a lil helper that unpacks Qt .qm files into a Python string. cute and useful ✨
🌸 qph
module for handling Qt Linguist Phrase Book (.qph) files.
straight from the Qt Linguist Manual: Translators: .qph Qt Phrase Book Files are human-readable XML files containing standard phrases and their translations. These files are created and updated by Qt Linguist and may be used by any number of projects and applications.
a DTD that defines the format doesn't seem to exist, but the following ‐ code provides the reference implementation for the Qt Linguist product. iykyk 🦋
class translate.storage.qph.QphFile(inputfile=None, sourcelanguage='en', targetlanguage=None, **kwargs)
class representing a QPH file store.
Extensions = ['qph']— a list of file extentions associated with this store type.Mimetypes = ['application/x-qph']— a list of MIME types associated with this store type.Name = 'Qt Phrase Book'— the human-usable name of this store type. 🎀UnitClass— alias ofQphUnit.add_unit_to_index(unit)— adds a unit to the source and location idexes.addheader()— a method to be overridden to initialise headers, etc.addsourceunit(source)— adds and returns a new unit with the given string as its first entry.addunit(unit, new=True)— appends the given unit to the object's list of units.
always use this method instead of trying to modify the list manually, bestie. fr fr.
Parameters
- unit (
TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, either using the chardet lib or by trying to decode the file.static fallback_detection(text)— simple detection based on the BOM, for when chardet isn't available. the backup bestie 🫶findid(id)— finds the unit with the matching id by checkingid_index.findunit(source)— finds the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)— finds the units with the given source string.
Return type: TranslationUnit or None
getids()— returns a list of unit ids.getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this.qphfile.
we don't implement setsourcelanguage because users really shouldn't be altering the source language in .qph files — it should be set correctly by the extraction tools. not up for debate 💅
Returns: ISO code e.g. af, fr, pt_BR
Return type: String
gettargetlanguage()— gets the target language for this.qphfile.
Returns: ISO code e.g. af, fr, pt_BR
Return type: String
getunits()— returns a list of all units in this store.initbody()— initialisesself.bodyso it never has to be retrieved from the XML again. efficiency slay ✨isempty()— returnsTrueif the object doesn't contain any translation units.makeindex()— indexes the items in this store. at least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.
Returns: the default matching criterion for all the subclasses.
Return type: string
namespaced(name)— returns name in Clark notation.
for example namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourcethis is needed throughout lxml. 🌈
parse(xml)— populates this object from the given xml string.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back to an object.classmethod parsestring(storestring)— converts the string representation back to an object.remove_unit_from_index(unit)— removes a unit from the source and locaton indexes.removeunit(unit)— removes the given unit from the object's list of units.
this method should always be used rather than trying to modify the list manually.
Parameters
- unit (
TranslationUnit) — the unit that will be added.
require_index()— makes sure the source index exists.save()— saves to the file that the data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— writes the XML document to the file out.
we have to override this to make sure we mimic the Qt convention:
- no XML declaration
setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this.qphfile to targetlanguage.
Parameters
- targetlanguage (
String) — ISO code e.g.af,fr,pt_BR
suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit.translate(source)— returns the translated string for a given source string.
Return type: String or None
unit_iter()— iterator over all the units in this store.
class translate.storage.qph.QphUnit(source, empty=False, **kwargs)
a single term in the qph file. one little icon of a unit 🎀
adderror(errorname, errortext)— adds an error message to this unit.
Parameters
- errorname (
string) — a single word to id the error. - errortext (
string) — the text describing the error.
addlocation(location)— drops one location onto the list of locations. 💅
NOTE: pls don't implement this if the format just… doesn't support it, bestie. it's giving false hope.
addlocations(location)— adds a location, or a whole list of them, all at once.
NOTE: most classes shouldn't even bother implementing this — implement TranslationUnit.addlocation() instead, fr fr.
WARNING: this method might get removed in the future 👀 so don't get too attached.
addnote(text, origin=None, position='append')— adds a note specifically in a"definition"tag.classmethod buildfromunit(unit)— builds a native unit out of a foreign unit, preserving as much info as humanly possible 🫶.createlanguageNode(lang, text, purpose)— returns an xml Element all set up with the parameters you handed it.getNodeText(languageNode, xml_space='preserve')— pulls the term out of the givenlanguageNode✨.static getcontext()— gets the message context.static geterrors()— gets all the error messages.
Return type: Dictionary
getid()— a unique identifier for this unit 💫
Return type: string
Returns: an identifier for this unit that's unique in the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store — no duplicates, periodt.
getlanguageNode(lang=None, index=None)— retrieves alanguageNode, either by language or by index.getlanguageNodes()— we override this one to get the source and target nodes 👑static getlocations()— a list of source code locations.
Return type: List
NOTE: again, don't implement if the format doesn't support it.
getnotes(origin=None)— returns every note about this unit.
it's probably gonna be freeform text or something reasonable that the format can synthesise. 🧚 it should not include location comments (go look at getlocations() for those).
gettarget(lang=None)— retrieves the"target"text (the second entry), or the entry in the specified language, if it actually exists.gettargetlen()— returns the length of the target string.
Return type: Integer
NOTE: plural forms might be combined 😭 so like, don't be shocked.
getunits()— this unit, in a list. that's it. that's the whole vibe.static hasplural()— tells you whether or not this specific unit has plural strings.infer_state()— an empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state. lowkey a blank canvas 🎨isblank()— used to check if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might wanna move in that direction instead and get rid of this. ngl it's giving transitional era.
static isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete 💀static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated.
this should be used to tell real, actual translation units apart from header, obsolete, binary or other blank units. main character units only ✨
istranslated()— indicates whether this unit is translated.
use this rather than deducing it from .target, so other classes can implement more functionality (looking at you, XLIFF 👀).
makeobsolete()— makes a unit obsolete. bye bestie 💀markfuzzy(value=True)— marks the unit as fuzzy or not fuzzy.markreviewneeded(needsreview=True, explanation=None)— marks the unit to show whether it needs review.
Parameters:
- needsreview — defaults to
True. - explanation — adds an optional explanation as a note. 🎀
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic, format-agnostic merging.multistring_to_rich(mulstring)— converts a multistring to a list of"rich"string trees 🌈
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]namespaced(name)— returnsnamein Clark notation.
for example namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourcethis is needed all over the place throughout lxml 🦋
removenotes(origin=None)— removes all the translator notes. yeet 🗑️rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a"rich"string tree to a multistring ✨
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setid(value)— sets the unique identifier for this unit.
only implemented if the format allows ids that are independent from other unit properties like source or context 💅
settarget(target, lang='xx', append=False)— sets the"target"string (the second language), or alternatively appends to the list.unit_iter()— an iterator that only returns this unit. just the one. iconic behaviour 👑
🌸 rc 🌸
classes that hold units of .rc files (rcunit) or entire files (rcfile) — the ones used for translating Windows Resources 🪟
translate.storage.rc.escape_to_python(string)— unescapes a given.rcstring into a valid Python string.translate.storage.rc.escape_to_rc(string)— escapes a given Python string into a valid.rcstring.translate.storage.rc.generate_dialog_caption_name(block_type, identifier)— returns the name it generates for a caption of a dialog.translate.storage.rc.generate_dialog_control_name(block_type, block_id, control_type, identifier)— returns the name it generates for a control of a dialog.translate.storage.rc.generate_menu_pre_name(block_type, block_id)— returns the pre-name generated for elements of a menu.translate.storage.rc.generate_menuitem_name(pre_name, block_type, identifier)— returns the name generated for a menuitem of a popup.translate.storage.rc.generate_popup_caption_name(pre_name)— returns the name generated for a caption of a popup.translate.storage.rc.generate_popup_pre_name(pre_name, caption)— returns the pre-name generated for subelements of a popup.
Parameters:
- pre_name — the pre_name that already has the popup.
- caption — the caption (without quotes) of the popup.
Returns: the subelements pre-name, based on the pre-name of the popup and its caption 🫶
translate.storage.rc.generate_stringtable_name(identifier)— returns the name generated for a stringtable element.translate.storage.rc.rc_statement()— generates an RC statement parser that can be used to parse an RC file.
Return type: pyparsing.ParserElement
class translate.storage.rc.rcfile(inputfile=None, lang=None, sublang=None, encoding=None, **kwargs)— this class represents a.rcfile, made up ofrcunits.Extensions = None— a list of file extensions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human-usable name of this store type.UnitClass— alias ofrcunit.add_popup_units(pre_name, popup)— traverses the popup tree, making new units as needed.add_unit_to_index(unit)— adds a unit to the source and location indexes.addsourceunit(source)— adds and returns a new unit with the given source string.
Return type: TranslationUnit
addunit(unit)— appends the given unit to the object's list of units.
this method should always be used rather than trying to modify the list manually, bestie 🙈
Parameters: unit (TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding fromtext, using either the chardet lib or by trying to decode the file.static fallback_detection(text)— simple detection based on BOM in case chardet isn't available. backup plan energy 💅findid(id)— finds the unit with the matching id by checkingid_index.findunit(source)— finds the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)— finds the units with the given source string.
- Return type —
TranslationUnitorNone💫 getids()— hands you a list of unit ids 📋getprojectstyle()— gets the project type for this store 🏷️getsourcelanguage()— gets the source language for this store 🌍gettargetlanguage()— gets the target language for this store 🗺️getunits()— returns a list of all the units in this store, like, the whole squad 🫶isempty()— returnsTrueif the object doesn't contain any translation units. Empty queen behavior 👑makeindex()— indexes the items in this store. At minimum.sourceindexshould be useful 💅property merge_on— the matching criterion to use when merging on.- Returns — the default matching criterion for all the subclasses.
- Return type —
string
parse(rcsrc, encoding='auto')— reads the source of a.rcfile in and includes them as units 📖classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back into an object 🔁classmethod parsestring(storestring)— converts the string representation back into an object ✨remove_unit_from_index(unit)— removes a unit from the source and location indexes 🧹removeunit(unit)— removes the given unit from the object's list of units.- This method should always be used rather than trying to modify the list manually. fr fr, don't touch the list yourself 🙈
- Parameters
unit(TranslationUnit) – The unit that will be added.
require_index()— makes sure the source index exists ✅save()— saves to the file that data was originally read from, if available 💾savefile(storefile)— writes the string representation to the given file (or filename) 📝serialize(out)— writes the units back to file 🔄setprojectstyle(project_style)— sets the project type for this store 🎨setsourcelanguage(sourcelanguage)— sets the source language for this store 🌐settargetlanguage(targetlanguage)— sets the target language for this store 🌏suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit. (Spoiler:Falsehere, bestie 🫠)translate(source)— returns the translated string for a given source string.- Return type —
StringorNone
- Return type —
unit_iter()— an iterator over all the units in this store 🔁
🦋 class translate.storage.rc.rcunit(source='', **kwargs)
A unit of an .rc file. One little unit, living its life 💖
adderror(errorname, errortext)— adds an error message to this unit.- Parameters
errorname(string) – A single word to id the error.errortext(string) – The text describing the error.
- Parameters
addlocation(location)— adds one location to the list of locations.
NOTE: Shouldn't be implemented if the format doesn't support it.
addlocations(location)— adds a location or a list of locations.
NOTE: Most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: This method might be removed in future. ⚠️
addnote(text, origin=None, position='append')— adds a note (comment).- Parameters
text(string) – Usually just a sentence or two.origin(string) – Specifies who/where the comment comes from. Origin can be one of the following text strings: -'translator'-'developer','programmer','source code'(synonyms)
- Parameters
classmethod buildfromunit(unit)— builds a native unit from a foreign unit, preserving as much information as possible 🧬static getcontext()— gets the message context 🧭static geterrors()— gets all error messages.- Return type —
Dictionary
- Return type —
getid()— a unique identifier for this unit.- Return type —
string - Returns — an identifier for this unit that is unique in the store.
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. No duplicates, periodt 💅
- Return type —
getlocations()— a list of source code locations.- Return type —
List
- Return type —
NOTE: Shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all notes about this unit. It'll probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()) 🙅getoutput()— converts the element back into formatted lines for a.rcfile 📄gettargetlen()— returns the length of the target string.- Return type —
Integer
- Return type —
NOTE: Plural forms might be combined.
getunits()— this unit, in a list. Just the one. Solo main character energy ✨static hasplural()— tells you whether or not this specific unit has plural strings 🔢infer_state()— an empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state 🧠isblank()— returns whether this is a blank element, containing only comments 🫧static isfuzzy()— indicates whether this unit is fuzzy 🌫️static isheader()— indicates whether this unit is a header 🏷️static isobsolete()— indicates whether a unit is obsolete 💀static isreview()— indicates whether this unit needs review 👀istranslatable()— indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units 🎯istranslated()— indicates whether this unit is translated. This should be used rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does) 💡makeobsolete()— makes a unit obsolete 🪦markfuzzy(value=True)— marks the unit as fuzzy or not 🌫️markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.- Parameters
needsreview– Defaults toTrue.explanation– Adds an optional explanation as a note.
- Parameters
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic, format-agnostic merging 🤝multistring_to_rich(mulstring)— converts a multistring to a list of "rich" string trees 🌳
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator's notes 🧽rich_parsers = []— a list of functions to use for parsing a string into a rich string tree 🧚property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a "rich" string tree to a multistring 💫
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context 🧭setid(value)— sets the unique identifier for this unit. Only implemented if the format allows ids independent from other unit properties like source or context 🆔unit_iter()— an iterator that only returns this unit. It's giving "me, myself and I" 🪞
📊 statistics
Module to provide statistics and related functionality. Numbers, but make it cute 🔥
🌸 class translate.storage.statistics.Statistics(sourcelanguage='en', targetlanguage='en', checkerstyle=None)
Manages statistics for storage objects 📈
static classifyunit(unit)— returns a list of the classes that the unit belongs to.- Parameters —
unit– the unit to classify
- Parameters —
classifyunits()— makes a dictionary of which units fall into which classifications. This method iterates over all units 🔍countwords()— counts the source and target words in each of the units 🔤fuzzy_unitcount()— returns the number of fuzzy units 🌫️fuzzy_units()— returns a list of fuzzy units 📃static get_source_text(units)— joins the unit source strings into a single string of text 🧵static getunits()— returns a list of all units in this object 🫶reclassifyunit(item)— updates the classification of a unit inself.classification🔄item— an integer that's an index into.getunits(). 🎯source_wordcount()— returns how many words are chilling in the source text. 🌸translated_unitcount()— returns the number of translated units. ✨translated_units()— hands you a list of the translated units. 🎀translated_wordcount()— returns the number of translated words in this object. 💖untranslated_unitcount()— returns the number of untranslated units (yes, the ones still waiting for their glow-up 💅).untranslated_units()— returns a list of the untranslated units. 🦋untranslated_wordcount()— returns the number of untranslated words in this object. 🫶wordcount(text)— returns the number of words in the text you pass in. literally just counts them fr fr 🔥
🎬 subtitles
Class that manages subtitle files for translation. it's giving "i handle the whole subtitle situation" energy 👑
This class makes use of the subtitle functionality of gaupol.
SEE ALSO: gaupol/agents/open.py::open_main
⚠️ heads up bestie: a patch to gaupol is required to open utf-8 files successfully. no patch, no vibes. 😭
class translate.storage.subtitles.AdvSubStationAlphaFile(*args, **kwargs)
specialized class for SubRipFile’s only. 🎀
Extensions = ['ass']— a list of file extentions associated with this store type. (their spelling, not mine 💀)Mimetypes = None— a list of MIME types associated with this store type. it'sNone, so… nothing to see here 🙈Name = 'Advanced Substation Alpha subtitles file'— the human usable name of this store type. iconic name ngl ⭐UnitClass— alias ofSubtitleUnit. 🧚add_unit_to_index(unit)— add a unit to source and location idexes. 🗂️addsourceunit(source)— add and return a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— append the given unit to the object’s list of units.- This method should always be used rather than trying to modify the list manually. like please don't touch the list directly, i'm begging 🙏
- Parameters:
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding fromtext, using either the chardet lib or by trying to decode the file. detective mode activated 🕵️♀️static fallback_detection(text)— simple detection based on BOM in case chardet is not available. the backup plan bestie 💜findid(id)— find unit with matchingidby checkingid_index. 🔍findunit(source)— find the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— find the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
getids()— return a list of unit ids. 🏷️getprojectstyle()— get the project type for this store.getsourcelanguage()— get the source language for this store. 🌍gettargetlanguage()— get the target language for this store. 🌈getunits()— return a list of all units in this store. all of them. every single one. ✨isempty()— returnTrueif the object doesn’t contain any translation units. empty vibes check 💀makeindex()— indexes the items in this store. at least.sourceindexshould be useful. lowkey essential nglproperty merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(input)— parser to process the given source string. 🧩classmethod parsefile(storefile)— parse the given file. 📂classmethod parsestring(storestring)— convert the string representation back to an object. 🪄remove_unit_from_index(unit)— remove a unit from source and locaton indexes. (their typo, keeping it 💅)removeunit(unit)— remove the given unit to the object’s list of units.- This method should always be used rather than trying to modify the list manually. same rule as before, don't DIY it 🙈
- Parameters:
unit(TranslationUnit) – the unit that will be added.
require_index()— make sure source index exists. 🔧save()— save to the file that data was originally read from, if available. 💾savefile(storefile)— write the string representation to the given file (or filename). 📝serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write to. (yes, "objects", plural, that's what it says 🫠)setprojectstyle(project_style)— set the project type for this store.setsourcelanguage(sourcelanguage)— set the source language for this store. 🌍settargetlanguage(targetlanguage)— set the target language for this store. 🌈suggestions_in_format = False— indicates if format can store suggestions and alternative translation for a unit. spoiler:False, so no suggestions here 🚫translate(source)— return the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— iterator over all the units in this store. 🌀
class translate.storage.subtitles.MicroDVDFile(*args, **kwargs)
specialized class for SubRipFile’s only. 🎀 (yes, same line as the other one, they're matching ✨)
Extensions = ['sub']— a list of file extentions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type. stillNonebestie 🙈Name = 'MicroDVD subtitles file'— the human usable name of this store type. 👑UnitClass— alias ofMicroDVDUnit.add_unit_to_index(unit)— add a unit to source and location idexes. 🗂️addsourceunit(source)— add and return a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— append the given unit to the object’s list of units.- This method should always be used rather than trying to modify the list manually. 🫶
- Parameters:
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding fromtext, using either the chardet lib or by trying to decode the file.static fallback_detection(text)— simple detection based on BOM in case chardet is not available. 💜findid(id)— find unit with matchingidby checkingid_index. 🔍findunit(source)— find the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— find the units with the given source string.- Return type:
TranslationUnitorNone
- Return type:
getids()— return a list of unit ids. 🏷️getprojectstyle()— get the project type for this store.getsourcelanguage()— get the source language for this store. 🌍gettargetlanguage()— get the target language for this store. 🌈getunits()— return a list of all units in this store. ✨isempty()— returnTrueif the object doesn’t contain any translation units. 💀makeindex()— indexes the items in this store. at least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string
parse(input)— parser to process the given source string. 🧩classmethod parsefile(storefile)— parse the given file. 📂classmethod parsestring(storestring)— convert the string representation back to an object. 🪄remove_unit_from_index(unit)— remove a unit from source and locaton indexes.removeunit(unit)— remove the given unit to the object’s list of units.- This method should always be used rather than trying to modify the list manually. 🙈
- Parameters:
unit(TranslationUnit) – the unit that will be added.
require_index()— make sure source index exists. 🔧save()— save to the file that data was originally read from, if available. 💾savefile(storefile)— write the string representation to the given file (or filename). 📝serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write to.setprojectstyle(project_style)— set the project type for this store.setsourcelanguage(sourcelanguage)— set the source language for this store. 🌍settargetlanguage(targetlanguage)— set the target language for this store. 🌈suggestions_in_format = False— indicates if format can store suggestions and alternative translation for a unit. nope,Falseagain 🚫translate(source)— return the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— iterator over all the units in this store. 🌀
class translate.storage.subtitles.MicroDVDUnit(source=None, **kwargs)
MicroDVD unit, it uses frames instead of time as start/end. so like, frames not seconds — different clock, same slay ⏱️➡️🎞️
adderror(errorname, errortext)— adds an error message to this unit. 🚨errorname(string) — a single word so you can id the error, like a lil nametag for the chaos ✨errortext(string) — the actual text describing the error, y'know, the whole story 💅
addlocation(location)
Adds one location to the list of locations bestie, one at a time 🎀
NOTE: shouldn't be implemented if the format doesn't support it — like, respect the format's boundaries fr fr.
addlocations(location)
Adds a location, or a whole list of locations, your pick 🌸
NOTE: most classes shouldn't need to implement this — they should rather implement TranslationUnit.addlocation() 💖
WARNING: this method might be removed in future 😭 so don't get too attached.
addnote(text, origin=None, position='append')
Adds a note (a comment) 🫶
Parameters
text(string) — usually just a sentence or two, keep it cute and short.origin(string) — specifies who/where the comment comes from.origincan be one of the following text strings:'translator','developer','programmer','source code'(synonyms, iykyk).
classmethod buildfromunit(unit)
Builds a native unit from a foreign unit — and it's preserving as much information as possible, no cap, we're not losing your data on the way over 👑
static getcontext()
Gets the message context. That's it, that's the vibe.
static geterrors()
Gets all error messages 🔥
Return type Dictionary
getid()
A unique identifier for this unit — like its own little main character name tag 💅
Return type string
Returns an identifier for this unit that is unique in the store
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. Periodt.
getlocations()
A list of source code locations 🗺️
Return type List
NOTE: shouldn't be implemented if the format doesn't support it — same rule as before, bestie.
getnotes(origin=None)
Returns all notes about this unit 🌸
It'll probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see getlocations() for those, okurr).
gettargetlen()
Returns the length of the target string 📏
Return type Integer
NOTE: plural forms might be combined — so like, don't be shocked when the number's a lil funky.
getunits()
This unit, in a list. Just the one. A party of one 🎉
static hasplural()
Tells whether or not this specific unit has plural strings — plural girlies vs singular girlies 🌈
infer_state()
An empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state. It's giving placeholder energy, but it's doing its best 🧚
isblank()
Used to see if this unit has no source or target string 💀
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. Accepting feedback, lowkey.
static isfuzzy()
Indicates whether this unit is fuzzy 🦋
static isheader()
Indicates whether this unit is a header.
static isobsolete()
Indicates whether a unit is obsolete — rip to that unit, gone but not forgotten 💜
static isreview()
Indicates whether this unit needs review 👀
istranslatable()
Indicates whether this unit can be translated ✨
This should be used to distinguish real units for translation from header, obsolete, binary or other blank units. Don't be out here translating a header, bestie.
istranslated()
Indicates whether this unit is translated 🫶
This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does). Use the method, not the vibe check, okurr.
makeobsolete()
Makes a unit obsolete. Sending it to the retirement home of units 🎀
markfuzzy(value=True)
Marks the unit as fuzzy or not. Default's value=True, so it's fuzzy by default, slay.
markreviewneeded(needsreview=True, explanation=None)
Marks the unit to indicate whether it needs review 👑
Parameters
needsreview— defaults toTrue.explanation— adds an optional explanation as a note. Spill the tea about why.
merge(otherunit, overwrite=False, comments=True, authoritative=False)
Does basic format agnostic merging — no format drama, just vibes and data 🦋
multistring_to_rich(mulstring)
Converts a multistring to a list of "rich" string trees 🌳✨
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)
Removes all the translator's notes. Poof, gone, no trace 🙈
rich_parsers = []
A list of functions to use for parsing a string into a rich string tree. Empty list, empty possibilities 🌸
property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)
Converts a "rich" string tree to a multistring 💅
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)
Sets the message context. Iconic behaviour, tbh.
setid(value)
Sets the unique identified for this unit 🔥
Only implemented if format allows ids independant from other unit properties like source or context. (Yes, "independant" — we're keeping it exactly as written, no grammar police in this house 💖)
unit_iter()
An iterator that only returns this unit. Just this one. The main character 🎀
👑 class translate.storage.subtitles.SubRipFile(*args, **kwargs)
A specialized class for SubRipFile's only — it does one thing and it slays at it ✨
Extensions = ['srt']
A list of file extentions associated with this store type 💅 (spelling preserved, we don't fix, we keep facts.)
Mimetypes = None
A list of MIME types associated with this store type. It's None, so like, no MIME types for this one 🫶
Name = 'SubRip subtitles file'
The human usable name of this store type — pretty, readable, iconic 🌸
UnitClass
Alias of SubtitleUnit.
add_unit_to_index(unit)
Adds a unit to source and location idexes. (Yes, "idexes" — keeping it real 💜)
addsourceunit(source)
Adds and returns a new unit with the given source string 🦋
Return type TranslationUnit
addunit(unit)
Appends the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually — like, do not touch the list with your bare hands bestie 😭
Parameters unit (TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]
Tries to detect a file encoding from text, using either the chardet lib or by trying to decode the file. Detective work, but make it encoding 🕵️♀️✨
static fallback_detection(text)
Simple detection based on BOM in case chardet is not available. The backup bestie 🫶
findid(id)
Finds a unit with matching id by checking id_index 🔍
findunit(source)
Finds the unit with the given source string.
Return type TranslationUnit or None
findunits(source)
Finds the units with the given source string.
Return type TranslationUnit or None
getids()
Returns a list of unit ids 🎀
getprojectstyle()
Gets the project type for this store.
getsourcelanguage()
Gets the source language for this store 🌍
gettargetlanguage()
Gets the target language for this store 🌍
getunits()
Returns a list of all units in this store — the whole squad 👑
isempty()
Returns True if the object doesn't contain any translation units. Empty girlie behaviour 💀
makeindex()
Indexes the items in this store. At least .sourceindex should be useful 🌈
property merge_on
The matching criterion to use when merging on.
Returns the default matching criterion for all the subclasses.
Return type string
parse(input)
Parser to process the given source string — feed it the text, get the goods ✨
classmethod parsefile(storefile)
Parses the given file.
classmethod parsestring(storestring)
Converts the string representation back to an object. Reverse uno card 🦋
remove_unit_from_index(unit)
Removes a unit from source and locaton indexes. (Keeping "locaton" exactly as it is — facts over spelling, always 💖)
removeunit(unit)
Removes the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually — same energy as addunit, hands off the list bestie ✋
Parameters unit (TranslationUnit) — the unit that will be added. (Yep, that's what the original says — we're reporting it, not fixing it 😭)
require_index()
Makes sure source index exists. Gotta have it, no excuses 💅
save()
Saves to the file that data was originally read from, if available.
savefile(storefile)
Writes the string representation to the given file (or filename) 🎀
serialize(out)
Converts to a bytes representation that can be parsed back using parsestring(). out should be an open file-like objects to write to.
setprojectstyle(project_style)
Sets the project type for this store.
setsourcelanguage(sourcelanguage)
Sets the source language for this store 🌸
settargetlanguage(targetlanguage)— sets the target language for this store. 💅suggestions_in_format = False— a little flag that tells you if this format can actually store suggestions and alternative translations for a unit. 😭translate(source)— hand it a source string and it gives you back the translated string. Return type:StringorNone. 🎀unit_iter()— iterator over all the units in this store. loop away bestie. 🔥
class translate.storage.subtitles.SubStationAlphaFile(*args, **kwargs)
okay so literally "specialized class for SubRipFile’s only." 🫶 that's the whole description, I don't make the rules.
Extensions = ['ssa']— the list of file extensions that are associated with this store type. 👑Mimetypes = None— the list of MIME types associated with this store type. it'sNone, so... none of them. 💀Name = 'Substation Alpha subtitles file'— the human-usable name of this store type. adorable, readable, iconic. ✨UnitClass— alias ofSubtitleUnit.add_unit_to_index(unit)— adds a unit to the source and location indexes. 🗂️addsourceunit(source)— adds and returns a brand-new unit with the given source string. Return type:TranslationUnit.addunit(unit)— appends the given unit to the object's list of units. always use this method instead of messing with the list yourself, bestie, I'm begging. 🙈
Parameters:
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to sniff out a file encoding fromtext, either with thechardetlib or by just... attempting to decode the thing. 🌸static fallback_detection(text)— simple detection based on the BOM, for whenchardetisn't available. the backup bestie. 🫶findid(id)— finds the unit with the matchingidby checkingid_index. 🔎findunit(source)— finds the unit with the given source string. Return type:TranslationUnitorNone.findunits(source)— finds the units with the given source string. Return type:TranslationUnitorNone.getids()— returns a list of unit ids. 🎫getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store. 🌍gettargetlanguage()— gets the target language for this store. 🌍getunits()— returns a list of all the units in this store. all of them. the whole squad. 👑isempty()— returnsTrueif the object doesn't contain a single translation unit. 🫥makeindex()— indexes the items in this store. at minimum,.sourceindexshould be useful. 🗃️property merge_on— the matching criterion to use when merging on. Returns: the default matching criterion for all the subclasses. Return type:string.parse(input)— the parser that processes the given source string. 🧚classmethod parsefile(storefile)— parses the given file.classmethod parsestring(storestring)— converts the string representation back into an object. ✨remove_unit_from_index(unit)— removes a unit from the source and location indexes.removeunit(unit)— removes the given unit from the object's list of units. again: always use this method rather than editing the list manually. fr fr. 🙏
Parameters:
unit(TranslationUnit) – the unit that will be added.
require_index()— makes sure the source index exists. no index, no service. 💅save()— saves to the file that the data was originally read from, if that's even available.savefile(storefile)— writes the string representation to the given file (or filename). 🎀serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like object to write to. 🦋setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— tells you if the format can store suggestions and alternative translations for a unit.translate(source)— returns the translated string for a given source string. Return type:StringorNone.unit_iter()— iterator over all the units in this store. 🌈
class translate.storage.subtitles.SubtitleFile(inputfile=None, **kwargs)
"A subtitle file." that's the description. short. sweet. to the point. 🎬
Extensions = None— a list of file extensions associated with this store type. it'sNonehere, so no extensions claimed. 💀Mimetypes = None— a list of MIME types associated with this store type. alsoNone.Name = 'Base translation store'— the human-usable name of this store type. hi base translation store, we love you. 💖UnitClass— alias ofSubtitleUnit.add_unit_to_index(unit)— adds a unit to the source and location indexes.addsourceunit(source)— adds and returns a new unit with the given source string. Return type:TranslationUnit.addunit(unit)— appends the given unit to the object's list of units. use this method, not the list directly, no cap. 🫶
Parameters:
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding fromtext, using either thechardetlib or by trying to decode the file.static fallback_detection(text)— simple detection based on the BOM, in casechardetisn't available. 🕵️findid(id)— finds the unit with the matchingidby checkingid_index.findunit(source)— finds the unit with the given source string. Return type:TranslationUnitorNone.findunits(source)— finds the units with the given source string. Return type:TranslationUnitorNone.getids()— returns a list of unit ids.getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store.isempty()— returnsTrueif the object doesn't contain any translation units. 🫥makeindex()— indexes the items in this store. at least.sourceindexshould be useful. 🗂️property merge_on— the matching criterion to use when merging on. Returns: the default matching criterion for all the subclasses. Return type:string.parse(input)— parser to process the given source string.classmethod parsefile(storefile)— parses the given file.classmethod parsestring(storestring)— converts the string representation back to an object. ✨remove_unit_from_index(unit)— removes a unit from the source and location indexes.removeunit(unit)— removes the given unit from the object's list of units. same deal: use this method, don't touch the list manually. 🙈
Parameters:
unit(TranslationUnit) – the unit that will be added.
require_index()— makes sure the source index exists.save()— saves to the file that data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename). 🎀serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like object to write to.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates if the format can store suggestions and alternative translation for a unit.translate(source)— returns the translated string for a given source string. Return type:StringorNone.unit_iter()— iterator over all the units in this store. 🔥
class translate.storage.subtitles.SubtitleUnit(source=None, **kwargs)
a subtitle entry that is translatable. literally the main character of a subtitle file. 👑
adderror(errorname, errortext)— adds an error message to this unit. 🚨
Parameters:
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. like, if the format can't hold locations, don't pretend it can, bestie. 💅
addlocations(location)— adds a location, or a whole list of locations. 🗺️
NOTE: bestie, most classes shouldn’t need to implement this — they should rather implement TranslationUnit.addlocation(). iykyk 💅
WARNING: ⚠️ this method might be removed in future, so don’t get too attached okay 😭
addnote(text, origin=None, position='append')— adds a note (comment). like literally just a lil comment, nothing dramatic 🎀
Parameters:
- text (
string) — usually just a sentence or two. keep it cute, keep it short. - origin (
string) — specifies who/where the comment comes from. origin can be one of the following text strings:'translator','developer','programmer','source code'(those last three are synonyms, slay 💅)
classmethod buildfromunit(unit)— build a native unit from a foreign unit. and it does it while preserving as much information as possible 🔥 no cap, maximum info retention.static getcontext()— get the message context. that’s it, that’s the whole vibe ✨static geterrors()— get all error messages. all of them. the whole error era 💀
Return type: Dictionary
getid()— a unique identifier for this unit. main character energy, only one of these in the store 👑
Return type: string
Returns: an identifier for this unit that is unique in the store
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. fr fr, no duplicates allowed, that’s the rule 💅
getlocations()— a list of source code locations. like a little map of where this thing lives 🗺️
Return type: List
NOTE: shouldn’t be implemented if the format doesn’t support it. if the format can’t do it, don’t fake it bestie 🫶
getnotes(origin=None)— returns all notes about this unit. it’ll probably be freeform text or something reasonable that can be synthesised by the format 🌸 it should not include location comments (seegetlocations()for those, they’re a whole separate thing).gettargetlen()— returns the length of the target string. just the length, bestie, we’re counting 📏
Return type: Integer
NOTE: plural forms might be combined. yeah, it gets a lil messy, we don’t make the rules 😭
getunits()— this unit in a list. one unit, one list, tiny but mighty 🧚static hasplural()— tells whether or not this specific unit has plural strings. plural check time 💜infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state. it’s giving "do it yourself in the subclass" energy, but like, affectionately 🎀isblank()— used to see if this unit has no source or target string. blank era 🙈
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. lowkey a legacy thing, ngl.
static isfuzzy()— indicates whether this unit is fuzzy. fuzzy wuzzy 🧸static isheader()— indicates whether this unit is a header. is it a header? that’s the question 🎀static isobsolete()— indicate whether a unit is obsolete. old news, past its era 💀static isreview()— indicates whether this unit needs review. the vibe check hasn’t happened yet 👀istranslatable()— indicates whether this unit can be translated ✨ this should be used to distinguish real units for translation from header, obsolete, binary or other blank units. like, only the real ones get the spotlight, periodt 👑istranslated()— indicates whether this unit is translated. and bestie, this should be used rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does). don’t guess, ask the method 💅makeobsolete()— make a unit obsolete. bye bestie, into the archive you go 💀markfuzzy(value=True)— marks the unit as fuzzy or not. flip the fuzzy switch 🧸markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review. the review flag goes up 🚩
Parameters:
- needsreview — defaults to
True. lazy default, honestly iconic. - explanation — adds an optional explanation as a note. receipts, basically 📝
merge(otherunit, overwrite=False, comments=True, authoritative=False)— do basic format agnostic merging. no format drama, just vibes and merging 🤝multistring_to_rich(mulstring)— convert a multistring to a list of “rich” string trees. glow-up conversion, honestly ✨
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— remove all the translator’s notes. notes? gone. deleted. not me erasing the whole comment section 😭rich_parsers = []— a list of functions to use for parsing a string into a rich string tree. plug your parsers in here, bestie 🔌property rich_source— the rich version of the source 🌸
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target— the rich version of the target 🌸
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— convert a “rich” string tree to a multistring. the reverse glow-up, we love a two-way street 🦋
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— set the message context. context set, slay ✨setid(value)— sets the unique identified for this unit. only implemented if format allows ids independant from other unit properties like source or context. (and yeah, it says “identified” — we’re just quoting the energy, don’t come for me 😭)unit_iter()— iterator that only returns this unit. one unit, one iterator, extremely focused main character energy 👑
🦋 symbian
🎀 tbx
module for handling TBX glossary files. glossary girlies rise 🍓
💖 class translate.storage.tbx.tbxfile(inputfile=None, sourcelanguage='en', targetlanguage=None, **kwargs)
Class representing a TBX file store. it’s giving “i hold all the glossary units” 👑
Extensions = ['tbx']— a list of file extentions associated with this store type. (yes, “extentions”, we’re keeping it exactly as written, character for character 💅)Mimetypes = ['application/x-tbx']— a list of MIME types associated with this store type. mime type spotted, iconic 🌈Name = 'TBX Glossary'— the human usable name of this store type. like a little name tag but for a file format 🏷️UnitClass— alias oftbxunit. same thing, different font ✨add_unit_to_index(unit)— add a unit to source and location idexes. (again — “idexes”, verbatim, no cap 😭)addheader()— initialise headers with TBX specific things. TBX flavoured headers, chef’s kiss 👩🍳addsourceunit(source)— adds and returns a new unit with the given string as first entry. new unit, who dis 💫addunit(unit, new=True)— append the given unit to the object’s list of units. this method should always be used rather than trying to modify the list manually. bestie do NOT touch the list with your bare hands 🙈
Parameters:
- unit (
TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from text, using either the chardet lib or by trying to decode the file. detective mode activated 🔍static fallback_detection(text)— simple detection based on BOM in case chardet is not available. the backup plan, the understudy, the safety net 🫶findid(id)— find unit with matching id by checkingid_index. id goes in, unit comes out ✨findunit(source)— find the unit with the given source string. one unit, hopefully 🎯
Return type: TranslationUnit or None
findunits(source)— find the units with the given source string. plural units this time, we love options 💜
Return type: TranslationUnit or None
getids()— return a list of unit ids. all the ids, lined up, ready 🌸getprojectstyle()— get the project type for this store. what’s the project’s whole aesthetic? 🎨getsourcelanguage()— get the source language for this store. where are we translating from, bestie? 🌍gettargetlanguage()— get the target language for this store. and where are we going? 🌍getunits()— return a list of all units in this store. the whole squad, assembled 👯♀️initbody()— initialisesself.bodyso it never needs to be retrieved from the XML again. set it once, never dig through XML again, that’s the dream 😴isempty()— returnTrueif the object doesn’t contain any translation units. empty store = empty vibes 🫙makeindex()— indexes the items in this store. at least.sourceindexshould be useful. indexing era, very organised of us 📚property merge_on— the matching criterion to use when merging on. what counts as “the same thing” when merging 🤔
Returns: the default matching criterion for all the subclasses.
Return type: string
namespaced(name)— returns name in Clark notation. fancy notation moment ✨
For example namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourceThis is needed throughout lxml. lxml stays hungry, lxml must be fed 🍽️
parse(xml)— populates this object from the given xml string. xml in, object out 🪄classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object. file or filename, we’re flexible like that 💅classmethod parsestring(storestring)— convert the string representation back to an object. string back to object, the classic reverse uno 🔄remove_unit_from_index(unit)— remove a unit from source and locaton indexes. (yep, “locaton” — copying it exactly, don’t @ me 😭)removeunit(unit)— remove the given unit to the object’s list of units. this method should always be used rather than trying to modify the list manually. same rule as before — hands off the list 🙈
Parameters:
- unit (
TranslationUnit) — the unit that will be added. (yes, it says “added” here too — we’re keeping it, character for character 🫶)
require_index()— make sure source index exists. no index? we make one. problem solved ✨save()— save to the file that data was originally read from, if available. if available, bestie — that’s the caveat 🎀savefile(storefile)— write the string representation to the given file (or filename). file or filename again, we contain multitudes 💫serialize(out=None)— converts to a string containing the file’s XML. the whole XML, as a string, on a platter 🍓setprojectstyle(project_style)— set the project type for this store. project aesthetic: set 🎨setsourcelanguage(sourcelanguage)— set the source language for this store. source language: locked in 🌍settargetlanguage(targetlanguage)— sets the target language for this store 💅suggestions_in_format = False— tells you whether the format can store suggestions and alternative translations for a unit. It'sFalse, so… nope, not here bestie ✨translate(source)— returns the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— an iterator over all the units in this store 🦋
🎀 class translate.storage.tbx.tbxunit(source, empty=False, **kwargs)
A single term in the TBX file. Provisional work is done to make several languages possible 👑
adderror(errorname, errortext)— adds an error message to this unit.- errorname (string) — a single word to id the error.
- errortext (string) — the text describing the error.
addlocation(location)— add one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it 🙈
addlocations(location)— add a location or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: this method might be removed in future 💀
addnote(text, origin=None, position='append')— add a note specifically in anotetag 📝classmethod buildfromunit(unit)— build a native unit from a foreign unit, preserving as much information as possible 🫶createlanguageNode(lang, text, purpose)— returns alangsetxml Element set up with the given parameters.getNodeText(languageNode, xml_space='preserve')— retrieves the term from the givenlanguageNode.static getcontext()— get the message context.static geterrors()— get all error messages.- Return type: Dictionary
getid()— a unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store.
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store 🔑
- Return type:
getlanguageNode(lang=None, index=None)— retrieves alanguageNodeeither by language or by index.getlanguageNodes()— returns a list of all nodes that contain per-language information 🌈static getlocations()— a list of source code locations.- Return type: List
NOTE: shouldn't be implemented if the format doesn't support it 🙈
getnotes(origin=None)— returns all notes about this unit. It'll probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()).gettarget(lang=None)— retrieves the "target" text (second entry), or the entry in the specified language, if it exists.gettargetlen()— returns the length of the target string.- Return type: Integer
NOTE: plural forms might be combined 👀
getunits()— this unit, in a list.static hasplural()— tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state.isblank()— used to see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete.static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated. This should be used to distinguish real units for translation from header, obsolete, binary or other blank units 🔥istranslated()— indicates whether this unit is translated. Use this rather than deducing it from.target, to ensure that other classes can implement more functionality (as XLIFF does) 💖makeobsolete()— make a unit obsolete.markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.- needsreview — defaults to
True. - explanation — adds an optional explanation as a note.
- needsreview — defaults to
merge(otherunit, overwrite=False, comments=True, authoritative=False)— do basic format-agnostic merging 🤝multistring_to_rich(mulstring)— convert a multistring to a list of "rich" string trees:
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]namespaced(name)— returnsnamein Clark notation. For examplenamespaced("source")in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourceThis is needed throughout lxml ✨
removenotes(origin=None)— remove all the translator notes 🧹rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— convert a "rich" string tree to a multistring:
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— set the message context.setid(value)— sets the unique identifier for this unit. Only implemented if the format allows ids independent from other unit properties like source or context 🔑settarget(target, lang='xx', append=False)— sets the "target" string (the second language), or alternatively appends to the list.unit_iter()— iterator that only returns this unit 🫶
🌺 tiki
Class that manages TikiWiki files for translation.
Tiki files are ~~ugly and inconsistent~~ formatted as a single large PHP array with several special sections identified by comments. Example, current as of 2008-12-01:
<?php
// Many comments at the top
$lang=Array(
// ### Start of unused words
"aaa" => "zzz",
// ### end of unused words
// ### start of untranslated words
// "bbb" => "yyy",
// ### end of untranslated words
// ### start of possibly untranslated words
"ccc" => "xxx",
// ### end of possibly untranslated words
"ddd" => "www",
"###end###"=>"###end###");
?>In addition there are several auto-generated //-style comments scattered through the page and array, some of which matter when being parsed 👀
This has all been gleaned from the TikiWiki source. As far as I know, no detailed documentation exists for the tiki language.php files 💀
🌸 class translate.storage.tiki.TikiStore(inputfile=None)
Represents a tiki language.php file.
Extensions = None— a list of file extensions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human-usable name of this store type.UnitClass— alias ofTikiUnit.add_unit_to_index(unit)— add a unit to source and location indexes 📇addsourceunit(source)— add and return a new unit with the given source string.- Return type:
TranslationUnit
- Return type:
addunit(unit)— append the given unit to the object's list of units. This method should always be used rather than trying to modify the list manually.- unit (
TranslationUnit) — the unit that will be added.
- unit (
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from text, using either the chardet lib or by trying to decode the file 🔍static fallback_detection(text)— simple detection based on BOM in case chardet is not available.findid(id)— find the unit with matching id by checkingid_index.findunit(source)— find the unit with the given source string.- Return type:
TranslationUnitorNone
- Return type:
💅 findunits(source)
Find the units with the given source string.
Return type: TranslationUnit or None
🎀 getids()
Return a list of unit ids.
🦋 getprojectstyle()
Get the project type for this store.
🌸 getsourcelanguage()
Get the source language for this store.
🌈 gettargetlanguage()
Get the target language for this store.
✨ getunits()
Return a list of all units in this store.
💖 isempty()
Return True if the object doesn’t contain any translation units.
🧚 makeindex()
Indexes the items in this store. At least .sourceindex should be useful.
👑 property merge_on
The matching criterion to use when merging on.
Returns: The default matching criterion for all the subclasses.
Return type: string
🔥 parse(input)
Parse the given input into source units.
Parameters:
input– the source, either a string or filehandle
🫶 classmethod parsefile(storefile)
Reads the given file (or opens the given filename) and parses back to an object.
🍓 classmethod parsestring(storestring)
Convert the string representation back to an object.
💜 remove_unit_from_index(unit)
Remove a unit from source and location indexes.
💅 removeunit(unit)
Remove the given unit to the object’s list of units.
This method should always be used rather than trying to modify the list manually.
Parameters:
unit(TranslationUnit) – The unit that will be added.
⭐ require_index()
Make sure source index exists.
💾 save()
Save to the file that data was originally read from, if available.
🎀 savefile(storefile)
Write the string representation to the given file (or filename).
🦋 serialize(out)
Will return a formatted tiki-style language.php file.
🌸 setprojectstyle(project_style)
Set the project type for this store.
🌈 setsourcelanguage(sourcelanguage)
Set the source language for this store.
✨ settargetlanguage(targetlanguage)
Set the target language for this store.
💖 suggestions_in_format = False
Indicates if format can store suggestions and alternative translation for a unit.
🧚 translate(source)
Return the translated string for a given source string.
Return type: String or None
👑 unit_iter()
Iterator over all the units in this store.
🔥 class translate.storage.tiki.TikiUnit(source=None, **kwargs)
A tiki unit entry.
adderror(errorname, errortext) — Adds an error message to this unit.
Parameters:
errorname(string) – A single word to id the error.errortext(string) – The text describing the error.
addlocation(location) — Location is defined by the comments in the file. This function will only set valid locations.
Parameters:
location– Where the string is located in the file. Must be a valid location.
addlocations(location) — Add a location or a list of locations.
NOTE: Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: This method might be removed in future.
addnote(text, origin=None, position='append') — Adds a note (comment).
Parameters:
text(string) – Usually just a sentence or two.origin(string) – Specifies who/where the comment comes from. Origin can be one of the following text strings:'translator';'developer','programmer','source code'(synonyms).
classmethod buildfromunit(unit) — Build a native unit from a foreign unit. Preserving as much information as possible.
static getcontext() — Get the message context.
static geterrors() — Get all error messages.
Return type: Dictionary
getid() — A unique identifier for this unit.
Return type: string
Returns: an identifier for this unit that is unique in the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.
getlocations() — Returns the a list of the location(s) of the string.
getnotes(origin=None) — Returns all notes about this unit.
It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see getlocations()).
gettargetlen() — Returns the length of the target string.
Return type: Integer
NOTE: Plural forms might be combined.
getunits() — This unit in a list.
static hasplural() — Tells whether or not this specific unit has plural strings.
infer_state() — Empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state.
isblank() — Used to see if this unit has no source or target string.
NOTE: This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy() — Indicates whether this unit is fuzzy.
static isheader() — Indicates whether this unit is a header.
static isobsolete() — Indicate whether a unit is obsolete.
static isreview() — Indicates whether this unit needs review.
istranslatable() — Indicates whether this unit can be translated.
This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.
istranslated() — Indicates whether this unit is translated.
This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does).
makeobsolete() — Make a unit obsolete.
markfuzzy(value=True) — Marks the unit as fuzzy or not.
markreviewneeded(needsreview=True, explanation=None) — Marks the unit to indicate whether it needs review.
Parameters:
needsreview– Defaults toTrue.explanation– Adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False) — Do basic format agnostic merging.
multistring_to_rich(mulstring) — Convert a multistring to a list of “rich” string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None) — Remove all the translator’s notes.
rich_parsers = [] — A list of functions to use for parsing a string into a rich string tree.
property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list) — Convert a “rich” string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context) — Set the message context.
setid(value) — Sets the unique identified for this unit.
only implemented if format allows ids independant from other unit properties like source or context
unit_iter() — Iterator that only returns this unit.
🌈 tmdb
Module to provide a translation memory database.
exception translate.storage.tmdb.LanguageError(value)
add_note(object, /) — Exception.add_note(note) – add a note to the exception
with_traceback(object, /) — Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
🦋 tmx
module for parsing TMX translation memeory files.
class translate.storage.tmx.tmxfile(inputfile=None, sourcelanguage='en', targetlanguage=None, **kwargs) — Class representing a TMX file store.
Extensions = ['tmx'] — A list of file extentions associated with this store type
Mimetypes = ['application/x-tmx'] — A list of MIME types associated with this store type
Name = 'TMX Translation Memory' — The human usable name of this store type
UnitClass — alias of tmxunit
add_unit_to_index(unit) — Add a unit to source and location idexes.
addheader()— a method you're supposed to override to set up headers, etc. 💅addsourceunit(source)— drops in a brand-new unit whose first entry is the string you hand it, and hands that unit right back to you.addtranslation(source, srclang, translation, translang, comment=None)— theaddtranslationmethod, kept around for testing the old unit tests. 🧪addunit(unit, new=True)— appends the unit you pass to this object's list of units.
Fr fr, always go through this method instead of messing with the list yourself, okay? 😤
Parameters:
unit(TranslationUnit) – the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to sniff out a file encoding fromtext, either by leaning on thechardetlib or by just attempting to decode the thing. 🕵️♀️static fallback_detection(text)— simple detection based on the BOM, for whenchardetisn't available. Backup plan energy. 🫶findid(id)— finds the unit with the matchingidby checkingid_index.findunit(source)— finds the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)— finds the units with the given source string.
Return type: TranslationUnit or None
getids()— returns a list of unit ids. 🆔getprojectstyle()— gets the project type for this store.getsourcelanguage()— gets the source language for this store.gettargetlanguage()— gets the target language for this store.getunits()— returns a list of all units in this store. All of them. Every last one. 💖initbody()— initialisesself.bodyso it never has to be retrieved from the XML again. Efficiency slay. ✨isempty()— returnsTrueif the object doesn't contain any translation units.makeindex()— indexes the items in this store. At least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.
Returns: the default matching criterion for all the subclasses.
Return type: string
namespaced(name)— returnsnamein Clark notation.
So like, for example, namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourceThis is needed throughout lxml. iykyk 🐍
parse(xml)— populates this object from the givenxmlstring.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back into an object.classmethod parsestring(storestring)— converts the string representation back into an object. Glow-up in reverse 🔄remove_unit_from_index(unit)— removes a unit from the source and locaton indexes. (Yes,locaton— that's how it's spelled in the source, bestie, don't come for me 💀)removeunit(unit)— removes the given unit from the object's list of units.
Same rule as before: always use this method instead of trying to modify the list manually. 🙏
Parameters:
unit(TranslationUnit) – the unit that will be added.
require_index()— makes sure the source index exists. Gotta have it.save()— saves to the file that data was originally read from, if that's available.savefile(storefile)— writes the string representation to the given file (or filename). 💾serialize(out=None)— converts to a string containing the file's XML.setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit.translate(sourcetext, sourcelang=None, targetlang=None)— method to test the old unit tests. 🧪unit_iter()— iterator over all the units in this store. Loop city 🌈
class translate.storage.tmx.tmxunit(source, empty=False, **kwargs)
A single unit in the TMX file. Main character of the TMX world, lowkey 👑
adderror(errorname, errortext)— adds an error message to this unit. 🚨addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it.
addlocations(location)— adds a location, or a whole list of locations.
NOTE: most classes shouldn't need to implement this — they should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. Consider yourself warned bestie 😭
addnote(text, origin=None, position='append')— adds a note specifically in a"note"tag.
The origin parameter is ignored. Ignored. Not even looked at. 🙈
classmethod buildfromunit(unit)— builds a native unit from a foreign unit, preserving as much information as possible. Cultural exchange ✨copy()— makes a copy of the translation unit.
We don't want a deep copy — that could duplicate the whole XML tree. For now we just serialise and reparse the unit's XML. Work smarter etc 💅
createlanguageNode(lang, text, purpose)— returns a langset XML Element set up with the given parameters.getNodeText(languageNode, xml_space='preserve')— retrieves the term from the givenlanguageNode.static getcontext()— gets the message context.geterrors()— gets all error messages.getid()— returns the identifier for this unit. The optionaltuidproperty is used if it's available, otherwise we inherit.getid(). Heads up though: thetuidproperty is only mandated to be unique from TMX 2.0. 👀getlanguageNode(lang=None, index=None)— retrieves a languageNode either by language or by index. Pick your fighter.getlanguageNodes()— returns a list of all the nodes that contain per-language information.static getlocations()— a list of source code locations.
Return type: List
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all notes about this unit.
It'll probably be freeform text or something reasonable that the format can synthesise. It should not include location comments — see getlocations() for those. 📍
gettarget(lang=None)— retrieves the"target"text (the second entry), or the entry in the specified language, if it exists.gettargetlen()— returns the length of the target string.
Return type: Integer
NOTE: plural forms might be combined. Math is mathing 🔢
getunits()— this unit, in a list. Just the one. Solo act 🎀static hasplural()— tells you whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— used to see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete.static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated.
This is the one to use for telling real, translatable units apart from header, obsolete, binary, or other blank units. ✨
istranslated()— indicates whether this unit is translated.
Use this rather than deducing it from .target, so that other classes can implement more functionality (as XLIFF does). Big brain move 🧠
makeobsolete()— makes a unit obsolete. Rip bozo 🪦markfuzzy(value=True)— marks the unit as fuzzy, or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.
Parameters:
needsreview– defaults toTrue.explanation– adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic, format-agnostic merging. 🤝multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]namespaced(name)— returnsnamein Clark notation.
Like, for example, namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourceThis is needed throughout lxml. Same deal as before, just saying it again for the people in the back 📣
removenotes(origin=None)— removes all the translator notes. Gone. Deleted. ✨🧹rich_parsers = []— a list of functions we use to parse a string into a rich string tree. It's giving "toolbox" energy 🔥property rich_source— the sourced-up rich version. SEE ALSO:rich_to_multistring(),multistring_to_rich(). 💅property rich_target— same vibe, but target-flavored. SEE ALSO:rich_to_multistring(),multistring_to_rich(). 🎀classmethod rich_to_multistring(elem_list)— converts a "rich" string tree into a multistring. Like, literally:
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context. Slay. ✨setid(value)— sets the unique identifier for this unit. Heads up bestie: only implemented if the format allows ids that are independent from other unit properties like source or context. iykyk 🫶settarget(target, lang='xx', append=False)— sets the "target" string (that's the second language, bestie), or alternatively appends to the list ifappendis true. 🦋unit_iter()— iterator that only returns this unit. Yes, just the one. Main character energy only. 👑
🧚 trados — the Trados .txt Translation Memory format
Okay so this module manages the Trados .txt Translation Memory format. 📁
A Trados file looks like this — no notes, this is verbatim:
<TrU>
<CrD>18012000, 13:18:35
<CrU>CAROL-ANN
<UsC>0
<Seg L=EN_GB>Association for Road Safety \endash Conference
<Seg L=DE_DE>Tagung der Gesellschaft für Verkehrssicherheit
</TrU>
<TrU>
<CrD>18012000, 13:19:14
<CrU>CAROL-ANN
<UsC>0
<Seg L=EN_GB>Road Safety Education in our Schools
<Seg L=DE_DE>Verkehrserziehung an Schulen
</TrU>translate.storage.trados.RTF_ESCAPES = {'\\-': '\xad', '\\_': '‑', '\\bullet': '•', '\\emdash': '—', '\\emspace': '\u2003', '\\endash': '–', '\\enspace': '\u2002', '\\ldblquote': '“', '\\lquote': '‘', '\\rdblquote': '”', '\\rquote': '’', '\\~': '\xa0'}— the RTF control to Unicode map. See http://msdn.microsoft.com/en-us/library/aa140283(v=office.10).aspx 🌈translate.storage.trados.TRADOS_TIMEFORMAT = '%d%m%Y, %H:%M:%S'— the time format used by Trados.txt. ⏰
💜 class translate.storage.trados.TradosTxtDate(newtime=None)
Manages the timestamps in the Trados .txt format of DDMMYYY, hh:mm:ss. (Yes, three Y's — we don't make the rules, we just vibe with them.)
get_time()— get thetime_structobject. 🕰️get_timestring()— get the time in the Trados time format. ✨set_time(newtime)— set thetime_structobject.
Parameters:
- newtime (
time.time_struct) — a new time object
set_timestring(timestring)— set thetime_structobject using a Trados time-formatted string.
Parameters:
- timestring (
String) — a Trados time string (DDMMYYYY, hh:mm:ss)
property time— get thetime_structobject.property timestring— get the time in the Trados time format.
🌸 class translate.storage.trados.TradosTxtTmFile(inputfile=None, **kwargs)
A Trados translation memory file. Obsessed, honestly. 💖
Extensions = ['txt']— a list of file extensions associated with this store type.Mimetypes = ['application/x-trados-tm']— a list of MIME types associated with this store type.Name = 'Trados Translation Memory'— the human-usable name of this store type.UnitClass— alias ofTradosUnit.add_unit_to_index(unit)— add a unit to source and location idexes. (Their spelling, not mine 🫶)addsourceunit(source)— add and return a new unit with the given source string.
Return type: TranslationUnit
addunit(unit)— append the given unit to the object's list of units. Always use this method rather than trying to modify the list manually, bestie — trust. 🙈
Parameters:
- unit (
TranslationUnit) — the unit that will be added
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from the text, using either the chardet lib or by trying to decode the file. 🔍static fallback_detection(text)— simple detection based on BOM in case chardet is not available. Backup plan iconique. 🧚findid(id)— find the unit with the matching id by checkingid_index.findunit(source)— find the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)— find the units with the given source string.
Return type: TranslationUnit or None
getids()— return a list of unit ids.getprojectstyle()— get the project type for this store.getsourcelanguage()— get the source language for this store.gettargetlanguage()— get the target language for this store.getunits()— return a list of all units in this store.isempty()— returnTrueif the object doesn't contain any translation units. Empty era. 💀makeindex()— indexes the items in this store. At minimum.sourceindexshould be useful. 🗂️property merge_on— the matching criterion to use when merging on.
Returns: the default matching criterion for all the subclasses.
Return type: string
parse(input)— parser to process the given source string.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object. 🎀classmethod parsestring(storestring)— convert the string representation back into an object.remove_unit_from_index(unit)— remove a unit from source and locaton indexes. (Again, their spelling — keep it cute, keep it exact. 💅)removeunit(unit)— remove the given unit from the object's list of units. Always use this rather than trying to modify the list manually, no cap.
Parameters:
- unit (
TranslationUnit) — the unit that will be added
require_index()— make sure the source index exists. 🫡save()— save to the file that data was originally read from, if available.savefile(storefile)— write the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like object to write to.setprojectstyle(project_style)— set the project type for this store.setsourcelanguage(sourcelanguage)— set the source language for this store.settargetlanguage(targetlanguage)— set the target language for this store.suggestions_in_format = False— indicates whether the format can store suggestions and alternative translations for a unit.translate(source)— return the translated string for a given source string.
Return type: String or None
unit_iter()— iterator over all the units in this store. 🌈
🦋 class translate.storage.trados.TradosUnit(source=None)
adderror(errorname, errortext)— adds an error message to this unit.
Parameters:
- errorname (
string) — a single word to id the error - errortext (
string) — the text describing the error
addlocation(location)— add one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. 🙅♀️
addlocations(location)— add a location, or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. Consider this your heads-up, bestie. ⚠️
addnote(text, origin=None, position='append')— adds a note (comment). 📝
Parameters:
- text (
string) — usually just a sentence or two - origin (
string) — specifies who/where the comment comes from. Origin can be one of the following text strings:'translator','developer','programmer','source code'(synonyms)
classmethod buildfromunit(unit)— build a native unit from a foreign unit, preserving as much information as possible. 🧚♀️static getcontext()— get the message context.static geterrors()— get all error messages.
Return type: Dictionary
getid()— a unique identifier for this unit.
Return type: string
Returns: an identifier for this unit that is unique in the store.
Derived classes should override this in a way that guarantees a unique identifier for each unit in the store. 👑
static getlocations()— a list of source code locations.
Return type: List
NOTE: shouldn't be implemented if the format doesn't support it. 💅
getnotes(origin=None)— returns all notes about this unit. It'll probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()). 🎀gettargetlen()— tells you how long the target string is, bestie. Return type: Integer.
NOTE: plural forms might get combined together, so don't come for me if the number looks a little sus 🙈
getunits()— returns this unit in a list. Yeah, just the one unit, in a list, vibing alone 🫶static hasplural()— tells you whether this specific unit has plural strings or not. Simple as that ✨infer_state()— an empty method that you're supposed to override in sub-classes so it can infer the current state (_n) of the unit from its current state. It's giving "template method, do it yourself" energy 💅isblank()— used to check if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and the devs might want to move in that direction rather and get rid of this. So like… don't get too attached 😭
static isfuzzy()— indicates whether this unit is fuzzy 👀static isheader()— indicates whether this unit is a header 👑static isobsolete()— indicates whether a unit is obsolete 💀static isreview()— indicates whether this unit needs review 🔍istranslatable()— indicates whether this unit can be translated.
This is the one you should use to distinguish real units for translation from header, obsolete, binary or other blank units. Basically it's the bouncer at the club 🌈
istranslated()— indicates whether this unit is translated.
Use this rather than deducing it from .target, so that other classes can implement more functionality (as XLIFF does). Trust the method, bestie 🫶
makeobsolete()— makes a unit obsolete. Bye boo 💀markfuzzy(value=True)— marks the unit as fuzzy or not fuzzy. Default'sTrue, so it's fuzzy unless you say otherwise 🦋markreviewneeded(needsreview=True, explanation=None)— marks the unit to show whether it needs review.
Parameters:
- needsreview — defaults to
True. - explanation — adds an optional explanation as a note.
Very "leave a comment for your future self" of it ngl 📝
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does a basic format-agnostic merge. No cap, that's the whole description 🔥multistring_to_rich(mulstring)— converts a multistring to a list of "rich" string trees. Watch this:
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]Iconic behaviour, honestly ✨
removenotes(origin=None)— removes all the translator's notes. All of them. Gone. Periodt 🧹rich_parsers = []— a list of functions to use for parsing a string into a rich string tree 🌸property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a "rich" string tree to a multistring. Like so:
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')The way it just glues it back together 😭💖
setcontext(context)— sets the message context 🎯setid(value)— sets the unique identifier for this unit.
Only implemented if the format allows ids independent from other unit properties like source or context. So like, format-dependent, iykyk 💅
unit_iter()— an iterator that only returns this unit. Just the one. Main character energy 👑
translate.storage.trados.escape(text)
Converts a Unicode string to Trodas escapes. (Yes, "Trodas" — that's exactly what the original says, I'm not fixing it 🙈)
translate.storage.trados.unescape(text)
Converts Trados text to a normal Unicode string 🦋
ts2
Module for handling Qt linguist (.ts) files.
This will eventually replace the older ts.py which only supports the older format. While converters haven't been updated to use this module, we retain both. So like, two of them, coexisting, iconic 🫶
TS file format 4.3, 4.8, 5. Example.
Specification of the valid variable entries, 2.
class translate.storage.ts2.tsfile(*args, **kwargs)
Class representing a TS file store. The whole vibe of a .ts file, basically ✨
Extensions = ['ts']— a list of file extentions associated with this store type. (Their spelling, not mine, bestie 💅)Mimetypes = ['application/x-linguist']— a list of MIME types associated with this store type 🎀Name = 'Qt Linguist Translation File'— the human usable name of this store type 👑UnitClass— alias oftsunit.add_unit_to_index(unit)— adds a unit to source and location idexes. (Again: "idexes". Keeping it exactly as written, no cap 😭)addheader()— a method to be overridden to initialise headers, etc.addsourceunit(source)— adds and returns a new unit with the given string as first entry 🍓addunit(unit, new=True, contextname=None, comment=None, createifmissing=True)— adds the given unit to the last used body node (a.k.a. the current context).
If the contextname is specified, it switches to that context (creating it if allowed by createifmissing). Very "make yourself at home" 🌸
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, using either thechardetlib or by trying to decode the file 🔍static fallback_detection(text)— simple detection based on BOM in casechardetis not available. The backup plan, the understudy, the reliable bestie 🫶findid(id)— finds the unit with a matching id by checkingid_index🎯findunit(source)— finds the unit with the given source string.
Return type: TranslationUnit or None
findunits(source)— finds the units with the given source string.
Return type: TranslationUnit or None
getids()— returns a list of unit ids 🔥getprojectstyle()— gets the project type for this store 👑getsourcelanguage()— gets the source language for this.tsfile.
The 'sourcelanguage' attribute was only added to the TS format in Qt v4.5. We return 'en' if there is no sourcelanguage set.
We don't implement setsourcelanguage as users really shouldn't be altering the source language in .ts files — it should be set correctly by the extraction tools. So hands off, bestie 🙈
Returns: ISO code e.g. af, fr, pt_BR
Return type: String
gettargetlanguage()— gets the target language for this.tsfile.
Returns: ISO code e.g. af, fr, pt_BR
Return type: String
getunits()— returns a list of all units in this store 💜initbody()— initialisesself.body✨isempty()— returnsTrueif the object doesn't contain any translation units 🌈makeindex()— indexes the items in this store. At least.sourceindexshould be useful.property merge_on— the matching criterion to use when merging on.
Returns: the default matching criterion for all the subclasses.
Return type: string
namespaced(name)— returns name in Clark notation.
For example namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourceThis is needed throughout lxml. So like, everywhere. It's the moment 🎀
parse(xml)— populates this object from the given xml string 🌸classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an object 🦋classmethod parsestring(storestring)— converts the string representation back to an object 🍓remove_unit_from_index(unit)— removes a unit from source and locaton indexes. (Yes, "locaton". Verbatim. Love that for us 😭)removeunit(unit)— removes the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually. Don't go rogue, bestie 💅
Parameters: unit (TranslationUnit) — the unit that will be added.
require_index()— makes sure the source index exists ✅save()— saves to the file that data was originally read from, if available 💾savefile(storefile)— writes the string representation to the given file (or filename) 🎀serialize(out)— writes the XML document to a file 🌈setprojectstyle(project_style)— sets the project type for this store 👑setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this.tsfile to targetlanguage.
Parameters: targetlanguage (String) — ISO code e.g. af, fr, pt_BR 🦋
suggestions_in_format = False— indicates if the format can store suggestions and alternative translation for a unit. Right now: nope 🙈translate(source)— returns the translated string for a given source string.
Return type: String or None
unit_iter()— an iterator over all the units in this store ✨
class translate.storage.ts2.tsunit(source, empty=False, **kwargs)
A single term in the TS file. One unit, living its life 💅
adderror(errorname, errortext)— adds an error message to this unit.
Parameters:
- errorname (
string) — a single word to id the error.
errortext(string) — the text that describes the error. 🫶
addlocation(location)— drop one location onto the list of locations. 💅
NOTE: don't even bother implementing this if the format doesn't support it. No cap.
addlocations(location)— adds a location, or a whole list of locations. ✨
NOTE: most classes really shouldn't implement this — go implement TranslationUnit.addlocation() instead, bestie.
WARNING: this method might get removed in the future. 😭 Consider yourself warned.
addnote(text, origin=None, position='append')— adds a note, specifically into the appropriate comment tag. 🎀buildfromunit(unit)(classmethod) — builds a native unit out of a foreign unit, preserving as much information as possible. 🧚createlanguageNode(lang, text, purpose)— returns an xml Element that's all set up with the parameters you gave it.getNodeText(languageNode, xml_space='preserve')— grabs the term out of the languageNode you hand it.getcontext()— get the message context. 💬geterrors()(static) — get all the error messages.
Return type: Dictionary
getid()— a unique identifier for this unit. 👑
Return type: string
Returns: an identifier for this unit that's unique in the store.
Derived classes should override this so it guarantees a unique identifier for each unit in the store. Periodt.
getlanguageNode(lang=None, index=None)— retrieves a languageNode, either by language or by index. 🦋getlanguageNodes()— we override this to get source and target nodes.getlocations()— a list of source code locations.
Return type: List
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all the notes about this unit. 📝
It'll probably be freeform text or something reasonable that the format can synthesise. It should not include location comments (peep getlocations() for those).
gettarget(lang=None)— retrieves the "target" text (that's the second entry), or the entry in the specified language, if it exists.gettargetlen()— returns the length of the target string.
Return type: Integer
NOTE: plural forms might be combined. 💀
getunits()— this unit, in a list.hasplural()— tells you whether or not this specific unit has plural strings.infer_state()— an empty method that you should override in sub-classes so it can infer the current state(_n) of the unit from its current state.isblank()— used to check if this unit has no source or target string. 🙈
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
isfuzzy()— indicates whether this unit is fuzzy.isheader()(static) — indicates whether this unit is a header.isobsolete()— indicates whether a unit is obsolete.isreview()— states whether this unit needs to be reviewed.istranslatable()— indicates whether this unit can be translated.
Use this to tell real units for translation apart from header, obsolete, binary or other blank units. 🎯
istranslated()— indicates whether this unit is translated.
Use this rather than deducing it from .target, so other classes can implement more functionality (like XLIFF does).
makeobsolete()— makes a unit obsolete.markfuzzy(value=True)— marks the unit as fuzzy or not. 🌸markreviewneeded(needsreview=True, explanation=None)— marks the unit to say whether it needs review.
Parameters
needsreview— defaults toTrue.explanation— adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic, format-agnostic merging.multistring_to_rich(mulstring)— converts a multistring into a list of "rich" string trees. 💖
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]namespaced(name)— returnsnamein Clark notation. 🪄
So for example, namespaced("source") in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourceThis is needed all over the place throughout lxml.
removenotes(origin=None)— removes all the translator notes. 🧹rich_parsers = [...]— a list of functions to use for parsing a string into a rich string tree. 👑
rich_parsers = [<bound method regex_parse of <class
'translate.storage.placeables.general.NewlinePlaceable'>>, <bound
method regex_parse of <class
'translate.storage.placeables.general.XMLTagPlaceable'>>, <bound
method regex_parse of <class
'translate.storage.placeables.general.AltAttrPlaceable'>>, <bound
method regex_parse of <class
'translate.storage.placeables.general.XMLEntityPlaceable'>>,
<bound method regex_parse of <class
'translate.storage.placeables.general.PythonFormattingPlaceable'>>,
<bound method regex_parse of <class
'translate.storage.placeables.general.JavaMessageFormatPlaceable'>>,
<bound method regex_parse of <class
'translate.storage.placeables.general.FormattingPlaceable'>>,
<bound method regex_parse of <class
'translate.storage.placeables.general.QtFormattingPlaceable'>>,
<bound method regex_parse of <class
'translate.storage.placeables.general.UrlPlaceable'>>, <bound
method regex_parse of <class
'translate.storage.placeables.general.FilePlaceable'>>, <bound
method regex_parse of <class
'translate.storage.placeables.general.EmailPlaceable'>>, <bound
method regex_parse of <class
'translate.storage.placeables.general.CapsPlaceable'>>, <bound
method regex_parse of <class
'translate.storage.placeables.general.CamelCasePlaceable'>>,
<bound method regex_parse of <class
'translate.storage.placeables.general.OptionPlaceable'>>, <bound
method regex_parse of <class
'translate.storage.placeables.general.PunctuationPlaceable'>>,
<bound method regex_parse of <class
'translate.storage.placeables.general.NumberPlaceable'>>]rich_source(property)
SEE ALSO:rich_to_multistring(),multistring_to_rich().
rich_target(property)
SEE ALSO:rich_to_multistring(),multistring_to_rich().
rich_to_multistring(elem_list)(classmethod) — converts a "rich" string tree into a multistring. 🦋
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(value)— sets the message context. 💬setid(value)— sets the unique identifier for this unit.
Only implement this if the format allows ids that are independent of other unit properties like source or context, iykyk.
settarget(target, lang='xx', append=False)— sets the "target" string (that's the second language), or alternatively appends to the list. ✨statemap = {'': 100, 'obsolete': -100, 'unfinished': 30, None: 100}— this maps the unit "type" attribute to state. 🗺️unit_iter()— an iterator that only returns this unit. Fr fr, just the one.
🌈 ts
Module for parsing Qt .ts files for translation. 🎀
Currently this module supports the old format of .ts files. Some applications use the newer .ts format, which is documented here: TS file format 4.3, Example
Specification of the valid variable entries, 2
🍓 txt
This class implements the functionality for handling plain text files, or similar wiki type files. 📄
Supported formats are
- Plain text
- dokuwiki
- MediaWiki
💜 class translate.storage.txt.TxtFile(inputfile=None, flavour=None, no_segmentation=False, **kwargs)
This class represents a text file, made up of txtunits.
Extensions = None— a list of file extentions associated with this store type.Mimetypes = None— a list of MIME types associated with this store type.Name = 'Base translation store'— the human usable name of this store type. 👑UnitClass— alias ofTxtUnit.add_unit_to_index(unit)— add a unit to the source and location idexes.addsourceunit(source)— add and return a new unit with the given source string. 🌱
Return type: TranslationUnit
addunit(unit)— append the given unit to the object's list of units.
This method should always be used rather than trying to modify the list manually. No cap, don't touch the list directly. 🙅♀️
Parameters
unit(TranslationUnit) — the unit that will be added.
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to sniff out a file's encoding from the text, either by leaning on thechardetlib or by just… attempting to decode the file and seeing what sticks. 💖static fallback_detection(text)— the lowkey backup plan: a super simple detection that just looks at the BOM, for whenchardetisn't installed. humble but it works ✨findid(id)— find the unit with a matchingidby checkingid_index. easy.findunit(source)— find the unit with the given source string. Return type:TranslationUnitorNone.findunits(source)— find the units (plural, bestie) with the given source string. Return type:TranslationUnitorNone. yes, same return type, don't come for me, that's what it says 🫶getids()— hands you a list of unit ids. that's it, that's the whole vibe.getprojectstyle()— get the project type for this store.getsourcelanguage()— get the source language for this store.gettargetlanguage()— get the target language for this store.getunits()— return a list of all units in this store. the whole squad 💅isempty()— returnsTrueif the object doesn't contain any translation units. empty store = no cap, nothing in there.makeindex()— indexes the items in this store. at least.sourceindexshould be useful. it's giving "get your house in order" energy.- property
merge_on— the matching criterion to use when merging on. Returns: the default matching criterion for all the subclasses. Return type:string. parse(lines)— reads in text lines and createstxtunits from the blocks of text.classmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses it back into an object. glowy little round trip ✨classmethod parsestring(storestring)— converts the string representation back into an object.remove_unit_from_index(unit)— removes a unit from the source and location indexes.removeunit(unit)— removes the given unit from the object's list of units. always use this method instead of trying to mess with the list by hand, fr fr — the manual literally says so.- Parameters:
unit(TranslationUnit) – The unit that will be added. (yes, that's genuinely what the param doc says on a remove method 💀 iconic of it)
- Parameters:
require_index()— makes sure the source index exists, like a little vibe check for your index 🧚save()— saves to the file that the data was originally read from, if available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like object to write to.setprojectstyle(project_style)— set the project type for this store.setsourcelanguage(sourcelanguage)— set the source language for this store.settargetlanguage(targetlanguage)— set the target language for this store.suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit. default: nope 🙈translate(source)— returns the translated string for a given source string. Return type:StringorNone.unit_iter()— iterator over all the units in this store. go lil iterator go 🦋
🎀 class translate.storage.txt.TxtUnit(source='', **kwargs)
okay so this class represents a block of text from a text file. simple as that, periodt.
adderror(errorname, errortext)— adds an error message to this unit.- Parameters:
errorname(string) – a single word to id the error.errortext(string) – the text describing the error.
- Parameters:
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it.
addlocations(location)— adds a location or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: this method might be removed in future. ⚠️ consider yourself warned, bestie.
addnote(text, origin=None, position='append')— adds a note (a comment).- Parameters:
text(string) – usually just a sentence or two.origin(string) – specifies who/where the comment comes from.origincan be one of the following text strings:'translator','developer','programmer','source code'(the last three are synonyms, iykyk).
- Parameters:
classmethod buildfromunit(unit)— builds a native unit from a foreign unit, preserving as much info as possible. 🫶static getcontext()— get the message context.static geterrors()— get all error messages. Return type:Dictionary.getid()— a unique identifier for this unit. Return type:string.- Returns: an identifier for this unit that is unique in the store.
- derived classes should override this in a way that guarantees a unique identifier for each unit in the store. no duplicates allowed, that's the rule 👑
getlocations()— a list of source code locations. Return type:List.
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all notes about this unit. it'll probably be freeform text or something reasonable that the format can synthesise. it should not include location comments (seegetlocations()).gettargetlen()— returns the length of the target string. Return type:Integer.
NOTE: plural forms might be combined.
getunits()— this unit, in a list. a list of one. main character energy, honestly 💅static hasplural()— tells you whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— used to see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. (the manual said it, not me 😭)
static isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete.static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated. this should be used to distinguish real units for translation from header, obsolete, binary or other blank units. we're filtering out the NPCs 🌈istranslated()— indicates whether this unit is translated. use this rather than deducing it from.target, so that other classes can implement more functionality (as XLIFF does).makeobsolete()— makes a unit obsolete. bye bestie ✨markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.- Parameters:
needsreview– defaults toTrue.explanation– adds an optional explanation as a note.
- Parameters:
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic, format-agnostic merging.multistring_to_rich(mulstring)— convert a multistring to a list of "rich" string trees:
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator's notes. poof, gone 🪄rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.- property
rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
- property
rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— convert a "rich" string tree to a multistring:
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— set the message context.setid(value)— sets the unique identifier for this unit. only implemented if the format allows ids independent from other unit properties like source or context.- property
target— gets the unquoted target string. unit_iter()— an iterator that only returns this unit. just the one. the moment is hers 👑
🌈 utx
manages the Universal Terminology eXchange (UTX) format. iconic little acronym, go off 💜
So like, UTX is this format for terminology exchange — and ngl it was clearly designed with Machine Translation (MT) as its main character consumer. The format comes from the Asia-Pacific Association for Machine Translation (AAMT). 💅
It's a bilingual base-class-derived kind of situation, with UtxFile and UtxUnit giving you file-level and unit-level access respectively. ✨
The format can technically manage monolingual dictionaries, but these classes don't implement that — so don't come for them about it. 🙈
🌸 Specification 🌸
The format is implemented according to UTX v1.0 — heads up: that spec is No longer available from their website. The current UTX version may be downloaded instead. 😭
🧚 Format Implementation 🧚
The UTX format is a Tab Seperated Value (TSV) file in UTF-8. The first two lines are headers, and the subsequent lines each contain a single source–target definition. 🎀
🔤 Encoding 🔤
The files are UTF-8 encoded with no BOM and CR+LF line terminators, periodt. ⭐
class translate.storage.utx.UtxDialect
Describes the properties of a UTX generated TAB-delimited dictionary file. 🦋
class translate.storage.utx.UtxFile(inputfile=None, **kwargs)
A UTX dictionary file. Your main character. 👑
Extensions=['utx']— a list of file extentions associated with this store typeMimetypes=['text/x-utx']— a list of MIME types associated with this store typeName='UTX Dictionary'— the human usable name of this store typeUnitClass— alias ofUtxUnitadd_unit_to_index(unit)— add a unit to source and location idexesaddsourceunit(source)— add and returns a new unit with the given source string- Return type:
TranslationUnit
- Return type:
addunit(unit)— append the given unit to the object's list of units. Lowkey this method should always be used rather than trying to modify the list manually, fr fr- Parameters:
unit(TranslationUnit) – The unit that will be added.
- Parameters:
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— try to detect a file encoding from text, using either the chardet lib or by trying to decode the filestatic fallback_detection(text)— simple detection based on BOM in case chardet is not availablefindid(id)— find unit with matching id by checkingid_indexfindunit(source)— find the unit with the given source string- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— find the units with the given source string- Return type:
TranslationUnitorNone
- Return type:
getids()— return a list of unit idsgetprojectstyle()— get the project type for this storegetsourcelanguage()— get the source language for this storegettargetlanguage()— get the target language for this storegetunits()— return a list of all units in this storeisempty()— returnTrueif the object doesn't contain any translation unitsmakeindex()— indexes the items in this store. At least.sourceindexshould be useful 👀property merge_on— the matching criterion to use when merging on- Returns: The default matching criterion for all the subclasses.
- Return type:
string
parse(input)— parsese the given file or file source stringclassmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an objectclassmethod parsestring(storestring)— convert the string representation back to an objectremove_unit_from_index(unit)— remove a unit from source and locaton indexesremoveunit(unit)— remove the given unit to the object's list of units. Again: this method should always be used rather than trying to modify the list manually, no cap- Parameters:
unit(TranslationUnit) – The unit that will be added.
- Parameters:
require_index()— make sure source index existssave()— save to the file that data was originally read from, if availablesavefile(storefile)— write the string representation to the given file (or filename)serialize(out)— converts to a bytes representation that can be parsed back usingparsestring().outshould be an open file-like objects to write tosetprojectstyle(project_style)— set the project type for this storesetsourcelanguage(sourcelanguage)— set the source language for this storesettargetlanguage(targetlanguage)— set the target language for this storesuggestions_in_format=False— indicates if format can store suggestions and alternative translation for a unittranslate(source)— return the translated string for a given source string- Return type:
StringorNone
- Return type:
unit_iter()— iterator over all the units in this store 🌀
class translate.storage.utx.UtxHeader
A UTX header entry. 💌
A UTX header is a single line that looks like this:
#UTX-S <version>; < source language >/< target language>; <date created>; <optional fields (creator, license, etc.)>Where:
- UTX-S version is currently 1.00.
- Source language/target language: ISO 639, 3166 formats. In the case of monolingual dictionary, target language should be omitted.
- Date created: ISO 8601 format
- Optional fields (creator, license, etc.)
class translate.storage.utx.UtxUnit(source=None)
A UTX dictionary unit. 🎀
adderror(errorname, errortext)— adds an error message to this unit- Parameters:
errorname(string) – A single word to id the error.errortext(string) – The text describing the error.
- Parameters:
addlocation(location)— add one location to the list of locations
NOTE: shouldn't be implemented if the format doesn't support it.
addlocations(location)— add a location or a list of locations
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation(). WARNING: this method might be removed in future. ⚠️
addnote(text, origin=None, position='append')— adds a note (comment)- Parameters:
text(string) – Usually just a sentence or two.origin(string) – Specifies who/where the comment comes from. Origin can be one of the following text strings: -'translator'-'developer','programmer','source code'(synonyms)
- Parameters:
classmethod buildfromunit(unit)— build a native unit from a foreign unit, preserving as much information as possibleproperty dict— get the dictionary of values for a UTX linestatic getcontext()— get the message contextgetdict()— get the dictionary of values for a UTX linestatic geterrors()— get all error messages- Return type:
Dictionary
- Return type:
getid()— a unique identifier for this unit- Return type:
string - Returns: an identifier for this unit that is unique in the store
- Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.
- Return type:
static getlocations()— a list of source code locations- Return type:
List
- Return type:
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all notes about this unit. It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (seegetlocations()).gettargetlen()— returns the length of the target string- Return type:
Integer
- Return type:
NOTE: plural forms might be combined.
getunits()— this unit in a liststatic hasplural()— tells whether or not this specific unit has plural stringsinfer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current stateisblank()— used to see if this unit has no source or target string
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
static isfuzzy()— indicates whether this unit is fuzzystatic isheader()— indicates whether this unit is a headerstatic isobsolete()— indicate whether a unit is obsoletestatic isreview()— indicates whether this unit needs reviewistranslatable()— indicates whether this unit can be translated 🌈
This should be used to distinguish real units for translation from header, obsolete, binary, or other blank units. Like, vibe-check your units before you trust them, bestie. 💅
istranslated()— Says whether this unit is translated. Use this instead of trying to deduce it from.target, so other classes can implement more functionality (as XLIFF does). ✨makeobsolete()— Makes a unit obsolete. Retire it, periodt. 🫶markfuzzy(value=True)— Marks the unit as fuzzy or not. Fuzzy = maybe translated, maybe not, iykyk. 🌀markreviewneeded(needsreview=True, explanation=None)— Marks the unit to show whether it needs review.needsreview— defaults toTrue.explanation— adds an optional explanation as a note. Receipts for the reviewer. 🧾
merge(otherunit, overwrite=False, comments=True, authoritative=False)— Does basic format-agnostic merging. No cap, it just merges. 💜multistring_to_rich(mulstring)— Converts a multistring to a list of “rich” string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— Removes all the translator’s notes. Clean slate energy. 🧼rich_parsers = []— A list of functions used for parsing a string into a rich string tree. Parser squad goes here. 🛠️property rich_source— The rich source property.
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target— The rich target property.
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— Converts a “rich” string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— Sets the message context. Context is the moment. 🫧setdict(newdict)— Sets the dictionary of values for a UTX line.newdict(Dict) — a new dictionary with UTX line elements.
setid(value)— Sets the unique identifier for this unit. Only implemented if the format allows ids independent from other unit properties like source or context. 🔑unit_iter()— Iterator that only returns this unit. Just this one, bestie. 🎀
🌸 wordfast
Manage the Wordfast Translation Memory format. Wordfast TM my beloved? 💜
Wordfast TM format is the Translation Memory format used by the Wordfast computer aided translation tool.
It is a bilingual base-class-derived format with WordfastTMFile and WordfastUnit providing file-level and unit-level access. File access + unit access = iconic duo. 🌈
Wordfast is a computer aided translation tool. It is an application built on top of Microsoft Word and is implemented as a rather sophisticated set of macros. Understanding that helps us understand many of the seemingly strange choices around this format, including: encoding, escaping, and file naming. Macros made it do what it does, okay? 💅
🛠️ Implementation
The implementation covers the full requirements of a Wordfast TM file. The files are simple Tab Separated Value (TSV) files that can be read by Microsoft Excel and other spreadsheet programs. They use the .txt extension, which does make it more difficult to automatically identify such files. Yes, .txt is a little chaotic of them. 🌀
The dialect of the TSV files is specified by WordfastDialect.
🔐 Encoding
The files are UTF-16 or ISO-8859-1 (Latin1) encoded. These choices are most likely because Microsoft Word is the base editing tool for Wordfast. Microsoft Word core, fr fr. 💖
The format is tab separated, so we are able to detect UTF-16 vs Latin-1 by searching for the occurrence of a UTF-16 tab character and then continuing with the parsing. Detective work but make it tabs. 🕵️
⏰ Timestamps
WordfastTime allows for the correct management of the Wordfast YYYYMMDD~HHMMSS timestamps. However, timestamps on individual units are not updated when edited. Time moves on, but those unit timestamps are staying put. 😭
📬 Header
WordfastHeader provides header management support. The header functionality is fully implemented through observing the behaviour of the files in real use cases, input from the Wordfast programmers, and public documentation. Receipts: real files, Wordfast devs, and docs. 🧾
🪄 Escaping
Wordfast TM implements a form of escaping that covers two aspects:
- Placeable: bold, formatting, etc. These are left as is and ignored. It is up to the editor and future placeable implementation to manage these. Future placeable implementation, you got this. ✨
- Escapes: items that may confuse Excel or translators are escaped as
&'XX;. These are fully implemented and are converted to and from Unicode. By observing behaviour and reading documentation, we were able to observe all possible escapes. Unfortunately the escaping differs slightly between Windows and Mac version. This might cause errors in future. Functions allow for<_wf_to_char>and back to Wordfast escape (<_char_to_wf>). Windows vs Mac escaping drama — not iconic. 💀
🧩 Extended Attributes
The last 4 columns allow users to define and manage extended attributes. These are left as is and are not directly managed by our implementation. We see them, we leave them. 🫶
translate.storage.wordfast.TAB_UTF16 = b'\x00\t'— the tab character as it would appear in UTF-16 encoding. 🌸
translate.storage.wordfast.WF_ESCAPE_MAP = (("&'26;", '&'), ("&'82;", '‚'), ("&'85;", '…'), ("&'91;", '‘'), ("&'92;", '’'), ("&'93;", '“'), ("&'94;", '”'), ("&'96;", '–'), ("&'97;", '—'), ("&'99;", '™'), ("&'A0;", '\xa0'), ("&'A9;", '©'), ("&'AE;", '®'), ("&'BC;", '¼'), ("&'BD;", '½'), ("&'BE;", '¾'), ("&'A8;", '®'), ("&'AA;", '™'), ("&'C7;", '«'), ("&'C8;", '»'), ("&'C9;", '…'), ("&'CA;", '\xa0'), ("&'D0;", '–'), ("&'D1;", '—'), ("&'D2;", '“'), ("&'D3;", '”'), ("&'D4;", '‘'), ("&'D5;", '’'), ("&'E2;", '‚'), ("&'E3;", '„'))Mapping of Wordfast &'XX; escapes to correct Unicode characters. Escape girlies, this is your map. 🗺️
translate.storage.wordfast.WF_FIELDNAMES = ['date', 'user', 'reuse', 'src-lang', 'source', 'target-lang', 'target', 'attr1', 'attr2', 'attr3', 'attr4', 'attr5']Field names for a Wordfast TU. 🎀
translate.storage.wordfast.WF_FIELDNAMES_HEADER = ['date', 'userlist', 'tucount', 'src-lang', 'version', 'target-lang', 'license', 'attr1list', 'attr2list', 'attr3list', 'attr4list', 'attr5list']Field names for the Wordfast header. 👑
translate.storage.wordfast.WF_FIELDNAMES_HEADER_DEFAULTS = {'attr1list': '', 'attr2list': '', 'attr3list': '', 'attr4list': '', 'attr5list': '', 'date': '%19000101~121212', 'license': '%---00000001', 'src-lang': '%EN-US', 'target-lang': '', 'tucount': '%TU=00000001', 'userlist': '%User ID,TT,TT Translate-Toolkit', 'version': '%Wordfast TM v.5.51w9/00'}Default or minimum header entries for a Wordfast file. Starter pack, but Wordfast. 💜
translate.storage.wordfast.WF_TIMEFORMAT = '%Y%m%d~%H%M%S'Time format used by Wordfast. ⏰
class translate.storage.wordfast.WordfastDialect— describes the properties of a Wordfast-generated TAB-delimited file. TAB-delimited, but make it Wordfast. 🦋class translate.storage.wordfast.WordfastHeader(header=None)— a Wordfast translation memory header.getheader()— gets the header dictionary. 🧾property header— gets the header dictionary. ✨
class translate.storage.wordfast.WordfastTMFile(inputfile=None, **kwargs)— a Wordfast translation memory file. 🌸Extensions = ['txt']— a list of file extensions associated with this store type. Yep,.txtagain. 🎀Mimetypes = ['application/x-wordfast']— a list of MIME types associated with this store type. 💅Name = 'Wordfast Translation Memory'— the human-usable name of this store type. 👑UnitClass— alias ofWordfastUnit. 🧚add_unit_to_index(unit)— adds a unit to source and location indexes. 🗂️addsourceunit(source)— adds and returns a new unit with the given source string.- Return type:
TranslationUnit.
- Return type:
addunit(unit)— appends the given unit to the object’s list of units. This method should always be used rather than trying to modify the list manually.unit(TranslationUnit) — the unit that will be added. 🫶
detect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, using either thechardetlib or by trying to decode the file. 🔍static fallback_detection(text)— simple detection based onBOMin casechardetis not available. 🧯findid(id)— finds the unit with matching id by checkingid_index. 🪪findunit(source)— finds the unit with the given source string.- Return type:
TranslationUnit or None.
- Return type:
findunits(source)— finds the units with the given source string.- Return type:
TranslationUnit or None.
- Return type:
getids()— returns a list of unit ids. 🔢getprojectstyle()— gets the project type for this store. 🎨getsourcelanguage()— gets the source language for this store. 🌍gettargetlanguage()— gets the target language for this store. 🗺️getunits()— returns a list of all units in this store. 📚isempty()— returnsTrueif the object doesn’t contain any translation units. 🫧makeindex()— indexes the items in this store. At least.sourceindexshould be useful. Index it, bestie. 🗂️property merge_on— the matching criterion to use when merging on.- Returns: the default matching criterion for all the subclasses.
- Return type:
string.
parse(input)— parses the given file or file source string. Read the file, understand the file, slay. 💖
classmethod parsefile(storefile)— reads the file you give it (or opens the filename you give it) and parses it straight back into an object. round-trip slay 💅classmethod parsestring(storestring)— converts the string representation right back into an object. reverse glow-up ✨remove_unit_from_index(unit)— removes a unit from the source and locaton indexes. (yes, "locaton" — the docs said it like that, we don't fix, we respect 😭)removeunit(unit)— removes the given unit from the object's list of units.
bestie pls always use this instead of trying to modify the list manually, i'm begging 🙏
unit(TranslationUnit) — the unit that will be added.
require_index()— makes sure the source index exists. no index no vibes ✨save()— saves to the file that the data was originally read from, if that's available.savefile(storefile)— writes the string representation to the given file (or filename).serialize(out)— converts to a bytes representation that can be parsed back withparsestring().outshould be an open file-like object to write to. 🫶setprojectstyle(project_style)— sets the project type for this store.setsourcelanguage(sourcelanguage)— sets the source language for this store.settargetlanguage(targetlanguage)— sets the target language for this store.suggestions_in_format = False— indicates if the format can store suggestions and alternative translations for a unit. 🎀translate(source)— returns the translated string for a given source string.- Return type:
StringorNone
- Return type:
unit_iter()— iterator over all the units in this store. every single one, no skips 💖
🕰️ class translate.storage.wordfast.WordfastTime(newtime=None)
manages time stamps in the Wordfast format of YYYYMMDD~hhmmss. lowkey iconic format ngl 🦋
get_time()— gets thetime_structobject.get_timestring()— gets the time in the Wordfast time format.set_time(newtime)— sets thetime_structobject.newtime(time.time_struct) — a new time object
set_timestring(timestring)— sets thetime_structobject using a Wordfast time formatted string. (the docs typed it "time_sturct" and honestly? relatable 💀)timestring(String) — a Wordfast time string (YYYMMDD~hhmmss)
property time— gets thetime_structobject.property timestring— gets the time in the Wordfast time format.
🧚 class translate.storage.wordfast.WordfastUnit(source=None)
a Wordfast translation memory unit. main character energy fr fr 👑
adderror(errorname, errortext)— adds an error message to this unit.errorname(string) — a single word to id the errorerrortext(string) — the text describing the error
addlocation(location)— adds one location to the list of locations.
NOTE: shouldn't be implemented if the format doesn't support it. 💀
addlocations(location)— adds a location or a list of locations.
NOTE: most classes shouldn't need to implement this, but should rather implement TranslationUnit.addlocation().
WARNING: this method might be removed in future. heads up bestie ⚠️
addnote(text, origin=None, position='append')— adds a note (comment). 📝text(string) — usually just a sentence or twoorigin(string) — specifies who/where the comment comes from. Origin can be one of the following text strings:'translator','developer','programmer','source code'(synonyms)
classmethod buildfromunit(unit)— builds a native unit from a foreign unit, preserving as much information as possible. cultural exchange moment 🌈property dict— gets the dictionary of values for a Wordfast line.static getcontext()— gets the message context.getdict()— gets the dictionary of values for a Wordfast line.static geterrors()— gets all error messages.- Return type:
Dictionary
- Return type:
getid()— a unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store
- Return type:
derived classes should override this in a way that guarantees a unique identifier for each unit in the store. no duplicates allowed, periodt ✨
static getlocations()— a list of source code locations.- Return type:
List
- Return type:
NOTE: shouldn't be implemented if the format doesn't support it.
getnotes(origin=None)— returns all notes about this unit.
it'll probably be freeform text or something reasonable that can be synthesised by the format. it should not include location comments (see getlocations()). keep those separate bestie 🫶
gettargetlen()— returns the length of the target string.- Return type:
Integer
- Return type:
NOTE: plural forms might be combined. 👀
getunits()— this unit, in a list. just the one. a solo moment 💅static hasplural()— tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isblank()— used to see if this unit has no source or target string.
NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this. 😭
static isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete.static isreview()— indicates whether this unit needs review.istranslatable()— indicates whether this unit can be translated.
this should be used to distinguish real units for translation from header, obsolete, binary or other blank units. only the real ones, bestie 🎀
istranslated()— indicates whether this unit is translated.
use this rather than deducing it from .target, so that other classes can implement more functionality (as XLIFF does). trust the method, not your detective work 🔍
makeobsolete()— makes a unit obsolete. it's giving retired 💀markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.needsreview— defaults toTrueexplanation— adds an optional explanation as a note
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic format agnostic merging. 🤝multistring_to_rich(mulstring)— converts a multistring to a list of "rich" string trees.
>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
<StringElem([<StringElem(['bar'])>])>,
<StringElem([<StringElem(['baz'])>])>]removenotes(origin=None)— removes all the translator's notes. clean slate energy 🧹rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source
SEE ALSO:rich_to_multistring(),multistring_to_rich().
property rich_target
SEE ALSO:rich_to_multistring(),multistring_to_rich().
classmethod rich_to_multistring(elem_list)— converts a "rich" string tree to a multistring.
>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')setcontext(context)— sets the message context.setdict(newdict)— sets the dictionary of values for a Wordfast line.newdict(Dict) — a new dictionary with Wordfast line elements
setid(value)— sets the unique identified for this unit. (their word, not mine 😭)
only implemented if the format allows ids independant from other unit properties like source or context. (again: their spelling, i'm just the messenger 💅)
unit_iter()— iterator that only returns this unit. one-unit parade 🎺
🔄 workflow
a workflow is defined by a set of states that a translation unit can be in, plus the (allowed) transitions between those states. a state is defined by a range between -128 and 127, indicating its level of "completeness". the range is closed at the beginning and open at the end. so like — if a workflow contains states A, B and C where A < B < C, then a unit with state number n is in state A if A <= n < B, in state B if B <= n < C, or in state C if C <= n < MAX. math but make it fashion 💅✨
a value of 0 is typically the "empty" or "new" state, with negative values reserved for states like "obsolete" or "do not use". negative numbers = the sad states, iykyk 🥀
so here's the vibe check on workflows: formats are supposed to line up their numbers so that similar states actually mean similar things across formats. Like, state 0 should be “untranslated” in PO and “new” or “empty” in XLIFF, and state 100 should be “translated” in PO and “final” in XLIFF. That way each format lowkey defines equivalent states just by being consistent with the numbering. iconic behaviour honestly ✨
exception translate.storage.workflow.InvalidStateObjectError(obj) 🫶
add_note(object, /)—Exception.add_note(note)– adds a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb)– setsself.__traceback__totband returnsself
exception translate.storage.workflow.NoInitialStateError 🫶
add_note(object, /)—Exception.add_note(note)– adds a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb)– setsself.__traceback__totband returnsself
class translate.storage.workflow.StateEnum — bestie this one only holds the constants for the default states, that's its whole personality 💅
exception translate.storage.workflow.StateNotInWorkflowError(state) 🫶
add_note(object, /)—Exception.add_note(note)– adds a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb)– setsself.__traceback__totband returnsself
exception translate.storage.workflow.TransitionError 🫶
add_note(object, /)—Exception.add_note(note)– adds a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb)– setsself.__traceback__totband returnsself
exception translate.storage.workflow.WorkflowError 🫶
add_note(object, /)—Exception.add_note(note)– adds a note to the exceptionwith_traceback(object, /)—Exception.with_traceback(tb)– setsself.__traceback__totband returnsself
xliff 🦋
This is the module that handles XLIFF files for translation, and the official recommendation is that you use the extension .xlf for XLIFF files. noted, bestie 📝
class translate.storage.xliff.xlifffile(*args, **kwargs) — the class that represents a XLIFF file store 👑
Extensions = ['xlf', 'xliff', 'sdlxliff']— a list of file extentions associated with this store typeMimetypes = ['application/x-xliff', 'application/x-xliff+xml']— a list of MIME types associated with this store typeName = 'XLIFF Translation File'— the human usable name of this store typeUnitClass— alias ofxliffunitadd_unit_to_index(unit)— add a unit to source and location idexesaddheader()— initialise the file headeraddsourceunit(source, filename='NoName', createifmissing=False)— adds the given trans-unit to the last used body node; if the filename has changed it uses the slow method instead (and it will create the nodes required if asked). Returns success 🫡addunit(unit, new=True)— append the given unit to the object’s list of units. Fr fr always use this method instead of trying to modify the list manually 🙈- Parameters:
unit(TranslationUnit) – the unit that will be added
- Parameters:
createfilenode(filename, sourcelanguage=None, targetlanguage=None, datatype='plaintext')— creates a filenode with the given filename. All parameters are needed for XLIFF compliance, no skipping 😤creategroup(filename='NoName', createifmissing=False, restype=None)— adds agrouptag into the specified filedetect_encoding(text: bytes, default_encodings: list[str] | None = None) -> tuple[str, str]— tries to detect a file encoding from text, either using thechardetlib or by trying to decode the filestatic fallback_detection(text)— simple detection based on BOM in casechardetis not availablefindid(id)— find unit with matching id by checkingid_indexfindunit(source)— find the unit with the given source string- Return type:
TranslationUnitorNone
- Return type:
findunits(source)— find the units with the given source string- Return type:
TranslationUnitorNone
- Return type:
getbodynode(filenode, createifmissing=False)— finds the body node for the given filenodegetdatatype(filename=None)— returns the datatype of the stored file. If no filename is given, the datatype of the first file is givengetdate(filename=None)— returns the date attribute for the file. If no filename is given, the date of the first file is given. If the date attribute is not specified,Noneis returned- Returns: date attribute of file
- Return type:
DateorNone
static getfilename(filenode)— returns the name of the given filegetfilenames()— returns all filenames in this XLIFF filegetfilenode(filename, createifmissing=False)— finds the filenode with the given namegetheadernode(filenode, createifmissing=False)— finds the header node for the given filenodegetids()— return a list of unit idsgetprojectstyle()— get the project type for this storegetsourcelanguage()— get the source language for this storegettargetlanguage()— get the target language for this storegetunits()— return a list of all units in this storeinitbody()— initialisesself.bodyso it never needs to be retrieved from the XML again 🎀isempty()— returnTrueif the object doesn’t contain any translation unitsmakeindex()— indexes the items in this store. At least.sourceindexshould be usefulproperty merge_on— the matching criterion to use when merging on- Returns: the default matching criterion for all the subclasses
- Return type:
string
namespaced(name)— returns name in Clark notation. For example,namespaced("source")in an XLIFF document might return:
{urn:oasis:names:tc:xliff:document:1.1}sourceThis is needed throughout lxml, iykyk 🌸
parse(xml)— populates this object from the given xml stringclassmethod parsefile(storefile)— reads the given file (or opens the given filename) and parses back to an objectclassmethod parsestring(storestring)— parses the string to return the correct file objectremove_unit_from_index(unit)— remove a unit from source and locaton indexesremovedefaultfile()— we want to remove the default file-tag as soon as possible if we know it's still present and emptyremoveunit(unit)— remove the given unit from the object’s list of units. Again: always use this method rather than trying to modify the list manually 🙈- Parameters:
unit(TranslationUnit) – the unit that will be added
- Parameters:
save()— save to the file that data was originally read from, if availablesavefile(storefile)— write the string representation to the given file (or filename)serialize(out)— converts to a string containing the file’s XMLstatic setfilename(filenode, filename)— set the name of the given filesetprojectstyle(project_style)— set the project type for this storesetsourcelanguage(language)— set the source language for this storesettargetlanguage(language)— set the target language for this storeproperty sourcelanguage— the type of theNonesingletonsuggestions_in_format = True— xliff units have alttrans tags which can be used to store suggestionsswitchfile(filename, createifmissing=False)— adds the given trans-unit (will create the nodes required if asked)- Returns: success
- Return type:
Boolean
property targetlanguage— the type of theNonesingletontranslate(source)— return the translated string for a given source string- Return type:
StringorNone
- Return type:
unit_iter()— iterator over all the units in this store
class translate.storage.xliff.xliffunit(source, empty=False, **kwargs) — a single term in the xliff file 🎀
addalttrans(txt, origin=None, lang=None, sourcetxt=None, matchquality=None)— adds an alt-trans tag and alt-trans components to the unit- Parameters:
txt(String) – alternative translation of the source text
- Parameters:
adderror(errorname, errortext)— adds an error message to this unitaddlocation(location)— add one location to the list of locations
NOTE: shouldn’t be implemented if the format doesn’t support it 🙅♀️
addlocations(location)— add a location or a list of locations
NOTE: most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation() WARNING: this method might be removed in future 💀
addnote(text, origin=None, position='append')— add a note specifically in a “note” tagclassmethod buildfromunit(unit)— builds a native unit out of a foreign unit, preserving as much information as possible. bestie behavior only 💅static correctorigin(node, origin)— checks against the node tag’s origin, likenoteoralt-trans.createcontextgroup(name, contexts=None, purpose=None)— adds the context group to the trans-unit.contextsis a list of(type, text)tuples describing each context.createlanguageNode(lang, text, purpose)— returns an xml Element set up with the given parameters ✨delalttrans(alternative)— removes the supplied alternative from the list ofalt-transtags.getNodeText(languageNode, xml_space='preserve')— retrieves the term from the givenlanguageNode.get_rich_target(lang=None)— retrieves the “target” text (the second entry), or the entry in the specified language if it exists.getalttrans(origin=None)— returns<alt-trans>for the given origin as a list of units. no origin means all alternatives, periodt.static getcontext()— gets the message context.getcontextgroups(name)— returns the contexts in the context groups with the specified name.getcontextgroupsbyattribute(attributeName, attributeValue)— returns the contexts in the context groups with the specifiedattributeNameandattributeValue.geterrors()— gets all error messages.getid()— a unique identifier for this unit.- Return type:
string - Returns: an identifier for this unit that is unique in the store
- derived classes should override this in a way that guarantees a unique identifier for each unit in the store. no cap, that’s the whole point 🫶
- Return type:
getlanguageNode(lang=None, index=None)— retrieves alanguageNodeeither by language or by index.getlanguageNodes()— we override this to get source and target nodes.getlocations()— returns a list of locations.getnotes(origin=None)— returns all notes about this unit.- it will probably be freeform text or something reasonable that can be synthesised by the format.
- it should not include location comments (see
getlocations()).
getrestype()— returns therestypeattribute in thetrans-unittag.gettarget(lang=None)— retrieves the “target” text (the second entry), or the entry in the specified language if it exists.gettargetlen()— returns the length of the target string.- Return type:
Integer - NOTE: plural forms might be combined.
- Return type:
getunits()— this unit in a list. tiny list, main character energy 👑static hasplural()— tells whether or not this specific unit has plural strings.infer_state()— empty method that should be overridden in sub-classes to infer the currentstate(_n)of the unit from its current state.isapproved()— states whether this unit is approved.isblank()— used to see if this unit has no source or target string.- NOTE: this is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.
isfuzzy()— indicates whether this unit is fuzzy.static isheader()— indicates whether this unit is a header.static isobsolete()— indicates whether a unit is obsolete.isreview()— states whether this unit needs to be reviewed.istranslatable()— indicates whether this unit can be translated.- this should be used to distinguish real units for translation from header, obsolete, binary or other blank units.
istranslated()— indicates whether this unit is translated.- this should be used rather than deducing it from
.target, to ensure that other classes can implement more functionality (as XLIFF does).
- this should be used rather than deducing it from
makeobsolete()— makes a unit obsolete.markapproved(value=True)— marks this unit as approved.markfuzzy(value=True)— marks the unit as fuzzy or not.markreviewneeded(needsreview=True, explanation=None)— marks the unit to indicate whether it needs review.- adds an optional explanation as a note.
merge(otherunit, overwrite=False, comments=True, authoritative=False)— does basic format-agnostic merging.classmethod multistring_to_rich(mstr)— overridesTranslationUnit.multistring_to_rich(), which is used by therich_sourceandrich_targetproperties.namespaced(name)— returnsnamein Clark notation.- for example
namespaced("source")in an XLIFF document might return:
- for example
{urn:oasis:names:tc:xliff:document:1.1}source- this is needed throughout lxml.
removenotes(origin=None)— removes all the translator notes.rich_parsers = []— a list of functions to use for parsing a string into a rich string tree.property rich_source- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
property rich_target- SEE ALSO:
rich_to_multistring(),multistring_to_rich().
- SEE ALSO:
classmethod rich_to_multistring(elem_list)— overridesTranslationUnit.rich_to_multistring(), which is used by therich_sourceandrich_targetproperties.setcontext(context)— sets the message context.setid(id)— sets the unique identifier for this unit.- only implemented if the format allows ids independent from other unit properties like source or context.
settarget(target, lang='xx', append=False)— sets the target string to the given value.unit_iter()— iterator that only returns this unit.
🌈 xml_extract
✨ extract
class translate.storage.xml_extract.extract.ParseState(no_translate_content_elements, inline_elements={}, nsmap={})— maintains constants and variables used during the walking of a DOM tree (via the functionapply).class translate.storage.xml_extract.extract.Translatable(placeable_name, xpath, dom_node, source, is_inline=False)— a node corresponds to a translatable element. a node may have children, which correspond to placeables.property has_translatable_text— checks if it contains any chunk of text with more than whitespace.- if not, then there’s nothing to translate. literally nothing, bestie 💀
translate.storage.xml_extract.extract.build_idml_store(odf_file, store, parse_state, store_adder=None)— builds a store for the given IDML file.translate.storage.xml_extract.extract.build_store(odf_file, store, parse_state, store_adder=None)— builds a store for the given XML file.translate.storage.xml_extract.extract.make_postore_adder(store, id_maker, filename)— returns a function which, when called with aTranslatable, will add a unit tostore. the placeables will be represented as strings according toplaceable_quoter.translate.storage.xml_extract.extract.process_translatable(dom_node, state)— processes a translatable DOM node.- any translatable content present in a child node is treated as a placeable.
🦋 generate
translate.storage.xml_extract.generate.find_dom_root(parent_dom_node, dom_node)- SEE ALSO:
find_placeable_dom_tree_roots().
- SEE ALSO:
translate.storage.xml_extract.generate.find_placeable_dom_tree_roots(unit_node)— for an inline placeable, finds the root DOM node for the placeable in its parent.- consider the diagram. in this pseudo-ODF example, there is an inline
spanelement. however, the span is contained in other tags (which we never process). when splicing the template DOM tree (that is, the DOM which comes from the XML document we’re using to generate a translated XML document), we’ll need to move DOM sub-trees around and we need the roots of these sub-trees:
- consider the diagram. in this pseudo-ODF example, there is an inline
<p> This is text \/ <- Paragraph containing an inline placeable
<blah> <- Inline placeable's root (which we want to find)
... <- Any number of intermediate DOM nodes
<span> bold text <- The inline placeable's Translatable
holds a reference to this DOM nodetranslate.storage.xml_extract.generate.get_xliff_source_target_doms(unit)— returns a tuple with unit source and target DOM objects.- this method is meant to provide a way to retrieve the DOM objects for the unit source and target for XLIFF stores.
translate.storage.xml_extract.generate.replace_dom_text(make_parse_state, dom_retriever=<function get_xliff_source_target_doms>, process_translatable=<function process_translatable>)— generates replacement functions.- returns a function:
action: etree_Element x base.TranslationUnit -> None- which takes a
dom_nodeand a translation unit. thedom_nodeis rearranged according to rearrangement of placeables inunit.target(relative to their positions inunit.source).
🍓 misc
translate.storage.xml_extract.misc.compose_mappings(left, right)— given two mappingsleft: A -> Bandright: B -> C, creates a hashresult_map: A -> C. only values inleft(i.e. things fromB) which have corresponding keys inrightwill have their keys mapped to values inright.translate.storage.xml_extract.misc.parse_tag(full_tag)— parses a full XML tag name into namespace and tag name. cute and useful, we love to see it 🌸
>>> parse_tag('{urn:oasis:names:tc:opendocument:xmlns:office:1.0}document-content')
('urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'document-content')
>>> parse_tag('document-content')
('', 'document-content')🌳 translate.storage.xml_extract.misc.reduce_tree(f, parent_unit_node, unit_node, get_children, *state)
Bestie, this one walks a whole tree and hits every single node with f, and it does it pre-order — like, it shows up to the party before the guests, you know? 💅
parent_unit_nodeholds the parent ofunit_node. Special case tho: for the root of the tree,parent_unit_node == unit_node— the root is its own parent, main character energy, no cap. 👑get_childrenis a single-argument function that you apply to aunit_nodeand it hands you back a list/iterator of that node's children.stateis whatfuses to tweak state info relating to whateverfis doing to the tree. It's giving "shared notebook, pass it around." 📓
🧚 unit_tree
translate.storage.xml_extract.unit_tree.build_unit_tree(store, filename=None)
Okay so this literally goes through a translation store and builds a whole tree where the nodes are XPath components, and a node contains a unit when the path from the tree root down to that node equals the XPath of the unit. Fr fr, that's the whole vibe check. ✨
The tree looks something like this:
root
`- ('document-content', 1)
`- ('body', 2)
|- ('text', 1)
| `- ('p', 1)
| `- <reference to a unit>
|- ('text', 2)
| `- ('p', 1)
| `- <reference to a unit>
`- ('text', 3)
`- ('p', 1)
`- <reference to a unit>Look at those little branch characters, they're so cute 🥺 the tree ate and left no crumbs.
🦋 xpath_breadcrumb
class translate.storage.xml_extract.xpath_breadcrumb.XPathBreadcrumb
A class that builds XPath-ish paths while a DOM tree gets walked. It keeps track of how many times it's seen a certain tag, so it can correctly create indices for tags. Bookkeeping icon 💅
Initially the path is empty, so:
>>> xb = XPathBreadcrumb()
>>> xb.xpath
""Now say we walk down a DOM node for the tag <foo> and we wanna record that — easy, we just do:
>>> xb.start_tag('foo')Now the path isn't empty anymore:
>>> xb.xpath
foo[0]And now suppose there are two <bar> tags under the tag <foo> (like <foo><bar></bar><bar></bar><foo>), the breadcrumb keeps track of how many times it sees <bar>. So:
>>> xb.start_tag('bar')
>>> xb.xpath
foo[0]/bar[0]
>>> xb.end_tag()
>>> xb.xpath
foo[0]
>>> xb.start_tag('bar')
>>> xb.xpath
foo[0]/bar[1]See how the index went bar[0] → bar[1]? Obsessed with the consistency. 🔥
🌸 xml_name
class translate.storage.xml_name.XmlNamer(dom_node)
Init me with a DOM node or a DOM document node (that's the toplevel node you get when you parse an XML file), then use me to generate fully qualified XML names. They could never make it simpler. 🫶
>>> xml = '<office:document-styles xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0"></office>'
>>> from lxml import etree
>>> namer = XmlNamer(etree.fromstring(xml))
>>> namer.name('office', 'blah')
{urn:oasis:names:tc:opendocument:xmlns:office:1.0}blah
>>> namer.name('office:blah')
{urn:oasis:names:tc:opendocument:xmlns:office:1.0}blahI can also hand you XmlNamespace objects if you give me the abbreviated namespace name — super useful if you need to reference a namespace over and over again:
>>> office_ns = name.namespace('office')
>>> office_ns.name('foo')
{urn:oasis:names:tc:opendocument:xmlns:office:1.0}foo🎀 zip
This module provides functionality to work with zip files. That's it, that's the tweet.
class translate.storage.zip.ZIPFile(filename=None)
This class represents a ZIP file like a directory. Honestly? Iconic behaviour. 📁✨
file_iter()
Iterator over (dir, filename) for all files in this directory. Go off.
getfiles()
Returns a list of (dir, filename) tuples for all the file names in this directory.
getunits()
A list of all the units in all the files in this directory. All of them. Every last one. 😭
scanfiles()
Populate the internal file data.
unit_iter()
Iterator over all the units in all the files in this zip file.
🧰 tools
Code to perform various operations, mostly on po files. Little toolbox, big energy. 🧰💜
🔨 build_tmdb
Import units from translations files into tmdb.
🍓 phppo2pypo
Convert PHP format .po files to Python format .po files. Glow-up in file format form tbh.
translate.tools.phppo2pypo.convertphp2py(inputfile, outputfile, template=None)
Converts from PHP .po format to Python .po format.
Parameters
inputfile– file handle of the sourceoutputfile– file handle to write totemplate– unused (bestie is just sitting there, being decorative 🎀)
translate.tools.phppo2pypo.main(argv=None)
Converts PHP .po files to Python .po files.
🧼 poclean
Produces a clean file from an unclean file (Trados/Wordfast) by stripping out the tw4win indicators. Detox era. ✨
Heads up tho — > this does not convert an RTF file to PO/XLIFF. What it does is produce the target file with only the target text in it, from a text version of the RTF. Don't get it twisted. 🙈
translate.tools.poclean.cleanfile(thefile)
Cleans the given file.
translate.tools.poclean.cleanunit(unit)
Cleans the targets in the given unit.
translate.tools.poclean.runclean(inputfile, outputfile, templatefile)
Reads in inputfile, cleans, writes to outputfile.
🔥 pocompile
Compile XLIFF and Gettext PO localization files into Gettext MO (Machine Object) files. Compression arc. 👑
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/pocompile.html for examples and usage instructions.
translate.tools.pocompile.convertmo(inputfile, outputfile, templatefile, includefuzzy=False)
Reads in a base class derived inputfile, converts using pocompile, writes to outputfile.
💀 poconflicts
Conflict finder for Gettext PO localization files. Drama detector, lowkey essential. 🕵️♀️
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/poconflicts.html for examples and usage instructions.
class translate.tools.poconflicts.ConflictOptionParser(formats, usetemplates=False, allowmissingtemplate=False, description=None)
a specialized Option Parser for the conflict tool…
add_option(Option)
Yep. That's the whole description. 😭
add_option(opt_str, ..., kwarg=val, ...) -> None
Also exactly that. iconic minimalism honestly.
buildconflictmap()
Work out which strings are conflicting. The whole point of the tool, slay.
check_values(values: Values, args: [string]) -> (values : Values, args : [string])
Check that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new — whatever you like). The default implementation just returns the passed-in values; subclasses may override as desired. Choose your own adventure. 🦋
checkoutputsubdir(options, subdir)
Checks to see if subdir under options.output needs to be created, and creates it if necessary. Very "if it doesn't exist, build it" of it.
static clean(string, options)
Returns the cleaned string that contains the text to be matched.
define_option(option)
Defines the given option, replacing an existing one of the same short name if necessary…
destroy()
Declare that you're done with this OptionParser. This cleans up reference cycles so the OptionParser (and all objects referenced by it) can be garbage-collected promptly.
⚠️ After callingdestroy(), theOptionParseris unusable. Like, gone gone. Don't come back asking it for favours. 💀
disable_interspersed_args()
Set parsing to stop on the first non-option. Use this if you've got a command processor that runs another command which has options of its own, and you wanna make sure these options don't get confused. Boundaries, bestie. ✋
enable_interspersed_args()
Set parsing to not stop on the first non-option, allowing interspersing switches with command arguments. This is the default behavior. See also disable_interspersed_args() and the class documentation description of the attribute allow_interspersed_args.
error(msg: string)
Print a usage message incorporating msg to stderr and exit. If you override this in a subclass, it should not return — it should either exit or raise an exception. No half-sending. 🚫
finalizetempoutputfile(options, outputfile, fulloutputpath)
Write the temp outputfile to its final destination. Moving day! 📦
static flatten(text, joinchar)
Flattens text to just be words. Nothing but words. Periodt.
format_manpage()
Returns a formatted manpage. (hi, that's me, I'm the manpage 💅)
static getformathelp(formats)
Make a nice help string for describing formats…
static getfullinputpath(options, inputpath)
Gets the full path to an input file.
static getfulloutputpath(options, outputpath)
Gets the full path to an output file.
getfulltemplatepath(options, templatepath)
Gets the full path to a template file.
getoutputname(options, inputname, outputformat)
Gets an output filename based on the input filename.
getoutputoptions(options, inputpath, templatepath)
Works out which output format and processor method to use…
getpassthroughoptions(options)
Get the options required to pass to the filtermethod…
gettemplatename(options, inputname)
Gets an output filename based on the input filename. (Yes, same vibe as getoutputname, but it's the template one. Different queen. 👑)
static getusageman(option)
Returns the usage string for the given option.
static getusagestring(option)
Returns the usage string for the given option.
static isexcluded(options, inputpath)
Checks if this path has been excluded. Sorry babe, not invited. 🙈
static isrecursive(fileoption, filepurpose='input')
Checks if fileoption is a recursive file.
isvalidinputname(inputname)
Checks if this is a valid input filename.
static mkdir(parent, subdir)
Makes a subdirectory (recursively if necessary).
static openinputfile(options, fullinputpath)
Opens the input file.
static openoutputfile(options, fulloutputpath)
Opens the output file.
opentemplatefile(options, fulltemplatepath)— opens the template file, but only if one is actually required 🎀static opentempoutputfile(options, fulloutputpath)— opens up a temporary output file so you've got somewhere to scribble before committing 💅outputconflicts(options)— saves the result of the conflict match ✨parse_args(args=None, values=None)— parses the command line options, and bestie it also handles the implicit input/output args for you, no cap 🌸print_help(file: file = stdout)— prints an extended help message tofile(defaultstdout), listing every single option plus whatever help text came bundled with it 👑print_manpage(file=None)— outputs a manpage for the program, built straight from the help information 📖print_usage(file: file = stdout)— prints the usage message for the current program (self.usage) tofile(defaultstdout). Any%proginself.usagegets swapped out for the current program's name (that's the basename ofsys.argv[0], iykyk). Does absolutely nothing ifself.usageis empty or undefined ¯\_(ツ)_/¯print_version(file: file = stdout)— prints the version message (self.version) tofile(defaultstdout). Same deal asprint_usage(): any%proginself.versionbecomes the current program's name. Does nothing ifself.versionis empty or undefined ✨processfile(fileprocessor, options, fullinputpath)— processes one individual file, one at a time, very demure 🦋recurseinputfilelist(options)— takes a list of files and finds a common base directory for all of them 🔥recurseinputfiles(options)— recurses through directories and returns the files that need processing 🌈recursiveprocess(options)— recurses through directories and actually processes the files 💜run()— parses the arguments, then runsrecursiveprocesswith the resulting options… ⭐set_usage(usage=None)— sets the usage string; if you don't hand it ausage, it usesgetusagestringfor each option 🎀seterrorleveloptions()— sets the errorlevel options 🫶setformats(formats, usetemplates)— sets the format options using the given format dictionary ✨
Parameters
formats(Dictionary or iterable) –
The dictionary keys should be:
- Single strings (or 1-tuples) containing an input format (if not
usetemplates) - Tuples containing an input format and template format (if
usetemplates) - Formats can be
Noneto indicate what to do with standard input
The dictionary values should be tuples of outputformat (string) and processor method 💅
setmanpageoption()— creates a manpage option that lets the optionparser generate a manpage 👑setprogressoptions()— sets the progress options ⏳static splitext(pathname)— splitspathnameinto name and ext, and removes theextsep🍓
Parameters
pathname(string) – A file path
Returns
root,ext
Return type
- tuple
splitinputext(inputpath)— splits aninputpathinto name and extension 🦋splittemplateext(templatepath)— splits atemplatepathinto name and extension 🌸templateexists(options, templatepath)— returns whether the given template exists… 🙈warning(msg, options=None, exc_info=None)— prints a warning message incorporatingmsgtostderr⚠️
🍓 pocount
Counts strings and words for supported localization files 💖
These include: XLIFF, TMX, Gettex PO and MO, Qt .ts and .qm, Wordfast TM, etc.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/pocount.html for examples and usage instructions ✨
class translate.tools.pocount.ConsoleColor— class to implement color mode 🌈class translate.tools.pocount.CsvRenderer(stats: 'StatCollector')— renders your stats, CSV style 🧾class translate.tools.pocount.FullRenderer(stats: StatCollector)— the full renderer, giving you everything 👑class translate.tools.pocount.Renderer(stats: 'StatCollector')— the base renderer vibe 💅class translate.tools.pocount.ShortStringsRenderer(stats: StatCollector, indent: int = 8)🌸
Parameters
indent– indentation of the 2nd column (length of longest filename)
class translate.tools.pocount.ShortWordsRenderer(stats: StatCollector, indent: int = 8)🌸
Parameters
indent– indentation of the 2nd column (length of longest filename)
translate.tools.pocount.file_extended_totals(units, wordcounts)— provides extended statuses (used by XLIFF) 🦋translate.tools.pocount.wordsinunit(unit)— counts the words in the unit's source and target, taking plurals into account. The target words are only counted if the unit is translated 🔥
🐛 podebug
Inserts debug messages into XLIFF and Gettext PO localization files 🎀
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/podebug.html for examples and usage instructions ✨
translate.tools.podebug.convertpo(inputfile, outputfile, templatefile, format=None, rewritestyle=None, ignoreoption=None, preserveplaceholders=None)— reads ininputfile, changes it so it has debug strings, and writes tooutputfile💅
🔎 pogrep
Greps XLIFF, Gettext PO and TMX localization files 👑
Matches get output to snippet files of the same type, which you can then review and later merge using pomerge — iconic workflow fr fr ✨
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/pogrep.html for examples and usage instructions 🌸
class translate.tools.pogrep.GrepMatch(unit, part='target', part_n=0, start=0, end=0)— just a small data structure that represents a search match 🍓class translate.tools.pogrep.GrepOptionParser(formats, usetemplates=False, allowmissingtemplate=False, description=None)— a specialized Option Parser for the grep tool… 🦋add_option(Option)— adds that option 💅add_option(opt_str, ..., kwarg=val, ...) -> None— the full-flavoured version of adding an option ✨check_values(values: Values, args: [string])→(values : Values, args : [string])
Checks that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new – whatever you like). The default implementation just returns the passed-in values; subclasses may override as desired 🫶
checkoutputsubdir(options, subdir)— checks to see ifsubdirunderoptions.outputneeds to be created, and creates it if neccessary 🎀define_option(option)— defines the given option, replacing an existing one of the same short name if neccessary… 💅destroy()— declares that you're done with this OptionParser. This cleans up reference cycles so the OptionParser (and all objects referenced by it) can be garbage-collected promptly. After callingdestroy(), the OptionParser is unusable 💀disable_interspersed_args()— sets parsing to stop on the first non-option. Use this if you have a command processor which runs another command that has options of its own and you want to make sure those options don't get confused 🙈enable_interspersed_args()— sets parsing to not stop on the first non-option, allowing switches to be interspersed with command arguments. This is the default behavior. See alsodisable_interspersed_args()and the class documentation description of the attributeallow_interspersed_args✨error(msg: string)— prints a usage message incorporatingmsgtostderrand exits. If you override this in a subclass, it should not return – it should either exit or raise an exception ⚠️finalizetempoutputfile(options, outputfile, fulloutputpath)— writes the tempoutputfileto its final destination 🏁format_manpage()— returns a formatted manpage 📖static getformathelp(formats)— makes a nice help string for describing formats… 💖static getfullinputpath(options, inputpath)— gets the full path to an input file 🦋static getfulloutputpath(options, outputpath)— gets the full path to an output file 🌸getfulltemplatepath(options, templatepath)— gets the full path to a template file 🎀getoutputname(options, inputname, outputformat)— gets an output filename based on the input filename 💅getoutputoptions(options, inputpath, templatepath)— works out which output format and processor method to use… ✨getpassthroughoptions(options)— gets the options required to pass to thefiltermethod… 🫶gettemplatename(options, inputname)— gets an output filename based on the input filename 🍓static getusageman(option)— returns the usage string for the given option 📜static getusagestring(option)— returns the usage string for the given option 📜static isexcluded(options, inputpath)— checks if this path has been excluded 🚫static isrecursive(fileoption, filepurpose='input')— checks iffileoptionis a recursive file 🔁isvalidinputname(inputname)— checks if this is a valid input filename ✅static mkdir(parent, subdir)— makes a subdirectory (recursively if neccessary) 📁static openinputfile(options, fullinputpath)— opens the input file 📥static openoutputfile(options, fulloutputpath)— opens the output file 📤opentemplatefile(options, fulltemplatepath)— opens the template file (if required) 🎀static opentempoutputfile(options, fulloutputpath)— opens a temporary output file 💅
okay bestie we're still deep in the method list, grab a snack and stay with me 💅✨
parse_args(args=None, values=None)— parses the command-line options and handles the implicit input/output args while it's at it. multitasking queen 👑print_help(file: file = stdout)— prints an extended help message — like, all the options plus any help text that came with them — intofile(which defaults tostdout).print_manpage(file=None)— outputs a whole manpage for the program using the help info it already has. it's giving self-publishing icon 💅print_usage(file: file = stdout)— prints the usage message for the current program (self.usage) tofile(defaultstdout). any time the string"%prog"shows up inself.usage, it gets swapped for the name of the current program (the basename ofsys.argv[0]). does absolutely nothing ifself.usageis empty or not defined. no vibes, no output 🙈print_version(file: file = stdout)— prints the version message for this program (self.version) tofile(defaultstdout). same deal asprint_usage(): every"%prog"inself.versiongets replaced with the current program's name. does nothing ifself.versionis empty or undefined.processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)— processes an individual file. one file, one moment, main character energy ⭐recurseinputfilelist(options)— uses a list of files and finds a common base directory for all of them. group project, but it actually works 🫶recurseinputfiles(options)— recurses through directories and returns the files that need processing.recursiveprocess(options)— recurses through directories and actually processes the files. go off 🔥run()— parses the args and runsrecursiveprocesswith whatever options came out of that. the whole plot in one method.set_usage(usage=None)— sets the usage string — and if you don't hand it ausage, it builds one withgetusagestringfor each option. DIY queen 💖seterrorleveloptions()— sets the errorlevel options.setformats(formats, usetemplates)— sets the format options using the given format dictionary.
Parameters
formats(Dictionary or iterable) —
the dictionary keys should be:
- single strings (or 1-tuples) containing an input format (if not
usetemplates) - tuples containing an input format and template format (if
usetemplates) - formats can be
Noneto indicate what to do with standard input
the dictionary values should be tuples of outputformat (string) and processor method.
iykyk, it's a whole schema 🧚
setmanpageoption()— creates a manpage option that lets the optionparser generate a manpage. literally the origin story of the page you're reading rn 😭setprogressoptions()— sets the progress options.static splitext(pathname)— splitspathnameinto name and ext, and removes theextsep.
Parameters
pathname(string) — a file path
Returns
root,ext
Return type
tuple
splitinputext(inputpath)— splits aninputpathinto name and extension.splittemplateext(templatepath)— splits atemplatepathinto name and extension. consistent, we love to see it ✨templateexists(options, templatepath)— returns whether the given template exists… yes bestie the ellipsis is canon 💀warning(msg, options=None, exc_info=None)— prints a warning message incorporatingmsgtostderr. gentle heads-up, not a scream 🍓
💜 pomerge
merges XLIFF and Gettext PO localization files. periodt.
a snippet file produced by e.g. pogrep and updated by a translator can be merged back into the original files. it's giving "return the borrowed hoodie" energy 🫶
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/pomerge.html for examples and usage instructions.
translate.tools.pomerge.mergestores(store1, store2, mergeblanks, mergefuzzy, mergecomments)— takes any new translations instore2and writes them intostore1. glow-up transplant 💅translate.tools.pomerge.str2bool(option)— converts a string value to boolean.
Parameters
option(String) —yes,true,1,no,false,0
Return type
- Boolean
🦋 porestructure
restructures Gettxt PO files produced by poconflicts into the original directory tree so they can be merged using pomerge. un-chaos-ing the chaos ✨
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/pomerge.html for examples and usage instructions.
class translate.tools.porestructure.SplitOptionParser(formats, usetemplates=False, allowmissingtemplate=False, description=None) — a specialized Option Parser for posplit. it has a type, it has defaults, it's booked and busy 🌈
add_option(Option)— adds that option.add_option(opt_str, ..., kwarg=val, ...) -> None— the other form of adding an option, kwargs and all.check_values(values: Values, args: [string]) -> (values : Values, args : [string])— checks that the supplied option values and leftover arguments are valid. returns the option values and leftover arguments (possibly adjusted, possibly completely new — whatever you like, honestly). the default implementation just returns the passed-in values; subclasses may override as desired. choose your own adventure 🔥checkoutputsubdir(options, subdir)— checks whethersubdirunderoptions.outputneeds to be created, and creates it if neccessary.define_option(option)— defines the given option, replacing an existing one with the same short name if neccessary… yes, again with the ellipsis 😭destroy()— declares that you are done with this OptionParser. this cleans up reference cycles so the OptionParser (and every object it references) can be garbage-collected promptly. after callingdestroy(), the OptionParser is unusable. it's giving "we're not getting back together" 💀disable_interspersed_args()— sets parsing to stop on the first non-option. use this if you've got a command processor that runs another command with its own options and you want to make sure those options don't get confused. boundaries, bestie 🎀enable_interspersed_args()— sets parsing to not stop on the first non-option, allowing switches interspersed with command arguments. this is the default behavior. see alsodisable_interspersed_args()and the class documentation description of the attributeallow_interspersed_args.error(msg: string)— prints a usage message incorporatingmsgtostderr, then exits. if you override this in a subclass, it should not return — it should either exit or raise an exception. no quiet quitting here 🙈
fr fr, it must not return. exit or raise, pick one.
finalizetempoutputfile(options, outputfile, fulloutputpath)— writes the tempoutputfileto its final destination. the grand finale ✨format_manpage()— returns a formatted manpage.static getformathelp(formats)— makes a nice help string for describing formats… (ellipsis #3, we're keeping count 💅)static getfullinputpath(options, inputpath)— gets the full path to an input file.static getfulloutputpath(options, outputpath)— gets the full path to an output file.getfulltemplatepath(options, templatepath)— gets the full path to a template file.getoutputname(options, inputname, outputformat)— gets an output filename based on the input filename.getoutputoptions(options, inputpath, templatepath)— works out which output format and processor method to use… (four 👑)getpassthroughoptions(options)— gets the options required to pass to thefiltermethod…gettemplatename(options, inputname)— gets an output filename based on the input filename. (yes, same description asgetoutputname— that's what the page says, don't come for me 💀)static getusageman(option)— returns the usage string for the given option.static getusagestring(option)— returns the usage string for the given option. twins, honestly 🫶static isexcluded(options, inputpath)— checks if this path has been excluded. you're not on the list, babe 🚫static isrecursive(fileoption, filepurpose='input')— checks iffileoptionis a recursive file.isvalidinputname(inputname)— checks if this is a valid input filename.static mkdir(parent, subdir)— makes a subdirectory (recursively if neccessary).static openinputfile(options, fullinputpath)— opens the input file.static openoutputfile(options, fulloutputpath)— opens the output file.opentemplatefile(options, fulltemplatepath)— opens the template file (if required).static opentempoutputfile(options, fulloutputpath)— opens a temporary output file.parse_args(args=None, values=None)— parses the command-line options, handling the implicit input/output args. yeah, it's back, it's a whole family reunion 😭print_help(file: file = stdout)— prints an extended help message, listing all options and any help text provided with them, tofile(defaultstdout).print_manpage(file=None)— outputs a manpage for the program using the help information.print_usage(file: file = stdout)— prints the usage message for the current program (self.usage) tofile(defaultstdout). any occurrence of the string"%prog"inself.usagegets replaced with the name of the current program (basename ofsys.argv[0]). does nothing ifself.usageis empty or not defined.print_version(file: file = stdout)— spits out the version message for this program (self.version) into*file*(stdout by default, bestie). Same energy asprint_usage(): every"%prog"inself.versiongets swapped for the current program's name. Ifself.versionis empty or undefined? It literally does nothing. Iconic minimalism 💅processfile(options, fullinputpath)— processes one single file. That's it, that's the job 🎀recurseinputfilelist(options)— takes a whole list of files and figures out a common base directory for all of them. Girl math but for paths ✨recurseinputfiles(options)— recurses through directories and hands back the files that need processing 🦋recursiveprocess(options)— recurses through directories and actually processes the files, no cap 🔥run()— parses the arguments, then runsrecursiveprocesswith the resulting options… and that's the whole vibe check 👑set_usage(usage=None)— sets the usage string. If you don't pass usage, it callsgetusagestringon each option like a good little overachiever 🌸seterrorleveloptions()— sets the errorlevel options. That's the whole description, she's lowkey mysterious 🙈setformats(formats, usetemplates)— sets the format options using the given format dictionary. Here's the tea on the params:
Parameters
- formats (Dictionary or iterable) — the dictionary keys should be:
- single strings (or 1-tuples) containing an input format (if
usetemplatesis not set) - tuples containing an input format and a template format (if
usetemplatesis set) - formats can be
None, which signals what to do with standard input
- single strings (or 1-tuples) containing an input format (if
- the dictionary values should be tuples of
outputformat(a string) and a processor method 🫶
setmanpageoption()— creates a manpage option so the optionparser can generate a manpage. Meta af. Manpage-ception 😭setprogressoptions()— sets the progress options ⭐static splitext(pathname)— splits pathname into name and ext, and removes the extsep.
Parameters
- pathname (string) — a file path
Returns
root, ext
Return type
tuple 🌈
splitinputext(inputpath)— splits an inputpath into name and extension 🎀splittemplateext(templatepath)— splits a templatepath into name and extension ✨templateexists(options, templatepath)— returns whether the given template exists… 💖warning(msg, options=None, exc_info=None)— prints a warning message withmsgbaked into it, to stderr. (stderr! not stdout! don't mix them up bestie 🙈)
🌸 posegment 🌸
Segments Gettext PO, XLIFF and TMX localization files at the sentence level. Sentence. Level. That's the whole personality 💅
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/posegment.html for examples and usage instructions 👑
translate.tools.posegment.segmentfile(inputfile, outputfile, templatefile, sourcelanguage='en', targetlanguage=None, stripspaces=True, onlyaligned=False)
Reads in inputfile, segments it, then writes to outputfile. Easy. Effortless. Slay 🦋
💜 poswap 💜
Builds a brand new translation file where the target of the input language becomes the source language. Flip it and reverse it 🎀
NOTE: make absolutely sure the two po files correspond 100% to the same pot file before you use this. Not 99%, not "close enough" — 100%. Fr fr, this one matters 😭
To translate Kurdish (ku) through French:
poswap -i fr/ -t ku -o fr-kuTo convert the fr-ku files back to en-ku:
poswap --reverse -i fr/ -t fr-ku -o en-kuSee: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/poswap.html for examples and usage instructions ⭐
translate.tools.poswap.convertpo(inputpofile, outputpotfile, template, reverse=False)
Reads in inputpofile, removes the header, and writes to outputpotfile. Bye bye header 👋
translate.tools.poswap.swapdir(store)
Swaps the source and target of each unit. The switcheroo, the classic 🫶
👑 poterminology 👑
Creates a terminology file by reading a set of .po or .pot files and producing a pootle-terminology.pot. Glossary glow-up, periodt 🔥
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/poterminology.html for examples and usage instructions ✨
class translate.tools.poterminology.TerminologyOptionParser(formats, usetemplates=False, allowmissingtemplate=False, description=None)
A specialized OptionParser for the terminology tool… it's giving main character parser energy 💅
add_option(Option)— adds an option.add_option(opt_str, ..., kwarg=val, ...) -> None— the full form, returnsNone.check_values(values: Values, args: [string])— signature result:-> (values : Values, args : [string]). Checks that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments — possibly adjusted, possibly completely new, whatever you like, bestie. The default implementation just returns the passed-in values; subclasses may override it as desired 🌸checkoutputsubdir(options, subdir)— checks whether subdir underoptions.outputneeds to be created, and creates it if neccessary. (That spelling is the original's, don't come for me 🙈)define_option(option)— defines the given option, replacing an existing one with the same short name if neccessary…destroy()— declares that you are done with thisOptionParser. It cleans up reference cycles so theOptionParser(and everything it references) can be garbage-collected promptly. ⚠️ After callingdestroy(), theOptionParseris unusable — she's gone, don't try her 💀disable_interspersed_args()— sets parsing to stop on the first non-option. Use this when you've got a command processor that runs another command with its own options and you want to make sure those options don't get confused. Boundary queen 🎀enable_interspersed_args()— sets parsing to not stop on the first non-option, letting you intersperse switches with command arguments. This is the default behavior. See alsodisable_interspersed_args()and the class documentation description of the attributeallow_interspersed_args✨error(msg: string)— prints a usage message withmsgin it to stderr and exits. If you override this in a subclass, it should not return — it should either exit or raise an exception. No half measures, bestie 😤finalizetempoutputfile(options, outputfile, fulloutputpath)— writes the temp outputfile to its final destination. Ship it 🚀format_manpage()— returns a formatted manpage.static getformathelp(formats)— makes a nice help string for describing formats… cute AND useful 🌈static getfullinputpath(options, inputpath)— gets the full path to an input file.static getfulloutputpath(options, outputpath)— gets the full path to an output file.getfulltemplatepath(options, templatepath)— gets the full path to a template file.getoutputname(options, inputname, outputformat)— gets an output filename based on the input filename.getoutputoptions(options, inputpath, templatepath)— works out which output format and processor method to use… the whole decision tree, handled 💜getpassthroughoptions(options)— gets the options required to pass to thefiltermethod…gettemplatename(options, inputname)— gets an output filename based on the input filename. (Yes, the description says output filename. Yes, the method's calledgettemplatename. I don't make the rules, I just report them 🙈)static getusageman(option)— returns the usage string for the given option.static getusagestring(option)— returns the usage string for the given option. Twin flame behavior 💖static isexcluded(options, inputpath)— checks if this path has been excluded. Not invited to the party 🚫static isrecursive(fileoption, filepurpose='input')— checks if fileoption is a recursive file.isvalidinputname(inputname)— checks if this is a valid input filename. Vibe check on the filename 👀static mkdir(parent, subdir)— makes a subdirectory (recursively if neccessary).static openinputfile(options, fullinputpath)— opens the input file.static openoutputfile(options, fulloutputpath)— opens the output file.opentemplatefile(options, fulltemplatepath)— opens the template file (if required).static opentempoutputfile(options, fulloutputpath)— opens a temporary output file 🎀outputterminology(options)— saves the generated terminology glossary. The grand finale of the whole glossary arc 👑parse_args(args=None, values=None)— parses the command line options, handling implicit input/output args. Sneaky but helpful ✨print_help(file: file = stdout)— prints an extended help message listing all options and any help text provided with them, to*file*(stdout by default) 💅print_manpage(file=None)— outputs a manpage for the program using the help information. Full circle, we're literally inside the thing it prints 🌀print_usage(file: file = stdout)— prints the usage message for the current program (self.usage) to*file*(stdout by default). Any occurrence of the string"%prog"inself.usagegets replaced with the name of the current program (basename ofsys.argv[0]). Does nothing ifself.usageis empty or not defined 🦋print_version(file: file = stdout)— prints the version message for this program (self.version) to*file*(stdout by default). As withprint_usage(), any occurrence of"%prog"inself.versionis replaced by the current program's name. Does nothing ifself.versionis empty or undefined 🌸processfile(fileprocessor, options, fullinputpath)— processes an individual file, this time with a fileprocessor in hand 💜recurseinputfilelist(options)— uses a list of files and finds a common base directory for them.recurseinputfiles(options)— recurses through directories and returns files to be processed.recursiveprocess(options)— recurses through directories and processes files.run()— parses the arguments, and runsrecursiveprocesswith the resulting options. Go girl, go 🏃♀️set_usage(usage=None)— Sets the usage string — and ifusageisn’t given, it usesgetusagestringfor each option. slay.seterrorleveloptions()— Sets the errorlevel options.setformats(formats, usetemplates)— Sets the format options using the given format dictionary.- Parameters:
formats(Dictionary or iterable) – - The dictionary keys should be:
- Single strings (or 1-tuples) containing an input format (if not
usetemplates) - Tuples containing an input format and template format (if
usetemplates) - Formats can be
Noneto indicate what to do with standard input
- Single strings (or 1-tuples) containing an input format (if not
- The dictionary values should be tuples of
outputformat(string) and processor method.
- Parameters:
setmanpageoption()— creates a manpage option that allows the optionparser to generate a manpage.setprogressoptions()— Sets the progress options.static splitext(pathname)— Splitspathnameinto name and ext, and removes theextsep.- Parameters:
pathname(string) – A file path - Returns:
root, ext - Return type:
tuple
- Parameters:
splitinputext(inputpath)— Splits aninputpathinto name and extension.splittemplateext(templatepath)— Splits atemplatepathinto name and extension.templateexists(options, templatepath)— Returns whether the given template exists…warning(msg, options=None, exc_info=None)— Print a warning message incorporatingmsgto stderr.
💾 pretranslate
Fill localization files with suggested translations based on translation memory and existing translations.
See: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/pretranslate.html for examples and usage instructions.
translate.tools.pretranslate.match_fuzzy(input_unit, matchers)— Return a fuzzy match from a queue of matchers.translate.tools.pretranslate.match_source(input_unit, template_store)— Returns a matching unit from a template. matching based on unit id.translate.tools.pretranslate.match_template_id(input_unit, template_store)— Returns a matching unit from a template. matching based on unit id.translate.tools.pretranslate.match_template_location(input_unit, template_store)— Returns a matching unit from a template. matching based on locations.translate.tools.pretranslate.memory(tmfiles, max_candidates=1, min_similarity=75, max_length=1000)— Returns the TM store to use. Only initialises on first call.translate.tools.pretranslate.pretranslate_file(input_file, output_file, template_file, tm=None, min_similarity=75, fuzzymatching=True)— Pretranslate any factory supported file with old translations and translation memory.translate.tools.pretranslate.pretranslate_store(input_store, template_store, tm=None, min_similarity=75, fuzzymatching=True)— Do the actual pretranslation of a whole store.translate.tools.pretranslate.pretranslate_unit(input_unit, template_store, matchers=None, mark_reused=False, merge_on='id')— Pretranslate a unit or return unchanged if no translation was found.- Parameters:
input_unit– Unit that will be pretranslated.template_store– Fill input unit with units matching in this store.matchers– List of fuzzy matcher objects.mark_reused– Whether to mark old translations as reused or not.merge_on– Where will the merge matching happen on.
- Parameters:
🧚 pydiff
diff tool like GNU diff, but lets you have special options that are useful in dealing with PO files.
class translate.tools.pydiff.DirDiffer(fromdir, todir, options)— generates diffs between directories.isexcluded(difffile)— Checks if the given filename has been excluded from the diff.writediff(outfile)— Writes the actual diff to the given file.
class translate.tools.pydiff.FileDiffer(fromfile, tofile, options)— generates diffs between files.get_from_lines(group)— Returns the lines referred to bygroup, from thefromfile.get_to_lines(group)— Returns the lines referred to bygroup, from thetofile.unified_diff(group)— takes the group of opcodes and generates a unified diff line by line.writediff(outfile)— Writes the actual diff to the given file.
translate.tools.pydiff.main()— Main program forpydiff.
🐘 pypo2phppo
Convert Python format .po files to PHP format .po files.
translate.tools.pypo2phppo.convertpy2php(inputfile, outputfile, template=None)— Converts from Python.poto PHP.po.- Parameters:
inputfile– file handle of the sourceoutputfile– file handle to write totemplate– unused
- Parameters:
translate.tools.pypo2phppo.main(argv=None)— Converts from Python.poto PHP.po.
👑 AUTHOR
Translate.org.za
💖 COPYRIGHT
2002-2023, Translate
That’s the whole vibe check, bestie — this was TRANSLATETOOLKIT(1), Translate Toolkit version 3.12.1, dated Apr 08, 2024. 💅✨
📦 where this page comes from
- opensuse-leap 16.0: python313-translate-toolkit 3.12.1-bp160.1.1 · GPL-2.0-or-later
- 💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash-1017 on : a language model's rewrite of the original, under its terms