mkgmap-splitter(1)

📦 version 654, in nixos 26.05, nixpkgs unstable

🎀 mkgmap-splitter(1) 🎀

okay bestie, welcome to the mkgmap-splitter manual page — she's the tile splitter for mkgmap, and honestly she's lowkey a legend. grab a snack, this one's long. 💅

✨ NAME ✨

mkgmap-splitter — tile splitter for mkgmap 🧚

🌸 SYNOPSIS 🌸

mkgmap-splitter [options] file.osm > splitter.log

that's it. that's the whole vibe. 💖

🦋 DESCRIPTION 🦋

so basically? mkgmap-splitter takes an .osm file that's got big, well mapped regions in it and splits it up into a bunch of smaller tiles, so everything fits inside the maximum size that the Garmin maps format will accept. slay. ✨

the two most important features, and ngl these are the ones you actually care about:

  • variable sized tiles so you don't end up with a million tiny useless files
  • tiles that join exactly — no overlap, no gaps, they just line up perfectly. it's giving precision. 👑

okay now the part where I have to be real with you: you will need a LOT of memory on your computer if you're planning to split a large area. 😭 there are a few options that let you configure how much memory you need. with the default parameters, you need about 2 bytes for every node and way. which sounds like nothing?? but there are about 4300 million nodes in the whole planet file (as of Jan 2018), so yeah — you cannot process the whole planet in one pass on a 32 bit machine using this utility, because the maximum java heap space is 2G. it is possible with 64 bit java and about 10GB of heap, or with multiple passes. 💀

on the other hand? a single country — even a well mapped one like Germany or the UK — is totally doable on a modest machine. even a netbook. we love an accessible queen. 🫶

⭐ USAGE ⭐

splitter requires java 1.6 or higher. basic usage goes like this:

mkgmap-splitter file.osm > splitter.log

if you've got less than 2 GB of memory on your computer, you should reduce the -Xmx option by setting the JAVA_OPTS environment variable:

JAVA_OPTS="-Xmx512m" mkgmap-splitter file.osm > splitter.log

look at you go, customizing your heap. 💅

🌈 the files it spits out

this will produce a number of .osm.pbf files that can be read by mkgmap(1). 🎀 and there are also other files produced, so let's go through them one by one because they're all doing their own little job:

  • template.args — this one is a file that can be used with the -c option of mkgmap and it'll compile all the files. you can use it as is, or edit it to include your own options. for example, instead of each description being "OSM Map" it could be "NW Scotland" as appropriate. main character energy, honestly. 👑
  • areas.list — the list of bounding boxes that were calculated. if you want, you can use this on subsequent calls of splitter using the --split-file option to use exactly the same areas as last time. super useful if you produce a map regularly and want to keep the tile areas the same from month to month. 🌙 it's also useful to avoid the time it takes to regenerate the file each time (currently about a third of the overall time taken to perform the split). of course, if the map grows enough that one of the tiles overflows, you'll have to re-calculate the areas again. no cap. 🙈
  • areas.poly — contains the bounding polygon of the calculated areas. see option --polygon-file for how this can be used. 🦋
  • densities-out.txt — written when no split-file is given, and contains debugging information only. literally just for debugging, bestie. that's it. 💀

🌸 compressed input, iykyk

you can also use a gzip'ed or bz2'ed compressed .osm file as the input file. 🎀 but heads up — this can slow down the splitter considerably (particularly true for bz2) because decompressing the .osm file can take quite a lot of CPU power. 😭

so if you're likely to be processing a file several times, you're probably better off converting the file to one of the binary formats pbf or o5m. the o5m format is faster to read, but requires more space on the disk. tradeoffs, bestie. always tradeoffs. ⚖️

✨ OPTIONS ✨

okay bestie, there are a bunch of options here to fine tune things that you might want to try, and honestly some of them are so useful. let's go through them one by one, no cap 💅

  • --boundary-tags=string — a comma separated list of tag values for relations. used to filter multipolygon and boundary relations for problem-list processing. see also option --wanted-admin-level. Default: use-exclude-list 🎀
  • --cache=string — deprecated, now does nothing. it's giving retired icon, lowkey just vibing there 💀
  • --description=string — sets the description to be written in to the template.args file.
  • --geonames-file=string — the name of a GeoNames file to use for determining tile names. typically cities15000.zip from geonames (http://download.geonames.org/export/dump). 🌍
  • --keep-complete=boolean — use --keep-complete=false to disable two additional program phases between the split and the final distribution phase (not recommended). the first phase, called gen-problem-list, detects all ways and relations that are crossing the borders of one or more output files. the second phase, called handle-problem-list, collects the coordinates of these ways and relations and calculates all output files that are crossed or enclosed. the information is passed to the final dist-phase in three temporary files. this avoids broken polygons, but be aware that it requires to read the input files at least two additional times. do not specify it with --overlap unless you have a good reason to do so. Default: true ✨
  • --mapid=int — set the filename for the split files. in the example the first file will be called 63240001.osm.pbf and the next one will be called 63240002.osm.pbf and so on. Default: 63240001 🎀
  • --max-areas=int — the maximum number of areas that can be processed in a single pass during the second stage of processing. this must be a number from 1 to 9999. higher numbers mean fewer passes over the source file and hence quicker overall processing, but also require more memory. if you find you are running out of memory but don't want to increase your --max-nodes value, try reducing this instead. changing this will have no effect on the result of the split, it's purely to let you trade off memory for performance. note that the first stage of the processing has a fixed memory overhead regardless of what this is set to so if you are running out of memory before the areas.list file is generated, you need to either increase your -Xmx value or reduce the size of the input file you're trying to split. Default: 2048 💖
  • --max-nodes=int — the maximum number of nodes that can be in any of the resultant files. the default is fairly conservative, you could increase it quite a lot before getting any 'map too big' messages. not much experimentation has been done. also the bigger this value, the less memory is required during the splitting stage. Default: 1600000 🔥
  • --max-threads=value — the maximum number of threads used by mkgmap-splitter. Default: 4 (auto) 🧚
  • --mixed=boolean — specify this if the input osm file has nodes, ways and relations intermingled or the ids are not strictly sorted. to increase performance, use the osmosis sort function. Default: false
  • --no-trim=boolean — don't trim empty space off the edges of tiles. this option is ignored when --polygon-file is used. Default: false 🦋
  • --num-tiles=valuestring — a target value that is used when no split-file is given. splitting is done so that the given number of tiles is produced. the --max-nodes value is ignored if this option is given.
  • --output=string — the format in which the output files are written. possible values are xml, pbf, o5m, and simulate. the default is pbf, which produces the smallest file sizes. the o5m format is faster to write, but creates around 40% larger files. the simulate option is for debugging purposes. 👑
  • --output-dir=path — the directory to which splitter should write the output files. if the specified path to a directory doesn't exist, mkgmap-splitter tries to create it. defaults to the current working directory. 🌸
  • --overlap=string — deprecated since r279. with --keep-complete=false, mkgmap-splitter should include nodes outside the bounding box, so that mkgmap can neatly crop exactly at the border. this parameter controls the size of that overlap. it is in map units, a default of 2000 is used which means about 0.04 degrees of latitude or longitude. if --keep-complete=true is active and --overlap is given, a warning will be printed because this combination rarely makes sense. ⚠️
  • --polygon-desc-file=path — an osm file (.o5m, .pbf, .osm) with named ways that describe bounding polygons with OSM ways having tags name and mapid.
  • --polygon-file=path — the name of a file containing a bounding polygon in the osmosis polygon file format. note that holes in the polygon are more or less ignored. mkgmap-splitter uses this file when calculating the areas. it first calculates a grid using the given --resolution. the input file is read and for each node, a counter is increased for the related grid area. if the input file contains a bounding box, this is applied to the grid so that nodes outside of the bounding box are ignored. next, if specified, the bounding polygon is used to zero those grid elements outside of the bounding polygon area. if the polygon file describes one or more rectilinear areas with no more than 40 vertices, mkgmap-splitter will try to create output files that fit exactly into each area, otherwise it will approximate the polygon area with rectangles. 📐
  • --precomp-sea=path — the name of a directory containing precompiled sea tiles. if given, mkgmap-splitter will use the precompiled sea tiles in the same way as mkgmap does. use this if you want to use a --polygon-file or --no-trim=true and mkgmap creates empty *.img files combined with a message starting "There is not enough room in a single garmin map for all the input data". 🌊
  • --problem-file=path — the name of a file containing ways and relations that are known to cause problems in the split process. use this option if --keep-complete requires too much time or memory and --overlap doesn't solve your problem. syntax of problem file:
way:<id> # comment...
rel:<id> # comment...

example:

way:2784765 # Ferry Guernsey - Jersey
  • --problem-report=path — the name of a file to write the generated problem list created with --keep-complete. the parameter is ignored if --keep-complete=false. you can reuse this file with the --problem-file parameter, but do this only if you use the same values for --max-nodes and --resolution. 📝
  • --resolution=int — the resolution of the density map produced during the first phase. a value between 1 and 24. default is 13. increasing the value to 14 requires four times more memory in the split phase. the value is ignored if a --split-file is given. ✨
  • --search-limit=int — search limit in split algo. higher values may find better splits, but will take longer. Default: 200000 🔎
  • --split-file=path — use the previously calculated tile areas instead of calculating them from scratch. the file can be in .list or .kml format.
  • --status-freq=int — displays the amount of memory used by the JVM every --status-freq seconds. set =0 to disable. Default: 120 💻
  • --stop-after=string — debugging: stop after a given program phase. can be split, gen-problem-list, or handle-problem-list. default is dist which means execute all phases.
  • --wanted-admin-level=int — specifies the lowest admin_level value of boundary relations that should be kept complete. used to filter boundary relations for problem-list processing. the default value 5 means that boundary relations are kept complete when the admin_level is 5 or higher (5..11). the parameter is ignored if --keep-complete=false. Default: 5 🫶
  • --write-kml=path — the name of a kml file to write out the areas to. this is in addition to areas.list (which is always written out). 🌈

🌟 Special options 🌟

  • --version — if the parameter --version is found somewhere in the options, mkgmap-splitter will just print the version info and exit. version info looks like this:
splitter 279 compiled 2013-01-12T01:45:02+0000
  • --help — if the parameter --help is found somewhere in the options, mkgmap-splitter will print a list of all known normal options together with a short help and exit. 🎀

🔧 TUNING 🔧

Okay bestie, this is the part where we make your splitter go zoom zoom 🌈✨

🌸 Tuning for best performance 🌸

A few little hints for anyone using mkgmap-splitter to split, like, huge files. Pay attention, this is the good stuff 💅

  • For faster processing with --keep-complete=true, convert that input file to o5m format using:
osmconvert --drop-version file.osm -o=file.o5m
  • The option --drop-version is optional — it just shrinks the file down to that data which is actually needed by mkgmap-splitter and mkgmap. Lowkey a glow-up for your file size 🎀
  • If you're still getting poor performance, go snooping in splitter.log 🔍 Search for the word Distributing. You may find something like this on the next line:
Processing 1502 areas in 3 passes, 501 areas at a time

Bestie, this means splitter has to read the input file input three times because the --max-areas parameter was much smaller than the number of areas 😭 If you've got enough heap, set the --max-areas value to something higher than the number of areas, e.g. --max-areas=2048. Then run mkgmap-splitter again and you should find:

Processing 1502 areas in a single pass

Single pass, single pass, single pass — that's the dream 🦋

  • More areas require more memory, no cap. Make sure mkgmap-splitter has enough heap (increase the -Xmx parameter) so it doesn't waste a bunch of time in the garbage collector (GC) — GC said NOT TODAY — but keep as much memory as possible for the system's I/O caches 💜
  • If available, use two different disks for the input file and the output directory, esp. when you use o5m format for input and output. Two disks, two vibes, no bottleneck ✨
  • If you use mkgmap r2415 or later and disk space is no concern, consider using --output=o5m to speed up processing. Speed queen energy 👑

🧚 Tuning for low memory requirements 🧚

If your machine has less than 1 GB free memory (eg. a netbook 🥺), you can still use mkgmap-splitter, but you might have to be patient if you use the parameter --keep-complete and want to split a file like germany.osm.pbf or a larger one. Girl, that's a whole country in there 😭

If needed, reduce the number of parallel processed areas to 50 with the --max-areas parameter. And heads up: you have to use --keep-complete=false when splitting an area like Europe. Europe. Not a drill. 🫶

📝 NOTES 📝

  • There is no longer an upper limit on the number of areas that can be output (previously it was 255) 🎉 More areas just mean potentially more passes being required over the .osm file, and hence the splitter will take longer to run. It's giving trade-off, iykyk.
  • There is no longer a limit on how many areas a way or relation can belong to (previously it was 4). Iconic. We love growth 🌱

👀 SEE ALSO 👀

Go read those too, bestie, they're literally part of the lore 📚✨

And that's the whole page, babes 🎀 This little number is mkgmap-splitter(1), dated 9 January 2015 — yes, she's a classic, vintage even, but she still ate 💅 💖 🌸 🦋

📦 where this page comes from