OSMFILTER(1)

💖 OSMFILTER 💖

the experimental OSM filter thingy, bestie. osmfilter — The experimental OSM filters data. she's a whole main character and she's built different, but also she bites, so read the warning, okay? 🫶

🌈 SYNOPSIS 🌈

osmfilter options [input file]

✨ DESCRIPTION ✨

THIS PROGRAM IS FOR EXPERIMENTAL USE ONLY. PLEASE EXPECT MALFUNCTION AND DATA LOSS. SAVE YOUR DATA BEFORE STARTING THIS PROGRAM.

No seriously. Not me being dramatic for once. EXPERIMENTAL. MALFUNCTION. DATA LOSS. Back up your files before you run this, periodt. 😭

So like, this program filters OpenStreetMap data. That's the whole vibe. 🗺️

The input file name has to be handed over as a command line argument — you literally cannot skip this. And the file must not be a stream. Redirecting from standard input? Not happening, sorry babe. The program needs random access to the file, so a stream just won't do. 💀

Good news tho: you do not need to tell her what the input format is, because osmfilter recognizes these all on her own:

• .osm (XML)
• .osc (OSM Change File)
• .osh (OSM Full History)
• .o5m (speed-optimized)
• .o5c (speed-optimized Change File)

The output format is .osm by default. Want something different? Specify it with the appropriate command line parameter and she'll switch it up. 💅

---

🎀 OPTIONS 🎀

• --keep=OBJECT_FILTER — Every object type (nodes, ways and relations) gets kept if it meets the filter criteria. Same energy for dependent objects too, like nodes inside ways, ways inside relations, relations inside other relations. Wanna know the OBJECT_FILTER syntax? It's down below, keep scrolling. 👀
• --keep-nodes=OBJECT_FILTER, --keep-ways=OBJECT_FILTER, --keep-relations=OBJECT_FILTER, --keep-nodes-ways=OBJECT_FILTER, --keep-nodes-relations=OBJECT_FILTER, --keep-ways-relations=OBJECT_FILTER — Same as the one above, but only for the object types you actually named. Cherry-picking, iconic. 🍒
• --drop=OBJECT_FILTER — Every object type (nodes, ways and relations) that meets the filter criteria gets dropped, no matter what — even if it also meets the criteria of a keep filter (see above). So yeah, drop beats keep. OBJECT_FILTER syntax is below. 🗑️
• --drop-nodes=OBJECT_FILTER, --drop-ways=OBJECT_FILTER, --drop-relations=OBJECT_FILTER, --drop-nodes-ways=OBJECT_FILTER, --drop-nodes-relations=OBJECT_FILTER, --drop-ways-relations=OBJECT_FILTER — Same as above, but just for the object types you specified.
• --keep-tags=TAG_FILTER — The tags you name in TAG_FILTER are the ones allowed on output. TAG_FILTER syntax lives below, go look. 🏷️
• --keep-node-tags=TAG_FILTER, --keep-way-tags=TAG_FILTER, --keep-relation-tags=TAG_FILTER, --keep-node-way-tags=TAG_FILTER, --keep-node-relation-tags=TAG_FILTER, --keep-way-relation-tags=TAG_FILTER — Same as above, but only for the object types you named.
• --drop-tags=TAG_FILTER — The tags you specify get dropped. And yes, this overrules the --keep-tags parameter we just talked about. She said what she said. TAG_FILTER syntax is below.
• --drop-node-tags=TAG_FILTER, --drop-way-tags=TAG_FILTER, --drop-relation-tags=TAG_FILTER, --drop-node-way-tags=TAG_FILTER, --drop-node-relation-tags=TAG_FILTER, --drop-way-relation-tags=TAG_FILTER — Same as above, but only for the object types you specified.
• --modify-tags=TAG_MODIFICATION_LIST — The tags you list get modified. This happens after any filtering (so after --keep, --keep-tags, --drop, --drop-tags — order matters, bestie). TAG_MODIFICATION_LIST is described below. ✏️
• --modify-node-tags=TAG_MODIFICATION_LIST, --modify-way-tags=TAG_MODIFICATION_LIST, --modify-relation-tags=TAG_MODIFICATION_LIST, --modify-node-way-tags=TAG_MODIFICATION_LIST, --modify-node-relation-tags=TAG_MODIFICATION_LIST, --modify-way-relation-tags=TAG_MODIFICATION_LIST — Same as above, but only for the object types you specified.
• --drop-author — For most applications the author tags are just not needed, fr fr. Flip this on and no author information gets written at all: no changeset, no user, no timestamp. Poof. 🫥
• --drop-version — Wanna exclude not just the author info but the version number too? This is your girl.
• --drop-nodes, --drop-ways, --drop-relations — Depending on which combo of these you pick, no members of the referred section get written.
• --emulate-osmosis, --emulate-pbf2osm — If the output format is .osm, the program will try to use the same data syntax as Osmosis, or respectively pbf2osm. Copying their whole flow. 💅
• --fake-author — If you dropped author info with --drop-author, that data is gone, obviously, that's what dropping means. But some programs require author information on input even though they don't actually need the data. So for that, you can fake it. o5mfiler will write changeset 1, timestamp 1970. ⏳
• --fake-version — Same as --fake-author, except that — if .osm XML is used as the output format — only the version number gets written (version 1). Super useful if you wanna inspect the data with JOSM. 🔍
• --fake-lonlat — Some programs depend on getting longitude/latitude values, even when the object in question is supposed to be deleted. With this option you can have osmfilter fake those values:

... lat="0" lon="0" ...

Heads up: this is for XML files only (.osc and .osh).

• -h — Displays a short parameter overview. Quick lil peek. 👀
• --help — Displays this help.
• --ignore-dependencies — Usually, all member nodes of a way that meets the filter criteria get included too. Same goes for members of included relations. Activate this option and all those dependencies between OSM objects get ignored. Cutting the strings. ✂️
• --out-key=KEYNAME — The output won't contain regular OSM data, just statistics: a list of all used keys gets assembled. To the left of each key, the number of occurrences is printed. If KEYNAME is given, the program lists all values used in connection with that key. You can use wildcard characters for KEYNAME, but only at the beginning and/or at the end. For example:

--out-key=addr:*

• --out-count=KEYNAME — Same as --out-key=, except the list is sorted by the number of occurrences of the keys, or respectively the values. Sorted by popularity, she's ranking them. 📊
• --out-osm — Data gets written in .osm format. This is the default output format.
• --out-osc — The OSM Change format gets used for output. Just so you know: OSM objects that are meant to be deleted are represented by their ids only.
• --out-osh — For every OSM object, the appropriate 'visible' tag gets added, to meet the 'full planet history' specification. 📜
• --out-o5m — The .o5m format gets used. Same structure as the conventional .osm format, but the data is stored as binary numbers, so it's much more compact than .osm. No packing is used, which means you can pack .o5m files with any file packer you want — lzo, bz2, etc. 🗜️
• --out-o5c — This is the change file format of the .o5m data format. All <delete> tags won't be performed as delete actions, they get converted into .o5c data format instead.
• -o=<outfile> — Standard output gets rerouted to the file you specify. And if no output format has been specified, the program figures it out from the file name extension. Clever girl. 🧠
• -t=<tempfile> — osmfilter uses a temporary file to process interrelational dependencies, and this parameter defines the name prefix. Default value is "osmfilter_tempfile".
• --parameter-file=FILE — Wanna supply one or more command line arguments through a parameter file? Use this option and give the file name. Inside the parameter file, parameters must be separated by empty lines. Line feeds inside a parameter get converted to spaces. And lines starting with "// " are treated as comments. 📝
• -v, --verbose — With 'verbose' mode activated, some statistical data and diagnosis data gets displayed. And if -v or --verbose is the first parameter in the line, osmfilter will display all input parameters. Spilling all the tea. ☕

---

🦋 OBJECT_FILTER 🦋

Some of the command line arguments need a filter, and this filter definition is made of key/val pairs. Here's the syntax:

"KEY1=VAL1 OP KEY2=VAL2 OP KEY3=VAL3 ..."

OP is the Boolean operator, and it must be either "and" or "or". As usual, "and" gets processed before "or". Wanna control the order of processing yourself? Use brackets. Just know that brackets always have to be padded by spaces. Example:

lit=yes and ( note=a or source=b )

Instead of every "=" you can enter one of these comparison operators: != (not equal), <, >, <=, >=. The program uses ASCII-alphabetic comparison, unless you're comparing against a value that starts with a digit.

If there are different possible values for the same key, you only need to write the key once. For example:

"amenity=restaurant =pub =bar"

It's also allowed to omit the value entirely — in that case the program accepts every value for the defined key. For example:

"highway= and lit=yes"

You may use wildcard characters for a key or a value, but again — only at the beginning and/or at the end. For example:

"wikipedia:*=highway=*ary  ref_name=*central*"

⚠️ Be careful with wildcards in keys, because only the first key that meets the pattern gets processed. First come, first served, and the rest are just standing there. 💀

There are three special keys that represent object id, user id and user name: @id, @uid and @user. They let you search for certain objects, or for edits made by specific users. 🕵️‍♀️

---

🌸 TAG_FILTER 🌸

The tag filter decides which tags get kept and which don't. For example:

--keep-tags="highway=motorway =primary"

...will not accept "highway" tags other than "motorway" or "primary". And note: neither the object itself gets deleted, nor the remaining tags. She's just being selective about the tags, not throwing the whole thing out.

If you wanna drop every tag that isn't mentioned in a list, use this example:

all highway= amenity= name=

---

💜 TAG_MODIFICATION_LIST 💜

The tag modification list decides which tags get modified. So like, this example:

--modify-tags="highway=primary to =secondary"

...will change every "primary" highway into "secondary". A total glow-up. ✨

You can also use comparisons, or add additional tags:

--modify-way-tags="maxspeed>200 add highspeed=yes"

🎚️ TUNING 🎚️

Okay so bestie, here's the tea on making this go fast ☕✨

To speed up the whole process, osmfilter uses some main memory for a hash table. By default it grabs 1200 MB to store a flag for every possible node, 150 for the way flags, and 10 relation flags. And here's the cute part — every single byte holds the flags for 8 ID numbers, which means in 1200 MB the program can store 9600 million flags. Iconic capacity, honestly. 💅

Now, since there are fewer than 5700 million IDs for nodes at present (that's May 2018, bestie, so like, ancient history but okay), 720 MB would totally suffice. So for example, you can shrink the hash sizes down to like 720, 80 and 2 MB (and heads up — for relations, 2 flags are needed each) using this option:

--hash-memory=720-80-2

But bestie, keep in mind the OSM database is continuously expanding. It never stops growing, she's always eating. 🍽️ For this reason the program's own default value is higher than what's shown in the example, and it may be appropriate to increase it in the future.

If you do not want to bother with all the details, you can just enter the amount of memory as a sum and the program will divide it by itself. Like:

--hash-memory=1000

Those 1000 MiB get split into three parts: 800 for nodes, 150 for ways, and 50 for relations. Slay, math done for you. 🧮✨

And because we're taking hashes, you don't have to provide all the suggested memory — the program will operate with less hash memory too. But (and this is the important bit, no cap) in that case the border filter will be less effective, i.e., some ways and some relations will be left in the output file even though they should have been excluded. So like, you get leftovers you didn't order. 🙈

The maximum value the program accepts for the hash size is 4000 MiB. If you exceed the maximum amount of memory available on your system, the program will try to reduce this amount and display a warning message. It's looking out for you fr fr. 🫶

---

🚧 LIMITATIONS 🚧

When you're filtering whole OSM objects (--keep...=, --drop...=), the input file must contain the objects ordered by their type: first, all nodes nodes, next, all ways, followed by all relations. Yes, that's the ordering. The order matters, bestie, don't fight it. 👑

Usual .osm, .osc, .o5m and o5c files adhere to this condition. So this means you do not have to worry about this limitation at all. Phew. 😌 And if this sequence does get broken, osmfilter will display an error message — it will absolutely call it out, periodt.

The number of key/val pairs in each filter parameter is limited to 1000, and the length of each key or val is limited to 100. That's the ceiling, babe. 🌈

---

📝 NOTES 📝

This program is for experimental use. Expect malfunctions and data loss. Do not use the program in productive or commercial systems.

Like, for real for real. Do not put this in prod. I am begging you. 😭💀

There is NO WARRANTY, to the extent permitted by law. Please send any bug reports to marqqs@gmx.eu — he wants to hear from you, bestie. 🐛📬

---

💖 EXAMPLE 💖

Here's the vibe, copy-paste ready:

osmfilter europe.o5m --keep=amenity=bar -o=new.o5m
osmfilter a.osm --keep-nodes=lit=yes --drop-ways -o=light.osm
osmfilter a.osm --keep="place=city or ( place=town and population>=10000
)" -o=b.osm
osmfilter region.o5m --keep="bridge=yes and layer>=2" -o=r.o5m

Look at those. Literally so satisfying. 🔥

---

👀 SEE ALSO 👀

• osmconvert(1)
• osmupdate(1)

Go read those too, they're part of the friend group. 🫂

---

👑 AUTHORS 👑

osmfilter was written by Markus Weber. Legend behaviour, honestly. 🌟

---

And that's the whole page, bestie! This one came from the source/version line and the date stamp at the bottom — September 2013, filed under OSMFILTER(1). Ancient relic, still serving looks. 🎀✨ Go filter something. Mwah. 💜