CONFUSABLE_HOMOGLYPHS(1)

πŸ“¦
version 3.3.1 Β· 2 versions
, in debian trixie, ubuntu noble, ubuntu questing

πŸ’– CONFUSABLE_HOMOGLYPHS(1) πŸ’–

okay bestie, welcome to confusable_homoglyphs(1) β€” the manual page that's gonna save you from getting catfished by a lookalike character. yes, really. this is the whole documentation, and we're gonna get through it together. grab a snack. πŸ“

✨ NAME ✨

confusable_homoglyphs β€” the confusable_homoglyphs documentation, babes.

Contents:

🌈 CONFUSABLE_HOMOGLYPHS [DOC] 🌈

so like, plot twist: this project has been adopted from the original confusable_homoglyphs by Victor Felder. iconic of them, no cap. πŸ‘‘

now. a homoglyph is one of two or more graphemes, characters, or glyphs with shapes that appear identical or very similar β€” wikipedia:Homoglyph. literally twins but make it unicode. πŸ™ˆ

Unicode homoglyphs can be a nuisance on the web. Your most popular client, AlaskaJazz, might be upset to be impersonated by a trickster who deliberately chose the username Ξ‘laskaJazz. the AUDACITY. 😭

  • AlaskaJazz is single script: only Latin characters.
  • Ξ‘laskaJazz is mixed-script: the first character is a greek letter.

You might also want to avoid people being tricked into entering their password on www.microsоft.com or www.faϲebook.com instead of www.microsoft.com or www.facebook.com. like, do NOT let them do that to you. 🚨 Here is a utility to play with these confusable homoglyphs.

Not all mixed-script strings have to be ruled out though β€” you could only exclude mixed-script strings containing characters that might be confused with a character from some unicode blocks of your choosing. selective toxicity, we love her. πŸ’…

  • Allo and ρττ are fine: single script.
  • AlloΞ“ is fine when our preferred script alias is β€˜latin’: mixed script, but Ξ“ is not confusable.
  • Alloρ is dangerous: mixed script and ρ could be confused with p.

This library is compatible with Python 3. slay. 🐍✨

πŸ¦‹ API documentation πŸ¦‹

πŸŽ€ Is the data up to date? πŸŽ€

Yep. πŸ’β™€οΈ

The unicode blocks aliases and names for each character are extracted from this file provided by the unicode consortium.

The matrix of which character can be confused with which other characters is built using this file provided by the unicode consortium.

This data is stored in two JSON files: categories.json and confusables.json. If you delete them, they will both be recreated by downloading and parsing the two abovementioned files and stored as JSON files again. self-healing behavior, honestly? obsessed. 🌸

πŸ’… INSTALLATION πŸ’…

If available, install an appropriate package from your distribution:

Otherwise you can install from PyPi:

at the command line:

$ easy_install confusable_homoglyphs

or, if you have virtualenvwrapper installed:

$ mkvirtualenv confusable_homoglyphs
$ pip install confusable_homoglyphs

⭐ USAGE ⭐

To use confusable_homoglyphs in a project:

pip install confusable_homoglyphs
import confusable_homoglyphs

To update the data files, you first need to install the β€œcli” bundle, then run the β€œupdate” command:

pip install confusable_homoglyphs[cli]
confusable_homoglyphs update
bestie we are SO not done yet β€” more of this page is coming. stay tuned. 🫢

✨ API DOCUMENTATION ✨

πŸ¦‹ confusable_homoglyphs package

✨ Submodules

πŸŽ€ confusable_homoglyphs.categories module

confusable_homoglyphs.categories.alias(chr)

Ok so like, this function? It’s giving script block alias retrieval for a unicode character. You hand it a character, it hands you the alias. Obsessed. πŸ’…

>>> categories.alias('A')
'LATIN'
>>> categories.alias('Ο„')
'GREEK'
>>> categories.alias('-')
'COMMON'

Parameters

  • chr (str) – A unicode character

Returns β€” The script block alias.

Return type β€” str

confusable_homoglyphs.categories.aliases_categories(chr)

This one is highkey a two-for-one: it gets you the script block alias and the unicode category for a unicode character. Iconic. ✨

>>> categories.aliases_categories('A')
('LATIN', 'L')
>>> categories.aliases_categories('Ο„')
('GREEK', 'L')
>>> categories.aliases_categories('-')
('COMMON', 'Pd')

Parameters

  • chr (str) – A unicode character

Returns β€” The script block alias and unicode category for a unicode character.

Return type β€” (str, str)

confusable_homoglyphs.categories.category(chr)

Bestie, this is the unicode category plug. Give it a character, it tells you the category. No cap. πŸŽ€

>>> categories.category('A')
'L'
>>> categories.category('Ο„')
'L'
>>> categories.category('-')
'Pd'

Parameters

  • chr (str) – A unicode character

Returns β€” The unicode category for a unicode character.

Return type β€” str

confusable_homoglyphs.categories.unique_aliases(string)

This one collects all the unique script block aliases used in a unicode string. Like a little alias detective. πŸ•΅οΈβ€β™€οΈ

>>> categories.unique_aliases('ABC')
{'LATIN'}
>>> categories.unique_aliases('ρAΟ„-')
{'GREEK', 'LATIN', 'COMMON'}

Parameters

  • string (str) – A unicode character

Returns β€” A set of the script block aliases used in a unicode string.

Return type β€” (str, str)

Yes, the return type in the original really says (str, str) even though it returns a set. We keep the lore exactly as written. πŸ’€

πŸŽ€ confusable_homoglyphs.cli module

confusable_homoglyphs.cli.generate_categories()

This one generates the categories JSON data file from the unicode specification. Slay. πŸ’–

Returns β€” True for success, raises otherwise.

Return type β€” bool

confusable_homoglyphs.cli.generate_confusables()

Same energy: generates the confusables JSON data file from the unicode specification.

Returns β€” True for success, raises otherwise.

Return type β€” bool

πŸŽ€ confusable_homoglyphs.confusables module

exception confusable_homoglyphs.confusables.Found

Bases: Exception

confusable_homoglyphs.confusables.is_confusable(string, greedy=False, preferred_aliases=[])

Ok so this is the main character. It checks if string contains characters which might be confusable with characters from preferred_aliases. πŸ”₯

If greedy=False, it will only return the first confusable character found without looking at the rest of the string, greedy=True returns all of them.

preferred_aliases=[] can take an array of unicode block aliases to be considered as your β€˜base’ unicode blocks:

  • considering paρa,
    • with preferred_aliases=['latin'], the 3rd character ρ would be returned because this greek letter can be confused with latin p.
    • with preferred_aliases=['greek'], the 1st character p would be returned because this latin letter can be confused with greek ρ.
    • with preferred_aliases=[] and greedy=True, you’ll discover the 29 characters that can be confused with p, the 23 characters that look like a, and the one that looks like ρ (which is, of course, p aka LATIN SMALL LETTER P).
>>> confusables.is_confusable('paρa', preferred_aliases=['latin'])[0]['character']
'ρ'
>>> confusables.is_confusable('paρa', preferred_aliases=['greek'])[0]['character']
'p'
>>> confusables.is_confusable('Abç', preferred_aliases=['latin'])
False
>>> confusables.is_confusable('AlloΞ“', preferred_aliases=['latin'])
False
>>> confusables.is_confusable('ρττ', preferred_aliases=['greek'])
False
>>> confusables.is_confusable('ρτ.Ο„', preferred_aliases=['greek', 'common'])
False
>>> confusables.is_confusable('ρττp')
[{'homoglyphs': [{'c': 'p', 'n': 'LATIN SMALL LETTER P'}], 'alias': 'GREEK', 'character': 'ρ'}]

Parameters

  • string (str) – A unicode string
  • greedy (bool) – Don’t stop on finding one confusable character - find all of them.
  • preferred_aliases (list(str)) – Script blocks aliases which we don’t want string’s characters to be confused with.

Returns β€” False if not confusable, all confusable characters and with what they are confusable otherwise.

Return type β€” bool or list

confusable_homoglyphs.confusables.is_dangerous(string, preferred_aliases=[])

Checks if string can be dangerous, i.e. is it not only mixed-scripts but also contains characters from other scripts than the ones in preferred_aliases that might be confusable with characters from scripts in preferred_aliases.

For preferred_aliases examples, see is_confusable docstring.

>>> bool(confusables.is_dangerous('Allo'))
False
>>> bool(confusables.is_dangerous('AlloΞ“', preferred_aliases=['latin']))
False
>>> bool(confusables.is_dangerous('Alloρ'))
True
>>> bool(confusables.is_dangerous('AlaskaJazz'))
False
>>> bool(confusables.is_dangerous('Ξ‘laskaJazz'))
True

Parameters

  • string (str) – A unicode string
  • preferred_aliases (list(str)) – Script blocks aliases which we don’t want string’s characters to be confused with.

Returns β€” Is it dangerous.

Return type β€” bool

confusable_homoglyphs.confusables.is_mixed_script(string, allowed_aliases=['COMMON'])

Checks if string contains mixed-scripts content, excluding script blocks aliases in allowed_aliases.

E.g. B. C is not considered mixed-scripts by default: it contains characters from Latin and Common, but Common is excluded by default.

>>> confusables.is_mixed_script('Abç')
False
>>> confusables.is_mixed_script('ρτ.Ο„')
False
>>> confusables.is_mixed_script('ρτ.Ο„', allowed_aliases=[])
True
>>> confusables.is_mixed_script('AlloΟ„')
True

Parameters

  • string (str) – A unicode string
  • allowed_aliases (list(str)) – Script blocks aliases not to consider.

Returns β€” Whether string is considered mixed-scripts or not.

Return type β€” bool

πŸŽ€ confusable_homoglyphs.utils module

confusable_homoglyphs.utils.delete(filename)

Deletes a JSON data file if it exists.

confusable_homoglyphs.utils.dump(filename, data)

No description in the original, bestie. It just vibes. πŸ’…

confusable_homoglyphs.utils.get(url, timeout=None)

Also no description. Just a signature serving looks. ✨

confusable_homoglyphs.utils.load(filename)

Loads a JSON data file.

Returns β€” A dict.

Return type β€” dict

confusable_homoglyphs.utils.path(filename)

Returns a file path relative to the data directory.

This is the package directory by default, or the env variable CONFUSABLE_DATA if set.

Returns β€” A file path string.

Return type β€” str

confusable_homoglyphs.utils.u(x)

No description. Short and mysterious. πŸ¦‹

πŸ“¦ Module contents

πŸ’– CONTRIBUTING πŸ’–

Bestie. Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given. Like, always always. No cap. 🫢

You can contribute in many ways:

πŸŽ€ Types of Contributions πŸŽ€

πŸ› Report Bugs

Report bugs at https://todo.sr.ht/~valhalla/confusable_homoglyphs ✨

If you're reporting a bug, please include:

  • Any details about your local setup that might be helpful in troubleshooting. (spill the tea about your environment, it genuinely helps)
  • Detailed steps to reproduce the bug. (receipts!! we need the receipts)

πŸ”§ Fix Bugs

Look through the sourcehut tickets for bugs. Anything tagged with β€œbug” is open to whoever wants to implement it. Go off, honestly. πŸ‘‘

✨ Implement Features

Look through the sourcehut tickets for features. Anything tagged with β€œfeature” is open to whoever wants to implement it. main character energy, take the ticket 🌟

πŸ“ Write Documentation

confusable_homoglyphs could always use more documentation, whether as part of the official confusable_homoglyphs docs, in docstrings, or even on the web in blog posts, articles, and such. fr fr, docs are the love language of open source πŸ’–

πŸ’Œ Submit Feedback

The best way to send feedback is to file an issue at https://todo.sr.ht/~valhalla/confusable_homoglyphs.

If you're proposing a feature:

  • Explain in detail how it would work. (be specific bestie, vague pitches are giving chaos)
  • Keep the scope as narrow as possible, to make it easier to implement.
  • Remember that this is a volunteer-driven project, and that contributions are welcome :) no one here is getting paid, be nice, be patient πŸ¦‹

🌸 Get Started! 🌸

Ready to contribute? Here's how to set up confusable_homoglyphs for local development. Let's gooo πŸ”₯

  1. Clone the git repository from sourcehut:
$ git clone https://git.sr.ht/~valhalla/confusable_homoglyphs
  1. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:
$ mkvirtualenv confusable_homoglyphs
$ cd confusable_homoglyphs/
$ python setup.py develop
  1. Create a branch for local development:
$ git checkout -b name-of-your-bugfix-or-feature

Now you can make your changes locally. tiny branch, big dreams πŸ’…

  1. When you're done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:
$ flake8 confusable_homoglyphs tests
$ python setup.py test
$ tox

To get flake8 and tox, just pip install them into your virtualenv. yes, into the virtualenv, not globally, we're not monsters πŸ™ˆ

  1. Commit your changes:
$ git add .
$ git commit -m "Your detailed description of your changes."
  1. Send the patch to mailto:~valhalla/confusable_homoglyphs-devel@lists.sr.ht:
$ git send-email \
  --to="mailto:~valhalla/confusable_homoglyphs-devel@lists.sr.ht" \
  HEAD^

you can see https://git-send-email.io/ for details on how to install and configure git-send-email. iykyk, the email workflow is a whole vibe once you get it πŸ“¬

Not me pointing out that the list jumps from 5 to 7 😭 β€” but it does, and we're keeping it exactly as it is, because the manual page says so and the manual page is the moment. πŸ’€

🌈 Pull Request Guidelines 🌈

Before you submit a pull request, check that it meets these guidelines:

  1. The pull request should include tests. always always always
  2. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst.
  3. The pull request should work for all supported Python versions. all of them, bestie, not just your fave

πŸ‘‘ CREDITS πŸ‘‘

🌟 Original Author and Former Maintainer

  • Victor Felder <victorfelder@gmail.com>

πŸ’œ Current Maintainer

  • Elena β€œof Valhalla” Grandi <valhalla@trueelena.org>

🫢 Contributors

  • Ryan P Kilby <rpkilby@ncsu.edu>

πŸ“œ HISTORY πŸ“œ

1.0.0

  • Initial release. the origin story ✨

2.0.0

  • allowed_categories renamed to allowed_aliases

2.0.1

3.0.0

Courtesy of Ryan P Kilby, via https://github.com/vhf/confusable_homoglyphs/pull/6 :

  • Changed file paths to be relative to the confusable_homoglyphs package directory instead of the user's current working directory.
  • Data files are now distributed with the packaging.
  • Fixes tests so that they use the installed distribution instead of the local files. (Originally, the data files were erroneously showing up during testing, despite not being included in the distribution).
  • Moves the data file generation into a simple CLI. This way, users have a method for controlling when the data files are updated.
  • Since the data files are now included in the distribution, the CLI is made optional. Its dependencies can be installed with the cli bundle, eg. pip install confusable_homoglyphs[cli].

3.1.0

  • Update unicode data

3.1.1

  • Update unicode data (via ftp)

3.2.0

  • Drop support for Python 3.3
  • Fix #11: work as expected when char not found in datafiles

3.3.0

  • Drop support for Python 2
  • Drop support for Python < 3.7, add support for Python up to 3.12
  • Allow using data files from a custom location set with the CONFUSABLE_DATA environment variable.
  • Fix the return value of confusables.is_dangerous() to the documented API of a boolean value. It used to return either False or the list output of confusable.is_confusable().
  • Added a check command for command line use.

3.3.1

  • Update unicode data

✍️ AUTHOR ✍️

Victor Felder

2024, Victor Felder

and that's the whole page, bestie!! πŸŽ€ This was CONFUSABLE_HOMOGLYPHS(1), version 3.3.1, dated Jan 30, 2024 β€” so if you're reading this from the future, hi, hope the unicode data got updated again πŸ’– slay, stay safe out there, and go report that bug πŸ¦‹βœ¨

πŸ“¦ where this page comes from