SKF(1)

πŸ“¦ version 2.10.16, in fedora 43, fedora 44

πŸ’– skf(1) β€” General Commands Manual πŸ’–

okay bestie, buckle up, because this is the manual page for skf and she is that girl. it's a simple Kanji Filter (v2.1) and we are going to read the whole thing together, no skipping, no skimming. ✨


🌸 NAME 🌸

skf β€” simple Kanji Filter (v2.1)


✨ SYNOPSIS ✨

skf [-EIJKNQRSXZbehjknqrsuvxz] [ long_format_options ] [infiles..]

πŸ¦‹ DESCRIPTION πŸ¦‹

so. skf is yet another i18n-capable kanji-filter, and she was literally designed for reading alllll the various CJK-coded files floating around on the Net. what she does: she takes input kanji texts or streams, converts them into a character stream using a designated codeset, and outputs them to standard output. basically, she's built to be a versatile filter for reading documents in a bunch of different code sets, and she does not do features that aren't related to code conversion. stay in your lane, periodt. πŸ’…

like nkf, skf automatically recognizes an input file code when it's a kind of ISO-2022 compliant code, and she also detects EUC-variant codes if the input file is Japanese text without X 0201 kanas. skf 2.1 can read various iso-2022 compliant character sets, including JIS Kanji codes (X 0208, X 0212 and X 0213), EUC encoding (euc-jp (with X 0213 support), euc-cn, euc-kr and euc-tw), ISO Europian latins (ISO-8859-1 to 11, 13/14/15/16) and many regional character sets. she can also read some non-iso2022 compliant sets, including Microsoft Shift-JIS code, KOI-8-R/U, GB2312 (HZ), big5, VISCII(rfc1456, include VIQR), Unicode standard (UCS2/UTF-16, UTF7 and UTF8), some of MS codesets (cp1250 etc.) and some other vendor specific codes (KEIS83, JEF etc). she contains multitudes fr fr. 🌈

supported output character sets of skf are more limited, but still include X 0208/X 0212/X 0213 JIS, X 0201 JIS, ASCII, Microsoft Shift-JIS, EUC-jp/-kr/-cn, HZ, iso-2022-jp/kr, big5, VISCII and Unicode.

skf also provides some basic decoding features for some common encodings including MIME, Punycode and URI codepoint. Unicode decomposition feature is also supported since 1.96. πŸŽ€

as noted above, skf is designed to convert input text into some kind of human-readable forms under a local environment (i.e. codeset), and has several extra conversion features like GNU recode type folding. such conversions include Windows/Macintosh specific code swaps and old-new jis glyph changes, html-format/TeX format conversion and variant unifications. iconic behavior honestly.

skf also can be compiled as an extension of some lightweight languages. see README.txt for details. πŸ‘‘

if one or more file names are given, skf reads the files and outputs the converted stream to stdout. if no file names are given, input is taken from stdin and output is also stdout. OPTIONS are taken from environment variables SKFENV, skfenv and command line, respectively in this order. environment variables are not used when skf is running as a priviledged user. skf does not use LOCALE-related environment variables for conversions, but output error messages are controlled by given LOCALES.

⚠️ lowkey important: those env vars get skipped entirely when you're a priviledged user. noted.

🌟 CODESET OPTIONS 🌟

skf is written from scratch, and inherits no code from nkf. however, skf is intended to be a drop-in replacement for nkf(v1.4) and has a similar commonly-used nkf option set.

skf 2.1 recognizes the following options. defaults are all off if not explicitly specified. (yes bestie, everything starts off. you gotta ask.) πŸ™ˆ

🧺 buffering control 🧺

  • -b β€” use buffered output. this is the default.
  • -u β€” use unbuffered output. code detection feature is disabled when this option is on.

πŸ”„ Input/Output codeset options πŸ”„

  • --ic= input_code_set β€” specify input codeset is input_code_set. possible candidates are shown below.
  • --oc= output_code_set β€” specify output codeset is output_code_set. possible candidates are shown below. default codeset in the distribution package is euc-jp, but it depends on compile option. the default codeset is shown by skf -h.

🧚 Supported codeset 🧚

skf recognizes the following codesets as an input/output codeset. these codeset names are case insensitive, and minus (-) and underscore (_) is ignored. note that iso-2022 escape-based input codeset (registered to IANA) is recognized automatically, even when non-iso2022 codeset (except Unicode and B-Right/V) is specified.

in the little flag pair below, o in the in-column means the named codeset can be specified as input, and x means the named codeset is not for input. the output-column is the same, except it's for output. slay. πŸ’œ

  • iso8859-1 (in o, out o) β€” ascii + iso-8859-1 (latin-1)
  • iso8859-2 (in o, out o) β€” ascii + iso-8859-2 (latin-2)
  • iso8859-3 (in o, out o) β€” ascii + iso-8859-3 (latin-3)
  • iso8859-4 (in o, out o) β€” ascii + iso-8859-4 (latin-4)
  • iso8859-5 (in o, out o) β€” ascii + iso-8859-5 (Cyrillic)
  • iso8859-6 (in o, out o) β€” ascii + iso-8859-6 (Arabic)
  • iso8859-7 (in o, out o) β€” ascii + iso-8859-7 (Greek)
  • iso8859-8 (in o, out o) β€” ascii + iso-8859-8 (Hebrew)
  • iso8859-9 (in o, out o) β€” ascii + iso-8859-9 (latin-5)
  • iso8859-10 (in o, out o) β€” ascii + iso-8859-10 (latin-6)
  • iso8859-11 (in o, out o) β€” ascii + iso-8859-11 (Thai)
  • iso8859-13 (in o, out o) β€” ascii + iso-8859-13 (Baltic Rim)
  • iso8859-14 (in o, out o) β€” ascii + iso-8859-14 (Celtic)
  • iso8859-15 (in o, out o) β€” ascii + iso-8859-15 (Latin-9)
  • iso8859-16 (in o, out o) β€” ascii + iso-8859-16
  • koi-8r (in o, out o) β€” koi-8r (Russian)
  • koi-8u (in o, out o) β€” koi-8r (Ukraina)
  • cp1251 (in o, out o) β€” Cyrillic latin MS cp1251
  • jis (in o, out o) β€” iso-2022-jp (rfc1496 7bit JIS)
  • iso-2022-jp-x0213 (in o, out o) β€” iso-2022-jp-3 (JIS X 0213:2000) a.k.a. jis-x0213
  • jis-x0213-strict (in o, out o) β€” iso-2022-jp-3-strict
  • iso-2022-jp-2004 (in o, out o) β€” iso-2022-jp-2004(JIS X 0213:2004) a.k.a. jis-x0213-2004
  • oldjis (in o, out o) β€” iso-2022-jp-1978(JIS X 0208:1978)
  • cp50220 (in o, out o) β€” Microsoft codepage 50220
  • cp50221 (in o, out o) β€” Microsoft codepage 50221
  • cp50222 (in o, out o) β€” Microsoft codepage 50222
  • euc-jp (in o, out o) β€” EUC-encoded JIS X 0208:1997
  • euc-x0213 (in o, out o) β€” EUC-encoded JIS X 0213:2000
  • euc-jis-2004 (in o, out o) β€” EUC-encoded JIS X 0213:2004
  • cp51932 (in o, out o) β€” EUC-encoded Microsoft codepage 932
  • euc-kr (in o, out o) β€” EUC-encoded KS X 1001 Korian
  • euc7-kr (in o, out o) β€” 7bit EUC-encoded KS X 1001 Korian
  • uhc (in o, out o) β€” Unified hangle (Windows cp949)
  • johab (in o, out o) β€” KS X 1001-johab Korian
  • euc-cn (in o, out o) β€” EUC-encoded GB2312 Chinese
  • euc7-cn (in o, out o) β€” 7bit EUC-encoded GB2312 Chinese
  • hz (in o, out o) β€” HZ-encoded GB2312 Chinese
  • euc-tw (in o, out o) β€” EUC-encoded CNS 11643 Chinese
  • gb12345 (in o, out o) β€” EUC-encoded GB12345 Chinese
  • gbk (in o, out o) β€” GB2312 Extension(cp936) Chinese
  • gb18030 (in o, out o) β€” GB18030 chinese
  • big5 (in o, out o) β€” BIG5 (with Eten extension + EURO)
  • cp950 (in o, out o) β€” BIG5 (Microsoft cp950 + EURO)
  • big5-hkscs (in o, out o) β€” BIG5 with HKSCS
  • big5-2003 (in o, out o) β€” BIG5-2003
  • big5-uao (in o, out o) β€” BIG5-Unicode at On
  • sjis (in o, out o) β€” Shift-jis (Microsoft cp943)
  • shiftjis-x0213 (in o, out o) β€” Shiftjis-encoded JIS X 0213:2000
  • shiftjis-2004 (in o, out o) β€” Shiftjis-encoded JIS X 0213:2004
  • sjis-docomo (in o, out o) β€” Shiftjis-encoded with NTT Docomo emoticons.
  • sjis-au (in o, out o) β€” Shiftjis-encoded with AU emoticons.
  • sjis-softbank (in o, out o) β€” Shiftjis-encoded with SoftBank emoticons.
  • oldsjis (in o, out o) β€” Shift-jis (JIS X 0208:1978)
  • cp932 (in o, out o) β€” Shift-jis-encoded MS cp932
  • cp932w (in o, out o) β€” Shift-jis-encoded MS cp932 with MS compatibility
  • viscii (in o, out o) β€” VISCII (rfc1456) Vietnamise
  • viqr (in o, out o) β€” VISCII (rfc1456-VIQR) Vietnamise
  • keis (in o, out o) β€” Hitachi KEIS83/90
  • jef (in o, out x) β€” Fujitsu JEF (basic support only)
  • ibm930 (in o, out x) β€” IBM EBCDIC DBCS Japanese
  • ibm931 (in o, out x) β€” IBM EBCDIC DBCS Japanese w.latin
  • ibm933 (in o, out x) β€” IBM EBCDIC DBCS Korian
  • ibm935 (in o, out x) β€” IBM EBCDIC DBCS Simpl. Chinese
  • ibm937 (in o, out x) β€” IBM EBCDIC DBCS Trad. Chinese
  • unicode (in o, out o) β€” Unicode(TM) UTF-16LE
  • unicodefffe (in o, out o) β€” Unicode(TM) UTF-16BE
  • utf7 (in o, out o) β€” Unicode(TM) UTF-7
  • utf8 (in o, out o) β€” Unicode(TM) UTF-8
  • utf8-bom (in o, out o) β€” Unicode(TM) UTF-8 with BOM
  • utf7-imap (in o, out o) β€” IMAP modified Unicode(TM) UTF-7 (RFC2060)
  • mutf8 (in o, out o) β€” Java modified Unicode(TM) UTF-8
  • cesu8 (in o, out o) β€” CESU-8 (Unicode Technical Report #26)
  • nyukan-utf-8, nyukan-utf-16 (in x, out o) β€” Nyukan-moji(Japanese nyukoku-kanrikyoku gaiji). Encoding is utf-8 and utf-16 respectively.
  • arib-b24 (in o, out x) β€” ARIB B24 8-bit JIS-based
  • arib-b24-sj (in o, out x) β€” ARIB B24 8-bit SJIS-based
  • transparent (in x, out o) β€” Transparent mode (see below)
  • x-iscii-de (in o, out x) β€” India ISCII-91(IS13194:1991)
  • asmiscii-8 (in o, out x) β€” Armenian ARMISCII 8
  • geostd8 (in o, out x) β€” Geogian Geogstd 8
  • mik (in o, out x) β€” Burgarian MIK
  • tscii (in o, out x) β€” Tamil TSCII 1.7
  • locale (in o, out o) β€” codeset specified in locale. see below.

πŸ“ Codeset explanations πŸ“

  • iso-8859-* β€” when specified as output, G0 = GL is ascii and G1 = GR is iso-8859-*. 8bit encoding is used.
  • iso-2022-jp, jis β€” encoding is iso-2022-jp-2 (RFC1496). G0 = GL is JIS X 0201 roman, G1 = GR is JIS X 0201 kana, G2 is iso-8859-1 and G3 is JIS X 0212:1990 Supplementary Kanji.
  • jis-x0213, iso-2022-jp-3 β€” encoding is iso-2022-jp-3 (JIS X 0213:2000 based). G0 = GL is JIS X 0201 roman, for output, G1 = GR is JIS X 0201 kana, G2 is iso-8859-1 and G3 is JIS X 0213 plane2 Kanji.
  • jis-x0213-strict β€” encoding is subset of iso-2022-jp-3-strict (uses Plane 1 only). for output, G0 = GL is JIS X 0201 roman, G1 = GR is JIS X 0201 kana, G2 is iso-8859-1 and G3 is not set. output code using JIS X 0208 whenever possible. JIS X 0213 input is automatically recognized.
  • jis-x0213-2004, iso-2022-jp-2004 β€” encoding is iso-2022-jp-2003:2004. for output, G0 = GL is JIS X 0201 roman, G1 = GR is JIS X 0201 kana, G2 is iso-8859-1 and G3 is JIS X 0213 plane2 Kanji.
  • oldjis β€” encoding is iso-2022-jp using old JIS X 0208:1978). G0 = GL is JIS X 0201 roman, G1 = GR is JIS X 0201 kana, G2 is iso-8859-1 and G3 is JIS X 0212 Supplementary Kanji.
  • euc-jp, euc β€” encoding is 8-bit EUC using JIS X 0208:1997 character set. G0 = GL is ascii, G1 = GR is JIS X 0208, G2 is JIS X 0201 kana and G3 is JIS X 0212 Supplementary Kanji.
  • euc-x0213, euc-jis-2003 β€” encoding is 8-bit EUC-based JIS X 0213:2000. G0 = GL is ascii, G1 = GR is X 0213:2000 plane 1, G2 is iso-8859-1 and G3 is JIS X 0213:2000 plane2 Kanji.
  • euc-jis-2004 β€” encoding is 8-bit EUC-based JIS X0213:2004. G0 = GL is ascii, G1 = GR is X0213:2004 plane 1, G2 is iso-8859-1 and G3 is JIS x0213:2004 plane2 Kanji.
  • euc-kr β€” encoding is 8-bit EUC using KS X 1001 Wansung character set. G0 = GR is KS X1003, G1 = GR is KS X1001, G2 and G3 is not set.
  • euc7-kr, iso-2022-kr β€” it's giving iso-2022-kr (rfc1557) energy: 7-bit EUC built on the KS X 1001 Wansung character set. G0 = GR is KS X1003, G1 is KS X1001, and G2 and G3 are not set, bestie πŸ’–
  • euc-cn β€” 8-bit EUC using the GB 2312 simplified chinese character set. G0 = GR is ASCII, G1 = GR is GB2312, G2 and G3 are not set ✨
  • euc7-cn β€” same vibe but 7-bit: EUC on GB 2312 simplified chinese. G0 = GR is ASCII, G1 = GR is GB2312, G2 and G3 are not set 🌸
  • hz β€” HZ encoded (rfc1842) GB 2312 simplified chinese character set. G0 = GR is ASCII, G1 = GR is GB2312, G2 and G3 are not set πŸ¦‹
  • euc-tw β€” EUC encoded CNS11643 Plane1/2 traditional chinese character set. It's a subset of iso-2022-cn, lowkey iconic. G0 = GR is ASCII, G1 = GR is CNS11643 plane 1, G2 is CNS11643 plane 2, and G3 is not set πŸ‘‘
  • gb12345 β€” 8-bit EUC using GB 12345 (GBF) traditional chinese character set. G0 = GR is ASCII, G1 = GR is GB12345, G2 and G3 are not set πŸŽ€
  • gbk, cp936 β€” GBK simplified chinese character set. G0 = GR is ASCII and G1 = GR is GBK. G2 and G3 are not set πŸ’…
  • gb18030 (experimental) β€” GB18030 (ibm-1392, Windows cp54936) chinese character set. Uses ASCII as the latin part ⭐
  • big5 β€” Big5 traditional chinese character set with the ETen extension. Includes Euro mapping. Uses ASCII as the latin part 🌈
  • cp950 β€” Microsoft cp950-Big5 traditional chinese character set. Uses ASCII as the latin part πŸ’œ
  • big5-hkscs (experimental) β€” cp950-Big5 traditional chinese character set with the HKSCS extension. Uses ASCII as the latin part ✨
  • big5-2003 (experimental) β€” Big5-2003 Taiwanese standard traditional chinese character set. Uses ASCII as the latin part πŸ¦‹
  • big5-uao (experimental) β€” Big5-UAO (http://uao.cpatch.org) traditional chinese character set. Uses ASCII as the latin part 🌸
  • VISCII (experimental) β€” Vietnamise VISCII (rfc1456) character set. Not TCVN-5712, okurrr πŸ™ˆ
  • VIQR (experimental) β€” Vietnamise VISCII character set with VIQR encoding (rfc1456) πŸ’–
  • sjis β€” Shift-encoded JIS X 0208:1997 character set. Note that this is not cp932, just so we're clear. Uses JIS X 0201 latin as the latin(GL) part πŸŽ€
  • sjis-x0213, shift_jis-2000 β€” Shift-encoded JIS using the JIS X 0213:2000 character set πŸ’…
  • sjis-x0213-2004, shift_jis-2004 β€” Shift-encoded JIS using the JIS X 0213:2004 character set. 10 newly defined character added, but the Unicode mapping is the same as JIS X 0213:2000. Uses JIS X 0201 latin as the latin(GL) part ⭐
  • sjis-cellular (experimental) β€” Shift-encoded JIS X 0208:1997 character set with NTT Docomo/Vodafone(SoftBank) cellular phone glyph mapping. Output is not supported, so don't try it 🫢
  • cp932, cp932w β€” Microsoft SJIS cp932 with the NEC/IBM gaiji area, based on Windows XP mapping. Uses ASCII as the latin(GL) part. --use-compat and --use-ms-compat is automatically enabled. cp932w provides further WideCharToMultiByte compatibility πŸ”₯
  • cp51932 β€” Microsoft EUC-based cp51932 with the NEC/IBM gaiji area, based on Windows XP mapping. Uses ASCII as G0 and JIS X 0201 kana as the EUC G2 part. G3 is not used for output, and JIS X 0212:2000 as input. --use-compat and --use-ms-compat is automatically enabled πŸ‘‘
  • cp50220, cp50221, cp50222 β€” Microsoft JIS-based cp50220, cp50221, cp50222 with the NEC/IBM gaiji area, based on Windows XP mapping. For input, skf accepts cp50220, 50221 and 50222. Note that this codeset is NOT compatible with iso-2022, fr fr. Uses ASCII as the default character set. --use-compat and --use-ms-compat is automatically enabled πŸ’œ
  • oldsjis β€” Microsoft SJIS (JIS X 0208:1978 a.k.a. old JIS). Uses JIS X 0201 latin as the latin(GL) part 🌸
  • johab β€” KS X1001(Johab) character set. Uses KS X1003 latin as the latin(GL) part πŸ¦‹
  • uhc β€” UHC (cp949) character set. Uses ASCII as the latin(GL) part ✨
  • unicode, unicodefffe, utf16, utf16le β€” Unicode UTF-16 (v15.0). Input/Output default byte-endian is little for unicode and big for unicodefffe, and the input byte order mark is recognized. utf16 and unicodefffe is big-endian. utf16le and unicode is little endian. Output includes the endian mark by default unless --disable-endian-mark is specified. Output range is within UTF-32 with surrogate pair unless --limit-to-ucs2 is specified. Note that ucs2 is not supported within the lightweight language extension in both in and output, because of SWIG's passing data structure limitation. Specify to ucs2 will generate error πŸ’€
  • utf8 β€” UTF-8 encoded Unicode (v15.0). Output doesn't include byte order mark unless --enable-endian-mark is specified. Output range is within UTF-32 unless --limit-to-ucs2 is specified. By default, CESU-8 is not accepted as input. Option --enable-cesu8 enables CESU-8 input for the utf-8 converter. CESU-8 output is not supported. For UTF-8, the endian mark (BOM) is always ignored πŸŽ€
  • utf7 β€” UTF-7 encoded Unicode (v15.0). Input/output range is limited to UTF-16, and any value above U+10000 is regarded as undefined. BOM is always ignored for input, and never used for output πŸ’…
  • utf7-imap β€” Modified utf-7 for the IMAP protocol described in RFC2060. BOM is always ignored for input, and never used for output ⭐
  • mutf8 β€” Modified utf-8 for the Java language. CESU-8 plus U-0000 encoding. BOM is always ignored for input, and never used for output 🌈
  • cesu-8 β€” Modified utf-8 described in unicode technical report #26. BOM is always ignored for input, and never used for output πŸ¦‹
  • keis (experimental) β€” Hitachi KEIS83/90. Output range is limited to the EBCDIK and JIS X 0208 area πŸ’–
  • jef (experimental) β€” Fujitsu JEF. Input only. Only the basic part is supported πŸ™ˆ
  • ibm930 (experimental) β€” IBM DBCS Japanese with EBCDIC Kana 🌸
  • ibm931 (experimental) β€” IBM DBCS Japanese with EBCDIC latin (ibm037) ✨
  • ibm933 (experimental) β€” IBM DBCS Korian with EBCDIC Wansung character set πŸŽ€
  • ibm935 (experimental) β€” IBM DBCS Simplified Chinese with EBCDIC Chinese πŸ’œ
  • ibm937 (experimental) β€” IBM DBCS Traditional Chinese with EBCDIC Chinese 🫢
  • koi8r β€” Russian KOI-8R code πŸ”₯
  • cp1250 β€” Central Europian latin Microsoft cp1250 code ⭐
  • cp1251 β€” Eastern Europian cyrillic Microsoft cp1251 code 🌈
  • arib-b24, arib-b24-sj β€” ARIB B24 code defined in ATIB-STD-B24 vol.1 part.2 chapt. 7.3. b24 is 8-bit jis based, and b24-sj is sjis based πŸ‘‘
  • nyukan-utf-8, nyukan-utf-16 β€” Normalized Unicode UTF-8/UTF-16 based on Japanese law ministry kokuji No. 582 πŸ’…
  • locale β€” Use the locale-specified codeset. Since locale only provides partial information as codeset, whether this option works as expected or not depends on environmental settings, so it's a vibe check every time 😭
  • transparent β€” Transparent mode. Various code control features, include folding and line end code conversion, is also ignored πŸ¦‹

βœ‚οΈ Shortcuts βœ‚οΈ

  • -j β€” same as --oc=jis πŸ’–
  • -s β€” same as --oc=sjis ✨
  • -e β€” same as --oc=euc-jp 🌸
  • -q β€” same as --oc=unicode πŸŽ€
  • -z β€” same as --oc=sjis πŸ’…
  • -E β€” same as --ic=euc-jp. Assume input codeset is EUC-JP ⭐
  • -J β€” same as --ic=jis. Assume input codeset is iso-2022-jp 🌈
  • -S β€” same as --ic=sjis. Assume input codeset is shift JIS πŸ¦‹
  • -Q β€” same as --ic=utf-16 --input-little-endian πŸ’œ
  • -Z β€” same as --ic=utf8 🫢

🌈 ISO-2022 Specific controls 🌈

Replaces G0-3 after setting up according to the specified input codeset by the assigned character set with this option. Note that this doesn't change any codeset properties of the original codeset, like language and encoding β€” it's just a lil glow-up for the planes, not a whole rebrand πŸ’…

  • --set-g0= charset name β€” Predefines the specified code set to plane 0 (G0). Also set to GL at initial state πŸ‘‘
  • --set-g1= charset name β€” Predefines the specified code set to right plane (G1). Also set to GR at initial state ⭐
  • --set-g2= charset name β€” Predefines the specified code set to right plane (G2) 🌸
  • --set-g3= charset name β€” Predefines the specified code set to right plane (G3) ✨

Supported char_set is as follows. 'o' means the codeset can be specified to set to the plane. 'x' means you can't. For unicode family codesets, this option is ignored. For other non-iso2022 categories, this option is not supported, and result is unpredictable πŸ’€

okay bestie, deep breath, because this next bit is the big codeset table and then a whole bunch of control switches πŸ’– the four little columns at the front are g0, g1, g2, g3, and each one is just o or x β€” I'm listing them all out so nothing gets lost, fr fr:

  • ascii β€” ANSI X3.4 ASCII Β· g0 o, g1 o, g2 o, g3 o
  • x0201 β€” JIS X 0201 (latin part) Β· g0 o, g1 o, g2 o, g3 o
  • iso8859-1 β€” ISO 8859-1 latin Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-2 β€” ISO 8859-2 latin Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-3 β€” ISO 8859-3 latin Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-4 β€” ISO 8859-4 latin Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-5 β€” ISO 8859-5 Cyrillic Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-6 β€” ISO 8859-6 Arabic Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-7 β€” ISO 8859-7 Greek-latin Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-8 β€” ISO 8859-8 Hebrew Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-9 β€” ISO 8859-9 latin Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-10 β€” ISO 8859-10 latin Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-11 β€” ISO 8859-11 Thai Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-13 β€” ISO 8859-13 latin Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-14 β€” ISO 8859-14 latin Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-15 β€” ISO 8859-15 latin Β· g0 x, g1 o, g2 o, g3 o
  • iso8859-16 β€” ISO 8859-16 latin Β· g0 x, g1 o, g2 o, g3 o
  • tcvn5712 β€” TCVN 5712 (Vietnamese) Β· g0 x, g1 o, g2 o, g3 o
  • ecma94 β€” ECMA 94 Cyrillic (KOI-8e) Β· g0 x, g1 o, g2 o, g3 o
  • x0212 β€” JIS X 0212:1990 Β· g0 o, g1 o, g2 o, g3 o
  • x0208 β€” JIS X 0208:1997 Β· g0 o, g1 o, g2 o, g3 o
  • x0213 β€” JIS X 0213 Plane 1:2000 Β· g0 o, g1 o, g2 o, g3 o
  • x0213-2 β€” JIS X 0213 Plane 2:2000 Β· g0 o, g1 o, g2 o, g3 o
  • x0213n β€” JIS X 0213 Plane 1:2004 Β· g0 o, g1 o, g2 o, g3 o
  • gb2312 β€” Simplified Chinese GB2312 Β· g0 o, g1 o, g2 o, g3 o
  • gb1988 β€” Chinese GB1988(latin) Β· g0 o, g1 o, g2 o, g3 o
  • gb12345 β€” Traditional Chinese GB12345 Β· g0 o, g1 o, g2 o, g3 o
  • ksx1003 β€” Korian KS X 1003(latin) Β· g0 o, g1 o, g2 o, g3 o
  • ksx1001 β€” Korian KS X 1001 Β· g0 o, g1 o, g2 o, g3 o
  • koi8-r β€” Cyrillic KOI-8R Β· g0 x, g1 o, g2 o, g3 o
  • koi8-u β€” Ukrainean Cyrillic KOI-8U Β· g0 x, g1 o, g2 o, g3 o
  • cns11643-1 β€” Traditional Chinese CNS11643-1 Β· g0 o, g1 o, g2 o, g3 o
  • viscii-r β€” RFC1496 VISCII (right plane) Β· g0 x, g1 o, g2 o, g3 o
  • viscii-l β€” RFC1496 VISCII (left plane) Β· g0 o, g1 o, g2 o, g3 o
  • cp437 β€” Microsoft cp437 (US latin) Β· g0 x, g1 o, g2 o, g3 o
  • cp737 β€” Microsoft cp737 Β· g0 x, g1 o, g2 o, g3 o
  • cp775 β€” Microsoft cp775 Β· g0 x, g1 o, g2 o, g3 o
  • cp850 β€” Microsoft cp850 Β· g0 x, g1 o, g2 o, g3 o
  • cp852 β€” Microsoft cp852 Β· g0 x, g1 o, g2 o, g3 o
  • cp855 β€” Microsoft cp855 Β· g0 x, g1 o, g2 o, g3 o
  • cp857 β€” Microsoft cp857 Β· g0 x, g1 o, g2 o, g3 o
  • cp860 β€” Microsoft cp860 Β· g0 x, g1 o, g2 o, g3 o
  • cp861 β€” Microsoft cp861 Β· g0 x, g1 o, g2 o, g3 o
  • cp862 β€” Microsoft cp862 Β· g0 x, g1 o, g2 o, g3 o
  • cp863 β€” Microsoft cp863 Β· g0 x, g1 o, g2 o, g3 o
  • cp864 β€” Microsoft cp864 Β· g0 x, g1 o, g2 o, g3 o
  • cp865 β€” Microsoft cp865 Β· g0 x, g1 o, g2 o, g3 o
  • cp866 β€” Microsoft cp866 Β· g0 x, g1 o, g2 o, g3 o
  • cp869 β€” Microsoft cp869 Β· g0 x, g1 o, g2 o, g3 o
  • cp874 β€” Microsoft cp874 Β· g0 x, g1 o, g2 o, g3 o
  • cp932 β€” Microsoft cp932 (Japanese) Β· g0 x, g1 o, g2 o, g3 o
  • cp1250 β€” Microsoft cp1250(Central Europe) Β· g0 x, g1 o, g2 o, g3 o
  • cp1251 β€” Microsoft cp1251 (Cyrillic) Β· g0 x, g1 o, g2 o, g3 o
  • cp1252 β€” Microsoft cp1252 (Latin-1) Β· g0 x, g1 o, g2 o, g3 o
  • cp1253 β€” Microsoft cp1253 (Greek) Β· g0 x, g1 o, g2 o, g3 o
  • cp1254 β€” Microsoft cp1254 (Turkish) Β· g0 x, g1 o, g2 o, g3 o
  • cp1255 β€” Microsoft cp1255 Β· g0 x, g1 o, g2 o, g3 o
  • cp1256 β€” Microsoft cp1256 Β· g0 x, g1 o, g2 o, g3 o
  • cp1257 β€” Microsoft cp1257 Β· g0 x, g1 o, g2 o, g3 o
  • cp1258 β€” Microsoft cp1258 Β· g0 x, g1 o, g2 o, g3 o

ok and then these three lil switches right after the table πŸ’…

  • --euc-protect-g1 β€” when you're in EUC input mode, this suppresses any sequences that try to set a charset to G1. Those sequences just get yeeted, discarded, gone.
  • --add-annon β€” adds an announcer for JIS X 0208:1997 to the X 0208 designate sequence. Heads up bestie, this one only works with iso-2022-based output.
  • --input-detect-jis78 β€” makes it tell JIS X 0208:1978 codeset and JIS X 0208:1997 codeset apart. By default those two charsets are both treated as X 0208:1997. This option is valid only when the input encoding is JIS (iso-2022-jp).

🌸 JIS X 0212(Supplement Kanji code) Support

  • --x0212-enable β€” okay so by default, skf does not output JIS X 0212 code in JIS/EUC mode. This option enables use of the JIS X 0212 part. Non-Japanese code, Shift_JIS variants, Unicode or KEIS output all ignore this option.
ngl this one is here for backward compatibility. It may not be supported in future versions, so don't get too attached 😭

πŸ’œ Unicode coding specific control options

skf-2.10 is conformed on Unicode 11.0 specification. periodt ✨

  • --use-compat, --suppress-compat β€” with --suppress-compat, skf substitutes characters in unicode compatibility planes (U+F900 - U+FFFD) for the appropriate characters in non-compatibility planes. If that substitution is enabled, these characters get converted to variants or undefined. With --use-compat, skf outputs characters in this area as they are. Default is --use-compat. Several codesets control this as a codeset feature (i.e. Use compatibility planes). See the codeset section πŸ‘€
  • --use-ms-compat β€” when the output is Unicode, this makes the Unicode map Microsoft Windows compatible). It only changes conversion for some symbols in JIS-Kanji, and adding --use-compat is recommended for roundtrip conversion. If you need more strict compatibility, try cp932w for input codeset.
  • --use-cde-compat β€” when the output is Unicode, makes the translation CDE standard codeset compatible.
  • --little-endian β€” when the output is UTF-16le/be, use little endian byte-order.
  • --big-endian β€” when the output is UTF-16le/be, use big endian byte-order.
  • --disable-endian-mark, --enable-endian-mark β€” when the output is UTF-16 or UTF-8, do not use/use byte order marking. To make UTF-16N, use this option together with --little-endian. By default, BOM is enabled for UTF-16 and disabled for UTF-8.
  • --input-little-endian β€” when the input is UTF-16le/be, assume the input is little endian byte-ordered.
  • --input-big-endian β€” when the input is UTF-16le/be, assume the input is big endian byte-ordered.
  • --endian-protect β€” do not use endian mark in the input stream. The endian mark is just discarded. This is off by default.
  • --limit-to-ucs2 β€” do not use the > 0x10000 area code in Unicode (i.e. limits code to the BMP area). This option doesn't limit the internal code range in skf. This is off by default.
  • --disable-cjk-extension β€” treat CJK extension A/B areas as undefined. This is off (i.e. these areas are enabled) by default.
  • --enable-cesu8 β€” enable CESU-8 input in the utf-8 codeset. Ignored for any other codesets.
  • --non-strict-utf8 β€” enable broken (decodable but not obeying specs.) utf-8 input.
bestie if you need this option, proceed with extra care πŸ™ˆ
  • --enable-nfd-decomposition, --disable-nfd-decomposition β€” enable/disable Unicode Normalized decomposition. Default is disabled.
  • --enable-nfda-decomposition, --disable-nfda-decomposition β€” enable/disable Apple-compatible Unicode Normalized decomposition. Default is disabled.
  • --oldcell-to-emoticon β€” convert the old cell-phone gaiji area in Unicode PUA to emoticon. Supported: NTT Docomo/AU emoticons. A reverse mapping is not supported.
  • --fix-ms-radical-bug β€” mscvrt has an infamous bug for Windows VISTA or later which converts some Kanji to Kanji radix. This option re-converts the radix area to appropriate Kanjis. This option is valid for Unicode output.
  • --old-nec-compat β€” enable old NEC kanji sequence (ESC-K,H). Needs the compile option --enable-oldnec at configuration.
  • --no-utf7 β€” assume the input codeset is NOT UTF-7 encoded Unicode. This option disables input utf7 testing.
  • --no-kana β€” assume the input codeset does NOT include JIS X 0201 kana.
  • --input-limit-to-jp β€” tell the detection mechanism that the input is some kind of Japanese codeset.

πŸ”₯ OUTPUT Conversions options πŸ”₯

skf is built to output a stream to stdout, buf an nkf-compatible file-encoding change option is also provided. slay πŸ’…

  • --overwrite[=SUFFIX], --in-place[=SUFFIX] β€” converts the encoding of the file(s) specified as input. --overwrite preserves the file change date. If the SUFFIX parameter is added, the input file is back-up'ed with a name appended with this SUFFIX.

skf has various features to fix output files appropriate in the local environment. Most of these are controlled by the extended control switches described in this section.

  • --use-g0-ascii β€” set G0(=GL) for the output encoding to ASCII, ignoring codeset designation.

πŸ¦‹ X-0201 Kana/latin conversions

ok so by default skf converts X-0201 kanas to X-0208 kanas. To output X-0201 kana as it is, use one of the following options. When the output is designated to EUC or SJIS, these three options enable X-0201 kana output by ways provided by each encoding. When Unicode output is specified, (equiv.) kana part output is controlled by --use-compat, not the following switches. Valid only when the output codeset is NOT Unicode family.

  • --kana-jis7 β€” use SI/SO locking shift sequence to designate X-0201 kana. This switch is valid for jis, jis-x0213 and cp50220 (i.e. cp50221) encoding. For other codesets, this option is ignored.
  • --kana-jis8 β€” output X-0201 kana using 8-bit code right plane. This switch is valid for jis and jis-x0213 encoding. For other codeset, this option is ignored.
  • --kana-esci, --kana-call β€” use ESC-(-I to designate X-0201 kana. This switch is valid for jis, jis-x0213 and cp50220 (i.e. cp50222) encoding. For other codeset, this option is ignored.
  • --kana-enable β€” if the output is EUC-JP or cp51932, use X-0201 kana with G2. If SJIS output, it's the same as --kana-jis8. When JIS output, it's the same as --kana-call.
  • --use-iso8859-1 β€” enable iso-8859-1 output. Iso-8859-1 is invoked to G1 and set to the GR plane.

🌈 URI/TeX format conversion feature options

With Unicode(tm) family output codings, skf outputs the non-ascii latin character part as it is, but with other output codings, skf converts these characters using the following rules:

  1. If a code is defined in a specified output codeset, the specified code point is used for output.
  2. If one of the following html convert modes are enabled (i.e. --convert-html, --convert-sgml) and the code is defined in html/sgml codeset, it is converted to entity-reference or codepoint reference.
  3. If tex convert mode is enabled and the code is defined in a tex expression, it is converted to tex format.
  4. If the code is a kind of combined ligatures, it is shown by a set of characters.
  5. A kind of replacement character is shown, with warning.
  • --convert-html --convert-sgml--convert-xml β€” enable html convert mode. This mode is cleared by --reset. These two options are synonyms, and are treated as same option.
  • --convert-html-decimal β€” turns ON html code-point decimal convert mode ✨ heads up bestie, this mode gets cleared by --reset.
  • --convert-html-hexadecimal β€” turns ON html code-point hexadecimal convert mode πŸ”₯ and yeah, --reset clears it too.
  • --convert-tex β€” TeX convert mode, ON πŸ’… cleared by --reset.
  • --convert-perl β€” Perl5 literal convert mode, ON πŸŽ€ cleared by --reset.
  • --convert-java β€” Java literal convert mode, ON β˜• cleared by --reset.
  • --convert-python β€” Python literal convert mode, ON 🐍 cleared by --reset.
  • --use-replace-char β€” in Unicode, use the unicode replacement character (U+fffc) for an undefined character 🫢.

πŸ”₯ EXTENDED OPTIONS πŸ”₯

πŸ’Œ Encoding/Decoding control options

  • --decode=encoding scheme
  • --encode=encoding scheme β€” pick a decoding/encoding scheme for your input stream, bestie 🌸

The decoding schemes skf has your back with are: hex, mime, mime_q, mime_b, uri, ace, hex_perc_encode, base64, qencode, rfc2231, rot and none. Each one means, in order: CAP hex-code, mime, mime Q-encoding, mime B-encoding, uri character reference, ACE punycode, uri percent notation, base64, Q-encoding, rfc2231 and rot13/47 respectively πŸŒ€ and none means no decode at all, periodt.

For encoding, the supported squad is hex, mime_b, mime_q, uri, ace, cap, hex_perc_encode, base64 and none. EBCDIC related codesets and some already ascii-encoded codesets (like UTF-7) can NOT do output with encoding β€” not supported, sorry bestie 😭

Only ONE decode/encode option is valid at a time, and if you specify more than one, the last one wins πŸ‘‘ When one of the mime decodings is specified, base text is assumed to be EUC encoding unless you say otherwise. Except rot, which assumes the input stream is Shift_JIS, EUC or iso-2022-jp β€” all these encodings assume the input stream is ascii (as defined in RFC2045). Some encodings may co-exist with encoding, but that is not guaranteed πŸ™ˆ And especially: if the input is UTF-16/UCS2 code, these encodings are ignored in skf. no cap.

  • --mime-ms-compat β€” treats japanese generic codesets as Microsoft cp932 compatible πŸ’€ More specifically, with this option skf treats iso-2022-jp as cp50220, euc-jp as cp51932 and Shift_JIS as cp932w. the way it just relabels everything.
  • --mime-persistent β€” so normally skf detects address-like strings and excludes them from mime encoding. This option disables that behavior 🚫 Default in nkf-compatible mode.
  • --mime-limit-aware β€” during address-like string detection, skf respects the character count limits for a line πŸ“ very mindful.

βœ‚οΈ Shortcut

  • -m β€” same as --decode=mime
  • -mB β€” same as --decode=mime_b
  • -mQ β€” same as --decode=qencode
  • -m0 β€” same as --decode=none
  • -M β€” same as --encode=mime_b
  • -MB β€” same as --encode=base64
  • -MQ β€” same as --encode=qencode

πŸ›οΈ End of line control options

  • --lineend-thru β€” output the end-of-line code exactly as it is 🫢 Also outputs ^Z code as it is. This is the default, bestie.
  • --lineend-cr --lineend-mac-Lm β€” use CR as the end-of-line code. Also deletes ^Z code from the input stream πŸ—‘οΈ
  • --lineend-lf --lineend-unix-Lu β€” use LF as the end-of-line code. Also deletes ^Z code from the input stream πŸ—‘οΈ
  • --lineend-crlf --lineend-windows-Lw β€” use CR+LF as the end-of-line code. Also deletes ^Z code from the input stream πŸ—‘οΈ Heads up: this option doesn't preserve the original order of cr and lf πŸ™ˆ
  • --input-cr β€” assume the input stream uses CR as the end-of-line code.
  • --input-lf β€” assume the input stream uses LF as the end-of-line code.
  • --input-crlf β€” assume the input stream uses CR+LF as the end-of-line code.
  • -F[line_length[-kinsoku]]
  • -f[line_length[-kinsoku]] -f[line_length[+kinsoku]] β€” wrap input lines by line_length columns πŸ“ The f option deletes CR/LF's in the input, and the F option doesn't delete them. For Japanese convention, both gyoutou-kinsoku (by burasage-gumi) and gyoumatsu-kinsoku (by oidasi-gumi) is supported πŸ’– The burasage-length is controlled by the kinsoku option. Default value for line_length is 66, and must be < 1000. Default value for kinsoku is 5, and must be <= 10. With the f option, skf autodetects paragraph and retains some CR/LF. The 2nd f option format (with +) disables this behaviour πŸ”•

In nkf compatible mode, some fold behaviors change as follows:

  1. Default line_length is set to 60, and kinsoku value is 10.
  2. alpha numeric characters become gyoutou-kinsoku characters.

πŸ“ File control options

  • --filewise-detect --force-reset β€” reset and re-detect the input code set at the start of each file πŸ”„
  • --linewise-detect β€” reset and re-detect the input code set at the start of each line πŸ”„

🧩 Compatibility options

  • --nkf-compat β€” interpret the following options as nkf compatible manners 🫑 -l, -d, -c, -x, -X, -w and -W work as nkf2.x. -f and -F behavior is changed as shown above. -T, -i, -o is not supported. Most of the other nkf options and switches also work like nkf, except in case of error.
  • --skf-compat β€” interpret the following options as skf-native manners πŸ’…
  • -r β€” nkf-compatible rot. Works only with --nkf-compat mode ⚠️ Allowed input encodings are limited to JIS/Shift_JIS/EUC.
  • -h[123] --hiragana --katakana --katakana-hiragana β€” -h, -h1 and --hiragana converts all kanas to hiragana 🌸 -h2 and --katakana convert all kanas to katakana 🌊 -h3 and --katakana-hiragana swap katakana and hiragana πŸ”
  • --nkf-help β€” show option difference/compatibility between skf and nkf πŸ“‹
  • --in-place[=SUF] --overwrite[=SUF] β€” replace specified file with converted codeset πŸ’Ύ overwrite retains the file create time stamp. If a suffix is given, the suffix is added to the output file name and the input file is not removed πŸ‘€

🧚 Lightweight language specific options

skf plugin for lightweight language has a subset of options. More specifically, file input/output related options (-b, -u, --overwrite, --in-place, --filewise-detect, --linewise-detect, --show-filename, --suppress-filename) and UTF-16 output is disabled (except ruby or python3). The calling methods differ depending on LWL, but each extension has two parameters: a option string and a string to convert. From 2.1.15, ruby is not supported πŸ’”

🐍 Python-3.x specific options

Since native codeset representation in python3.x is ATIN-1/UCS2/UCS4, skf behaves differently with the output codeset option. If the output codeset is either ASCII, UTF-16 or UTF-32 (in wide mode), skf returns a Unicode object, and for all other codesets skf returns a binary array object. The following options change this behavior. Codesets assumed as ascii (UTF-7) and MIME encoded strings are returned as strings.

  • --py-out-binary β€” use psuede unicode binary array stream to output. BOM is enabled ✨
  • --py-out-string β€” use binary array object on ASCII, UTF-16/32 output. This is default 🫢 skf accepts either a binary array or an unicode object for input. BOM is disabled.

πŸŽ›οΈ Misc. Control options

  • --disable-space-convert --enable-space-convert β€” skf converts an ideographic space into two ascii spaces πŸ‘€ the disable option disables, and the enable option enables this behavior. Default is disabled.
  • --html-sanitize β€” convert several characters in an HTML document to entity reference expression. Specifically, !#$&%()/<>:;?Β΄ are escaped by entity-references 🧼
  • --filewise-detect --force-reset β€” if multiple input files are given, detect input codeset for each file πŸ“‚
  • --linewise-detect β€” detect input code line-wise. Note this option weakens code detect correctness ⚠️
  • --reset β€” reset all flags specified by extended controls and enviroment variables πŸ”„
  • --inquiry --guess β€” skf detects code and outputs the detect result to stdout. No filtering output is performed πŸ” If multiple input files are given, --show-filename is automatically enabled.
  • --hard-inquiry β€” similar as inquiry, but reports both code and an end-of-line character πŸ•΅οΈ
  • --suppress-filename β€” when inquiry (--inquiry) is on, this option disables file name output. This option overrides --show-filename πŸ™…
  • --show-filename β€” when inquiry (--inquiry) is on, this option adds each file name to output πŸ“›
  • --invis-strip β€” delete all escape sequences not belonging to ISO-2022 code extension. This is intended to replace the invisstrip command bundled in the inews package 🧹
  • -I β€” warn if input has unassigned code points ⚠️
  • -v β€” print version information and exit πŸ‘‹
  • --help β€” print brief help and exit.
  • --show-supported-codeset β€” display supported codesets (input) and exit. Both canonical names (left side) and detailed names are shown. This canonical name can be used as MIME charset and also as ic-option code specification πŸ“œ
  • --show-supported-charset β€” display supported character sets (output) and exit. Both canonical names and detailed names are shown. Some charsets with special treatments (i.e. meaningless as set-g* parameters) intensionally lacks addressable cnames πŸ™ƒ

πŸ“‚ FILES πŸ“‚

  • /usr/(local/)share/skf/lib/ (Unices)
  • /Program Files/skf/share/lib (MS Windows)

These directories are where external codeset conversion tables go πŸ—‚οΈ The location that current skf assumes are shown by -h option.


πŸ‘‘ AUTHOR πŸ‘‘

skf is written by Seiji Kaneko (efialtes@osdn.jp) based on idea from nkf written by Itaru Ichikawa (ichikawa@flab.fujitsu.co.jp). X 0213 code table is derived from work of earthian@tama.or.jp πŸ’– Some codeset mapping is derived from various sources. Detailed origin is shown in copyright document included in this distribution. Unicode Database is copyrighted(c) by Unicode(R), Inc.


🫢 ACKNOWLEDGEMENT 🫢

skf is inspired by works or requests by shinoda@cs.titech, kato@cs.titech, uematsu@cs.titech, void@global ohta@ricoh, Hinata(HKE) Ashizawa(CRL) Kunimoto(SDL) Oohara(Univ of Kyoto), Jokagi(elf2000) and Naruse (at osdn.jp). Thanks 🌈

πŸ’€ BUGS AND LIMITATIONS πŸ’€

Okay bestie, real talk β€” here's what skf can't do (or does in a sus way). No cap, we're keeping it 100. 🫢

  1. skf can handle mixed coding with some limitations. But ngl, code detection tends to fail for mixed code, and giving explicit input code set is strongly encouraged if codeset is known beforehand. In case of need, --linewise-detect option may help, but code detecting will more likely fail. 😭
  2. skf implements ISO-2022 with following exceptions:
    • i) GL 0x20 is always space. Even when 96-character codeset is invoked to GL.
    • ii) Sequences for setting codes to C1 and C2 are ignored.
    • iii) If unknown sequence is given to G0, G0 is set to ascii, and locking/single shift is cleared. Unknown sequence call to set to G1-G3 is just ignored. Private charset is also not supported and is ignored.
    • iv) Sequences for 96 character multibyte coding is ignored (Currently, no codeset is registered).
    • v) Calling UTF-8, UTF-16 coding system from iso-2022 is supported, and returns to previous coding system by standard return. Callings and returns to/from other coding schemes are ignored.
    • vi) For supporting some of cellular phone glyphs, several private (not registered) codesets are defined in skf, and can be called by appropriate sequences.
  3. Error output coding is controlled by LOCALE environment variables in UN*X system. skf doesn't take care of situations like stdout and stderr are redirecting into a same stream. Such case should be handled by user side. πŸ™ˆ
  4. skf converts KEIS/JIS X 0213 code using CJK-extension B area and CJK compatibility area. For this reason, X 0213 and KEIS convert result varies depending on --use-compat and --limit-to-ucs2 switches. πŸ”„
  5. JIS X 0207:1979 is not supported. JIS X 0211:1987 is designed to be supported (i.e. common terminal control sequence will be transparently passed to output). ✨
  6. Even if unbuffer option (-u) is specified, some code-translation related bufferings are still performed (in MIME, kana, VIQR etc.). πŸ’€
  7. skf-1.9x or later recognizes and handles languages in iso639-1 (alpha 2). iso639-2 is not supported as a valid language set. πŸ—£οΈ
  8. Unicode IVS is not supported. Sequences are just discarded. πŸ—‘οΈ
  9. skf-1.9x or later does not retain Macintosh RLO-ordered character property. Codesets with this kind of codes are not supported. 🍎
  10. CNS11643 4th, 5th, 6th planes are not supported. 🚫
  11. In python 3 extension, a detected codeset by inquiry for input unicode strings are always UTF-32be. 🐍
  12. In lightweight language extension except ruby and python, UCS2/UTF-16 are not supported. πŸ’Ž

πŸ“ Notes πŸ“

More tea, bestie! β˜• Here's the 411 on some behind-the-scenes stuff. πŸ’…

  1. Extended options are changed extensively since skf-1.9. Some archaic options (eg. -B, -@ and -r) have been deleted from this version. πŸ‘‹
  2. skf is originally forked project from nkf, but doesn't contain any nkf codes now. Copyright notice is retained by honor. 🫑
  3. From version 1.9, default Japanese character set assumed by skf has changed to JIS X 0208:1990 with Microsoft Japanese Windows gaiji (i.e. CP932). πŸ‡―πŸ‡΅
  4. Code autodetection is not perfect by design. If it has failed to detect input code properly, please give input code information explicitly. πŸ”
  5. Some ligatures in Unicode, cp932 gaiji and KEIS83 are converted using JIS X 0124 and other convention. During this conversion, its byte length is not preserved. ⚠️
  6. skf is intended to pass ANSI compatible terminal control codes transparently, but this is not guaranteed. πŸ€·β€β™€οΈ
  7. nkf's -i and -o options works only in nkf-compat mode. It is obsolete option in 1.97, and valid only when iso-2022-jp and without considering output codeset specifications. πŸ‘΅
  8. For unconverted character, skf uses geta and undefined character as --use-replace-char option. If output codeset doesn't contain geta code, skf prefers 'black square character', then uses . respectively. ⬛
  9. There are some undocumented options. These options should be considered as highly experimental. πŸ§ͺ
  10. In lineend_thru mode and using folding, skf remembers order of cr and lf appears in stream, and use that order. For this design, if skf needs to output line-end character before any line-end character appears in input stream, input order may not be preserved. πŸ”„
  11. NKF-compatibility
    1. --prefix, some --fb's and --no-best-fit-chars are not supported. Error behaviors are not compatible.
    2. -r option and --decode=rot is different. See each option description.
    3. MSDOS (and -T), --exec-in and --exec-out are not supported. -O is supported.
    4. MIME decoding/encoding handling behaviors differ in various ways.
    5. lineend conversion acts differently. Results may not be same for text with multiple lineend characters.
    6. detected codeset name is not compatible with nkf. --help and --version return different results.
    7. in-place and overwrite suffix with * is not supported.
  12. Conversion to NYUUKAN GAIJI is as follows:
    1. Kanji codes in JIS X0208(1997), JIS X0212(1990), JIS X0213(2004/2012), Houmusho-kokuji No.582 beppyou No.1 are sent to output as it is.
    2. Kanji codes in beppyou No.4-2 leftmost columns are converted to the first priority character in the table. If the second priority characters appear, the codes are sent to output as it is.
    3. Other kanji codes are converted as undefined codes. See above conversion method. Non-kanji codes (latins, glyphs etc.) are sent to output as it is.
  13. ARIB B24 compatibility
    1. Input only. ARIB B24 output is not supported.
    2. Neither international encoding nor X0213 extension are supported.
    3. Macro define sequences are suppressed. These sequences are recognized and discarded.
    4. Without specifying arib codeset, skf treats Arib-defined codepage as follows:
      • i) private codepage are supported. ascii/jis x-0201 0x5f is not modified.
      • ii) macro define/invoke and rpc invoke does not work. These characters are discarded.
  14. option mnemonic table for -v option
    • AA: aware ascii-art in code detection
    • DBG: Debugging feature enabled
    • F64: Large file enabled(default)
    • NE: Environment variable handling disabled
    • NFJ: suppress fj-newsgroup convension
    • NLS: Native language messaging enabled(default)
    • NN: detect skf is called under nkf name
    • OMST: Have mkstemp
    • PEP: Python3 PEP393 support enabled
    • SG: Slow getc enabled
    • SPNC: Space convert disabled.
    • STT: Use Static codeset table
    • UFY_A_J: Unify JIS x-0201 to ascii
    • UID/EUID: Have UID/EUID.
    • ULM: UCS2 generic latin support.
    • WIN32: Windows environment.
  15. feature mnemonic table for -v option
    • 98: old-nec-compat (ESC-H/ESC-K) feature enabled
    • ACE: punycode support enabled
    • ARIB: ARIB B24 support enabled
    • FD: fold feature enabled
    • KD: KEIS90 auto-detect enabled
    • KX: KEIS90 extra region enabled
    • MIMEREC: Mime recovery feature anabled
    • NFD: Unic*de decompose enabled
    • ROT: rot13/47 support enabled
    • UK: UTF16 hankaku-kana disabled
    • UN: UTF16 normalize enabled
    • ONKF: nkf old -i, -o option enabled
    • LE_*: lineend handling.

πŸ“’ Notice πŸ“’

Legal bestie moment, sorry! πŸ˜… Gotta say the trademark tea:

Unicode(TM) is a trademark of Unicode, Inc. Microsoft and Windows are registered trademarks of Microsoft corporation. Macintosh is a registered trademark of Apple Inc. Vodafone is a trademark of Vodafone K.K. Other names and terms may be trademarks or registered trademarks of their respective owner. Trademark symbol (TM) may be omitted in this manual page. πŸ’…


That's the whole vibe, bestie! This was SKF(1) from 10/Aug/2018. Stay iconic, stay slaying! πŸ’–βœ¨πŸŒˆ

πŸ“¦ where this page comes from