SKF(1)
π skf(1) β General Commands Manual π
okay bestie, buckle up, because this is the manual page for skf and she is that girl. it's a simple Kanji Filter (v2.1) and we are going to read the whole thing together, no skipping, no skimming. β¨
πΈ NAME πΈ
skf β simple Kanji Filter (v2.1)
β¨ SYNOPSIS β¨
skf [-EIJKNQRSXZbehjknqrsuvxz] [ long_format_options ] [infiles..]
π¦ DESCRIPTION π¦
so. skf is yet another i18n-capable kanji-filter, and she was literally designed for reading alllll the various CJK-coded files floating around on the Net. what she does: she takes input kanji texts or streams, converts them into a character stream using a designated codeset, and outputs them to standard output. basically, she's built to be a versatile filter for reading documents in a bunch of different code sets, and she does not do features that aren't related to code conversion. stay in your lane, periodt. π
like nkf, skf automatically recognizes an input file code when it's a kind of ISO-2022 compliant code, and she also detects EUC-variant codes if the input file is Japanese text without X 0201 kanas. skf 2.1 can read various iso-2022 compliant character sets, including JIS Kanji codes (X 0208, X 0212 and X 0213), EUC encoding (euc-jp (with X 0213 support), euc-cn, euc-kr and euc-tw), ISO Europian latins (ISO-8859-1 to 11, 13/14/15/16) and many regional character sets. she can also read some non-iso2022 compliant sets, including Microsoft Shift-JIS code, KOI-8-R/U, GB2312 (HZ), big5, VISCII(rfc1456, include VIQR), Unicode standard (UCS2/UTF-16, UTF7 and UTF8), some of MS codesets (cp1250 etc.) and some other vendor specific codes (KEIS83, JEF etc). she contains multitudes fr fr. π
supported output character sets of skf are more limited, but still include X 0208/X 0212/X 0213 JIS, X 0201 JIS, ASCII, Microsoft Shift-JIS, EUC-jp/-kr/-cn, HZ, iso-2022-jp/kr, big5, VISCII and Unicode.
skf also provides some basic decoding features for some common encodings including MIME, Punycode and URI codepoint. Unicode decomposition feature is also supported since 1.96. π
as noted above, skf is designed to convert input text into some kind of human-readable forms under a local environment (i.e. codeset), and has several extra conversion features like GNU recode type folding. such conversions include Windows/Macintosh specific code swaps and old-new jis glyph changes, html-format/TeX format conversion and variant unifications. iconic behavior honestly.
skf also can be compiled as an extension of some lightweight languages. see README.txt for details. π
if one or more file names are given, skf reads the files and outputs the converted stream to stdout. if no file names are given, input is taken from stdin and output is also stdout. OPTIONS are taken from environment variables SKFENV, skfenv and command line, respectively in this order. environment variables are not used when skf is running as a priviledged user. skf does not use LOCALE-related environment variables for conversions, but output error messages are controlled by given LOCALES.
β οΈ lowkey important: those env vars get skipped entirely when you're a priviledged user. noted.
π CODESET OPTIONS π
skf is written from scratch, and inherits no code from nkf. however, skf is intended to be a drop-in replacement for nkf(v1.4) and has a similar commonly-used nkf option set.
skf 2.1 recognizes the following options. defaults are all off if not explicitly specified. (yes bestie, everything starts off. you gotta ask.) π
π§Ί buffering control π§Ί
-bβ use buffered output. this is the default.-uβ use unbuffered output. code detection feature is disabled when this option is on.
π Input/Output codeset options π
--ic=input_code_set β specify input codeset is input_code_set. possible candidates are shown below.--oc=output_code_set β specify output codeset is output_code_set. possible candidates are shown below. default codeset in the distribution package iseuc-jp, but it depends on compile option. the default codeset is shown byskf -h.
π§ Supported codeset π§
skf recognizes the following codesets as an input/output codeset. these codeset names are case insensitive, and minus (-) and underscore (_) is ignored. note that iso-2022 escape-based input codeset (registered to IANA) is recognized automatically, even when non-iso2022 codeset (except Unicode and B-Right/V) is specified.
in the little flag pair below, o in the in-column means the named codeset can be specified as input, and x means the named codeset is not for input. the output-column is the same, except it's for output. slay. π
iso8859-1(in o, out o) β ascii + iso-8859-1 (latin-1)iso8859-2(in o, out o) β ascii + iso-8859-2 (latin-2)iso8859-3(in o, out o) β ascii + iso-8859-3 (latin-3)iso8859-4(in o, out o) β ascii + iso-8859-4 (latin-4)iso8859-5(in o, out o) β ascii + iso-8859-5 (Cyrillic)iso8859-6(in o, out o) β ascii + iso-8859-6 (Arabic)iso8859-7(in o, out o) β ascii + iso-8859-7 (Greek)iso8859-8(in o, out o) β ascii + iso-8859-8 (Hebrew)iso8859-9(in o, out o) β ascii + iso-8859-9 (latin-5)iso8859-10(in o, out o) β ascii + iso-8859-10 (latin-6)iso8859-11(in o, out o) β ascii + iso-8859-11 (Thai)iso8859-13(in o, out o) β ascii + iso-8859-13 (Baltic Rim)iso8859-14(in o, out o) β ascii + iso-8859-14 (Celtic)iso8859-15(in o, out o) β ascii + iso-8859-15 (Latin-9)iso8859-16(in o, out o) β ascii + iso-8859-16koi-8r(in o, out o) β koi-8r (Russian)koi-8u(in o, out o) β koi-8r (Ukraina)cp1251(in o, out o) β Cyrillic latin MS cp1251jis(in o, out o) β iso-2022-jp (rfc1496 7bit JIS)iso-2022-jp-x0213(in o, out o) β iso-2022-jp-3 (JIS X 0213:2000) a.k.a.jis-x0213jis-x0213-strict(in o, out o) β iso-2022-jp-3-strictiso-2022-jp-2004(in o, out o) β iso-2022-jp-2004(JIS X 0213:2004) a.k.a.jis-x0213-2004oldjis(in o, out o) β iso-2022-jp-1978(JIS X 0208:1978)cp50220(in o, out o) β Microsoft codepage 50220cp50221(in o, out o) β Microsoft codepage 50221cp50222(in o, out o) β Microsoft codepage 50222euc-jp(in o, out o) β EUC-encoded JIS X 0208:1997euc-x0213(in o, out o) β EUC-encoded JIS X 0213:2000euc-jis-2004(in o, out o) β EUC-encoded JIS X 0213:2004cp51932(in o, out o) β EUC-encoded Microsoft codepage 932euc-kr(in o, out o) β EUC-encoded KS X 1001 Korianeuc7-kr(in o, out o) β 7bit EUC-encoded KS X 1001 Korianuhc(in o, out o) β Unified hangle (Windows cp949)johab(in o, out o) β KS X 1001-johab Korianeuc-cn(in o, out o) β EUC-encoded GB2312 Chineseeuc7-cn(in o, out o) β 7bit EUC-encoded GB2312 Chinesehz(in o, out o) β HZ-encoded GB2312 Chineseeuc-tw(in o, out o) β EUC-encoded CNS 11643 Chinesegb12345(in o, out o) β EUC-encoded GB12345 Chinesegbk(in o, out o) β GB2312 Extension(cp936) Chinesegb18030(in o, out o) β GB18030 chinesebig5(in o, out o) β BIG5 (with Eten extension + EURO)cp950(in o, out o) β BIG5 (Microsoft cp950 + EURO)big5-hkscs(in o, out o) β BIG5 with HKSCSbig5-2003(in o, out o) β BIG5-2003big5-uao(in o, out o) β BIG5-Unicode at Onsjis(in o, out o) β Shift-jis (Microsoft cp943)shiftjis-x0213(in o, out o) β Shiftjis-encoded JIS X 0213:2000shiftjis-2004(in o, out o) β Shiftjis-encoded JIS X 0213:2004sjis-docomo(in o, out o) β Shiftjis-encoded with NTT Docomo emoticons.sjis-au(in o, out o) β Shiftjis-encoded with AU emoticons.sjis-softbank(in o, out o) β Shiftjis-encoded with SoftBank emoticons.oldsjis(in o, out o) β Shift-jis (JIS X 0208:1978)cp932(in o, out o) β Shift-jis-encoded MS cp932cp932w(in o, out o) β Shift-jis-encoded MS cp932 with MS compatibilityviscii(in o, out o) β VISCII (rfc1456) Vietnamiseviqr(in o, out o) β VISCII (rfc1456-VIQR) Vietnamisekeis(in o, out o) β Hitachi KEIS83/90jef(in o, out x) β Fujitsu JEF (basic support only)ibm930(in o, out x) β IBM EBCDIC DBCS Japaneseibm931(in o, out x) β IBM EBCDIC DBCS Japanese w.latinibm933(in o, out x) β IBM EBCDIC DBCS Korianibm935(in o, out x) β IBM EBCDIC DBCS Simpl. Chineseibm937(in o, out x) β IBM EBCDIC DBCS Trad. Chineseunicode(in o, out o) β Unicode(TM) UTF-16LEunicodefffe(in o, out o) β Unicode(TM) UTF-16BEutf7(in o, out o) β Unicode(TM) UTF-7utf8(in o, out o) β Unicode(TM) UTF-8utf8-bom(in o, out o) β Unicode(TM) UTF-8 with BOMutf7-imap(in o, out o) β IMAP modified Unicode(TM) UTF-7 (RFC2060)mutf8(in o, out o) β Java modified Unicode(TM) UTF-8cesu8(in o, out o) β CESU-8 (Unicode Technical Report #26)nyukan-utf-8,nyukan-utf-16(in x, out o) β Nyukan-moji(Japanese nyukoku-kanrikyoku gaiji). Encoding is utf-8 and utf-16 respectively.arib-b24(in o, out x) β ARIB B24 8-bit JIS-basedarib-b24-sj(in o, out x) β ARIB B24 8-bit SJIS-basedtransparent(in x, out o) β Transparent mode (see below)x-iscii-de(in o, out x) β India ISCII-91(IS13194:1991)asmiscii-8(in o, out x) β Armenian ARMISCII 8geostd8(in o, out x) β Geogian Geogstd 8mik(in o, out x) β Burgarian MIKtscii(in o, out x) β Tamil TSCII 1.7locale(in o, out o) β codeset specified in locale. see below.
π Codeset explanations π
iso-8859-*β when specified as output, G0 = GL is ascii and G1 = GR isiso-8859-*. 8bit encoding is used.iso-2022-jp,jisβ encoding isiso-2022-jp-2(RFC1496). G0 = GL is JIS X 0201 roman, G1 = GR is JIS X 0201 kana, G2 isiso-8859-1and G3 is JIS X 0212:1990 Supplementary Kanji.jis-x0213,iso-2022-jp-3β encoding isiso-2022-jp-3(JIS X 0213:2000 based). G0 = GL is JIS X 0201 roman, for output, G1 = GR is JIS X 0201 kana, G2 isiso-8859-1and G3 is JIS X 0213 plane2 Kanji.jis-x0213-strictβ encoding is subset ofiso-2022-jp-3-strict(uses Plane 1 only). for output, G0 = GL is JIS X 0201 roman, G1 = GR is JIS X 0201 kana, G2 isiso-8859-1and G3 is not set. output code using JIS X 0208 whenever possible. JIS X 0213 input is automatically recognized.jis-x0213-2004,iso-2022-jp-2004β encoding isiso-2022-jp-2003:2004. for output, G0 = GL is JIS X 0201 roman, G1 = GR is JIS X 0201 kana, G2 isiso-8859-1and G3 is JIS X 0213 plane2 Kanji.oldjisβ encoding isiso-2022-jpusing old JIS X 0208:1978). G0 = GL is JIS X 0201 roman, G1 = GR is JIS X 0201 kana, G2 isiso-8859-1and G3 is JIS X 0212 Supplementary Kanji.euc-jp,eucβ encoding is 8-bit EUC using JIS X 0208:1997 character set. G0 = GL is ascii, G1 = GR is JIS X 0208, G2 is JIS X 0201 kana and G3 is JIS X 0212 Supplementary Kanji.euc-x0213,euc-jis-2003β encoding is 8-bit EUC-based JIS X 0213:2000. G0 = GL is ascii, G1 = GR is X 0213:2000 plane 1, G2 isiso-8859-1and G3 is JIS X 0213:2000 plane2 Kanji.euc-jis-2004β encoding is 8-bit EUC-based JIS X0213:2004. G0 = GL is ascii, G1 = GR is X0213:2004 plane 1, G2 isiso-8859-1and G3 is JIS x0213:2004 plane2 Kanji.euc-krβ encoding is 8-bit EUC using KS X 1001 Wansung character set. G0 = GR is KS X1003, G1 = GR is KS X1001, G2 and G3 is not set.euc7-kr,iso-2022-krβ it's giving iso-2022-kr (rfc1557) energy: 7-bit EUC built on the KS X 1001 Wansung character set.G0 = GRis KS X1003,G1is KS X1001, andG2andG3are not set, bestie πeuc-cnβ 8-bit EUC using the GB 2312 simplified chinese character set.G0 = GRis ASCII,G1 = GRis GB2312,G2andG3are not set β¨euc7-cnβ same vibe but 7-bit: EUC on GB 2312 simplified chinese.G0 = GRis ASCII,G1 = GRis GB2312,G2andG3are not set πΈhzβ HZ encoded (rfc1842) GB 2312 simplified chinese character set.G0 = GRis ASCII,G1 = GRis GB2312,G2andG3are not set π¦euc-twβ EUC encoded CNS11643 Plane1/2 traditional chinese character set. It's a subset of iso-2022-cn, lowkey iconic.G0 = GRis ASCII,G1 = GRis CNS11643 plane 1,G2is CNS11643 plane 2, andG3is not set πgb12345β 8-bit EUC using GB 12345 (GBF) traditional chinese character set.G0 = GRis ASCII,G1 = GRis GB12345,G2andG3are not set πgbk,cp936β GBK simplified chinese character set.G0 = GRis ASCII andG1 = GRis GBK.G2andG3are not set πgb18030(experimental) β GB18030 (ibm-1392, Windows cp54936) chinese character set. Uses ASCII as the latin part βbig5β Big5 traditional chinese character set with the ETen extension. Includes Euro mapping. Uses ASCII as the latin part πcp950β Microsoft cp950-Big5 traditional chinese character set. Uses ASCII as the latin part πbig5-hkscs(experimental) β cp950-Big5 traditional chinese character set with the HKSCS extension. Uses ASCII as the latin part β¨big5-2003(experimental) β Big5-2003 Taiwanese standard traditional chinese character set. Uses ASCII as the latin part π¦big5-uao(experimental) β Big5-UAO (http://uao.cpatch.org) traditional chinese character set. Uses ASCII as the latin part πΈVISCII(experimental) β Vietnamise VISCII (rfc1456) character set. Not TCVN-5712, okurrr πVIQR(experimental) β Vietnamise VISCII character set with VIQR encoding (rfc1456) πsjisβ Shift-encoded JIS X 0208:1997 character set. Note that this is notcp932, just so we're clear. Uses JIS X 0201 latin as the latin(GL) part πsjis-x0213,shift_jis-2000β Shift-encoded JIS using the JIS X 0213:2000 character set πsjis-x0213-2004,shift_jis-2004β Shift-encoded JIS using the JIS X 0213:2004 character set. 10 newly defined character added, but the Unicode mapping is the same as JIS X 0213:2000. Uses JIS X 0201 latin as the latin(GL) part βsjis-cellular(experimental) β Shift-encoded JIS X 0208:1997 character set with NTT Docomo/Vodafone(SoftBank) cellular phone glyph mapping. Output is not supported, so don't try it π«Άcp932,cp932wβ Microsoft SJIS cp932 with the NEC/IBM gaiji area, based on Windows XP mapping. Uses ASCII as the latin(GL) part.--use-compatand--use-ms-compatis automatically enabled.cp932wprovides furtherWideCharToMultiBytecompatibility π₯cp51932β Microsoft EUC-based cp51932 with the NEC/IBM gaiji area, based on Windows XP mapping. Uses ASCII asG0and JIS X 0201 kana as the EUCG2part.G3is not used for output, and JIS X 0212:2000 as input.--use-compatand--use-ms-compatis automatically enabled πcp50220,cp50221,cp50222β Microsoft JIS-based cp50220, cp50221, cp50222 with the NEC/IBM gaiji area, based on Windows XP mapping. For input,skfaccepts cp50220, 50221 and 50222. Note that this codeset is NOT compatible with iso-2022, fr fr. Uses ASCII as the default character set.--use-compatand--use-ms-compatis automatically enabled πoldsjisβ Microsoft SJIS (JIS X 0208:1978 a.k.a. old JIS). Uses JIS X 0201 latin as the latin(GL) part πΈjohabβ KS X1001(Johab) character set. Uses KS X1003 latin as the latin(GL) part π¦uhcβ UHC (cp949) character set. Uses ASCII as the latin(GL) part β¨unicode,unicodefffe,utf16,utf16leβ Unicode UTF-16 (v15.0). Input/Output default byte-endian is little forunicodeand big forunicodefffe, and the input byte order mark is recognized.utf16andunicodefffeis big-endian.utf16leandunicodeis little endian. Output includes the endian mark by default unless--disable-endian-markis specified. Output range is within UTF-32 with surrogate pair unless--limit-to-ucs2is specified. Note thatucs2is not supported within the lightweight language extension in both in and output, because of SWIG's passing data structure limitation. Specify toucs2will generate error πutf8β UTF-8 encoded Unicode (v15.0). Output doesn't include byte order mark unless--enable-endian-markis specified. Output range is within UTF-32 unless--limit-to-ucs2is specified. By default, CESU-8 is not accepted as input. Option--enable-cesu8enables CESU-8 input for the utf-8 converter. CESU-8 output is not supported. For UTF-8, the endian mark (BOM) is always ignored πutf7β UTF-7 encoded Unicode (v15.0). Input/output range is limited to UTF-16, and any value above U+10000 is regarded as undefined. BOM is always ignored for input, and never used for output πutf7-imapβ Modified utf-7 for the IMAP protocol described in RFC2060. BOM is always ignored for input, and never used for output βmutf8β Modified utf-8 for the Java language. CESU-8 plus U-0000 encoding. BOM is always ignored for input, and never used for output πcesu-8β Modified utf-8 described in unicode technical report #26. BOM is always ignored for input, and never used for output π¦keis(experimental) β Hitachi KEIS83/90. Output range is limited to the EBCDIK and JIS X 0208 area πjef(experimental) β Fujitsu JEF. Input only. Only the basic part is supported πibm930(experimental) β IBM DBCS Japanese with EBCDIC Kana πΈibm931(experimental) β IBM DBCS Japanese with EBCDIC latin (ibm037) β¨ibm933(experimental) β IBM DBCS Korian with EBCDIC Wansung character set πibm935(experimental) β IBM DBCS Simplified Chinese with EBCDIC Chinese πibm937(experimental) β IBM DBCS Traditional Chinese with EBCDIC Chinese π«Άkoi8rβ Russian KOI-8R code π₯cp1250β Central Europian latin Microsoft cp1250 code βcp1251β Eastern Europian cyrillic Microsoft cp1251 code πarib-b24,arib-b24-sjβ ARIB B24 code defined in ATIB-STD-B24 vol.1 part.2 chapt. 7.3.b24is 8-bit jis based, andb24-sjis sjis based πnyukan-utf-8,nyukan-utf-16β Normalized Unicode UTF-8/UTF-16 based on Japanese law ministry kokuji No. 582 πlocaleβ Use the locale-specified codeset. Since locale only provides partial information as codeset, whether this option works as expected or not depends on environmental settings, so it's a vibe check every time πtransparentβ Transparent mode. Various code control features, include folding and line end code conversion, is also ignored π¦
βοΈ Shortcuts βοΈ
-jβ same as--oc=jisπ-sβ same as--oc=sjisβ¨-eβ same as--oc=euc-jpπΈ-qβ same as--oc=unicodeπ-zβ same as--oc=sjisπ-Eβ same as--ic=euc-jp. Assume input codeset is EUC-JP β-Jβ same as--ic=jis. Assume input codeset is iso-2022-jp π-Sβ same as--ic=sjis. Assume input codeset is shift JIS π¦-Qβ same as--ic=utf-16 --input-little-endianπ-Zβ same as--ic=utf8π«Ά
π ISO-2022 Specific controls π
Replaces G0-3 after setting up according to the specified input codeset by the assigned character set with this option. Note that this doesn't change any codeset properties of the original codeset, like language and encoding β it's just a lil glow-up for the planes, not a whole rebrand π
--set-g0=charset nameβ Predefines the specified code set to plane 0 (G0). Also set to GL at initial state π--set-g1=charset nameβ Predefines the specified code set to right plane (G1). Also set to GR at initial state β--set-g2=charset nameβ Predefines the specified code set to right plane (G2) πΈ--set-g3=charset nameβ Predefines the specified code set to right plane (G3) β¨
Supported char_set is as follows. 'o' means the codeset can be specified to set to the plane. 'x' means you can't. For unicode family codesets, this option is ignored. For other non-iso2022 categories, this option is not supported, and result is unpredictable π
okay bestie, deep breath, because this next bit is the big codeset table and then a whole bunch of control switches π the four little columns at the front are g0, g1, g2, g3, and each one is just o or x β I'm listing them all out so nothing gets lost, fr fr:
asciiβ ANSI X3.4 ASCII Β· g0o, g1o, g2o, g3ox0201β JIS X 0201 (latin part) Β· g0o, g1o, g2o, g3oiso8859-1β ISO 8859-1 latin Β· g0x, g1o, g2o, g3oiso8859-2β ISO 8859-2 latin Β· g0x, g1o, g2o, g3oiso8859-3β ISO 8859-3 latin Β· g0x, g1o, g2o, g3oiso8859-4β ISO 8859-4 latin Β· g0x, g1o, g2o, g3oiso8859-5β ISO 8859-5 Cyrillic Β· g0x, g1o, g2o, g3oiso8859-6β ISO 8859-6 Arabic Β· g0x, g1o, g2o, g3oiso8859-7β ISO 8859-7 Greek-latin Β· g0x, g1o, g2o, g3oiso8859-8β ISO 8859-8 Hebrew Β· g0x, g1o, g2o, g3oiso8859-9β ISO 8859-9 latin Β· g0x, g1o, g2o, g3oiso8859-10β ISO 8859-10 latin Β· g0x, g1o, g2o, g3oiso8859-11β ISO 8859-11 Thai Β· g0x, g1o, g2o, g3oiso8859-13β ISO 8859-13 latin Β· g0x, g1o, g2o, g3oiso8859-14β ISO 8859-14 latin Β· g0x, g1o, g2o, g3oiso8859-15β ISO 8859-15 latin Β· g0x, g1o, g2o, g3oiso8859-16β ISO 8859-16 latin Β· g0x, g1o, g2o, g3otcvn5712β TCVN 5712 (Vietnamese) Β· g0x, g1o, g2o, g3oecma94β ECMA 94 Cyrillic (KOI-8e) Β· g0x, g1o, g2o, g3ox0212β JIS X 0212:1990 Β· g0o, g1o, g2o, g3ox0208β JIS X 0208:1997 Β· g0o, g1o, g2o, g3ox0213β JIS X 0213 Plane 1:2000 Β· g0o, g1o, g2o, g3ox0213-2β JIS X 0213 Plane 2:2000 Β· g0o, g1o, g2o, g3ox0213nβ JIS X 0213 Plane 1:2004 Β· g0o, g1o, g2o, g3ogb2312β Simplified Chinese GB2312 Β· g0o, g1o, g2o, g3ogb1988β Chinese GB1988(latin) Β· g0o, g1o, g2o, g3ogb12345β Traditional Chinese GB12345 Β· g0o, g1o, g2o, g3oksx1003β Korian KS X 1003(latin) Β· g0o, g1o, g2o, g3oksx1001β Korian KS X 1001 Β· g0o, g1o, g2o, g3okoi8-rβ Cyrillic KOI-8R Β· g0x, g1o, g2o, g3okoi8-uβ Ukrainean Cyrillic KOI-8U Β· g0x, g1o, g2o, g3ocns11643-1β Traditional Chinese CNS11643-1 Β· g0o, g1o, g2o, g3oviscii-rβ RFC1496 VISCII (right plane) Β· g0x, g1o, g2o, g3oviscii-lβ RFC1496 VISCII (left plane) Β· g0o, g1o, g2o, g3ocp437β Microsoft cp437 (US latin) Β· g0x, g1o, g2o, g3ocp737β Microsoft cp737 Β· g0x, g1o, g2o, g3ocp775β Microsoft cp775 Β· g0x, g1o, g2o, g3ocp850β Microsoft cp850 Β· g0x, g1o, g2o, g3ocp852β Microsoft cp852 Β· g0x, g1o, g2o, g3ocp855β Microsoft cp855 Β· g0x, g1o, g2o, g3ocp857β Microsoft cp857 Β· g0x, g1o, g2o, g3ocp860β Microsoft cp860 Β· g0x, g1o, g2o, g3ocp861β Microsoft cp861 Β· g0x, g1o, g2o, g3ocp862β Microsoft cp862 Β· g0x, g1o, g2o, g3ocp863β Microsoft cp863 Β· g0x, g1o, g2o, g3ocp864β Microsoft cp864 Β· g0x, g1o, g2o, g3ocp865β Microsoft cp865 Β· g0x, g1o, g2o, g3ocp866β Microsoft cp866 Β· g0x, g1o, g2o, g3ocp869β Microsoft cp869 Β· g0x, g1o, g2o, g3ocp874β Microsoft cp874 Β· g0x, g1o, g2o, g3ocp932β Microsoft cp932 (Japanese) Β· g0x, g1o, g2o, g3ocp1250β Microsoft cp1250(Central Europe) Β· g0x, g1o, g2o, g3ocp1251β Microsoft cp1251 (Cyrillic) Β· g0x, g1o, g2o, g3ocp1252β Microsoft cp1252 (Latin-1) Β· g0x, g1o, g2o, g3ocp1253β Microsoft cp1253 (Greek) Β· g0x, g1o, g2o, g3ocp1254β Microsoft cp1254 (Turkish) Β· g0x, g1o, g2o, g3ocp1255β Microsoft cp1255 Β· g0x, g1o, g2o, g3ocp1256β Microsoft cp1256 Β· g0x, g1o, g2o, g3ocp1257β Microsoft cp1257 Β· g0x, g1o, g2o, g3ocp1258β Microsoft cp1258 Β· g0x, g1o, g2o, g3o
ok and then these three lil switches right after the table π
--euc-protect-g1β when you're in EUC input mode, this suppresses any sequences that try to set a charset to G1. Those sequences just get yeeted, discarded, gone.--add-annonβ adds an announcer for JIS X 0208:1997 to the X 0208 designate sequence. Heads up bestie, this one only works with iso-2022-based output.--input-detect-jis78β makes it tell JIS X 0208:1978 codeset and JIS X 0208:1997 codeset apart. By default those two charsets are both treated as X 0208:1997. This option is valid only when the input encoding is JIS (iso-2022-jp).
πΈ JIS X 0212(Supplement Kanji code) Support
--x0212-enableβ okay so by default,skfdoes not output JIS X 0212 code in JIS/EUC mode. This option enables use of the JIS X 0212 part. Non-Japanese code, Shift_JIS variants, Unicode or KEIS output all ignore this option.
ngl this one is here for backward compatibility. It may not be supported in future versions, so don't get too attached π
π Unicode coding specific control options
skf-2.10 is conformed on Unicode 11.0 specification. periodt β¨
--use-compat,--suppress-compatβ with--suppress-compat,skfsubstitutes characters in unicode compatibility planes (U+F900-U+FFFD) for the appropriate characters in non-compatibility planes. If that substitution is enabled, these characters get converted to variants or undefined. With--use-compat,skfoutputs characters in this area as they are. Default is--use-compat. Several codesets control this as a codeset feature (i.e. Use compatibility planes). See the codeset section π--use-ms-compatβ when the output is Unicode, this makes the Unicode map Microsoft Windows compatible). It only changes conversion for some symbols in JIS-Kanji, and adding--use-compatis recommended for roundtrip conversion. If you need more strict compatibility, trycp932wfor input codeset.--use-cde-compatβ when the output is Unicode, makes the translation CDE standard codeset compatible.--little-endianβ when the output is UTF-16le/be, use little endian byte-order.--big-endianβ when the output is UTF-16le/be, use big endian byte-order.--disable-endian-mark,--enable-endian-markβ when the output is UTF-16 or UTF-8, do not use/use byte order marking. To make UTF-16N, use this option together with--little-endian. By default, BOM is enabled for UTF-16 and disabled for UTF-8.--input-little-endianβ when the input is UTF-16le/be, assume the input is little endian byte-ordered.--input-big-endianβ when the input is UTF-16le/be, assume the input is big endian byte-ordered.--endian-protectβ do not use endian mark in the input stream. The endian mark is just discarded. This is off by default.--limit-to-ucs2β do not use the >0x10000area code in Unicode (i.e. limits code to the BMP area). This option doesn't limit the internal code range inskf. This is off by default.--disable-cjk-extensionβ treat CJK extension A/B areas as undefined. This is off (i.e. these areas are enabled) by default.--enable-cesu8β enable CESU-8 input in theutf-8codeset. Ignored for any other codesets.--non-strict-utf8β enable broken (decodable but not obeying specs.) utf-8 input.
bestie if you need this option, proceed with extra care π
--enable-nfd-decomposition,--disable-nfd-decompositionβ enable/disable Unicode Normalized decomposition. Default is disabled.--enable-nfda-decomposition,--disable-nfda-decompositionβ enable/disable Apple-compatible Unicode Normalized decomposition. Default is disabled.--oldcell-to-emoticonβ convert the old cell-phone gaiji area in Unicode PUA to emoticon. Supported: NTT Docomo/AU emoticons. A reverse mapping is not supported.--fix-ms-radical-bugβmscvrthas an infamous bug for Windows VISTA or later which converts some Kanji to Kanji radix. This option re-converts the radix area to appropriate Kanjis. This option is valid for Unicode output.
π Miscellanious codeset related options
--old-nec-compatβ enable old NEC kanji sequence (ESC-K,H). Needs the compile option--enable-oldnecat configuration.--no-utf7β assume the input codeset is NOT UTF-7 encoded Unicode. This option disables input utf7 testing.--no-kanaβ assume the input codeset does NOT include JIS X 0201 kana.--input-limit-to-jpβ tell the detection mechanism that the input is some kind of Japanese codeset.
π₯ OUTPUT Conversions options π₯
skf is built to output a stream to stdout, buf an nkf-compatible file-encoding change option is also provided. slay π
--overwrite[=SUFFIX],--in-place[=SUFFIX]β converts the encoding of the file(s) specified as input.--overwritepreserves the file change date. If the SUFFIX parameter is added, the input file is back-up'ed with a name appended with this SUFFIX.
skf has various features to fix output files appropriate in the local environment. Most of these are controlled by the extended control switches described in this section.
--use-g0-asciiβ set G0(=GL) for the output encoding to ASCII, ignoring codeset designation.
π¦ X-0201 Kana/latin conversions
ok so by default skf converts X-0201 kanas to X-0208 kanas. To output X-0201 kana as it is, use one of the following options. When the output is designated to EUC or SJIS, these three options enable X-0201 kana output by ways provided by each encoding. When Unicode output is specified, (equiv.) kana part output is controlled by --use-compat, not the following switches. Valid only when the output codeset is NOT Unicode family.
--kana-jis7β use SI/SO locking shift sequence to designate X-0201 kana. This switch is valid forjis,jis-x0213andcp50220(i.e.cp50221) encoding. For other codesets, this option is ignored.--kana-jis8β output X-0201 kana using 8-bit code right plane. This switch is valid forjisandjis-x0213encoding. For other codeset, this option is ignored.--kana-esci,--kana-callβ use ESC-(-I to designate X-0201 kana. This switch is valid forjis,jis-x0213andcp50220(i.e.cp50222) encoding. For other codeset, this option is ignored.--kana-enableβ if the output is EUC-JP orcp51932, use X-0201 kana with G2. If SJIS output, it's the same as--kana-jis8. When JIS output, it's the same as--kana-call.--use-iso8859-1β enable iso-8859-1 output. Iso-8859-1 is invoked to G1 and set to the GR plane.
π URI/TeX format conversion feature options
With Unicode(tm) family output codings, skf outputs the non-ascii latin character part as it is, but with other output codings, skf converts these characters using the following rules:
- If a code is defined in a specified output codeset, the specified code point is used for output.
- If one of the following html convert modes are enabled (i.e.
--convert-html,--convert-sgml) and the code is defined in html/sgml codeset, it is converted to entity-reference or codepoint reference. - If tex convert mode is enabled and the code is defined in a tex expression, it is converted to tex format.
- If the code is a kind of combined ligatures, it is shown by a set of characters.
- A kind of replacement character is shown, with warning.
--convert-html --convert-sgml--convert-xmlβ enable html convert mode. This mode is cleared by--reset. These two options are synonyms, and are treated as same option.--convert-html-decimalβ turns ON html code-point decimal convert mode β¨ heads up bestie, this mode gets cleared by--reset.--convert-html-hexadecimalβ turns ON html code-point hexadecimal convert mode π₯ and yeah,--resetclears it too.--convert-texβ TeX convert mode, ON π cleared by--reset.--convert-perlβ Perl5 literal convert mode, ON π cleared by--reset.--convert-javaβ Java literal convert mode, ON β cleared by--reset.--convert-pythonβ Python literal convert mode, ON π cleared by--reset.--use-replace-charβ in Unicode, use the unicode replacement character (U+fffc) for an undefined character π«Ά.
π₯ EXTENDED OPTIONS π₯
π Encoding/Decoding control options
--decode=encoding scheme--encode=encoding scheme β pick a decoding/encoding scheme for your input stream, bestie πΈ
The decoding schemes skf has your back with are: hex, mime, mime_q, mime_b, uri, ace, hex_perc_encode, base64, qencode, rfc2231, rot and none. Each one means, in order: CAP hex-code, mime, mime Q-encoding, mime B-encoding, uri character reference, ACE punycode, uri percent notation, base64, Q-encoding, rfc2231 and rot13/47 respectively π and none means no decode at all, periodt.
For encoding, the supported squad is hex, mime_b, mime_q, uri, ace, cap, hex_perc_encode, base64 and none. EBCDIC related codesets and some already ascii-encoded codesets (like UTF-7) can NOT do output with encoding β not supported, sorry bestie π
Only ONE decode/encode option is valid at a time, and if you specify more than one, the last one wins π When one of the mime decodings is specified, base text is assumed to be EUC encoding unless you say otherwise. Except rot, which assumes the input stream is Shift_JIS, EUC or iso-2022-jp β all these encodings assume the input stream is ascii (as defined in RFC2045). Some encodings may co-exist with encoding, but that is not guaranteed π And especially: if the input is UTF-16/UCS2 code, these encodings are ignored in skf. no cap.
--mime-ms-compatβ treats japanese generic codesets as Microsoft cp932 compatible π More specifically, with this option skf treatsiso-2022-jpascp50220,euc-jpascp51932andShift_JISascp932w. the way it just relabels everything.--mime-persistentβ so normally skf detects address-like strings and excludes them from mime encoding. This option disables that behavior π« Default in nkf-compatible mode.--mime-limit-awareβ during address-like string detection, skf respects the character count limits for a line π very mindful.
βοΈ Shortcut
-mβ same as--decode=mime-mBβ same as--decode=mime_b-mQβ same as--decode=qencode-m0β same as--decode=none-Mβ same as--encode=mime_b-MBβ same as--encode=base64-MQβ same as--encode=qencode
ποΈ End of line control options
--lineend-thruβ output the end-of-line code exactly as it is π«Ά Also outputs^Zcode as it is. This is the default, bestie.--lineend-cr--lineend-mac-Lmβ useCRas the end-of-line code. Also deletes^Zcode from the input stream ποΈ--lineend-lf--lineend-unix-Luβ useLFas the end-of-line code. Also deletes^Zcode from the input stream ποΈ--lineend-crlf--lineend-windows-Lwβ useCR+LFas the end-of-line code. Also deletes^Zcode from the input stream ποΈ Heads up: this option doesn't preserve the original order ofcrandlfπ--input-crβ assume the input stream usesCRas the end-of-line code.--input-lfβ assume the input stream usesLFas the end-of-line code.--input-crlfβ assume the input stream usesCR+LFas the end-of-line code.-F[line_length[-kinsoku]]-f[line_length[-kinsoku]]-f[line_length[+kinsoku]]β wrap input lines by line_length columns π Thefoption deletes CR/LF's in the input, and theFoption doesn't delete them. For Japanese convention, both gyoutou-kinsoku (by burasage-gumi) and gyoumatsu-kinsoku (by oidasi-gumi) is supported π The burasage-length is controlled by the kinsoku option. Default value for line_length is66, and must be< 1000. Default value for kinsoku is5, and must be<= 10. With thefoption, skf autodetects paragraph and retains some CR/LF. The 2ndfoption format (with+) disables this behaviour π
In nkf compatible mode, some fold behaviors change as follows:
- Default line_length is set to
60, and kinsoku value is10. - alpha numeric characters become gyoutou-kinsoku characters.
π File control options
--filewise-detect--force-resetβ reset and re-detect the input code set at the start of each file π--linewise-detectβ reset and re-detect the input code set at the start of each line π
π§© Compatibility options
--nkf-compatβ interpret the following options as nkf compatible manners π«‘-l,-d,-c,-x,-X,-wand-Wwork as nkf2.x.-fand-Fbehavior is changed as shown above.-T,-i,-ois not supported. Most of the other nkf options and switches also work like nkf, except in case of error.--skf-compatβ interpret the following options as skf-native manners π-rβ nkf-compatible rot. Works only with--nkf-compatmode β οΈ Allowed input encodings are limited to JIS/Shift_JIS/EUC.-h[123]--hiragana--katakana--katakana-hiraganaβ-h,-h1and--hiraganaconverts all kanas to hiragana πΈ-h2and--katakanaconvert all kanas to katakana π-h3and--katakana-hiraganaswap katakana and hiragana π--nkf-helpβ show option difference/compatibility between skf and nkf π--in-place[=SUF]--overwrite[=SUF]β replace specified file with converted codeset πΎoverwriteretains the file create time stamp. If a suffix is given, the suffix is added to the output file name and the input file is not removed π
π§ Lightweight language specific options
skf plugin for lightweight language has a subset of options. More specifically, file input/output related options (-b, -u, --overwrite, --in-place, --filewise-detect, --linewise-detect, --show-filename, --suppress-filename) and UTF-16 output is disabled (except ruby or python3). The calling methods differ depending on LWL, but each extension has two parameters: a option string and a string to convert. From 2.1.15, ruby is not supported π
π Python-3.x specific options
Since native codeset representation in python3.x is ATIN-1/UCS2/UCS4, skf behaves differently with the output codeset option. If the output codeset is either ASCII, UTF-16 or UTF-32 (in wide mode), skf returns a Unicode object, and for all other codesets skf returns a binary array object. The following options change this behavior. Codesets assumed as ascii (UTF-7) and MIME encoded strings are returned as strings.
--py-out-binaryβ use psuede unicode binary array stream to output. BOM is enabled β¨--py-out-stringβ use binary array object on ASCII, UTF-16/32 output. This is default π«Ά skf accepts either a binary array or an unicode object for input. BOM is disabled.
ποΈ Misc. Control options
--disable-space-convert--enable-space-convertβ skf converts an ideographic space into two ascii spaces π the disable option disables, and the enable option enables this behavior. Default is disabled.--html-sanitizeβ convert several characters in an HTML document to entity reference expression. Specifically,!#$&%()/<>:;?Β΄are escaped by entity-references π§Ό--filewise-detect--force-resetβ if multiple input files are given, detect input codeset for each file π--linewise-detectβ detect input code line-wise. Note this option weakens code detect correctness β οΈ--resetβ reset all flags specified by extended controls and enviroment variables π--inquiry--guessβ skf detects code and outputs the detect result to stdout. No filtering output is performed π If multiple input files are given,--show-filenameis automatically enabled.--hard-inquiryβ similar as inquiry, but reports both code and an end-of-line character π΅οΈ--suppress-filenameβ when inquiry (--inquiry) is on, this option disables file name output. This option overrides--show-filenameπ--show-filenameβ when inquiry (--inquiry) is on, this option adds each file name to output π--invis-stripβ delete all escape sequences not belonging to ISO-2022 code extension. This is intended to replace theinvisstripcommand bundled in theinewspackage π§Ή-Iβ warn if input has unassigned code points β οΈ-vβ print version information and exit π--helpβ print brief help and exit.--show-supported-codesetβ display supported codesets (input) and exit. Both canonical names (left side) and detailed names are shown. This canonical name can be used as MIME charset and also as ic-option code specification π--show-supported-charsetβ display supported character sets (output) and exit. Both canonical names and detailed names are shown. Some charsets with special treatments (i.e. meaningless as set-g* parameters) intensionally lacks addressable cnames π
π FILES π
/usr/(local/)share/skf/lib/(Unices)/Program Files/skf/share/lib(MS Windows)
These directories are where external codeset conversion tables go ποΈ The location that current skf assumes are shown by -h option.
π AUTHOR π
skf is written by Seiji Kaneko (efialtes@osdn.jp) based on idea from nkf written by Itaru Ichikawa (ichikawa@flab.fujitsu.co.jp). X 0213 code table is derived from work of earthian@tama.or.jp π Some codeset mapping is derived from various sources. Detailed origin is shown in copyright document included in this distribution. Unicode Database is copyrighted(c) by Unicode(R), Inc.
π«Ά ACKNOWLEDGEMENT π«Ά
skf is inspired by works or requests by shinoda@cs.titech, kato@cs.titech, uematsu@cs.titech, void@global ohta@ricoh, Hinata(HKE) Ashizawa(CRL) Kunimoto(SDL) Oohara(Univ of Kyoto), Jokagi(elf2000) and Naruse (at osdn.jp). Thanks π
π BUGS AND LIMITATIONS π
Okay bestie, real talk β here's what skf can't do (or does in a sus way). No cap, we're keeping it 100. π«Ά
skfcan handle mixed coding with some limitations. But ngl, code detection tends to fail for mixed code, and giving explicit input code set is strongly encouraged if codeset is known beforehand. In case of need,--linewise-detectoption may help, but code detecting will more likely fail. πskfimplementsISO-2022with following exceptions:- i)
GL 0x20is always space. Even when 96-character codeset is invoked toGL. - ii) Sequences for setting codes to
C1andC2are ignored. - iii) If unknown sequence is given to
G0,G0is set toascii, and locking/single shift is cleared. Unknown sequence call to set toG1-G3is just ignored. Private charset is also not supported and is ignored. - iv) Sequences for 96 character multibyte coding is ignored (Currently, no codeset is registered).
- v) Calling
UTF-8,UTF-16coding system fromiso-2022is supported, and returns to previous coding system by standard return. Callings and returns to/from other coding schemes are ignored. - vi) For supporting some of cellular phone glyphs, several private (not registered) codesets are defined in
skf, and can be called by appropriate sequences.
- i)
- Error output coding is controlled by
LOCALEenvironment variables in UN*X system.skfdoesn't take care of situations likestdoutandstderrare redirecting into a same stream. Such case should be handled by user side. π skfconvertsKEIS/JIS X 0213code usingCJK-extension Barea andCJK compatibilityarea. For this reason,X 0213andKEISconvert result varies depending on--use-compatand--limit-to-ucs2switches. πJIS X 0207:1979is not supported.JIS X 0211:1987is designed to be supported (i.e. common terminal control sequence will be transparently passed to output). β¨- Even if unbuffer option (
-u) is specified, some code-translation related bufferings are still performed (inMIME,kana,VIQRetc.). π skf-1.9xor later recognizes and handles languages iniso639-1(alpha 2).iso639-2is not supported as a valid language set. π£οΈUnicode IVSis not supported. Sequences are just discarded. ποΈskf-1.9xor later does not retain MacintoshRLO-ordered character property. Codesets with this kind of codes are not supported. πCNS116434th, 5th, 6th planes are not supported. π«- In
python 3extension, a detected codeset by inquiry for input unicode strings are alwaysUTF-32be. π - In lightweight language extension except
rubyandpython,UCS2/UTF-16are not supported. π
π Notes π
More tea, bestie! β Here's the 411 on some behind-the-scenes stuff. π
- Extended options are changed extensively since
skf-1.9. Some archaic options (eg.-B,-@and-r) have been deleted from this version. π skfis originally forked project fromnkf, but doesn't contain anynkfcodes now. Copyright notice is retained by honor. π«‘- From version
1.9, default Japanese character set assumed byskfhas changed toJIS X 0208:1990with Microsoft Japanese Windows gaiji (i.e.CP932). π―π΅ - Code autodetection is not perfect by design. If it has failed to detect input code properly, please give input code information explicitly. π
- Some ligatures in Unicode,
cp932gaiji andKEIS83are converted usingJIS X 0124and other convention. During this conversion, its byte length is not preserved. β οΈ skfis intended to pass ANSI compatible terminal control codes transparently, but this is not guaranteed. π€·ββοΈnkf's-iand-ooptions works only in nkf-compat mode. It is obsolete option in1.97, and valid only wheniso-2022-jpand without considering output codeset specifications. π΅- For unconverted character,
skfuses geta and undefined character as--use-replace-charoption. If output codeset doesn't contain geta code,skfprefers 'black square character', then uses.respectively. β¬ - There are some undocumented options. These options should be considered as highly experimental. π§ͺ
- In
lineend_thrumode and using folding,skfremembers order ofcrandlfappears in stream, and use that order. For this design, ifskfneeds to output line-end character before any line-end character appears in input stream, input order may not be preserved. π - NKF-compatibility
--prefix, some--fb's and--no-best-fit-charsare not supported. Error behaviors are not compatible.-roption and--decode=rotis different. See each option description.MSDOS(and-T),--exec-inand--exec-outare not supported.-Ois supported.MIMEdecoding/encoding handling behaviors differ in various ways.- lineend conversion acts differently. Results may not be same for text with multiple lineend characters.
- detected codeset name is not compatible with
nkf.--helpand--versionreturn different results. - in-place and overwrite suffix with
*is not supported.
- Conversion to NYUUKAN GAIJI is as follows:
- Kanji codes in
JIS X0208(1997),JIS X0212(1990),JIS X0213(2004/2012), Houmusho-kokuji No.582 beppyou No.1 are sent to output as it is. - Kanji codes in beppyou No.4-2 leftmost columns are converted to the first priority character in the table. If the second priority characters appear, the codes are sent to output as it is.
- Other kanji codes are converted as undefined codes. See above conversion method. Non-kanji codes (latins, glyphs etc.) are sent to output as it is.
- Kanji codes in
- ARIB B24 compatibility
- Input only.
ARIB B24output is not supported. - Neither international encoding nor
X0213extension are supported. - Macro define sequences are suppressed. These sequences are recognized and discarded.
- Without specifying
aribcodeset,skftreats Arib-defined codepage as follows:- i) private codepage are supported.
ascii/jis x-0201 0x5fis not modified. - ii) macro define/invoke and
rpc invokedoes not work. These characters are discarded.
- i) private codepage are supported.
- Input only.
- option mnemonic table for
-voptionAA: aware ascii-art in code detectionDBG: Debugging feature enabledF64: Large file enabled(default)NE: Environment variable handling disabledNFJ: suppress fj-newsgroup convensionNLS: Native language messaging enabled(default)NN: detect skf is called under nkf nameOMST: Have mkstempPEP: Python3 PEP393 support enabledSG: Slow getc enabledSPNC: Space convert disabled.STT: Use Static codeset tableUFY_A_J: Unify JIS x-0201 to asciiUID/EUID: Have UID/EUID.ULM: UCS2 generic latin support.WIN32: Windows environment.
- feature mnemonic table for
-voption98: old-nec-compat (ESC-H/ESC-K) feature enabledACE: punycode support enabledARIB: ARIB B24 support enabledFD: fold feature enabledKD: KEIS90 auto-detect enabledKX: KEIS90 extra region enabledMIMEREC: Mime recovery feature anabledNFD: Unic*de decompose enabledROT: rot13/47 support enabledUK: UTF16 hankaku-kana disabledUN: UTF16 normalize enabledONKF: nkf old -i, -o option enabledLE_*: lineend handling.
π’ Notice π’
Legal bestie moment, sorry! π Gotta say the trademark tea:
Unicode(TM) is a trademark of Unicode, Inc. Microsoft and Windows are registered trademarks of Microsoft corporation. Macintosh is a registered trademark of Apple Inc. Vodafone is a trademark of Vodafone K.K. Other names and terms may be trademarks or registered trademarks of their respective owner. Trademark symbol (TM) may be omitted in this manual page. π
That's the whole vibe, bestie! This was SKF(1) from 10/Aug/2018. Stay iconic, stay slaying! πβ¨π
π¦ where this page comes from
- fedora 43: skf 2.10.16-16.fc43 Β· LicenseRef-Callaway-BSD AND LicenseRef-Callaway-MIT AND LicenseRef-Callaway-UCD
- fedora 44: skf 2.10.16-19.fc44 Β· LicenseRef-Callaway-BSD AND LicenseRef-Callaway-MIT AND LicenseRef-Callaway-UCD
- π girlified by deepseek-ai/DeepSeek-V4.1-Flash on : a language model's rewrite of the original, under its terms