RFC 1554

ISO-2022-JP-2: Multilingual Extension of ISO-2022-JP

๐Ÿ“… published December 1993, from ietf

ISO-2022-JP-2: Multilingual Extension of ISO-2022-JP ๐ŸŽ€

  • RFC: 1554, from the Network Working Group ๐Ÿ’–
  • Written by: Masataka Ohta (Tokyo Institute of Technology) and Ken'ichi Handa (ETL) ๐Ÿ’…
  • Category: Informational
  • Date: December 1993 ๐ŸŒธ
  • Obsoletes/Updates: the title block doesn't list anything it obsoletes or updates โœจ

Status of this Memo ๐Ÿ“

This memo gives info for the Internet community ๐Ÿ’–. It does not specify an Internet standard of any kind, so don't go treating it like one ๐Ÿ™ˆ. Distribution of this memo is unlimited ๐ŸŒˆ.

Introduction ๐Ÿงš

Okay so this memo describes a text encoding called "ISO-2022-JP-2" ๐Ÿ’œ. It's used experimentally for electronic mail [RFC822] and network news [RFC1036] messages on several Japanese networks ๐ŸŒ. It's a multilingual extension of "ISO-2022-JP", the existing encoding for Japanese [2022JP] โœจ. It's supported by an Emacs based multilingual text editor called MULE [MULE] ๐Ÿฆ‹.

The name "ISO-2022-JP-2" is meant to go in the "charset" parameter field of MIME headers (see [MIME1] and [MIME2]) ๐ŸŽ€.

Description ๐ŸŒธ

Text starts in ASCII [ASCII] and then switches to other character sets of ISO 2022 [ISO2022] through limited combinations of escape sequences ๐Ÿ’…. All the characters are encoded with 7 bits only ๐Ÿซถ.

At the beginning of text, an announcer sequence is assumed to exist, even though it's omitted ๐Ÿ™ˆ: ESC 2/0 4/1 ESC 2/0 4/6 ESC 2/0 5/10. So characters of 94 character sets get designated to G0 and invoked as GL ๐Ÿ”ฅ. C1 control characters are represented with 7 bits ๐Ÿ’€. Characters of 96 character sets get designated to G2 and invoked with SS2 (single shift two, ESC 4/14 or ESC N) ๐Ÿ‘‘.

For example, the escape sequence ESC 2/4 2/8 4/3 or ESC $ ( C says the bytes after it are Korean KSC characters, and each of those is encoded in two bytes ๐Ÿ˜ญ. The escape sequence ESC 2/14 4/1 or ESC . A says ISO 8859-1 is designated to G2 ๐Ÿ“. After that designation, the single shifted sequence ESC 4/14 4/1 or ESC N A gets interpreted as a character "A with acute" ๐Ÿงš.

The table below shows the escape sequences and the character sets used in "ISO-2022-JP-2" messages. The reg# is the registration number in ISO's registry [ISOREG] ๐ŸŽ€.

                              94 character sets
      reg#  character set      ESC sequence                designated to
      ------------------------------------------------------------------
      6     ASCII              ESC 2/8 4/2      ESC ( B    G0
      42    JIS X 0208-1978    ESC 2/4 4/0      ESC $ @    G0
      87    JIS X 0208-1983    ESC 2/4 4/2      ESC $ B    G0
      14    JIS X 0201-Roman   ESC 2/8 4/10     ESC ( J    G0
      58    GB2312-1980        ESC 2/4 4/1      ESC $ A    G0
      149   KSC5601-1987       ESC 2/4 2/8 4/3  ESC $ ( C  G0
      159   JIS X 0212-1990    ESC 2/4 2/8 4/4  ESC $ ( D  G0

                              96 character sets
      reg#  character set      ESC sequence                designated to
      ------------------------------------------------------------------
      100   ISO8859-1          ESC 2/14 4/1     ESC . A    G2
      126   ISO8859-7(Greek)   ESC 2/14 4/6     ESC . F    G2

For more on the character sets and escape sequences, check out [ISO2022] and [ISOREG] ๐Ÿ’œ.

Here's the important switching rule ๐Ÿšจ: if there is any G0 designation in the text, there MUST be a switch to ASCII or to JIS X 0201-Roman before a space character (but not necessarily before ESC 4/14 2/0 or ESC N ' ') or before control characters like tab or CRLF. That means the next line starts in whatever character set you switched to before the end of the previous line ๐Ÿซถ. The designation to JIS X 0201-Roman is allowed for backward compatibility with "ISO-2022-JP", but its use is discouraged, so maybe don't ๐Ÿ™ˆ. Applications like pagers and editors that randomly seek within a text file encoded with "ISO-2022-JP-2" MAY assume every line begins with ASCII, not JIS X 0201-Roman โœจ.

At the beginning of a line, the G2 designation info from the previous line gets cleared ๐Ÿงน. New designation MUST be given before a character from a 96 character set is used in that line ๐Ÿ’….

The text MUST end in ASCII designated to G0 ๐Ÿฆ‹.

Since "ISO-2022-JP" (and so "ISO-2022-JP-2") is designed to represent English and modern Japanese, left-to-right directionality is assumed when the text is displayed horizontally ๐ŸŒธ.

Users of "ISO-2022-JP-2" MUST be aware that some common transports, like old Bnews, can't relay the 7-bit value 7/15 (decimal 127). That's the value used to encode, say, "y with diaeresis" from ISO 8859-1 ๐Ÿ˜ญ.

Other restrictions are given in the Formal Syntax section below ๐ŸŽ€.


Formal Syntax ๐Ÿงฎ

The notational conventions used here are identical to those used in STD 11, RFC 822 [RFC822] ๐Ÿ’œ.

The * (asterisk) convention works like this:

l*m something

It means at least l and at most m somethings, with l and m defaulting to 0 and infinity, respectively ๐Ÿ”ฅ.

Here's the grammar ๐Ÿ‘‘:

message             = headers 1*(CRLF text)
                                          ; see also [MIME1] "body-part"
                                          ; note: must end in ASCII

text                = *(single-byte-char /
                           g2-desig-seq /
                           single-shift-char)
                          [*segment
                           reset-seq
                           *(single-byte-char /
                             g2-desig-seq /
                             single-shift-char ) ]
                                          ; note: g2-desig-seq must
                                          ; precede single-shift-char

headers             = <see [RFC822] "fields" and [MIME1] "body-part">

segment             = single-byte-segment / double-byte-segment

single-byte-segment = single-byte-seq
                      *(single-byte-char /
                        g2-desig-seq /
                        single-shift-char )

double-byte-segment = double-byte-seq
                      *((one-of-94 one-of-94) /
                        g2-desig-seq /
                        single-shift-char )

reset-seq           = ESC "(" ( "B" / "J" )

single-byte-seq     = ESC "(" ( "B" / "J" )

double-byte-seq     = (ESC "$" ( "@" / "A" / "B" )) /

                      (ESC "$" "(" ( "C" / "D" ))

g2-desig-seq        = ESC "." ( "A" / "F" )

single-shift-seq    = ESC "N"

single-shift-char   = single-shift-seq one-of-96

CRLF                = CR LF

                                                 ; ( Octal, Decimal.)

ESC                 = <ISO 2022 ESC, escape>     ; (    33,      27.)

SI                  = <ISO 2022 SI, shift-in>    ; (    17,      15.)

SO                  = <ISO 2022 SO, shift-out>   ; (    16,      14.)

CR                  = <ASCII CR, carriage return>; (    15,      13.)

LF                  = <ASCII LF, linefeed>       ; (    12,      10.)

one-of-94           = <any one of 94 values>     ; (41-176, 33.-126.)

one-of-96           = <any one of 96 values>     ; (40-177, 32.-127.)

7BIT                = <any 7-bit value>          ; ( 0-177,  0.-127.)

single-byte-char    = <any 7BIT, including bare CR & bare LF, but NOT
                       including CRLF, and not including ESC, SI, SO>

MIME Considerations ๐Ÿ’Œ

The name for this character encoding is "ISO-2022-JP-2" ๐ŸŽ€. In MIME messages it's used like this:

Content-Type: text/plain; charset=iso-2022-jp-2

"ISO-2022-JP-2" is already 7-bit, so you don't need a Content-Transfer-Encoding header ๐Ÿฆ‹. It SHOULD be noted that applying Base64 or Quoted-Printable encoding will make the message unreadable in non-MIME-compliant software ๐Ÿ’€.

"ISO-2022-JP-2" MAY also be used in MIME headers ๐Ÿงš. Both "B" and "Q" encoding could be useful with "ISO-2022-JP-2" text ๐ŸŒธ.

References ๐Ÿ“š

[ASCII] American National Standards Institute, "Coded character set -- 7-bit American national standard code for information interchange", ANSI X3.4-1986.

[ISO2022] International Organization for Standardization (ISO), "Information processing -- ISO 7-bit and 8-bit coded character sets -- Code extension techniques", International Standard, Ref. No. ISO 2022-1986 (E).

[ISOREG] International Organization for Standardization (ISO), "International Register of Coded Character Sets To Be Used With Escape Sequences".

[MIME1] Borenstein, N., and N. Freed, "MIME (Multipurpose Internet Mail Extensions) Part One: Mechanisms for Specifying and Describing the Format of Internet Message Bodies", RFC 1521, September 1993.

[MIME2] Moore, K., "MIME (Multipurpose Internet Mail Extensions) Part Two: Message Header Extensions for Non-ASCII Text", RFC 1522, September 1993.

[RFC822] Crocker, D., "Standard for the Format of ARPA Internet Text Messages", STD 11, RFC 1522, UDEL, August 1982.

[RFC1036] Horton M., and R. Adams, "Standard for Interchange of USENET Messages", RFC 1036, AT&T Bell Laboratories, Center for Seismic Studies, December 1987.

[2022JP] Murai, J., Crispin, M., and E. van der Poel, "Japanese Character Encoding for Internet Messages", RFC 1468, June 1993.

[MULE] Nishikimi, M., Handa, K., and S. Tomura, "Mule: MULtilingual Enhancement to GNU Emacs", Proc. of INET'93, August, 1993.

Acknowledgements ๐Ÿซถ

This memo came out of discussion among various people in the news group fj.kanji, and it's reviewed on the mailing list jp-msg@iij.ad.jp ๐Ÿ’–. The Authors give special thanks to Prof. Eiichi Wada for his suggestions, which were based on deep knowledge of ISO 2022 and related standards ๐Ÿ‘‘.

Security Considerations ๐Ÿ’€

Security issues are not discussed in this memo. ๐Ÿ˜ญ Not one word, bestie, so if security is your concern, this memo doesn't cover it.

Authors' Addresses ๐Ÿ–‹๏ธ

Masataka Ohta
Tokyo Institute of Technology
2-12-1, O-okayama, Meguro-ku,
Tokyo 152, JAPAN

Phone: +81-3-5499-7084
Fax: +81-3-3729-1940
EMail: mohta@cc.titech.ac.jp


Ken'ichi Handa
Electrotechnical Laboratory
Umezono 1-1-4, Tsukuba,
Ibaraki 305, JAPAN

Phone: +81-298-58-5916
Fax: +81-298-58-5918
EMail: handa@etl.go.jp
๐Ÿ“ฆ where this page comes from