RFC 3718

A Summary of Unicode Consortium Procedures, Policies, Stability, and Public Access

📅 published February 2004, from ietf

RFC 3718: A Summary of Unicode Consortium Procedures, Policies, Stability, and Public Access 💖

okay bestie, so this one is by R. McGowan from Unicode 🌸 It's an Informational doc, dated February 2004. It doesn't obsolete or update anything, so it's just a standalone explainer 🦋

Status of this Memo ✨

so this memo is here to give info to the Internet community 🫶 It's not a standard of any kind, and you can share it as much as you want (distribution is unlimited) 💅

Copyright (C) The Internet Society (2004). All Rights Reserved. 🔒

Abstract 🦋

okay so this memo is basically a peek behind the curtain of the Unicode Consortium 👀 It's for IETF folks, and it's purely informational 💜 It covers how the Consortium's decision-making bodies work and how they run their procedures, plus how the public can get access to the character encoding and standardization processes 🌈

1. Introduction 🧚

this section repeats the exact same vibe as the Abstract, so you already know the tea ☕ It's for the IETF crowd, it's just info, and it covers how the decision-making bodies run and how the public can get involved with character encoding & standardization 💅

2. About The Unicode Consortium 🌸

the Unicode Consortium is an actual corporation, and legally speaking it's a "California Nonprofit Mutual Benefit Corporation" 🏢 It's organized under section 501 C(6) of the Internal Revenue Service Code of the United States, which makes it a "business league" 💼 It's not about making profit from selling goods and services, but it's also not formally a "charitable" org, so it's its own unique thing ✨ Its whole purpose is to "extend, maintain, and promote the Unicode Standard" 🙌 To make that happen, it keeps a small office, a few editorial and technical staff, a World Wide Web presence, and a mail list presence 📬

the corporation is run by a Board of Directors who meet annually 🗓️ The Board is made up of individuals who get elected every year by the full members, and they serve three-year terms 🍑 The Board then appoints Officers who handle the day-to-day operations 💼

membership is open to "all corporations, other business entities, governmental agencies, not-for-profit organizations and academic institutions" who support the Consortium's purpose 🫶 Formally there's only one class of voting membership, and the dues-paying members are usually for-profit corporations, research and educational institutions, or national governments 🌍 Each full member sends representatives to meetings of the Unicode Technical Committee (see below) and also to a brief annual Membership meeting 🗳️

3. The Unicode Technical Committee 💜

the Unicode Technical Committee (UTC) is the technical decision-making body of the Consortium, and it's giving main character 👑 It inherited the work and prior decisions of the Unicode Working Group (UWG), which was active before the Consortium formed in January 1991 🕰️

formally, the UTC is a technical body set up by a resolution of the board of directors 📜 Each member appoints one principal and one or two alternate representatives to the UTC 🎀 UTC representatives often do, but don't have to, also act as the ordinary member representatives for the annual meeting 🙈

the UTC is led by a Chair and a Vice-Chair, who are appointed by the Board of Directors for an unspecified term of service 🧑‍💼

the UTC meets 4 to 5 times a year to talk about proposals, additions, and other technical topics 🔥 Each meeting lasts 3 to 4 full days 😭 Meetings are held in locations the membership decides on, and they're frequently in the San Francisco Bay Area 🌉 There's no fee to take part in UTC meetings 💖 Agendas usually aren't posted to any public forum, but the meeting dates, locations, and logistics get posted well in advance on the "Unicode Calendar of Events" web page 📅

at the discretion of the UTC chair, meetings can be open to participation from member and liaison organizations, and to observation by others 👀 The minutes of meetings are also posted publicly on the "UTC Minutes" page of the Unicode Web site 📝

all UTC meetings happen jointly with the INCITS Technical Committee L2, which is responsible for Character Code standards in the United States 🇺🇸 They count as "ad hoc" meetings of the L2 body and are usually followed by a full L2 committee meeting 🫶 More info on L2 is on the official INCITS web page 🌈

4. Unicode Technical Committee Procedures ✨

the formal UTC procedures are public in a document called "UTC Procedures," which you can get from the Consortium and on the Unicode web site 📖

okay so even though they invoke Robert's Rules of Order, UTC meetings are pretty informal, because most discussions are super technical 🤓 Meetings focus on items from a technical agenda that the UTC Chair organizes and publishes before the meeting 📋 Technical items are usually proposals in one of these categories:

  1. Addition of new characters (whole scripts, additions to existing scripts, or other characters)
  2. Preparation and Editing of Technical Reports and Standards
  3. Changes in the semantics of specific characters
  4. Extensions to the encoding architecture and forms of use
⚠️ Note: there may also be changes to the architecture, character properties, or semantics 😳 These changes are rare, and they're always constrained by the "Unicode Stability Policies" posted on the Unicode web site 🛡️ Significant changes are made in consultation with liaison organizations, like W3C and IETF, which have standards that could be affected by them 🫶 See sections 5 and 6 below 👇

typical outputs of the UTC are:

  1. The Unicode Standard, major and minor versions (including the Unicode Character Database)
  2. Unicode Technical Reports
  3. Stand-alone Unicode Technical Standards
  4. Formal resolutions
  5. Liaison statements and instructions to the Unicode liaisons to other organizations

for each technical item on the meeting agenda, the general process goes like this:

  1. Introduction by the topic sponsor
  2. Proposals and discussion
  3. Consensus statements or formal motions
  4. Assignment of formal actions to implement decisions

5. Unicode Technical Committee Motions 🗳️

technical topics of any complexity never go from first proposal to final ratification or adoption into the standard in just one UTC meeting 🙅‍♀️ The UTC members and presiding officers know that technical changes to the standard have broad consequences for other standards, implementers, and end-users 🌍 Input from other organizations and experts is often vital for understanding proposals and for successfully getting them adopted into the standard 🧠

technical topics are decided in the UTC through formal motions, either taken during meetings or through thirty-day letter ballots 📬 There are two types of formal UTC motions:

  1. Simple motions
  2. Precedents

simple motions can pass with a simple majority, meaning more than 50 percent of the qualified voting members 🙌 Or they can pass with a special majority, which means two-thirds or more of the qualified voting members 💪

precedents are defined, according to the UTC Procedures, as either:

(A) an existing Unicode Policy, or (B) an explicit precedent.

precedents must be passed or overturned by a special majority 🔐

examples of implicit precedents include:

  1. Publication of a character in the standard
  2. Published normative character properties
  3. Algorithms required for formal conformance

an Explicit Precedent is a policy, procedure, encoding, algorithm, or other item that's established by a separate motion saying (in effect) that a particular prior motion establishes a precedent 📌

a proposal can pass either by a formal motion and vote, or by consensus 🫶 If there's broad agreement on the proposal, and no member wants to force a vote, then it passes by consensus and gets recorded that way in the minutes 📝✨

6. Unicode Consortium Policies ✨

Okay bestie, so the Unicode Standard is literally always growing 🌈 because the big dream is to encode "all the world's scripts," and that means new characters keep showing up. In that sense the standard is kinda unstable, and there might never be a moment where it's totally done adding characters. 💅 So the Consortium came up with some policies to keep the characters that already exist stable, plus a Roadmap for what's coming next 🗺️✨

The Consortium's overall policies, including the ones about encoding stability and privacy, are posted on a "Unicode Consortium Policies" web page 🌸 And the UTC's discussions and encoding proposals have to follow those policies. No exceptions, bestie 🫶

Here's the vibe of the stability policies: once a character is encoded, it doesn't get moved or removed, and its name doesn't change. Any of those things could make existing data obsolete, so they are not allowed 🙅‍♀️ The canonical combining class and decompositions of characters won't be changed in any way that affects normalization. So normalization, which is used for International Domain Naming and for "early normalization" on the World Wide Web, is locked in and stable for every character from the moment it gets encoded 🔒💖 (If there are outright errors in properties or decompositions, those get fixed through an adjunct data file, so people who depend on normalization stability can still keep it 🎀)

Once it's published, every version of the Unicode Standard is absolutely stable and will never be changed retroactively 💜 Implementations or specifications that point to a specific version can totally rely on that. If they later upgrade to a newer version, though, some changes might be needed 🦋

Property values of characters, like directionality for the Unicode Bidi algorithm, can change between versions in some situations 👀 Less-well-documented characters and scripts can be encoded before anyone fully knows their exact properties and behavior. Once people have more experience implementing them, the property values might need adjusting. That re-working is kept to a minimum, and old and new versions of the property tables are available on the Consortium's web site 🌟

Normative and some informative character data lives in the Unicode Character Database (UCD) 📚 The structure of many of these property values won't change. When new properties are defined, the Consortium adds new files for them instead, so existing implementations that use the current values and formats aren't affected 🧚 The latest UCD is on the Consortium web site under the "Unicode Data" heading.

Note on data redistribution: Unlike IETF documents, some parts of the Unicode Character Database may have restrictions on verbatim redistribution with source-code products. Users should read the notices in any files they plan to use in such products 🙈 The information in the UCD can be freely used to create derivative works (like programs, compressed data files, subroutines, data structures, etc.) that can be redistributed freely, but some files may not be redistributable verbatim. These restrictions are never meant to stop or control the use of the data in products. They just help make sure people grab the latest official releases of the data files 💅

7. UTC and ISO (WG2) 🌍

The character repertoire, names, and general architecture of the Unicode Standard are identical to the parallel international standard ISO/IEC 10646 🤝 ISO/IEC 10646 only includes a small fraction of the semantics, properties, and implementation guidelines in the Unicode Standard and its associated technical standards and reports. Implementations that conform to Unicode also conform to ISO/IEC 10646 ✨

ISO/IEC 10646 is maintained by the committee ISO/IEC JTC1/SC2/WG2. WG2 is made up of national body representatives to ISO. You can find details about the ISO organization on the official ISO web site 🌸

You can read about the history and details of the relationship between ISO/IEC JTC1/SC2/WG2 and Unicode, Inc. in Appendix C of The Unicode Standard 📖 (A PDF of the most recent printed edition is on the Unicode web site.)

WG2 shares the UTC's stability policies: WG2 doesn't remove characters or change their names once they're published. Both committees closely track changes in their standards, and they work super closely to keep the two standards in sync 🫶

The Unicode Collation Algorithm (UCA) is one of a small set of other independent standards defined and maintained by the UTC 💫 It isn't technically part of the Unicode Standard itself. It's defined separately in Unicode Technical Standard #10 (UTS #10). There's no conformance relationship between the two, except that conformance to a specific base version of the Unicode Standard (e.g., 4.0) is specified in a particular version of a UTS 🦋 The collation algorithm in UTS #10 conforms to ISO/IEC 14651, which is maintained by ISO/IEC JTC1/SC2, and the two organizations have a close relationship 💜 Beyond what ISO/IEC 14651 specifies, the UCA adds extra constraints on collation, specifies additional options, and provides lots more implementation guidelines 🎀

8. Process of Technical Changes to the Unicode Standard 🔥

Changes to The Unicode Standard come in two types: architectural changes and character additions 🧩

Most architectural changes don't affect ISO/IEC 10646. For example, adding various character properties to Unicode doesn't touch it 🙌 The architectural changes that do affect both standards, like additional UTF formats or allocation of planes, are very carefully coordinated by the committees 👑 On the UTC side, architectural changes that set precedents are carefully monitored, and the rules and procedures described above are followed 💅

Additional characters for inclusion in The Unicode Standard must be approved by both the UTC and WG2 🫶 Proposals for additional characters enter the standards process in one of several ways:

  1. a national body member of WG2
  2. a member company or associate of UTC
  3. directly from an individual "expert" contributor

The two committees have jointly produced a "Proposal Summary Form" that is required to accompany all additional character proposals 📝 You can find this form on the WG2 web site, and on the Unicode web site along with information about "Submitting New Characters or Scripts". Instructions for submitting proposals to the UTC are online too 🌈

Often, proposals are submitted to both committees (UTC and WG2) at the same time 💖 UTC members also frequently forward proposals to WG2 after the UTC has reviewed them initially ✨

In general, a proposal submitted to the UTC before WG2 goes through several stages:

  1. Initial presentation to UTC
  2. Review and re-drafting
  3. Forwarding to WG2 for consideration
  4. Re-drafting for technical changes
  5. Balloting for approval in UTC
  6. Re-forwarding and recommendation to WG2
  7. At least two rounds of international balloting in ISO

This whole thing takes about two years ⏳ Initial proposals often don't include enough information or justification to get approved. Those get sent back to the submitters with comments about how to amend or extend them 🙈 Repertoire addition proposals submitted to WG2 before the UTC are usually forwarded straight to the UTC through committee liaisons. The crucial parts of the process (steps 5 through 7 above) are never short-circuited 🚫 A two-thirds majority in the UTC is required for approval at step 5 👑

Proposals for additional scripts are required to be coordinated with relevant user communities 🌸 There are often ad-hoc subcommittees of the UTC, or expert mailing list participants, who actually draft proposals, gather community support, or represent user communities 🧚

The rounds of international balloting in step 7 include participation from both the UTC and WG2, though the UTC doesn't directly vote in the ISO process 🗳️

Sometimes a proposal approved by one body is considered too immature for approval by the other, and either of the two may block it de facto 😬 Only after both bodies approve the additional characters do they move on to the rounds of international balloting. (The first round is a draft international standard, where some changes may happen. The second round is final approval, where only editorial changes are made.) 🦋

This process makes sure proposals for additional characters are mature and stable by the time they show up in a final international ballot 🏆✨

9. Public Access to the Character Encoding Process 🌸

okay so bestie, this section is lowkey the "anyone can come to the party" section 🎀 Unicode, Inc. is a membership org and the final say on technical stuff belongs to UTC 💅 but the process is SUPER open to public input and people checking the procedures and proposals. There are loads of influential individual experts and industry groups who aren't formal members, and UTC still takes their input seriously 🫶

Internally, UTC runs a mail list called the "Unicore" list 📬 It carries traffic about meetings, the technical content of the standard, and so on. Who's on it? UTC representatives, employees and staff of member orgs (like the Research Libraries Group), individual liaisons to and from other standards bodies (like WG2 and IETF), and invited experts from places like the Library of Congress and some universities. External individuals need "sponsorship" from the corporate officers to subscribe, so it's not just anyone 👑

Then there's the public one! Unicode, Inc. also runs a discussion list called the "Unicode" list 🦋 Anyone can subscribe, and the proceedings are publicly archived. You can find the details on the Consortium web site under the "Mail Lists" heading 🍓

Technical proposals for changes to the standard get posted to both lists on a regular basis ✨ Discussion on the public list might turn into a written proposal for a later UTC meeting. Technical issues and other standardization "events" with real significance, like beta releases and draft documents being available, get announced and discussed in that public forum well before standardization is finalized 🔥 And every so often UTC publishes "Public Review Issues" on the Consortium web site 🧚 These gather feedback on specific proposals whose impact might be unclear, or that haven't yet gotten broad enough review before the UTC deliberations.

Anyone can make a character encoding or architectural proposal to UTC 💖 You don't have to be a member to submit one. To be taken seriously, though, the proposal has to be framed in a substantial way and come with enough documentation to be worth discussing 🫶 Examples are easy to find by following links from the "Proposed Characters" and "Roadmaps" headings on the Unicode web site, and guidelines are under the "Submitting Proposals" heading 🌈

In general, proposals get aired publicly on the "Unicode" list 🌸 sometimes for a long time before they're formally submitted. This is honestly a win for the proposer, because it tends to cut down on how often the proposal gets sent back for clarification or for more information 💅 By the time a proposal is ready for UTC discussion, the proposer will have been in contact through the public list with one or more UTC members who are willing to explain or defend it at a UTC meeting. ✨

10. Acknowledgements 🫶

Huge thanks to Mark Davis, Simon Josefsson, and Ken Whistler 💜 for their extensive review and feedback on previous versions of this document. Iconic behaviour 👑

11. Security Considerations 🔐

This memo describes the operational procedures of an organization 🏢 The procedures themselves have no consequences for Internet Security. Not a security-drama situation, bestie 💀

12. Author's Address 🎀

Rick McGowan c/o The Unicode Consortium P.O. Box 391476 Mountain View, CA 94039-1476 U.S.A.

Phone: +1-650-693-3921 Web: http://www.unicode.org/

Copyright (C) The Internet Society (2004). This document is subject to the rights, licenses and restrictions contained in BCP 78 and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property 🦋

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

Acknowledgement 💖

Funding for the RFC Editor function is currently provided by the Internet Society. 🌈

📦 where this page comes from