10.6 Handling Different Character Sets

The DEC C RTL supports a number of state- independent codesets and codeset encoding schemes that contain the ASCII encoded Portable Character Set. It does not support state- dependent codesets. The codesets supported are:

• ISO8859-n

  where n = 1,2,5,7,8 or 9. This covers codesets for North America, Europe (West and East), Israel, and Turkey.

• eucJP, SJIS, DECKANJI, SDECKANJI: Codesets used in Japan.

• eucTW, DECHANYU, BIG5, DECHANZI: Chinese codesets used in China (PRC), Hong-Kong, and Taiwan.

• DECKOREAN: Codeset used in Korea.

10.6.1 Charmap File

The characters in a codeset are defined in a charmap file. The charmap files supplied by Digital are located in the directory defined by the SYS$I18N_LOCALE logical name. The file type for a charmap file is .CMAP.

10.6.2 Converter Functions

As well as supporting different coded character sets, the DEC C RTL provides the following converter functions that enable you to convert characters from one codeset to another:

• iconv_open-specifies the type of conversion. It allocates a conversion descriptor required by the iconv function.

• iconv-converts characters in a file to the equivalent characters in a different codeset. The converted characters are stored in a separate file.

• iconv_close-deallocates a conversion descriptor and the resources allocated to the descriptor.

10.6.3 Using Codeset Converter Files

The file naming convention for codeset converters is:

fromcode_tocode.iconv

Where fromcode is the name of the source codeset, and tocode is the name of the codeset to which characters are converted.

You can add codeset converters to a given system by installing the converter files in the directory pointed by the logical name SYS$I18N_ICONV.

Codeset converter files can be implemented either as table-based conversion files or as algorithm-based converter files created as OpenVMS shareable images.

Creating a Table-based Conversion File

The following summarizes the necessary steps to create a table-based codeset converter file:

1. Create a text file that describes the mapping between any character from the source codeset to the target codeset. For the format of this file, see the DCL command ICONV COMPILE in the OpenVMS New Features Manual, which processes such a file and creates a codeset converter table file.

2. Copy the resulting file from the previous step to the directory pointed by the logical SYS$I18N_ICONV, assuming you have the privilege to do so.

Creating an Algorithm-based Conversion File

Use the following steps to create an algorithm-based codeset converter file implemented as a shareable image:

1. Create C source files that implement the codeset converter. The API is documented in the public header file <iconv.h> as follows:

   • The universal entry point _u_iconv_open is called by the DEC C RTL routine iconv_open to initialize a conversion.

   • _u_iconv_open returns to iconv_open a pointer to the structure __iconv_extern_obj_t.

   • Within this structure, the converter exports its own conversion entry point and conversion close routine, which are called by the DEC C RTL routines iconv and iconv_close, respectively.

   • The major and minor identifier fields are required by iconv_open to test for a possible mismatch between the library and the converter. The converter usually assigns the constants __ICONV_MAJOR and __ICONV_MINOR, defined in the <iconv.h> header file.

   • The field tcs_mb_cur_max is used only by the DCL command ICONV CONVERT to optimize its buffer usage. This field reflects the maximum number of bytes that comprise a single character in the target codeset, including the shift sequence (if any).

2. Compile and link the modules that comprise the codeset converter as an OpenVMS shareable image, making sure that the file name adheres to the preceding conventions.

3. Copy the resulting file from the previous step to the directory pointed by the logical SYS$I18N_ICONV, assuming you have the privilege to do so.

Some Final Notes

SYS$I18N_ICONV is by default a search list where the first directory in the list SYS$SYSROOT:[SYS$I18N.ICONV.USER] is meant for use as a site-specific repository for iconv codeset converters.

The number of codesets and locales installed vary from system to system. Check the SYS$I18N directory tree for the codesets, converters, and locales installed on your system.