Encryption for OpenVMS Installation and Reference Manual


Previous Contents Index


Chapter 4
Programming with Encryption for OpenVMS Routines

To program encryption operations into applications, use the Encryption for OpenVMS callable routines. Encryption provides the following routines, listed by function:

Installation Note

To use the programming interface from an image that was installed with privileges, ensure that the system startup procedure installs the ENCRYPSHR shareable image as a known image (see Section 2.4).

4.1 How the Routines Work

You can call the Encryption for OpenVMS routines from any language that supports the OpenVMS Procedure Calling Standard. After it is called, each routine:

The callable routines do not provide all the options of the file selection qualifiers available with the DCL ENCRYPT and DECRYPT commands. The functions of /BACKUP, /BEFORE, /BY_OWNER, /CONFIRM, /EXCLUDE, /EXPIRED, /SINCE, and /SHOW are supported only at the DCL-interface level. For more information, see the Guide to Creating OpenVMS Modular Procedures.

4.1.1 DES Key and Data Semantics

The NBS document FIPS-PUB-46 describes the operation of the DES algorithm in detail. The bit-numbering conventions in the NBS document are different from OpenVMS numbering conventions.

If you are using Encryption for OpenVMS routines in conjunction with an independently developed DES encryption system, ensure that you are familiar with the relationship between the NBS and OpenVMS numbering conventions. Table 4-1 highlights the differences.

Table 4-1 Comparison of NBS and OpenVMS Numbering Conventions
NBS Encryption for OpenVMS
Numbers bits from left to right. Numbers bits from right to left.
Displays bytes in memory from left to right. Displays bytes in memory from right to left.
Handles keys and data in 8-byte blocks (see Figure 4-1). Handles 8-byte blocks in OpenVMS display order (see Figure 4-2).
Treats keys and data as byte strings. Treats keys and data as character strings.
The "most significant byte" is byte 1. Same.
In DES keys, the parity bits are DES bits 8, 16, 24, and so forth. In DES keys, the parity bits are OpenVMS bits 0, 8, 16, and so forth.
DES keys, when expressed as strings of hexadecimal digits, are given starting with the high digit of byte 1, then the low digit of byte 1, then the high digit of byte 2, and so forth, through the low digit of byte 8. Same.

To convert a hexadecimal key string into the 8-byte binary key, convert from hex to binary one byte at a time. For example, a quadword hex-to-binary conversion, using the library subroutine OTS$CVT_TZ_L, yields an incorrect, byte-reversed key.

Figure 4-1 OpenVMS Numbering Overlay on FIPS-46 Numbering


Figure 4-2 NBS Numbering Overlay on an OpenVMS Quadword


4.2 Maintaining Keys

When you use the Encryption routines, first define the key that will be used in the encryption operation. Similarly, to decrypt a file specify the same key. Table 4-2 describes the callable routines that maintain keys.

Table 4-2 Routines for Maintaining Keys
Routine Description
ENCRYPT$DEFINE_KEY Creates a key definition with a key name and a key value. Puts the definition into a key storage table.

Similar to the ENCRYPT /CREATE_KEY command.
ENCRYPT$DELETE_KEY Removes a key definition from a key storage table. Uses the key name to identify the key to be removed.

Similar to the ENCRYPT /REMOVE_KEY command.
ENCRYPT$GENERATE_KEY Generates random key values.

When you call these routines, use the following arguments:

4.3 Operations on Files

The ENCRYPT$ENCRYPT_FILE routine is similar to the DCL ENCRYPT and DECRYPT commands in that you use this routine with entire files.

The ENCRYPT$ENCRYPT_FILE routine specifies the key, the input file specification, the output file specification, and other file operation information.

Specify the type of operation, either encryption or decryption, with the file-flags argument.

ENCRYPT$ENCRYPT_FILE does not require a prior call to ENCRYPT$INIT.

4.4 Operations on Records and Blocks

To operate on small records or blocks of data, use the following routines:

These routines are a shorthand form of the ENCRYPT$INIT, ENCRYPT$ENCRYPT, ENCRYPT$DECRYPT, ENCRYPT$FINI sequence of calls.

Do not use these routines for data larger than a few records.

4.5 Routine Descriptions

This section describes the syntax of each callable routine. The routines are listed alphabetically.

4.5.1 Specifying Arguments

Each routine's argument list shows the mandatory arguments first, followed by the optional arguments. Brackets ( [ ] ) identify optional arguments in the argument list.

For example, this format line shows that the required arguments are context, input, and output, and that the optional arguments are output-length and p1:

ENCRYPT$DECRYPT context ,input ,output [,output-length] [,p1]

When you specify arguments, follow these guidelines:

4.5.2 Bitmasks

Constants are associated with the symbolic names of the bitmasks used by the Encryption routines. These constants are defined in the ENCRYPT_STRUCTURES files that are provided with the kit.

The examples directory, ENCRYPT$EXAMPLES, has a copy of the ENCRYPT_STRUCTURES file in each supported programming language.

4.5.3 Error Handling

By default, Encryption signals error conditions with messages. To intercept a message that is inappropriate for your application, supply a condition handler.

For information about implementing condition handlers, see your programming language reference manual. For additional documentation, see:


ENCRYPT$DECRYPT

Decrypts the next record of ciphertext according to the algorithm specified in the ENCRYPT$INIT call.

Format

ENCRYPT$DECRYPT context ,input ,output [,output-length] [,p1]


Arguments

context


VMS usage:
type: longword integer (signed)
access: write only
mechanism: by reference

Context area initialized when ENCRYPT$INIT completes execution. The context argument is the address of a longword of unspecified interpretation that is used to convey context between encryption operations.

input


VMS usage:
type: char_string
access: read only
mechanism: by descriptor

Ciphertext record that ENCRYPT$DECRYPT is to decrypt. The input argument is the address of a descriptor pointing to a byte-aligned buffer containing the input record to the decryption operation.

output


VMS usage:
type: char_string
access: write only
mechanism: by descriptor

Plaintext record that results when ENCRYPT$DECRYPT completes execution. The output argument is the address of a descriptor pointing to a byte-aligned buffer that will contain the output record from the decryption operation.

If the descriptor is dynamic and insufficient space is allocated to contain the output record, storage will be allocated from dynamic memory. If insufficient space exists to contain the output of the operation, then an error status is returned.

The ENCRYPT$DECRYPT routine adjusts the length of the output descriptor, if possible, to reflect the actual length of the output string. If the descriptor type is not DSC$K_DTYPE_VS (varying string), DSC$K_DTYPE_V (varying), or DSC$K_DTYPE_D (dynamic), the routine takes the actual output count from the output-length argument.

output-length


VMS usage:
type: word integer
access: write only
mechanism: by reference

Optional argument.

Number of bytes that ENCRYPT$DECRYPT wrote to the output buffer. The output-length argument is the address of a word containing the number of bytes written to the output buffer, including any bytes of pad characters generated by the selected algorithm to meet length requirements of the input buffer, if any. Output length does not count padding in the case of a fixed-length string.

Some encryption algorithms have specific requirements for the length of the input and output strings. In particular, DESECB and DESCBC pad input data with from 1 to 7 bytes to form complete 64-bit blocks for operation. The values of the pad characters are indeterminate.

When you decrypt fewer than 8 bytes, present the full 8 bytes resulting from the ENCRYPT$ENCRYPT to ENCRYPT$DECRYPT. Retain the byte count of the input data in order to strip trailing pad bytes after a subsequent decryption operation.

p1


VMS usage:
type: quadword
access: read only
mechanism: by reference

Optional argument. The p1 argument is the address of a quadword initialization vector used to seed the two modes of the DES algorithm for which it is applicable (DESECB and DESCFB). If this argument is omitted, the initialization vector used is the residue of the previous use of the specified context block. ENCRYPT$INIT initializes the context block with an initialization vector of zero.

Description

The ENCRYPT$DECRYPT routine decrypts the next record of ciphertext according to the algorithm specified in the ENCRYPT$INIT call. Any errors encountered in the operation are returned as status values. The message authentication mode (DESMAC) is not supported by ENCRYPT$DECRYPT.

The ENCRYPT$DECRYPT routine returns a 32-bit status code indicating the success or failure of the routine's operation.


Condition Values Returned

SS$_NORMAL Record successfully decrypted.
ENCRYPT$ xyz An error reported by the Encryption software. xyz identifies the message (see Appendix B).
SS$_ xyz A return status from a called system service. xyz identifies the return status.

ENCRYPT$DECRYPT_ONE_RECORD

Decrypts a small amount of data on a decrypt stream.

Format

ENCRYPT$DECRYPT_ONE_RECORD input ,output ,key-name ,algorithm


Arguments

input


VMS usage:
type: char_string
access: read only
mechanism: by descriptor

Ciphertext record to be decrypted. The input argument is the address of a string descriptor pointing to a byte-aligned buffer containing the input record to be decrypted.

output


VMS usage:
type: char_string
access: write only
mechanism: by descriptor

Plaintext record resulting when ENCRYPT$DECRYPT_ONE_RECORD completes execution. The output argument is the address of a string descriptor pointing to a byte-aligned buffer that will contain the plaintext record.

If the descriptor is dynamic and insufficient space is allocated to contain the output record, storage is allocated from dynamic memory. If insufficient space exists to contain the output of the operation, an error is returned.

The ENCRYPT$DECRYPT_ONE_RECORD routine adjusts the length of the output descriptor, if possible, to reflect the actual length of the output string.

key-name


VMS usage:
type: char_string
access: read only
mechanism: by descriptor

Key used to initialize the decrypt stream. The key-name argument is the address of a string descriptor pointing to the name of the previously defined user key to be used.

algorithm


VMS usage:
type: char_string
access: read only
mechanism: by descriptor

Algorithm used for the decryption operation. The algorithm argument is the address of a string descriptor pointing to a code for the selected algorithm. The algorithm code is an ASCII string. Specify the descriptor type value as one of the following:

The following algorithms are valid:


Description

In some applications, only a small amount of data needs to be decrypted on a particular decrypt stream. The ENCRYPT$DECRYPT_ONE_RECORD routine allows you to perform such a decryption operation.

The ENCRYPT$DECRYPT_ONE_RECORD routine is a shorthand form of the ENCRYPT$INIT, ENCRYPT$DECRYPT, and ENCRYPT$FINI sequence of calls. However, using ENCRYPT$DECRYPT_ONE_RECORD repeatedly to decrypt records of a file is extremely inefficient.

The ENCRYPT$DECRYPT_ONE_RECORD routine returns a 32-bit status code indicating the success or failure of the routine's operation.


Condition Values Returned

SS$_NORMAL Operation performed.
ENCRYPT$ xyz An error reported by the Encryption software. xyz identifies the message (see Appendix B).
SS$_ xyz A return status from a called system service. xyz identifies the return status.

ENCRYPT$DEFINE_KEY

Places a key definition into the process, group, job, or system key storage table.

Format

ENCRYPT$DEFINE_KEY key-name ,key-value ,key-flags


Arguments

key-name


VMS usage:
type: char_string
access: read only
mechanism: by descriptor

Name of the key defined when ENCRYPT$DEFINE_KEY completes execution. The key-name argument is the address of a string descriptor pointing to a char_string that is interpreted as the name of the key to be defined. A maximum of 243 characters is permitted.

Note

Key names beginning with ENCRYPT$ are reserved for Compaq.

key-value


VMS usage:
type: char_string
access: read only
mechanism: by descriptor

Key value defined when ENCRYPT$DEFINE_KEY completes execution. The key-value argument is the address of a string descriptor pointing to a vector of unsigned byte values that are assigned to the named key. A maximum of 240 bytes may be assigned.

key-flags


VMS usage:
type: longword
access: read only
mechanism: by reference

Flags that ENCRYPT$DEFINE_KEY uses when defining a key. The key-flags argument is the address of a longword containing flags that control the key definition process.

Each flag has a symbolic name. The constants associated with these names are defined in the ENCRYPT$EXAMPLES:ENCRYPT_STRUCTURES files in various programming languages. Table 4-3 defines the function of each flag.

Table 4-3 ENCRYPT$DEFINE_KEY Flags
Flag Function
Symbolic Name Function
ENCRYPT$M_KEY_PROCESS Places definition in process table
ENCRYPT$M_KEY_GROUP Places definition in group table
ENCRYPT$M_KEY_JOB Places definition in job table
ENCRYPT$M_KEY_SYSTEM Places definition in system table
ENCRYPT$M_KEY_LITERAL Stores key without compressing


Description

The ENCRYPT$DEFINE_KEY routine places a key definition into the process, group, job, or system key storage table. The key value supplied with the routine is processed as specified and placed in the key storage table under the indicated name. The ENCRYPT$DEFINE_KEY routine does not interpret the key value.

By default, keys are treated as char_string keys, using the Digital Multinational Character Set and are compressed before being inserted into the key storage table. The compression proceeds as follows:

  1. The string is converted to uppercase characters.
  2. The digits 0 through 9 are left unchanged.
  3. All characters except letters, digits, dollar signs, periods, and underscores are converted to spaces.
  4. All sequences of multiple spaces (or characters that have been converted into spaces) are converted into single spaces.

When a char_string key is retrieved from key storage for use as a DES key, it is folded into an 8-byte key by exclusive OR-ing 8-byte segments of the key string together, and then applying odd parity to each byte by modifying the sign bit (bit 7).

The key flag ENCRYPT$M_KEY_LITERAL specifies that the key string supplied is a binary key. A binary key is not compressed, but is placed into key storage as is. When a binary key is used as a DES key, it is likewise folded into an 8-byte key by exclusive OR-ing 8-byte segments together. Odd parity is then applied by modifying the low bit (bit 0) of each byte.

The ENCRYPT$DEFINE_KEY routine returns a 32-bit status code indicating the success or failure of the routine's operation.


Condition Values Returned

SS$_NORMAL Key has been defined.
ENCRYPT$ xyz An error reported by the Encryption software. xyz identifies the message (see Appendix B).
SS$_ xyz A return status from a called system service. xyz identifies the return status.


Previous Next Contents Index