Updated: 11 December 1998

OpenVMS Programming Concepts Manual

A protected subsystem is a set of application programs that allow controlled access to objects. It has under its control one or more protected objects and a gatekeeper application. Users cannot access the objects within the subsystem unless they execute the gatekeeper application. Once users have successfully executed the application, their process rights list acquires the identifiers necessary to access objects owned by the subsystem. The identifiers allow processes to use the resources of the subsystem. When the application completes execution or the user exits, the identifiers are removed from the user's process rights list. Protected subsystems are an alternative to creating privileged images and protected shareable images (user-written system services), and help prevent the overuse of privileges.

Roles and Responsibilities

You should think of a protected subsystem as an isolated security domain where the system manager creates and grants SUBSYSTEM identifiers using the Authorize utility as shown in the following example:

UAF> ADD/IDENTIFIER FOO/ATTRIBUTES=SUBSYSTEM UAF> GRANT/IDENTIFIER FOO FRANK /ATTRIBUTES=SUBSYSTEM

The system manager can delegate responsibility for the maintenance of the subsystem to subsystem managers who can associate existing identifiers with the subsystem executable and its data. In the following example, the manager of a protected subsystem creates an ACE in a subsystem's image and data files:

$ SET SECURITY BLOP.EXE - _$ /ACL=(SUBSYSTEM, IDENTIFIER=FOO) - $ SET SECURITY BLOP.DAT - _$ /ACL=(IDENTIFIER=FOO, ACCESS=READ+WRITE) - $ SET SECURITY BLOP.EXE - _$ /ACL=(IDENTIFIER=HARRY, ACCESS=EXECUTE) -

Finally, a user uses the protected subsystem to access data available only through the subsystem.

Subsystem Security

During the execution of a protected subsystem, $IMGACT adds subsystem identifiers to the image rights list. What happens if the user presses the Ctrl/Y key sequence during execution? Will the user retain whatever privileges were granted by the subsystem? If Ctrl/Y is pressed, image identifiers are removed from the process. Also, subprocesses do not inherit image identifiers by default. However, SYS$CREPRC and LIB$SPAWN do contain flags PRC$M_SUBSYSTEM and SUBSYSTEM, respectively, that allow subprocesses to inherit image identifiers.

25.7 Security Auditing

Auditing is the recording of security-relevant activity as it occurs on a system. See the OpenVMS Guide to System Security for a list of all types of security-relevant activity or classes of events that are audited. The following table describes the security services that provide security auditing:

Service	Description
SYS$AUDIT_EVENT	Appends an event message to the system audit log file or sends an alarm to a security operator terminal
SYS$CHECK_PRIVILEGE	Determines whether the caller has the specified privileges or identifiers

The system service SYS$AUDIT_EVENT is used to report security events to the auditing system. It examines the settings of the DCL command SET AUDIT to determine if an event is enabled for auditing. If the event is enabled for alarms or audits, SYS$AUDIT_EVENT generates an audit record and appends it to the system audit log file (or sends an alarm to a security operator terminal) that identifies the process involved and lists information supplied by its caller.

25.8 Checking Access Protection

The operating system provides two system services that allow a process to check access to objects on the system: SYS$CHKPRO and SYS$CHECK_ACCESS. The SYS$CHKPRO service performs the system access protection check on a user attempting direct access to an object on the system; SYS$CHECK_ACCESS performs a similar check on a third party attempting access to an object. The following table describes the security services that provide access checking:

Service	Description
SYS$CHECK_ACCESS	Invokes a system access protection check on behalf of another user
SYS$CHKPRO	Invokes a system access protection check

The SYS$CHKPRO and SYS$CHECK_ACCESS system services have been extended to support auditing. The OpenVMS Guide to System Security describes how to use the auditing function. The OpenVMS System Services Reference Manual: A--GETMSG describes how to use the two system services. These services are described in the following sections.

25.8.1 Creating a Security Profile

The SYS$CREATE_USER_PROFILE system service returns a user profile, using information in the rights database and the system authorization database to generate the profile. The system services SYS$CHECK_ACCESS or SYS$CHKPRO accept as input the profile from SYS$CREATE_USER_PROFILE.

25.8.2 SYS$CHKPRO System Sevice

The SYS$CHKPRO system service invokes the access protection check used by the system. The service does not grant or deny access; rather, it performs the protection check. Subsequently, an application might grant or deny access to the specified object.

To pass the input and output information to SYS$CHKPRO, use the itmlst argument, which is the address of an item list of descriptors. The SYS$CHKPRO service compares the item list of the rights and privileges of the accessor to a list of the protection attributes of the object to be accessed. If the accessor can access the object, SYS$CHKPRO returns the status SS$_NORMAL; if the accessor cannot access the object, SYS$CHKPRO returns the status SS$_NOPRIV. The SYS$CHKPRO service does not grant or deny access. The subsystem itself must grant or deny access based on the output (SS$_NORMAL or SS$_NOPRIV) from SYS$CHKPRO.

The SYS$CHKPRO service also returns an item list of the rights or privileges that allowed the accessor access to the object, as well as the names of security alarms raised by the access attempt. For information about the item codes defined for SYS$CHKPRO, see the description of SYS$CHKPRO in the OpenVMS System Services Reference Manual.

See the OpenVMS Guide to System Security for a flowchart describing how SYS$CHKPRO evaluates an access request attempt.

25.8.3 SYS$CHECK_ACCESS System Service

The SYS$CHECK_ACCESS service performs a protection check on a third-party accessor. An example of this is a file server program that uses SYS$CHECK_ACCESS to ensure that an accessor (the third party) requesting a file has the required privileges to do so.

You pass the input and output information to SYS$CHECK_ACCESS by using the itmlst argument, which is the address of an item list of descriptors. You also pass the name of the accessor and the name and type of the object being accessed by using the usrnam, objnam, and objtyp arguments, respectively. The SYS$CHECK_ACCESS service compares the rights and privileges of the accessor to a list of the protection attributes of the object to be accessed. If the accessor can access the object, SYS$CHECK_ACCESS returns the status SS$_NORMAL; if the accessor cannot access the object, SYS$CHECK_ACCESS returns the status SS$_NOPRIV.

The SYS$CHECK_ACCESS service does not grant or deny access. The subsystem itself must explicitly grant or deny access based on the output (SS$_NORMAL or SS$_NOPRIV) from SYS$CHECK_ACCESS.

The SYS$CHECK_ACCESS service also returns an item list of the rights or privileges that allowed the accessor to access the object, as well as the names of security alarms raised by the access attempt. For information about the item codes defined for SYS$CHECK_ACCESS, see the description of SYS$CHECK_ACCESS in the OpenVMS System Services Reference Manual.

25.9 SYS$CHECK_PRIVILEGE

The SYS$CHECK_PRIVILEGE system service determines whether the caller has the specified privileges or identifiers. The service performs the privilege check and looks at the SET AUDIT settings to determine whether the system administrator enabled privilege auditing. When privilege auditing is enabled, SYS$CHECK_PRIVILEGE generates an audit record. The audit record identifies the process (subject) and privilege involved, provides the result of the privilege check, and lists supplemental event information supplied by its caller. Privilege audit records usually contain the DCL command line or the system service name associated with the privilege check.

SYS$CHECK_PRIVILEGE completes asynchronously; that is, it does not wait for final status. For synchronous completion, use the SYS$CHECK_PRIVILEGEW service.

25.10 Implementing Site-Specific Security Policies

Occasionally, you may need to write routines that implement site-specific policies or special algorithms. The routines that you write can either replace or augment built-in operating system policies. This section contains instructions for replacing key operating system security routines with routines that are specific to your site. Two types of routines are discussed: loadable system services and shareable images.

25.10.1 Creating Loadable Security Services

This section describes how to create a system service image and how to update the SYS$LOADABLE_IMAGES:VMS$SYSTEM_IMAGES.DATA file, which controls site-specific loading of system images. These procedures update the loading of system images for all nodes of a cluster.

Currently, you can replace the following three system services with services specific to your site:

Service	Description
SYS$ERAPAT	Generates a security erasure pattern
SYS$MTACCESS	Controls magnetic tape access
SYS$HASH_PASSWORD	Applies a hash algorithm to an ASCII password

When you create the system service, you code the source module and define the vector offsets, the entry point, and the program sections for the system service. Then, you can assemble and link the module to create a loadable image.

Once you have created the loadable image, you install it. First, you copy the image into the SYS$LOADABLE_IMAGES directory and add an entry for it in the operating system's images file using the System Management utility (SYSMAN). Next, you invoke the system images command procedure to generate a new system image data file. Finally, you reboot the system to load your service.

The following sections describe how to create and load the the Get Security Erase Pattern (SYS$ERAPAT) system service.

On VAX only systems, you can find an example of the SYS$ERAPAT system service in SYS$EXAMPLES:DOD_ERAPAT.MAR on the operating system. The description here also applies to the Hash Password (SYS$HASH_PASSWORD) and Magnetic Tape Accessibility (SYS$MTACCESS) system services. You can find an example of how to prepare and load the SYS$HASH_PASSWORD service in SYS$EXAMPLES:HASH_PASSWORD.MAR.

25.10.1.1 Preparing and Loading a System Service

With the following VAX only example, use this procedure to prepare and load a system service, in this case SYS$ERAPAT:

1. Create the source module.
   1. Include the following macro to define system service vector offsets:

      $SYSVECTORDEF ; Define system service vector offsets

   2. Use the following macro to define the system service entry point:

      SYSTEM_SERVICE ERAPAT, - ; Entry point name <R4>, - ; Register to save MODE=KERNEL,- ; Mode of system service NARG=3 ; Number of arguments

      
      (The code immediately following this macro is the first instruction of the SYS$ERAPAT system service.)

   3. Use the following macros to declare the desired program sections:

      DECLARE_PSECT EXEC$PAGED_CODE ; Pageable code PSCET DECLARE_PSECT EXEC$PAGED_DATA ; Pageable data PSECT DECLARE_PSECT EXEC$NONPAGED_DATA ; Nonpageable data PSECT DECLARE_PSECT EXEC$NONPAGED_CODE ; Nonpageable code PSCET

2. Assemble the source module by using the following command:

   $ MACRO DOD_ERAPAT+SYS$LIBRARY:LIB.MLB/LIB

3. Link the module to create a SYS$ERAPAT.EXE executive loaded image. You can link the module using the command procedure DOD_ERAPAT_LNK.COM in SYS$EXAMPLES. (A command procedure is also available to link the SYS$HASH_PASSWORD example.) To link the SYS$ERAPAT module, enter the following command:

   $ @SYS$EXAMPLES:DOD_ERAPAT_LNK.COM

4. Prepare the operating system image to be loaded.
   1. Copy the SYS$ERAPAT.EXE image produced by the link command into the SYS$COMMON:[SYS$LDR] directory. Note that privilege is required to put files into this directory.
   2. Add an entry for the SYS$ERAPAT.EXE image in the SYS$UPDATE:VMS$SYSTEM_IMAGES.IDX data file.
      You add an entry by using the SYSMAN command SYS_LOADABLE ADD. (See the OpenVMS System Management Utilities Reference Manual for a description of this command.) For example, the following commands add an entry in VMS$SYSTEM_IMAGES.IDX for SYS$ERAPAT.EXE:

      $ RUN SYS$SYSTEM:SYSMAN SYSMAN> SYS_LOADABLE ADD _LOCAL_ SYS$ERAPAT - _SYSMAN> /LOAD_STEP = SYSINIT - _SYSMAN> /SEVERITY = WARNING - _SYSMAN> /MESSAGE = "failure to load SYS$ERAPAT.EXE"

      
      This entry specifies that the SYS$ERAPAT.EXE image is to be loaded by the SYSINIT process during the bootstrap. If there is an error loading the image, the following messages are printed on the console terminal:

      %SYSINIT-E-failure to load SYS$ERAPAT.EXE -SYSINIT-E-error loading <SYS$LDR>SYS$ERAPAT.EXE, status = "status"

      
      The following table shows other error messages that may be returned:

      Message	Meaning	User Action
      NO_PHYSICAL_MEMORY	Physical memory is not available.	Check SYSGEN parameters.
      NO_POOL	Amount of nonpaged pool is insufficient.	Check SYSGEN parameters.
      MULTIPLE_ISDS	Encountered more than one image section of a given type.	Check link options.
      BAD_GSD	An inconsistency was detected.	Verify that the image was linked properly.
      NO_SUCH_IMAGE	The requested image cannot be located.	Check image name against images in SYS$LOADABLE_IMAGES.

   3. Invoke the SYS$UPDATE:VMS$SYSTEM_IMAGES.COM command procedure to generate a new system image data file (VMS$SYSTEM_IMAGES.DATA). The system bootstrap uses this image data file to load the appropriate images into the system.
   4. Reboot the system, which loads the original SYS$ERAPAT.EXE image into the system. Subsequent calls to the SYS$ERAPAT system service use the normal operating system routine.
      As the default, the system bootstrap loads all images described in VMS$SYSTEM_IMAGES.DATA. You can disable this feature by setting the special system parameter LOAD_SYS_IMAGES to 0.

With the following VAX only example, use this procedure to remove an executive loaded image; in this case, SYS$ERAPAT.EXE:

1. Enter the following SYSMAN command:

   SYSMAN> SYS_LOADABLE REMOVE _LOCAL_ SYS$ERAPAT

2. Invoke the SYS$UPDATE:VMS$SYSTEM_IMAGES.COM command procedure to generate a new system image data file (VMS$SYSTEM_IMAGES.DATA). The system bootstrap uses this image data file to load the appropriate images into the system.
3. Reboot the system, which loads the installation-specific SYS$ERAPAT.EXE image into the system. Subsequent calls to the SYS$ERAPAT system service use the installation-specific routine.
   As the default, the system bootstrap loads all images described in the system image data file (VMS$SYSTEM_IMAGES.DATA). You can disable this functionality by setting the special system parameter LOAD_SYS_IMAGES to 0.

A site security administrator can screen new passwords to make sure they comply with a site-specific password policy. (See the OpenVMS Guide to System Security for more information.) This section describes how a security administrator can encode the policy, create a shareable image and install it in SYS$LIBRARY, and enable the policy by setting a SYSGEN parameter.

Installing and enabling a site-specific password policy image requires both SYSPRV and CMKRNL privileges.

25.10.2.1 Creating a Shareable Image

To compile and link a shareable image that filters passwords for words that are sensitive to your site, perform the following steps:

1. Create the source module VMS$PASSWORD_POLICY.*.
   Bliss and Ada examples of the policy module's interface, called VMS$PASSWORD_POLICY.*, are located in SYS$EXAMPLES.
   Define two routine names in the source module: POLICY_PLAINTEXT and POLICY_HASH. These routines must be global (see your language reference for instructions on defining a global routine). The Set Password utility looks for these routine names and displays the message SYMNOTFOU if the names are missing or if the routines are not defined as global.
2. Link the source file.
   For VAX only examples, use the VMS$PASSWORD_POLICY_LNK.COM command procedure, located in SYS$EXAMPLES.

To install a shareable image, perform the following steps:

1. Copy the file to SYS$LIBRARY and install it using the following commands:

   $ COPY VMS$PASSWORD_POLICY.EXE SYS$COMMON:[SYSLIB]/PROTECTION=(W:RE) $ INSTALL ADD SYS$LIBRARY:VMS$PASSWORD_POLICY/OPEN/HEAD/SHARE

2. Set the system parameter LOAD_PWD_POLICY to 1 as follows:

   $ RUN SYS$SYSTEM:SYSGEN SYSGEN> USE ACTIVE SYSGEN> SET LOAD_PWD_POLICY 1 SYSGEN> WRITE ACTIVE SYSGEN> WRITE CURRENT

3. To make the changes permanent, add the INSTALL command from step 1 to the SYS$SYSTEM:SYSTARTUP_VMS.COM file and modify the system parameter file, MODPARAMS.DAT, so that the LOAD_PWD_POLICY parameter is set to 1.
4. Run AUTOGEN as follows to ensure that the system parameters are set correctly on subsequent system startups:

   $ @SYS$UPDATE:AUTOGEN SAVPARAMS SETPARAMS

Copyright © Compaq Computer Corporation 1998. All rights reserved. Legal

5841PRO_067.HTML