Updated: 11 December 1998

OpenVMS Guide to System Security

This section describes additional ways to restrict the data and resources available to users.

8.9.1 Precautions to Take when Installing New Software

When you install new software, you must address several security concerns. You want to ensure that you are not admitting software that will in any way corrupt or undermine your usual security precautions. You must also consider whether to install the software with any privileges. This section discusses the security aspects of installing new software.

8.9.1.1 Potentially Harmful Programs

New software can contain programs that are potentially harmful to your system. These programs, called Trojan horse programs, are designed to do damage and frequently include features that do the following:

• Pass privileges of the person running the program back to the author of the program
• Allow unauthorized access to the system
• Change protection of system files
• Patch the system (add special software to the operating system)
• Create jobs that scan for easily guessed passwords

To protect your system from this type of intrusion, always buy software from reputable sources. When training new users, stress the importance of avoiding use of software from an unknown source.

Another risk to programs and directories is known as the virus. While Trojan horse software must rely on the innocent user to unwittingly accept the damaging software by using it, the virus requires no user cooperation. It is a program that takes advantage of faulty file protection, working its way through your system and modifying command procedures and executable programs. By modifying command procedures, it can propagate by making use of user access rights and privileges.

Viruses are less of a problem in the OpenVMS environment than in an environment of personal computers. The OpenVMS protection features and the environment's larger scale and diversity make virus attacks more difficult. However, no environment that permits the sharing of software and data is immune from virus attacks.

The user's login command procedure is a prime target for this type of security breach. Login command procedures generally contain easily modified DCL commands and are executed regularly.

ACLs are also targets. File protection designed with users sharing access privileges allows this type of program to run through many users' programs, acquiring new privileges along the way.

Well-designed file protection is critical for protection from this type of security breach. Make sure that likely targets cannot be modified by users. For example, set up file protection so that your login command procedure permits at most read access to all other users. Also make sure the directory containing the login command procedure permits write access only to users in the system and owner categories.

Because most damage occurs when programs like these reach a target account with privileges, users with privileges should be especially cautious with the protection of their root directory, executable files, and command procedures. To deter Trojan horse attacks, users should never execute a command procedure or run an image in a privileged account without inspecting the command procedure or the image's sources. Application images should be rebuilt from source to ensure that the binary image reflects the accompanying source.

8.9.1.2 Installing Programs with Privilege

Some software requires privilege to run. You can extend the privilege to all users you expect will need to run the software, or you can install the program with the required privileges. When you install privileged software, you allow users to execute it whether or not they personally possess the required privilege. In effect, you extend the privilege to the process while it runs the software. While this offers some advantages, it also introduces several security-related dangers. Section 8.7 describes these options in greater detail.

8.9.2 Protecting System Files

Even on the most open system, you will want protection for the system software. Normally, Compaq delivers system programs and databases with adequate UIC protection. However, if for any reason you are dissatisfied with the default protection, you can change it with the techniques outlined in Chapter 4, provided you have the necessary SYSPRV privilege. You might also add an ACL to any file that you decide needs additional protection.

You can obtain a full listing of system files from the system manager's account during an OpenVMS installation with the following DCL command:

$ DIRECTORY/SECURITY/OUTPUT=SYSTEM_FILES.LIS SYS$SYSROOT:[*...]

Compaq recommends you generate such a listing and store it for reference. Regularly compare these values with current system file protection to ensure that no tampering has occurred. (The DCL commands DIRECTORY/SECURITY/OUTPUT and DIFFERENCES facilitate such checks.)

On Alpha systems, you can obtain a listing of system files and their protections from the read-only compact disc distribution media. Your OpenVMS software should have this set of protection codes following a correct installation.

On VAX systems, refer to Appendix B for a listing of system files and their protections. Your OpenVMS software should have this set of protection codes following a correct installation.

Table 8-4 provides a summary of DCL commands you use to set up and display file protection; these commands are described in the OpenVMS DCL Dictionary.

Table 8-4 DCL Commands Used to Protect Files

Command	Function
DIRECTORY/ACL	Displays the ACL for the file
DIRECTORY/OWNER	Displays the file owner's UIC
DIRECTORY/PROTECTION	Displays the file's protection code
DIRECTORY/SECURITY	Combines and displays file information produced by DIRECTORY/ACL, DIRECTORY/OWNER, and DIRECTORY/PROTECTION
EDIT/ACL	Invokes the access control list editor (ACL editor)
SET PROTECTION/DEFAULT	Establishes the default protection to be applied to all files subsequently created
SET SECURITY	Modifies the security profile of any object: the owner, protection code, and ACL
SHOW SECURITY	Displays the ownership, UIC protection code, and ACL of a protected object

The OpenVMS installation procedure does not initially install MAIL.EXE with any privileges (because MAIL.EXE does not require privileges to perform its functions). Prior versions of the OpenVMS operating system did include mechanisms that allowed MAIL.EXE to check, ignore, grant, or override certain privileges that a system manager might assign when reinstalling MAIL.EXE. Because these regulatory mechanisms sometimes created unexpected or undesirable conditions, they have been removed.

As indicated, Compaq provides default protection for the system programs that it provides. However, if you have a special requirement, you might examine the potential of ACLs for your needs. For example, you might use ACLs to restrict the use of system programs such as compilers. (Any number of considerations might prompt this action, ranging from performance to licensing issues.)

You might also ask if there are cases where you do not want some or all of your users to be able to initialize media. If there are, you can put an ACL to good use on the system program SYS$SYSTEM:INIT.EXE. Ensure that you grant no access to the world category in the UIC-based protection code. Then create an ACL for the file that grants access to specific users.

Similarly, if a department in your company has paid for a license to a software product, you may want to make that software available to them but not to others. Ensure that the world category receives no access through the standard UIC-based protection code, and create an entry in the ACL for that file that allows access through the department's identifier.

You may also find that ACL protection is relevant to protect your applications databases, limiting the access to certain users or to protected subsystems.

8.9.3 Restricting DCL Command Usage

There are several ways that you can affect the use of DCL commands by your users. Among them are the following:

• Impose ACLs on the system program files in the directories SYS$SYSROOT:[SYSEXE] and SYS$SYSROOT:[SYSLIB].
• Set the AUTHORIZE flag DISIMAGE to prevent use of the MCR or the RUN command. This prevents users from executing system or user-written images or from executing images defined as foreign commands. Because the DISIMAGE flag is enforced by the DCL command language interpreter (CLI), you must ensure that the account for which the DISIMAGE flag is set has access to the DCL CLI only. Use the DISIMAGE flag in conjunction with the AUTHORIZE flag DEFCLI or within a restricted account. (Setting the RESTRICTED flag for an account implicitly sets the DEFCLI flag.)
• Remove or modify DCL command definitions, and rebuild the DCL tables. (The OpenVMS System Management Utilities Reference Manual describes how to create command definitions.) Use the /CLITABLES qualifier in the user's UAF record to specify the modified tables. Also specify /FLAGS=DEFCLI to ensure that the user can log in only with the specified command language interpreter (CLI) and tables. Protect the original DCL tables from unauthorized access by imposing ACLs on the system program files in the directories SYS$SYSROOT:[SYSEXE] and SYS$SYSROOT:[SYSLIB]. In particular, protect SYS$LIBRARY:DCLTABLES.EXE and SYS$SYSTEM:CDU.EXE.

File encryption refers to the process of applying an algorithm to data to conceal its content. Decryption reverses the operation and converts encoded information back to its original content. If you need to copy proprietary software onto media for removal to another site, you might use file encryption. The software on the media is useless without the correct decryption code.

Different file encryption systems, both software and hardware, are available. Consult your Compaq support channel for information on which products are available in your country.

8.9.5 Protecting Disks

Disk scavenging is the process of reading magnetic imprints of data after deletion of the file header following a purge or delete operation. (When users delete files from the system, only the file header is deleted.) Until the data is overwritten, it is a potential target for disk scavenging. Sites with medium or high security needs should be concerned about this procedure.

After establishing overall security features, restrict access to disks containing valuable information by using UIC-based volume protection. Because disk scavenging is frequently performed by authorized users, consider implementing erasure patterns and high-water marking, as described in the following sections.

8.9.5.1 Erasing Techniques

There are several ways to implement erasing of disks.

• The inclusion of the /ERASE qualifier with the DELETE or the PURGE command causes the system to write an erasure pattern of zeros over the entire file location when you delete or purge that file. You can encourage users to use this qualifier voluntarily or make inclusion automatic by including the following command definitions in the system login command procedure (usually SYS$MANAGER:SYLOGIN.COM):

  DEL*ETE :== "DELETE/ERASE" PUR*GE :== "PURGE/ERASE"

  
  However, any user can bypass these definitions by adding the /NOERASE qualifier to the DELETE or the PURGE command.

• To guarantee erase-on-delete, turn on the feature for the entire volume by using the DCL command SET VOLUME/ERASE_ON_DELETE. When files are deleted, this command overwrites all files on the volume with the erasure pattern of zeros.
• To completely erase the volume and enable erase-on-delete for the volume at volume initialization, use the DCL command INITIALIZE/ERASE.
  By default, when erase-on-delete is enabled, the operating system writes a default data security erase (DSE) pattern of zeros, applied during a single write operation over the area. If you feel that the default pattern of zeros or the single rather than multiple number of erasures does not suit your requirements, you can use the $ERAPAT (Get Security Erase Pattern) system service to write a customized erasure pattern. See the description of $ERAPAT in the OpenVMS System Services Reference Manual for more information.

Generally, for sites with high-level security requirements, a random pattern is preferable to a fixed pattern. The technology is already available that can detect and use faint residual magnetic impressions. Thus, if you conclude there is sufficient danger that a disk might be removed and read by some of this specialized analysis equipment, you may need to rewrite the erasure pattern several times. You can learn how to customize the data security erase pattern to fit your needs by studying the information provided in the file SYS$EXAMPLES:DOD_ERAPAT.MAR.

Employ erasing patterns only on disks where the security needs are the greatest. Erasures are time-consuming and affect system performance.

8.9.5.2 Prevention Through High-water Marking

High-water marking refers to a technique that tracks the furthest extent to which each file has been written and prohibits user attempts at reading data beyond that point.

The operating system implements true high-water marking for all sequential, exclusively accessed files, such as the set of files output from various text editors, compilers, and linkers, that is, most files a process writes. The high-water mark is updated in the file header whenever the logical end-of-file mark is updated (usually when the file is closed).

For shared files (both indexed and sequential), the operating system uses the principle of erase-on-allocate to achieve a result similar to true high-water marking. When a file is about to be created or extended, the system determines how much disk space (the extent of the file) is required and applies the security erasure pattern of zeros to the areas (extents) it allocates for writing. The file is then written into the area just erased for it. Thus, if any user gains access to the file (including its full extent) and attempts to read the area beyond where the file has been written, only the data security erase pattern is readable.

By default, the operating system turns on high-water marking for all volumes. High-water marking is a deterrent to disk scavenging attempts. However, it does require additional I/O, which affects system performance.

You can turn off high-water marking and erase-on-allocate on a volume-by-volume basis by specifying the DCL command SET VOLUME/NOHIGHWATER_MARKING.

8.9.5.3 Summary of Prevention Techniques

As security administrator, you can apply the following controls to discourage disk scavengers:

• Provide tight physical security, particularly on those disks with the most valuable information.
• Provide tight volume protection through UIC-based protection.
• Encourage the use of the /ERASE qualifier when key files are purged or deleted through user participation or volume enforcement.
• Permit default high-water marking on your most valuable disks.

You can guard against data loss or corruption by creating copies of your files, directories, and disks. In case of a problem, you can restore the backup copy and continue your work. Secure media storage and controlled access to media are essential parts of the process. It is best to store backup media off site.

8.9.6.1 Backing Up Disks

Having an effective backup schedule is critical to protect your data. By performing regularly scheduled backup operations, you prevent the loss of accidentally deleted or damaged files.

Refer to the OpenVMS System Management Utilities Reference Manual for information about performing backups and setting up backup schedules. Be aware that the Backup utility (BACKUP) does not implement security policy; you must direct it explicitly. It runs with the security profile of the operator, which can often be privileged.

8.9.6.2 Protecting a Backup Save Set

Limiting access to backup save sets is an important part of system security. The file system treats a backup save set as a single file, whether it is stored on disk or on magnetic tape. Therefore, anyone with access to a save set can read any file in the save set. BACKUP does not check protection on individual files.

To maintain system security, it is crucial that you protect save sets adequately. Assign restrictive protection to save sets on disk and to magnetic tape volumes by using the output save-set qualifiers /BY_OWNER and /PROTECTION. Sufficient protection can prevent nonprivileged users from mounting a save-set volume or from reading files from a save set. You should also take physical security precautions with save sets stored off line by keeping backup media in locked cabinets.

When you write a save set to a Files--11 disk or a sequential disk and do not specify the /PROTECTION qualifier, BACKUP applies the process default protection to the save set. If you specify /PROTECTION, any protection categories that you do not specify default to your default process protection.

Protection information is written to the volume header record of a magnetic tape and applies to all save sets stored on the tape. Therefore, the output save-set qualifiers /BY_OWNER and /PROTECTION are effective on magnetic tape save sets only if you specify the output save-set qualifier /REWIND. This qualifier allows the tape to rewind to its beginning, to write the protection data to the volume header record, and to initialize the tape. If you specify /PROTECTION, any protection categories that you do not specify default to your default process protection. If you do not specify /REWIND with the /PROTECTION and /BY_OWNER qualifiers, the magnetic tape retains its existing protection. However, specifying /REWIND alone results in a magnetic tape without any protection.

The following example illustrates how a directory is backed up to tape:

$ BACKUP _FROM: [PAYROLL] _TO: MFA2:KNOX.BCK/LABEL=BANK01 - (1) _$ /REWIND/BY_OWNER_UIC=[030,003] - (2) _$ /TAPE_EXPIRATION=15-JAN-1993 - (3) _$ /PROTECTION=(S:RWE,O:RWED,G:RE,W) (4)

1. The contents of the directory [PAYROLL] is copied to file KNOX.BCK on the magnetic tape drive MFA2. The output save-set qualifier /LABEL provides the label BANK01 for the tape.
2. The output save-set qualifier /BY_OWNER assigns an owner UIC of [030,003] to the save set.
3. The output save-set qualifier /TAPE_EXPIRATION assigns an expiration date of January 15, 1993 to the tape.
4. The output save-set qualifier /PROTECTION assigns the owner of the volume read, write, execute, and delete access. System users are assigned read, write, and execute access; group users are assigned read and execute access; world users are assigned no access.

Anyone who has access to a save set can read any file in the save set. Never give a copy of your backup media to a user; a malicious user could restore the files from the tape or disk and compromise the security of the system.

When a nonprivileged user wants to restore a particular file, do not lend the volume containing the save set. You could give away access to all the files on the volume. The safest way to restore a particular file is to restore the file selectively, as shown in the following example:

$ BACKUP MTA0:JULY.BCK/SELECT=[JONES.TEXTPROC]LASTMONTH.DAT - _$ [*...]/BY_OWNER=ORIGINAL

The selected file is restored with its original directory, ownership, and protection. In this way, the file system determines if the user is permitted access to the file.

8.9.7 Protecting Terminals

The next sections describe the controls available for restricting the use of terminals.

8.9.7.1 Restricting Terminal Use

Through the device object class template TERMINAL, the operating system sets up terminals to be accessible to the SYSTEM account only. When a user logs in, the operating system transfers ownership from a system UIC to the UIC of the current process.

You can limit logins on specific terminals in the following ways:

• Assign a system password.
• Set the terminal to /NOTYPE_AHEAD, making it impossible to log in.

The application of system passwords limits the use of those terminals to users who know the system password.

8.9.7.2 Restricting Applications Terminals and Miscellaneous Devices

To make terminals accessible to certain users as applications terminals, you may want to change any or all of the device's security characteristics. You can include the DCL command SET SECURITY/CLASS=DEVICE for specific terminals (with appropriate protection codes) in the command procedure SYS$MANAGER:SYSTARTUP_VMS.COM. This DCL command can limit access to any device that is not file structured. You might also place an ACL on the device to limit user access.

Copyright © Compaq Computer Corporation 1998. All rights reserved. Legal

6346PRO_020.HTML