Document revision date: 15 July 2002

COM Version 1.2 for OpenVMS, COM Version 1.1-B for OpenVMS, and COM Version 1.1-A for OpenVMS support NTLM (NT LAN Manager) authentication for controlling access to COM objects.

Processes that execute client and server applications must obtain Windows NT credentials in order to be authenticated. Processes created automatically by DCOM$RPCSS to execute server applications obtain Windows NT credentials based on the Registry settings for the server being launched. Interactive processes that are used to execute client and server applications must obtain Windows NT credentials by running the NTA$LOGON utility (see Section 8.2).

This chapter applies to COM for OpenVMS in authenticated mode. See Appendix E for information about running COM for OpenVMS in an unauthenticated environment.

This chapter discusses the following topics:

• How to configure an OpenVMS system for NTLM authentication
• How to acquire Windows NT credentials
• The way security affects COM applications
• The way your domain configuration affects COM applications
• The Application Server run-time environment

NTLM authentication on OpenVMS is implemented in three major components of the operating system (see Section 4.8).

• ACME server --- controls the granting of credentials
• RPC and SSPI --- provide remote procedure calls and Windows NT-style authentication
• Advanced Server for OpenVMS --- maintains Windows NT accounts and provides mapping of Windows NT accounts to OpenVMS accounts

The ACME server, RPC, and SSPI are installed as part of the OpenVMS operating system and require no special configuration. Advanced Server for OpenVMS must be installed as a layered product and must be configured to support NTLM authentication for COM applications (see Section 4.4).

After installing Advanced Server for OpenVMS, you must create network accounts that will be used to execute COM applications. You must also map the network accounts to OpenVMS accounts.

The Advanced Server ADMINISTER utility is used to create network accounts. For example, to create the network account NTUSER1, use the following command:

$ ADMINISTER ADD USER NTUSER1 /PASSWORD="pppppp" /FLAG=NOPWDEXPIRED

The password is case sensitive, so it is enclosed in quotation marks in order to maintain case. A password without quotation marks is converted to uppercase. By default, network accounts are created with the password pre-expired, thus forcing the user to change the password at the first login. The NOPWDEXPIRED flag overrides this default.

A hostmap entry defines the association between a Windows NT user account and a local OpenVMS user account. When OpenVMS authenticates a Windows NT user, OpenVMS uses the hostmap entry to map the OpenVMS user account to the Windows NT user account and build the local OpenVMS user profile and the Windows NT user profile. If no hostmap entry exists, OpenVMS uses the Windows NT user account name as the local OpenVMS user account name.

Use the Compaq Advanced Server for OpenVMS ADMINISTER utility to define hostmap information. For example, to map the network account NTUSER1 to the OpenVMS account VMSUSER1, use the following command:

$ ADMINISTER ADD HOSTMAP NTUSER1 VMSUSER1

If the OpenVMS account does not already exist, you must create the account using the OpenVMS Authorize utility (AUTHORIZE). The OpenVMS account must have the EXTAUTH flag set, or the IGNORE_EXTAUTH flag (bit 11, %X0800) must be set in the SECURITY_POLICY SYSGEN parameter (see Section 5.1). This policy allows the OpenVMS system manager to control which OpenVMS user accounts can be used with Windows NT authentication. For example, to set the EXTAUTH flag for an OpenVMS account VMSUSER1, use the following command. For example:

$ MCR AUTHORIZE MODIFY VMSUSER1 /FLAG=EXTAUTH

5.1.1 LOGINOUT.EXE Use of External Authentication

The EXTAUTH flag also directs LOGINOUT.EXE to use external authentication to authenticate an OpenVMS user during the login process (that is, local, dialup, remote, interactive, and network logins). When you set the EXTAUTH flag, LOGINOUT.EXE uses external authentication, not the password in the SYSUAF.DAT record, to verify the OpenVMS user name and password.

LOGINOUT external authentication always requires that you set the EXTAUTH flag in the SYSUAF account record. Unlike NTA$LOGON and authenticated RPC, you cannot override this requirement using the IGNORE_EXTAUTH flag.

5.1.2 DCE Integrated Login Restriction

A site cannot use both external authentication and the older LGI-callout feature on the same system. If you have an LGI-callout image installed, external authentication is disabled for login purposes. Because DCE integrated login uses the LGI-callout mechanism, OpenVMS does not allow logins using Windows NT-based external authentication if DCE integrated login is enabled.

5.2 Cross-Domain Configuration

You can run a COM application on a system in one domain and have the application authenticated by a system in a second domain.

To configure authentication across Windows NT domains, you must do the following:

1. Set up trust relationships between domains.
   For more information, see the Compaq Advanced Server for OpenVMS Server Administrator's Guide.
2. Set up the HostMapDomains parameter on Compaq Advanced Server for OpenVMS domains (see Example 5-1).
   For more information, see the Compaq Advanced Server for OpenVMS Server Administrator's Guide.
3. Set up account hostmap entries between the Windows NT user account and a local OpenVMS user account.

Example 5-1 shows how you can set up the HostMapDomains parameter. In this example, there are two domains: DOMAIN_1 and DOMAIN_2 . Domain DOMAIN_2 is running Compaq Advanced Server for OpenVMS; domain DOMAIN_1 is a Windows NT domain. The commands in Example 5-1 introduce DOMAIN_2 to DOMAIN_1 .

Example 5-1 Sample: Setting Up HostMapDomains

SYSJANE$ show sym regutl REGUTL == "$SYS$SYSTEM:PWRK$REGUTL.EXE" SYSJANE$ regutl REGUTL> SET PARAM /CREATE VMSSERVER HOSTMAPDOMAINS DOMAIN_1 REGUTL> SHOW VALUE * HOSTMAPDOMAINS Key: SYSTEM\CurrentControlSet\Services\AdvancedServer\UserServiceParameters Value: HostmapDomains Type: String Current Data: DOMAIN_1

5.3 Acquiring Windows NT Credentials

After the network account and the OpenVMS account have been set up as described in Section 5.1, you can log in to the OpenVMS account using the usual OpenVMS login procedures. You can then acquire Windows NT credentials using the NTA$LOGON utility. For example:

$ MCR NTA$LOGON NTUSER1 "pppppp"

In this format, pppppp is the password you specified when you created the network account. The password is enclosed in quotation marks to preserve case. A password without quotation marks is converted to lowercase. If the user name or password is not specified on the command line, the program prompts the user for the required input (see Section 8.2).

To acquire Windows NT credentials for a network account using NTA$LOGON , you must be logged in to the OpenVMS account that is mapped to the network account. Alternatively, if you are logged in to a different OpenVMS account, you must have the IMPERSONATE privilege and use the /OVERRIDE_MAPPING switch. For example:

$ MCR NTA$LOGON /OVERRIDE_MAPPING NTUSER2 "pppppp"

To determine whether a process has Windows NT credentials, use the NTA$LOGON utility with the /LIST switch. For example:

$ MCR NTA$LOGON /LIST

5.4 Application Security

The COM security model allows the creation of secure distributed applications. COM security can be enabled by using settings in the OpenVMS Registry and by using COM security APIs and interfaces. There are two primary areas of security that can be applied to COM applications: launch security and activation security.

Launch security and activation security have system default settings; application-specific settings override these defaults. The settings are stored in the Registry and are maintained by using the DCOMCNFG utility on Windows NT and by using the DCOMCNFG option of DCOM$SETUP.COM on OpenVMS. The COM API CoInitializeSecurityEx can be used from within an application to enhance or override the Registry settings.

5.4.1 Launch Security

Launch security determines which network accounts can be used to create, or "launch" server processes. The launch security settings are referenced when a COM request is received on a system that will result in the launching of a server process to satisfy the request. These settings can explicitly or implicitly allow or disallow a user request to launch a server. The DCOM$RPCSS process authenticates the incoming request to determine the identity of the client. If DCOM$RPCSS determines that it needs to launch a server process to satisfy the request, DCOM$RPCSS allows or disallows the launching of the server based on the identity of the client and the launch security settings.

5.4.2 Activation Security

Activation security determines which network accounts can be used to execute method calls in server applications. The activation security settings are referenced when a COM request is received on a system for a method call in an existing server process. The server process authenticates the incoming request to determine the identity of the client. The server process allows or disallows the execution of the method call based on the identity of the client and the activation security settings.

5.4.3 Server Process Identity

A server process created by DCOM$RPCSS on OpenVMS is a detached process that has an OpenVMS identity and follows all the OpenVMS security rules for a detached process. In addition, it has a network identity that is used to enforce the COM security model (see Section 5.5).

COM servers create separate server threads to execute each client request. These server threads have their own OpenVMS identity and network identity, based on the identity of the client. When a server thread is executing a request on behalf of a client, it is the thread's identities, not the process' identities, that are used to enforce security.

5.4.4 Domain Issues

Two systems running COM client and server applications can exist in one of three possible domain configurations:

• Systems are in the same domain
• Systems are in separate domains with trusts established between the two domains
• Systems are in separate domains without trusts, or systems are not in a domain

The ability for servers and DCOM$RPCSS to authenticate client requests are affected by the domain configurations. When both systems are in the same domain or when the systems are in separate but trusted domains there is no problem authenticating. The trusted domain configuration is a bit more complex and requires that the trusts and mappings be configured correctly but once configured, there is no trouble authenticating (see Section 5.2).

Systems in separate, nontrusted domains or systems not in any domain cannot be authenticated using the normal mechanisms. To run authenticated COM applications between such systems, you must pass authentication information (user name and password) from the client to the server. COM provides this capability in the CoCreateInstanceEx API. The pServerInfo parameter of the CoCreateInstanceEx API allows you to specify a user name and password to be used for authentication on the remote server system. The user name and password are part of the COAUTHIDENTITY structure, within the COAUTHINFO structure within the COSERVERINFO structure, that is passed as the pServerInfo parameter to CoCreateInstanceEx.

Section C.3 shows how you can authenticate a remote client that is neither in the server's domain nor in a domain that has a trust with the server's domain.

The current NTLM security implementation on OpenVMS does not support this feature for COM client applications on OpenVMS. This feature is supported for COM clients on Windows NT that communicate with COM servers on OpenVMS. To run COM client applications on OpenVMS where the server is not in the same domain or in a trusted domain, you must disable authentication for the application, as described in Section 5.4.5.

5.4.5 Disabling Authentication

Under certain conditions, you may want to disable authentication between a client and server applications. This feature disables many of the security features of COM and of the operating system and should not be used in an environment where security is required. There are two ways to disable authentication for COM applications:

• Use DCOMCNFG to change the default authentication level to None on both systems.
• Add a call to CoInitializeSecurity in both the client and server applications and set the dwAuthnLevel parameter to RPC_C_AUTHN_LEVEL_NONE.

The server must be configured to run with a specific NTLM account identity. Since the client will not be authenticated, there is no way for the server to run with a client's identity. To configure a server to run with a specific NTLM identity, use DCOMCNFG and change the application properties to select the NTLM account.

5.4.6 Access Denied Problems (80070005)

The most common security error a COM application will encounter is access denied (error status value 80070005). The following is a list of the most common causes of this error:

• Client process on OpenVMS does not have Windows NT credentials. Run NTA$LOGON to acquire Windows NT credentials.
• Application-specific launch or access permissions do not allow access. Check the application settings using DCOMCNFG (see Section 6.3.1).
• System default launch or access permissions do not allow access. Check the system defaults using DCOMCNFG (see Section 6.3.6).
• CoInitializeSecurityEx API call is incorrect. Verify that the client and server security calls are valid.
• Server process or thread does not have permission to perform a particular operation. Verify that the OpenVMS identity being used to execute a client request has the necessary privileges to perform the operation.
• Server process does not have access to the server images. Verify that the OpenVMS identity used to launch a server process has read and execute permissions for the server image and any dynamically loaded images (.EXE files).
• Advanced Server hostmap entry problems. In order for NTLM authentication to work correctly, network accounts must be mapped to OpenVMS accounts (see Section 5.1). To verify the mapping of network accounts to OpenVMS accounts, use the ADMINISTER command:

  $ ADMINISTER SHOW HOSTMAP

  
  In a multiple-domain environment with trusts established between domains, cross-domain mappings must be created (see Section 5.2). Under certain conditions, the Advanced Server is unable to verify a user account associated with a hostmap entry. Any attempt to display the hostmap entry of a user name that can not be verified will result in the user name being displayed using its eight-digit hexadecimal internal representation (for example, "DOMAINNAME\000003fd"). If this happens, verify that the Advanced Server is running on each machine. You should also verify the trusts between domains using the following ADMINISTER command:

  $ ADMINISTER SHOW TRUST

  
  If the trusts are valid and the hostmap entries are still displayed with the numeric format, you should shut down and restart the Advanced Server.

When DCOM$RPCSS launches a server in response to a client request for a COM object, DCOM$RPCSS creates a detached process and executes either the server image or server command file in the context of the detached process. The image or command file that is executed is determined by the value of the Registry key HKEY_CLASSES_ROOT\CLSID\{iid}\LocalServer32, where iid is the unique identifier of the COM object.

The run-time environment of the detached process is as follows:

• Default directory
  The default directory of the detached process is the same as the default directory of DCOM$RPCSS . This is determined by the default directory of the process that executed the DCOM$STARTUP command file. If DCOM$STARTUP is executed by the system startup procedure, then the default directory will be SYS$SYSTEM .
• Windows NT identity
  Depends on the application identity setting. This setting is made using DCOMCNFG .
  If the application identity is set to "launching user" (the default), then the Windows NT identity of the detached process is the same as the Windows NT identity of the client.
  If the application identity is set to a specific NTLM account then the Windows NT identity of the detached process is that of the NTLM account.
• OpenVMS user name
  The OpenVMS user name of the detached process is the user name that is mapped to the Windows NT identity of the detached process. The mapping of OpenVMS user name to Windows NT identity is established using the Advanced Server ADMINISTER utility (see Section 5.1).
• OpenVMS privileges
  Depends on the application identity setting. This setting is made using DCOMCNFG .
  If the application identity is set to "launching user" (the default), then the privileges depend on the location of the client. If the client is running on the same system as the server, then the privileges of the detached process will match the privileges of the client. If the client is running on a different system from the server, then the privileges of the detached process will be the default privileges of the OpenVMS user name account.
  If the application identity is set to a specific NTLM account then the privileges of the detached process will be the default privileges of the OpenVMS user name account.
• Process logicals
  SYS$INPUT and SYS$OUTPUT are defined to the disk device where the server image or command file is located. For example, if the server image is DKA0:[TEST]CMPNT.EXE , then SYS$INPUT and SYS$OUTPUT are defined to DKA0: . SYS$SCRATCH is not defined.
  If these environment settings are not sufficient for the successful execution of your server, then you should explicitly define the environment settings you need. One way to easily set up an environment for your server is to create a command file to run your server, and register the command file, instead of the executable image, as the file to be executed when the server is launched. You can define the environment in the command file prior to executing the server image. For example, if you build SAMPLE1 in the directory DKA0:[SAMPLE1] and register it using the BUILD_SAMPLE1 command file, then the server image will be named DKA0:[SAMPLE1]CMPNT.EXE . The Registry key HKEY_CLASSES_ROOT\CLSID\{0C092C21-882C-A6BB-0080C7B2D682}\LocalServer32 will have a value of DKAO:[SAMPLE1]CMPNT.EXE . You can change the value of that key to DKA0:[SAMPLE1]RUN_CMPNT.COM and create the command file, as follows:

  $! RUN_CMPNT.COM $! Command file to run SAMPLE1 $ set default DKA0:[SAMPLE1] $ define sys$output DKA0:[SAMPLE1]SAMPLE1.LOG $ ! Other definitions as needed $ RUN CMPNT.EXE $ exit

  
  When DCOM$RPCSS receives a request for SAMPLE1 and launches a server, the server executes this command file in the detached process.

	privacy and legal statement
6539PRO_004.HTML