Updated: 11 December 1998

OpenVMS Alpha Galaxy Guide

OpenVMS supports several methods of managing CPU resources. The console establishes the default owner instance for each CPU using the console environment variables. This allows the CPU resources to be statically assigned, providing a precise initial configuration. In the event of a cold boot (that is, power cycle or initialization), this default configuration is restored from console nonvolatile RAM.

Once a configuration has been booted, OpenVMS provides more elegant means of assigning resources for users with CMKRNL (Change Mode to Kernel) privilege. The following sections describe these methods.

11.1 DCL Reassignment

Users with CMKRNL privilege can perform CPU reassignment operations using the following DCL command:

$ STOP/CPU/MIGRATE=instance-or-id cpu-id

The user must provide the target instance name (SCSNAME) or numeric ID (0, 1, and so on), and the numeric ID of the CPU being reassigned. The following examples show a few forms of this command.

$ STOP/CPU/MIGRATE=0 4 !Reassign CPU 4 to instance 0 $ STOP/CPU/MIGRATE=1 3,4,5 !Reassign CPUs 3,4,5 to instance 1 $ STOP/CPU 7/MIGRATE=BIGBNG !ReassignCPU 7 to instance BIGBNG $ STOP/CPU/ALL/MIGRATE=0 !Reassign all secondary CPUs to instance 0

These commands can be inserted in to command procedures. For example, you might want to move extra CPU resources to an instance in a startup procedure of an application with known processing requirements. Similarly, you may want to reassign CPUs away from an instance that is about to perform lengthy, I/O intensive operations (such as backups) so that the CPUs are available to other instances. When the job completes, you may reassign them back. Or you may reassign CPUs away from an instance that is shutting down.

Note that you can only reassign resources away from an instance. This is the push model defined by the Galaxy Software Architecture. This model prevents resources from being "stolen" by other instances that may not be aware of their current usage. To effectively manage the entire Galaxy system using DCL, you must either log in to each of the involved instances or use the SYSMAN utility to execute the commands on the owner instance.

11.2 GCU Drag-and-Drop Reassignment

The GCU provides an interactive visual interface for managing Galaxy resources. Using the GCU, you can reassign CPUs by simply dragging and dropping them among instances. Additionally, the GCU allows you to draw charts of various configurations (known as configuration models), and save them as files. At any time, you can load and engage a configuration model and the system will reassign resources as needed to achieve the desired model.

11.3 Intermodal Reassignment

Because of the resource push model defined by the Galaxy Software Architecture, resources must be given away by the Galaxy instance that currently owns them. For a utility or user to effectively manage resource assignment in a multiple-instance Galaxy configuration, you must establish some means of executing commands on each instance.

One such means is to open a window or terminal session on each of the Galaxy instances and perform your resource management operations in each of these windows.

Another method is use the SYSMAN utility and its underlying SMI-Server to establish a command environment on the owner instance. Using this method, a fairly simple command procedure can be written to perform specific resource management operations. This method has some limitations, however. First, it requires that the involved Galaxy instances must be in a cluster. Also, a command procedure cannot effectively pass variable parameters to SYSMAN environment scripts, and you cannot specify a remote system password within a SYSMAN script. So it is cumbersome to generate a general-purpose command procedure interface that uses SYSMAN.

The GCU does, in fact, use SYSMAN wherever possible to accomplish its management actions. When a system is not configured to support SYSMAN, the GCU will attempt to use DECnet task-to-task comminations among proxy accounts as its management transport. If that approach also fails (that is, if the system is not running DECnet or if the necessary proxy accounts are not set up), the GCU will not be able to manage Galaxy instances other than the one on which the GCU is currently running. You could certainly run multiple copies of the GCU if you choose, one per Galaxy instance. However, you can assume that OpenVMS Galaxy systems are likely to be clustered or to use DECnet.

The GCUs management actions are based in the SYS$MANAGER:GCU$ACTIONS.COM command procedure. You can modify this file to customize actions for your own environment. For example, in a TCP/IP environment, you may choose to use REXEC or similar utility for your management transport, or you may want to include some form of notification or logging whenever a management action is executed.

The GCU$ACTIONS.COM file is somewhat unusual in the way it operates. When using SYSMAN, the procedure builds small SYSMAN command scripts in temporary files to deal with variable parameters that SYSMAN Cannot handle. When SYSMAN is not available, the procedure attempts to open a DECnet task-to-task connection to a proxy account on the owner instance. If successful, it uses this connection to shuffle command parameters to the copy of GCU$ACTIONS.COM that resides on the owner instance. The end result is execution of the command locally by the owner instance.

11.4 Software Reassignment Using Galaxy Services

Perhaps the best method for managing resource assignment is to use the Galaxy APIs to write your own resource management routines. This allows you to base your decisions for resource management on your own criteria and application environment. The same push-model restriction described in Section 11.3 still exists, however, so your routines will need to be Galaxy aware, possibly using shared memory to coordinate their operations.

For information about CPU reassignment APIs, see Chapter 17.

11.5 Reassignment Faults

CPU reassignment can fail or be blocked, for several reasons. Because the GCU buries its management actions in SYSMAN or DCL scripts, it may not always identify and report the reasons for a reassignment fault. The GCU does perform certain checks prior to allowing reassignment actions in order, for example, to prevent attempts to reassign the primary CPU. Other reasons exist for reassignment faults that can only be detected by the operating system or console firmware. For example, if the operating system detects a fault attempting to reassign a CPU that currently has process affinity or Fast Path duties, a DCL message will be displayed on both the console and the users terminal.

The Galaxy APIs for reassignment are capable of reporting most faults to the caller. However, even using the reassignment services, the console may reject a reassignment because of hardware platform dependencies not readily visible to the operating system.

The following DCL Commands are useful for managing an OpenVMS Galaxy:

• SHOW MEMORY
• STOP/CPU/MIGRATE
• SHOW CPU
• CONFIGURE Galaxy
• Lexical functions

CPUs are assignable resources in an OpenVMS Galaxy.

12.1.1 STOP/CPU/MIGRATE

The STOP/CPU/MIGRATE command stops and removes the specified secondary processor(s) from the active set in an OpenVMS SMP system.

For example:

When a user enters:

GLX1$ stop/cpu/migrate=GLX0 4

The following message is displayed at the user's terminal:

%SYSTEM-I-CPUSTOPPING, trying to stop CPU 4 after it reaches quiescent state

The source console displays:

%SMP-I-STOPPED, CPU #04 has been stopped.

The destination console displays:

%SMP-I-SECMSG, CPU #04 message: P04>>>START %SMP-I-CPUTRN, CPU #04 has joined the active set.

12.1.2 SHOW CPU

The SHOW CPU command displays information about the status, characteristics, and capabilities of the specified processor(s).

For example:

GLX0$ show cpu GLX0, AlphaServer 8400 Model 5/440 Multiprocessing ENABLED. Full checking synchronization image loaded. Minimum multiprocessing revision levels: CPU = 1 PRIMARY CPU = 00 Active CPUs: 00 01 Configured CPUs: 00 01 Potential CPUs: 00 01 03 04 05 06 07

12.2 SHOW MEMORY

The SHOW MEMORY command displays the uses of memory by the system.

For example:

CLSSIC$ SHOW MEMORY/physical System Memory Resources on 5-OCT-1999 20:50:19.03 Physical Memory Usage (pages): Total Free In Use Modified Main Memory (2048.00Mb) 262144 228183 31494 2467 Of the physical pages in use, 11556 pages are permanently allocated to OpenVMS. GALAXY$ show memory/physical System Memory Resources on 5-OCT-1999 07:55:14.68 Physical Memory Usage (pages): Total Free In Use Modified Private Memory (512.00Mb) 65536 56146 8875 515 Shared Memory (1024.00Mb) 131072 130344 728 Of the physical pages in use, 6421 pages are permanently allocated to OpenVMS. $

12.3 Lexical Function Example

Lexical Function Example Command Procedure

$ write sys$output "" $ write sys$output "Instance = ",f$getsyi("scsnode") $ write sys$output "Platform = ",f$getsyi("galaxy_platform") $ write sys$output "Sharing Member = ",f$getsyi("galaxy_member") $ write sys$output "Galaxy ID = ",f$getsyi("galaxy_id") $ write sys$output "Community ID = ",f$getsyi("community_id") $ write sys$output "Partition ID = ",f$getsyi("partition_id") $ write sys$output "" $ exit

Lexical Function Command Procedure Output

COBRA2$ @shoglx Instance = COBRA2 Platform = 1 Sharing Member = 1 Galaxy ID = 5F5F30584C47018011D3CC8580F40383 Community ID = 0 Partition ID = 0 COBRA2$

12.4 INSTALL

The LIST option now returns the galaxywide sections as well as standard global sections.

12.5 SET CPU

• SET CPU n/FAILOVER=y
  Establishes instance-specific failover relationships for each CPU in the instances potential set.
  Where n is the CPU number, a comma-separated list of CPUs, or /ALL and y is the instance number or name.
• SET CPU n/NOFAILOVER
  Removes any instance-specific failover relationship for the specified CPU/s.
• SET CPU n/[NO]AUTOSTART
  Sets or clears the instance-specific autostart flag for the specified CPU/s.
  Where n is the CPU number, a comma-separated list of CPUs, or /ALL.

The CONFIGURE command invokes the Galaxy Configuration Utility (GCU) to monitor, display, and interact with an OpenVMS Galaxy system. The GCU requires DECwindows Motif V1.2-4 or greater and ALPHA OpenVMS V7.2 or greater.

The optional model parameter specifies the location and name of a Galaxy Configuration Model to load and display. If no model is provided and the system is running as an OpenVMS Galaxy, the current active configuration is displayed.

If the system is not running as an OpenVMS Galaxy, the GCU will assist the user in creating a single-instance OpenVMS Galaxy system.

OpenVMS Galaxy Configuration Models are created with the Galaxy Configuration Utility. Refer to the OpenVMS Galaxy Guide and GCU online help for more information.

Format:

CONFIGURE GALAXY [model.gcm]

Parameters:

GALAXY [model.gcm] Specifies the location and name of a Galaxy configuration model to load and display. If no model is provided and the system is running as an OpenVMS Galaxy, the current active configuration is displayed.

Qualifiers:

/ENGAGE Causes the GCU to engage (load, validate, and activate) the specified OpenVMS Galaxy Configuration Model without displaying the graphical user interface. After validation, the specified model becomes the active system configuration. This qualifier allows system managers to restore the OpenVMS Galaxy system to a known configuration, regardless of what dynamic resource reassignments may have occurred since the system was booted. This command can be embedded in DCL command procedures to automate configuration operations. /VIEW When used in conjunction with /ENGAGE and a model parameter, causes the GCU to load, validate, activate, and display the specified configuration model.

$ CONFIGURE GALAXY Displays the GCU's graphical user interface. If the system is currently configured as an OpenVMS Galaxy, the active system configuration is displayed. $ CONFIGURE GALAXY model.GCM Displays the GCU's graphical user interface. The specified OpenVMS Galaxy Configuration Model is loaded and displayed, but does not become the active configuration until the user chooses to engage it. $ CONFIGURE GALAXY/ENGAGE model.GCM Invokes the GCU command line interface to engage the specified OpenVMS Galaxy Configuration Model without displaying the GCU's graphical user interface. $ CONFIGURE GALAXY/ENGAGE/VIEW model.GCM Invokes the GCU command line interface to engage the specified OpenVMS Galaxy Configuration Model and display the GCU's graphical user interface.

This chapter describes the following two OpenVMS internal mechanisms that use shared memory to communicate between instances in an OpenVMS Galaxy computing environment:

• Shared Memory Cluster Interconnect (SMCI)
• Local Area Network (LAN) Shared Memory Device Driver

The Shared Memory Cluster Interconnect (SMCI) is a System Communications Services (SCS) port for communications between Galaxy instances. When an OpenVMS instance is booted as both a Galaxy and as an OpenVMS Cluster member, the SMCI driver is loaded. This SCS port driver communicates with other cluster instances in the same Galaxy through shared memory. This capability provides one of the major performance benefits of the OpenVMS Galaxy Software Architecture. The ability to communicate to another clustered instance through shared memory provides dramatic performance benefits over traditional cluster interconnects.

13.1.1 SYS$PBDRIVER Port Devices

When booting as both a Galaxy and a cluster member, SYS$PBDRIVER is loaded by default. The loading of this driver creates a device PBAx, where x represents the Galaxy partition ID. As other instances are booted, they also create PBAx devices. The SMCI quickly identifies the other instances and creates communications channels to them. Unlike traditional cluster interconnects, a new device is created to communicate with the other instances. This device also has the name PBAx, where x represents the Galaxy partition ID for the instance with which this device is communicating.

For example, consider an OpenVMS Galaxy that consists of two instances: MILKY and WAY. MILKY is instance 0 and WAY is instance 1. When node MILKY boots, it creates device PBA0. When node WAY boots, it creates PBA1. As the two nodes "find" each other, MILKY creates PBA1 to talk to WAY and WAY creates PBA0 to talk to MILKY.

MILKY WAY PBA0: PBA1: PBA1: <-------> PBA0:

13.1.2 Multiple Clusters in a Single Galaxy

SYS$PBDRIVER can support multiple clusters in the same Galaxy. This is done in the same way that SYS$PEDRIVER allows support for multiple clusters on the same LAN. The cluster group number and password used by SYS$PEDRIVER are also used by SYS$PBDRIVER to distinguish different clusters in the same Galaxy community. If your Galaxy instances are also clustered with other OpenVMS instances over the LAN, the cluster group number are set appropriately by CLUSTER_CONFIG. To determine the current cluster group number:

$ MCR SYMAN SYSMAN> CONFIGURATION SHOW CLUSTER_AUTHORIZATION Node: MILKY Cluster group number: 0 Multicast address: xx-xx-xx-xx-xx-xx SYSMAN>

If you are not clustering over a LAN and you want to run multiple clusters in the same Galaxy community, then you must set the cluster group number. You must ensure that the group number and password are the same for all Galaxy instances that you want to be in the same cluster.

$ MCR SYSMAN SYSMAN> CONFIGURATION SET CLUSTER_AUTHORIZATION/GROUP_NUMBER=222/PASSWORD=xxxx SYSMAN>

If your Galaxy instances are also clustering over the LAN, CLUSTER_CONFIG asks for a cluster group number, and the Galaxy instances utilize those group numbers. If you are not clustering over a LAN, the group number defaults to zero. This means that all instances in the Galaxy will be in the same cluster.

13.1.3 SYSGEN Parameters for SYS$PBDRIVER

In most cases, the default settings for SYS$PBDRIVER should be appropriate. However, several SYSGEN parameters are provided. Two SYSGEN parameters control SYS$PBDRIVER: SMCI_PORTS and SMCI_FLAGS.

13.1.3.1 SMCI_PORTS

The SMCI_PORTS SYSGEN parameter controls initial loading of SYS$PBDRIVER. This parameter is a bitmask in which bits 0 through 25 each represent a controller letter. If bit 0 is set, PBAx will be loaded; this is the default setting. If bit 1 is set, PBBx will be loaded, and so on all the way up to bit 25 which will cause PBZx to be loaded. For OpenVMS Alpha Version 7.2, Compaq recommends leaving this parameter at the default value of 1.

Loading additional ports allows for multiple paths between Galaxy instances. For OpenVMS Alpha Version 7.2, having multiple communications channels does not provide any advantages because SYS$PBDRIVER will initially not support Fast Path. A future release of OpenVMS will provide Fast Path support for SYS$PBDRIVER. When Fast Path support is enabled, instances with multiple CPUs can achieve improved throughput by having multiple communications channels between instances.

13.1.4 SMCI_FLAGS

The SMCI_FLAGS SYSGEN parameter controls operational aspects of SYS$PBDRIVER. The only currently defined flag is bit 1. This controls whether or not the port device supports communications with itself. Supporting SCS communications to itself is primarily used for test purposes. By default, this bit will be turned off and thus support for SCS communication locally is disabled which saves system resources. This parameter is dynamic and by turning this bit on, an SCS virtual circuit should soon form.

Bit	Mask	Description
0	0	0 = Do not create local communications channels (SYSGEN default). Local SCS communications are primarily used in test situations and not needed for normal operations. Leaving this bit off saves resources and overhead. 1 = Create local communications channels.
1	2	0 = Load SYS$PBDRIVER if booting into both a Galaxy and a Cluster (SYSGEN Default). 1 = Load SYS$PBDRIVER if booting into a Galaxy.
2	4	0 = Minimal console output (SYSGEN default) 1 = Full console output, SYS$PBDRIVER will display console messages when creating communication channels and tearing down communication channels.

13.2 LAN Shared Memory Device Driver

Local Area Network (LAN) communications between OpenVMS Galaxy instances are supported by the Ethernet LAN shared memory driver. This LAN driver communicates to other instances in the same OpenVMS Galaxy system through shared memory. Communicating with other instances through shared memory provides performance benefits over traditional LANs.

To load the LAN shared memory driver SYS$EBDRIVER, enter the following command:

$ MCR SYSMAN SYSMAN> IO CONN EBA/DRIVER=SYS$EBDRIVER/NOADAPTER

For OpenVMS Version 7.2, in order for LAN protocols to automatically start over this LAN device (EBAn, where n is the unit number), the procedure for loading this driver should be added to the configuration procedure: SYS$MANAGER:SYCONFIG.COM.

The LAN driver emulates an Ethernet LAN with frame formats the same as Ethernet/IEEE 802.3 but with maximum frame size increased from 1518 to 7360 bytes. The LAN driver presents a standard OpenVMS QIO and VCI interface to applications. All existing QIO and VCI LAN applications should work unchanged.

In a future release, the SYS$EBDRIVER device driver will be loaded automatically.

Copyright © Compaq Computer Corporation 1998. All rights reserved. Legal

6512PRO_007.HTML