Updated: 11 December 1998

Guidelines for OpenVMS Cluster Configurations

In a multipath configuration, the first path to a device that OpenVMS configures is chosen as the initial current path. Note that this overrides the selection of a preferred path through HSx console commands such as the following:

SET UNIT PREFERRED_PATH=this_controller or other_controller

For this reason, it is not recommended that the OpenVMS system manager set preferred paths at the HSx console. Instead the manual path switching commands, described in Section 6.8.7 should be used.

In case of failover, OpenVMS chooses an alternate path according to the following order of precedence:

• The path specified by the system manager in a manual switch command
• A direct path where failover from one HSx controller module to another is not required
• A direct path, where HSx failover is required
• MSCP served path

OpenVMS avoids unnecessary failover from one HSx controller module to another because:

• Failover from one HSx controller module to another causes a delay of approximately 1 to 15 seconds, depending on the amount of cached data that needs to be synchronized.
• Other nodes that share access to the device must reestablish communication using an alternate path

When an I/O operation to a multipath disk fails and the failure suggests that a retry (on the current path or an alternate path) might succeed, then failover proceeds as follows:

• Mount verification is invoked.
• Mount verification attempts to communicate with the device on the current path and to validate that the volume is still correct.
• If mount verification fails on the current path, then a new path is sought according to the following order:
  • Direct paths that do not require an HSx failover are attempted a few times to maximize their chance of selection (the number of retries is controlled by the SYSGEN parameter MPDEV_LCRETRIES).
  • Any direct path is attempted.
  • Any path, including the served path is attempted.
  
  These steps execute in a loop, until a working path is located, or mount verification times out.
• A successful mount verification restarts all failed I/Os and allows new ones to proceed.

6.4 Configuration Requirements and Restrictions

Table 6-1 Multipath SCSI Configuration Requirements

Component	Description
Alpha console firmware	For systems with HSZ70 and HSZ80, the minimum revision level is 5.3. For systems with HSG80, the minimum revision level is 5.4
Controller firmware	For HSZ70, the minimum revision level is 7.3; for HSZ80, it is 8.3; for HSG80, it is 8.4.
Controller module mode	Must be set to multibus mode. The selection is made at the HS x console.
Full connectivity	All hosts that are connected to an HS x in multibus mode must have a path to both HS x controller modules. This is because hosts that are connected exclusively to different controllers will switch the logical unit back and forth between controllers, preventing any I/O from executing. To prevent this from happening, always provide full connectivity from hosts to controller modules. If a host's connection to a controller fails, then take one of the following steps to avoid indefinite path switching: Repair the connection promptly. Prevent the other hosts from switching to the partially-connected controller. This is done by either disabling switching to the paths that lead to the partially-connected controller (see Section 6.8.8), or by shutting down the partially-connected controller. Disconnect the partially-connected host from the both controllers.
Allocation classes	A valid HSZ allocation class is required (refer to Section 6.6.3). If a SCSI bus is configured with HSZ controllers only, and all the controllers have a valid HSZ allocation class, then it is not necessary to adhere to the older SCSI device naming rules for that bus. That is, the adapters do not require a matching port allocation class, or a matching node allocation class and matching OpenVMS adapter device names. However, if there are non-HSZ devices on the bus, or HSZ controllers without an HSZ allocation class, then the standard rules for node and port allocation class assignments and controller device names for shared SCSI buses must be followed.

The restrictions for multipath SCSI configurations are presented in Table 6-2.

Table 6-2 Multipath SCSI Configuration Restrictions

Component	Description
Devices supported	DKDRIVER disk devices attached to HSZ70, HSZ80, and HSG80 controller modules are supported. Other device types, such as tapes, and class drivers, such as GKDRIVER, are not supported.
Mixed version and mixed architecture clusters	All hosts that are connected to an HSZ or HSG in multibus mode must be running OpenVMS Version 7.2 or higher. All Version 6.2 systems must have installed the cluster compatibility kit, as described in the OpenVMS Version 7.2 Release Notes.
Host based volume shadowing	In the initial release of OpenVMS Version 7.2, multipath devices can not be members of host-based shadow sets. This restriction will be removed by a Version 7.2 update kit.
SCSI to MSCP failover	In the initial release of OpenVMS Version 7.2, the MSCP path can not be included in the multipath set. This is accomplished by setting the SYSGEN parameter MPDEV_REMOTE = 0. This is the default setting in OpenVMS Version 7.2. This restriction will be removed by a Version 7.2 update kit.

6.5 Parallel SCSI Multipath Configurations

Prior to the introduction of multipath failover support, parallel SCSI configurations using HSZ70 storage controllers already provided transparent failover. Transparent failover is failover from one controller port to the corresponding port on the other controller module. Both ports must be on the same host bus.

Multipath failover offers another level of failover, from one SCSI bus to another.

The figures in this section show systems configured for transparent failover, and for multipath failover. The special considerations for controller modules that have multiple ports, like the HSZ80 are also described.

6.5.1 Transparent Failover

Figure 6-4 Parallel SCSI Configuration With Transparent Failover

In this configuration:

• Each logical unit is visible to the host on only one controller module at a time. The other controller module does not answer at the same SCSI address, but it can be used for other SCSI addresses.
• One HSZ controller module detects the failure of the other controller and fails over the logical unit to itself. The surviving controller takes over the SCSI address or addresses of the failed controller.

6.5.2 Multibus Failover and Multiple Paths

Figure 6-5 Parallel SCSI Configuration With Multibus Failover and Multiple Paths

Note the following about this configuration:

• Each logical unit is visible to the host at the same ID on both controller modules so it can be configured. The logical unit responds to read/write I/O on only one controller at a time, the controller to which it is "online".
• The controller modules must be on different SCSI buses to prevent a bus ID conflict.
• The HSZ moves a logical unit to the other controller if either of the following events occurs:
  • HSZ detects a controller failure.
  • Host sends a SCSI START command for the logical unit to the other controller.

6.5.3 Configurations Using Multiported Storage Controllers

This section shows three configurations that use multiported storage controllers. The configurations are presented in order of increasing availability.

Figure 6-6 shows a single host with a single interconnect, using an HSZ80 in transparent mode.

Figure 6-6 Multiported Parallel SCSI Configuration With Single Interconnect in Transparent Mode

Note the following about this configuration:

• Each logical unit is visible on one port per storage controller.
• If a port fails, the HSZ80 fails over the traffic to the corresponding port of the other HSZ80.

Figure 6-7 shows a system configured in transparent mode using two paths from the host.

Figure 6-7 Multiported Parallel SCSI Configuration With Multiple Paths in Transparent Mode

In this configuration:

• Physically corresponding ports must be on the same SCSI bus.
• A maximum of two buses can be connected to each storage controller.

Note that in this configuration, although there are two buses, there is only one path from the host to a particular logical unit. When a controller fails, the logical unit moves to the corresponding port on the other controller. Both ports are on the same host bus.

This configuration has better performance than the one in Figure 6-6 because both SCSI buses can be simultaneously active. This ocnfiguration does not have higher availability, however, because there is still only one path from the host to the logical unit.

Figure 6-8 shows a system using the multiported HSZ80 storage controller configured in multibus mode.

Figure 6-8 Multiported Parallel SCSI Configuration With Multiple Paths in Multibus Mode

In this configuration:

• Each logical unit is visible to the host at the same ID on all ports (so they all will be configured by the host).
• All the ports must be on different SCSI buses.
• The host uses one path at a time.
• Each logical unit can execute I/O simultaneously over the two ports of the controller to which it is "on line." This means that if there are multiple hosts, then two paths to the storage device may be simultaneously active.

6.6 Device Naming for Parallel SCSI Multipath Configurations

• The node name can not be used when there are multiple nodes with direct access to a device.
• The host adapter's controller letter can not be used when the controller letters on a shared bus do not match.
• The host adapter's controller letter can not be used when a node is connected to a device with multiple adapters.

The first two of these issues were addressed by the use of the node allocation class and the port allocation class. The third issue requires the introduction of an HSZ controller-based allocation class. These three allocation classes are reviewed in the following sections.

6.6.1 Review of Node Allocation Classes

A node allocation class is used in a device name in place of a node name. A node allocation class is needed to produce a unique device name when multiple nodes have a direct connection to the same SCSI device.

A node allocation class can only be used in a device name when all nodes that share access to a SCSI storage device:

• Have only one direct path to the device.
• Use the same host controller name on the shared bus.
• Have sufficient SCSI IDs to produce unique names for nonshared devices.

Figure 6-9 shows a configuration whose devices are named using a node allocation class.

Figure 6-9 Devices Named Using a Node Allocation Class

6.6.2 Review of Port Allocation Classes

The port allocation class can be used when SCSI systems need more SCSI IDs to produce unique device names, or when the controller letter of the adapters on a shared bus do not match. A port allocation class can only be used in a device name when all nodes that share access to a SCSI storage device have only one direct path to the device.

Figure 6-10 shows a configuration whose devices are named using a port allocation class.

Figure 6-10 Devices Named Using a Port Allocation Class

6.6.3 Device Naming Using HSZ Allocation Classes

Figure 6-11 Devices Named Using an HSZ Allocation Class

An HSZ allocation class is similar to the HSC, HSD, and HSJ allocation classes. The device name, using an HSZ allocation class number, takes the following form:

$HSZ-allocation-class$ddcu

where:

• HSZ-allocation-class is a decimal value from 1 to 999, assigned to a particular HSZ storage controller by the system manager
• dd represents the device class, which is DK for disk
• c represents the controller, which must be A when using an HSZ allocation class
• u represents the device unit number, which is determined by the SCSI bus ID and the logical unit number (LUN) of the device

The system manager sets an HSZ allocation class from the HSZ console, using the following command:

SET this_controller or other_controller ALLOCATION_CLASS = n

where n is a value from 1 to 999.

When the allocation class is set on one controller module in a dual redundant configuration, it is automatically set to the same value on the other controller.

In the following example, the allocation class is set to 199. The example shows that the value is set for both controllers.

z70_B => set this allo=199 z70_B => sho this Controller: HSZ70 ZG64100136 Firmware XB32-0, Hardware CX25 Configured for MULTIBUS_FAILOVER with ZG64100160 In dual-redundant configuration Device Port SCSI address 6 Time: NOT SET Host port: SCSI target(s) (0, 2, 3, 4, 5, 6) TRANSFER_RATE_REQUESTED = 20MHZ Host Functionality Mode = A Allocation class 199 Command Console LUN is target 0, lun 1 Cache: 32 megabyte write cache, version 4 Cache is GOOD Battery is GOOD No unflushed data in cache CACHE_FLUSH_TIMER = DEFAULT (10 seconds) NOCACHE_UPS z70_B => sho other Controller: HSZ70 ZG64100160 Firmware XB32-0, Hardware CX25 Configured for MULTIBUS_FAILOVER with ZG64100136 In dual-redundant configuration Device Port SCSI address 7 Time: NOT SET Host port: SCSI target(s) (0, 2, 3, 4, 5, 6) TRANSFER_RATE_REQUESTED = 20MHZ Host Functionality Mode = A Allocation class 199 Command Console LUN is target 0, lun 1 Cache: 32 megabyte write cache, version 4 Cache is GOOD Battery is GOOD No unflushed data in cache CACHE_FLUSH_TIMER = DEFAULT (10 seconds) NOCACHE_UPS

The following rules pertain to the use of an HSZ allocation class in SCSI device names:

1. In multibus mode, an HSZ allocation class must be used in a device name (otherwise, the device is not configured).
2. In transparent mode, an HSZ allocation class can be used in a device name but it is not required.
3. The HSZ allocation class number must be the same for both controllers of an HSZ. This is handled automatically by the HSZ firmware.
4. The number must be unique among all types of allocation classes throughout the cluster. (Note that this requirement is specific to HSZ allocation classes; it does not apply to MSCP controller allocation classes.)
5. The HSZ allocation class must be specified when referring to devices that have an HSZ allocation class. For example, the names DKA500 and NODE10$DKA500 can not be used. In addition, the $GETDVI system service will only return the fully specified name, including the HSZ allocation class, for these devices.

6.7 Fibre Channel Multipath Configurations

Figure 6-12 shows a single system with redundant paths to each HSG80 storage controller. The storage controllers are configured in transparent mode. Each HSG80 provides a standby port to which I/O can fail over.

Figure 6-12 Single Host With Two Dual-Ported Storage Controllers on a Single Bus

Note the following about this configuration:

• Host has one adapter; thus it is connected to only one bus.
• Each HSG80 storage controller has two ports.
• Only one port per storage controller is active.
• Both storage controllers provide access to the same storage.

In this configuration, if one path fails, the standby port takes over the other port's Fibre Channel address and port worldwide ID (WWID).

Figure 6-13 shows a multipath configuration with the storage controllers configured in multibus mode. This means that each HSG80 port has its own Fibre Channel address and Fibre Channel port WWID.

This configuration is not possible using parallel SCSI as the interconnect because the SCSI address for a port is part of the device name. Because of this difference in naming, an additional configuration option is available with Fibre Channel---multiple ports on the same Fibre Channel.

Figure 6-13 Single Host With Two Dual-Ported Storage Controllers and Two Buses

Note the following about this configuration:

• Host has two adapters, each attached to a different bus.
• Each port on each HSG80 storage controller is attached to a different interconnect.
• Both storage controllers can access the same disk.

Because of the second bus, this configuration provides a higher level of availability than the configuration in Figure 6-12.

Figure 6-14 shows another multipath configuration with its storage controllers configured in multibus mode.

Figure 6-14 Single Host With Two Dual-Ported Storage Controllers and Four Buses

Note the following about this configuration:

• Host has four adapters, each attached to a different interconnect.
• Each port on each HSG80 storage controller is attached to a different interconnect.
• Host has four paths to the same logical unit.

6.8 Implementing Multipath Configurations

Implementing multiple paths to devices consists of the following steps:

1. Configuring a system or systems with multiple physical paths to those devices for which you want multipath support
2. Setting the HSx controller to multibus mode
3. Optionally qualifying multipath support by setting certain multipath system and console parameters, as appropriate for your configuration
4. Optionally tailoring the operation of multipath functionality, using the DCL command SET DEVICE/qualifier/PATH=path-identifier

Figure 6-15 shows a valid multipath, multihost configuration.

Figure 6-15 Two Hosts With Shared Buses and Shared Storage Controllers

Note the following about this configuration:

• Each host has two adapters.
• Both hosts are connected to the same two buses.
• Both hosts share the storage.
• Each storage controller is connected to one bus only.
• The two storage controllers are connected to the same disks.

This configuration provides each host with two direct paths and one MSCP served path to each device.

Figure 6-16 shows a valid multipath configuration for systems that are not configured on the same bus.

Figure 6-16 Two Hosts With Shared, Multiported Storage Controllers

Note the following about this configuration:

• Each host has two adapters.
• Each host is connected to two buses but the hosts do not share a bus.
• Both hosts share the storage.
• Each storage controller has two connections, one to each host.
• The two storage controllers are connected to the same disks.

This configuration provides each host with two direct paths, one to each storage controller, and one MSCP served path to each device.

Copyright © Compaq Computer Corporation 1998. All rights reserved. Legal

6318PRO_006.HTML