Document revision date: 19 July 1999

Figure 6-20 shows an invalid multipath configuration. The configuration is invalid because, if multiple hosts in a cluster are connected to an HSZ or HSG, they must all have connections to the same controller modules (see Table 6-1). In this configuration, each host is connected to a different controller module.

Figure 6-20 Invalid Multipath Configuration

Multipath support is enabled and qualified by the use of four system parameters, as described in Table 6-3. (A fifth multipath system parameter is reserved for the operating system.)

If a node has MPDEV_ENABLE or MPDEV_REMOTE set to OFF, and the node has one or more direct paths and served paths to its system disk, then the node must only boot on the direct paths.

6.7.4 Path Identifiers

The system management commands described in the following sections allow you to monitor and control the operation of multipath failover. These commands provide a path identifier to uniquely specify each path in a multipath set.

Direct Fibre Channel paths are identified by the local host adapter name and the remote Fibre Channel port WWID --- that is, the initiator and the target. For example, in Figure 6-21, the path identifier for the path from the host adapter on the left to the HSG storage controller on the left is PGB0.5000-1FE1-0000-0200. (The second port on each HSG is omitted for convenience.) You can obtain the WWID for a storage controller from its console.

Figure 6-21 Fibre Channel Path Naming

Direct parallel SCSI paths are identified by the local host adapter name and the remote SCSI bus ID --- that is, the initiator and the target. For example, in Figure 6-22, the path identifiers for node Edgar's two direct paths to the disk would be named PKB0.5 and PKC0.5.

The path identifier for MSCP served paths is MSCP.

Figure 6-22 Configuration With Multiple Direct Paths

When multipath support is enabled, you can display the multiple paths to a device using either of the following variants of the SHOW DEVICE DCL command:

SHOW DEVICE/FULL device-name SHOW DEVICE/MULTIPATH_SET device-name

The SHOW DEVICE/FULL device-name command displays the traditional information about the device first and then lists all the paths to a device by their path identifiers (described in Section 6.7.4).

The SHOW DEVICE/MULTIPATH_SET device-name command lists only the multiple paths to a device.

Multipath information is displayed only on nodes that are directly connected to the multipath device.

6.7.5.1 Displaying Paths With SHOW DEVICE/FULL

The following example shows the output of a SHOW DEVICE/FULL device-name command. Note that the use of multiple paths is shown at the beginning of the display (device has multiple I/O paths), and the multiple path descriptions are shown toward the end of the display, beneath I/O paths to device. Note, too, that the values for Error count and Operations completed shown at the beginning of the display are the sums of the counts for each path.

$ SHOW DEVICE/FULL $70$DKA100: Disk $70$DKA100: (FLAM10), device type DEC HSZ70, is online, mounted, file- oriented device, shareable, device has multiple I/O paths, served to cluster via MSCP Server, error logging is enabled. Error count 2 Operations completed 123471 Owner process "" Owner UIC [SYSTEM] Owner process ID 00000000 Dev Prot S:RWPL,O:RWPL,G:R,W Reference count 1 Default buffer size 512 Total blocks 4109470 Sectors per track 85 Total cylinders 3022 Tracks per cylinder 16 Allocation class 70 Volume label "X6MV_SYS_1" Relative volume number 0 Cluster size 9 Transaction count 1 Free blocks 2040228 Maximum files allowed 205524 Extend quantity 5 Mount count 4 Mount status System Cache name "_$600$DKA0:XQPCACHE" Extent cache size 64 Maximum blocks in extent cache 204022 File ID cache size 64 Blocks currently in extent cache 0 Quota cache size 0 Maximum buffers in FCP cache 2594 Volume owner UIC [1,1] Vol Prot S:RWCD,O:RWCD,G:RWCD,W:RWCD Volume Status: ODS-2, subject to mount verification, file high-water marking, write-back caching enabled. Volume is also mounted on FIBRE3, COB2, FLAM10. I/O paths to device 2 Path PKC0.1 (SISKO), primary path, current path. Error count 1 Operations completed 121001 Path PKB0.1 (SISKO). Error count 1 Operations completed 2470

For each path of the multipath device, the path identifier, the host name associated with that path, the path status, the error count, and the operations count are displayed.

The terms that may appear in the multiple paths portion of the display are described in Table 6-4.

Table 6-4 SHOW DEVICE/FULL Multipath Terms

Term	Description
Primary path	This was the first path to the device found by the operating system.
Current path	This path is currently used for I/O.
User disabled	The DCL command SET DEVICE/NOENABLE has been executed for this path.
Polling disabled	The DCL command SET DEVICE/NOPOLL has been executed for this path.
Unavailable	The path is unavailable because the software driver has disconnected from the path.
MSCP host unavailable	The path is unavailable because the connection to the MSCP server on the remote host has been broken.

6.7.5.2 Displaying Paths With SHOW DEVICE/MULTIPATH_SET

You can obtain a brief listing of multiple paths for a specific device, for all the devices in an allocation class, or for all devices with the DCL command:

SHOW DEVICE/MULTIPATH_SET [device-name]

The device name is optional; when omitted, all devices that have formed multipath sets are shown. For each multipath device found, the device name, host name, device status, error count, number of accessible paths, total number of paths, and the current path's path identifier are displayed.

The host name displayed is the host name of the current path. For direct paths, this will be the local system's host name. For MSCP served paths, this will be the host name of the remote system which is serving access to the device.

The following example shows the output of a SHOW DEVICE /MULTIPATH command.

$ SHO DEV /MULTIPATH Device Device Error Current Name Status Count Paths path $70$DKA0: (FIBRE3) Online 0 3/ 3 PKB0.0 $70$DKA100: (FIBRE3) MountVerify 11 2/ 3 PKB0.1 $70$DKA200: (FIBRE3) Mounted 0 3/ 3 PKC0.2 $70$DKA300: (FIBRE3) Mounted 2 1/ 3 PKC0.3 $70$DKA400: (FIBRE3) Mounted 4 3/ 3 PKC0.4 $70$DKA500: (FIBRE3) Mounted 7 3/ 3 PKC0.5 $70$DKA1400: (FIBRE3) Mounted 0 3/ 3 PKC0.14 $70$DKA1500: (FIBRE3) Mounted 4 3/ 3 PKB0.15 $100$DKA100: (FIBRE3) Online 0 2/ 2 PKA0.1 $100$DKA200: (FIBRE3) Mounted 3 2/ 2 PKA0.2 $100$DKA300: (FIBRE3) Online 0 2/ 2 PKA0.3 $100$DKA400: (FIBRE3) Online wrtlck 0 2/ 2 PKA0.4

If you choose to specify a partial device name, such as $70$DKA, the display shows all devices with multiple paths whose names begin with $70$DKA.

6.7.6 Path Polling

When SCSI multipath support is in effect, the system periodically polls all the I/O paths from each host adapter to each HSZ or HSG controller to determine the status of each I/O path. If the system detects any changes to a path, it outputs a message, similar to the following messages, to the console and to the operator's log:

All multipath devices on path PKB0.5 are either disabled or not reachable.

or

At least one multipath device on path PKB0.5 is enabled and reachable.

If all the devices on a path are removed, a path failure is reported. The path from the host to the HSx controller may still function, but this cannot be determined when there are no devices to poll.

You can turn polling on or off with the following command:

SET DEVICE device/[NO]POLL/PATH=path-identifier

Turning off polling for a path that will be out of service for a prolonged period is useful because it can reduce system overhead.

6.7.7 Switching Current Paths Manually

You can switch a device's current path manually using the SET DEVICE command with the /SWITCH qualifier. The most common reason for doing this is to balance the aggregate I/O load across multiple HSx controller modules and buses.

The command syntax for switching the current path is:

SET DEVICE device-name/SWITCH/PATH=path-identifier

The following command switches the path of device $2$DKA502 to an MSCP served path.

$ SET DEVICE $2$DKA502/SWITCH/PATH=MSCP

Note that this command initiates the process of switching the path and then returns to the DCL prompt immediately. A delay may occur between when the DCL prompt reappears and when the path switch is complete.

A manual path switch is accompanied by mount verification messages and by a status message indicating the result of the path switch request. This is shown in Example 6-1:

Example 6-1 Messages Resulting from Manual Path Switch

%%%%%%%%%%% OPCOM 17-MAR-1999 20:48:56.66 Device $1$DGA50: (FCNOD1 PGA, FCNOD2) is offline. Mount verification is in progress. _$ %%%%%%%%%%% OPCOM 17-MAR-1999 20:49:12.30 Mount verification has completed for device $1$DGA50: (FCNOD1 PGA, FCNOD2) %%%%%%%%%%% OPCOM 17-MAR-1999 20:49:12.31 Multipath access to device $1$DGA50: has been manually switched from path PGB0.5000-1FE1-0000-0D13 to path PGA0.5000-1FE1-0000-0D12

You can check for completion of a path switch by issuing the SHOW DEVICE/FULL command, or the SHOW DEVICE/MULTIPATH command.

Note that if the path that is designated in a manual path switch fails during the switch operation, then automatic path switching takes over. This can result in a switch to a path different from the one designated in the command.

If a manual path switch causes a virtual unit to switch from one HSG80 controller to another controller, then the command can affect other nodes in the cluster. These nodes will experience a mount verification on their current path. This will cause an automatic switch to a path on the other HSG80 controller. The messages indicating this event are shown in Example 6-2:

Example 6-2 Messages Displayed When Other Nodes Detect a Path Switch

%%%%%%%%%%% OPCOM 17-MAR-1999 20:50:56.66 Device $1$DGA50: (FCNOD1 PGA, FCNOD2) is offline. Mount verification is in progress. _$ %%%%%%%%%%% OPCOM 17-MAR-1999 20:51:12.30 Mount verification has completed for device $1$DGA50: (FCNOD1 PGA, FCNOD2) %%%%%%%%%%% OPCOM 17-MAR-1999 20:51:12.31 Multipath access to device $1$DGA50: has been automatically switched from path PGB0.5000-1FE1-0000-0D13 to path PGA0.5000-1FE1-0000-0D11

6.7.8 Path Selection by OpenVMS

In a multipath configuration, the first path to a device that OpenVMS configures is chosen as the initial current path. Note that this overrides the selection of a preferred path through HSx console commands such as the following:

HSG> SET UNIT PREFERRED_PATH=THIS_CONTROLLER

or

HSG> SET UNIT PREFERRED_PATH=OTHER_CONTROLLER

For this reason, it is not recommended that the OpenVMS system manager set preferred paths at the HSx console. Instead the manual path switching commands, described in Section 6.7.7 should be used.

In case of failover, OpenVMS chooses an alternate path according to the following order of precedence:

1. The path specified by the system manager in a manual switch command
2. A direct path where failover from one HSx controller module to another is not required
3. A direct path, where HSx failover is required
4. MSCP served path

OpenVMS avoids unnecessary failover from one HSx controller module to another because:

• Failover from one HSx controller module to another causes a delay of approximately 1 to 15 seconds, depending on the amount of cached data that needs to be synchronized.
• Other nodes that share access to the device must reestablish communication using an alternate path

When an I/O operation to a multipath disk fails and the failure suggests that a retry (on the current path or an alternate path) might succeed, then failover proceeds as follows:

1. Mount verification is invoked.
2. Mount verification attempts to communicate with the device on the current path and to validate that the volume is still correct.
3. If mount verification fails on the current path, then a new path is sought according to the following order:
   • Direct paths that do not require an HSx failover are attempted a few times to maximize their chance of selection (the number of retries is controlled by the SYSGEN parameter MPDEV_LCRETRIES).
   • Any direct path is attempted.
   • Any path, including the served path is attempted.
   
   These steps execute in a loop, until a working path is located, or mount verification times out.
4. A successful mount verification restarts all failed I/Os and allows new ones to proceed.

By default, all paths are candidates for path switching. You can disable or re-enable a path as a switch candidate by using the SET DEVICE command with the /[NO]ENABLE qualifier. The reasons you might want to do this include the following:

• You know a specific path is broken, or that a failover to that path will cause some members of the cluster to lose access.
• To prevent automatic switching to a selected path while it is being serviced.

Note that the current path cannot be disabled.

The command syntax for enabling a disabled path is:

$ SET DEVICE device-name/[NO]ENABLE/PATH=path-identifier

The following command enables the MSCP served path of device $2$DKA502.

$ SET DEVICE $2$DKA502/ENABLE/PATH=MSCP

The following command disables a local path of device $2$DKA502.

$ SET DEVICE $2$DKA502/ENABLE/PATH=PKC0.5

Be careful when disabling paths. Avoid creating an invalid configuration, such as the one shown in Figure 6-20

6.7.11 Console Considerations

This section describes how to use the console with parallel SCSI multipath devices. Refer to Section 7.5 for information on using the console with FC multipath devices.

The console uses traditional, path-dependent, SCSI device names. For example, the device name format for disks is DK, followed by a letter indicating the host adapter, followed by the SCSI target ID, and the LUN.

This means that a multipath device will have multiple names, one for each host adapter it is accessible through. In the following sample output of a console show device command, the console device name is in the left column. The middle column and the right column provide additional information, specific to the device type.

Notice, for example, that the devices dkb100 and dkc100 are really two paths to the same device. The name dkb100 is for the path through adapter PKB0, and the name dkc100 is for the path through adapter PKC0. This can be determined by referring to the middle column, where the informational name includes the HSZ allocation class. The HSZ allocation class allows you to determine which console "devices" are really paths to the same HSZ device.

>>>sho dev dkb0.0.0.12.0 $55$DKB0 HSZ70CCL XB26 dkb100.1.0.12.0 $55$DKB100 HSZ70 XB26 dkb104.1.0.12.0 $55$DKB104 HSZ70 XB26 dkb1300.13.0.12.0 $55$DKB1300 HSZ70 XB26 dkb1307.13.0.12.0 $55$DKB1307 HSZ70 XB26 dkb1400.14.0.12.0 $55$DKB1400 HSZ70 XB26 dkb1500.15.0.12.0 $55$DKB1500 HSZ70 XB26 dkb200.2.0.12.0 $55$DKB200 HSZ70 XB26 dkb205.2.0.12.0 $55$DKB205 HSZ70 XB26 dkb300.3.0.12.0 $55$DKB300 HSZ70 XB26 dkb400.4.0.12.0 $55$DKB400 HSZ70 XB26 dkc0.0.0.13.0 $55$DKC0 HSZ70CCL XB26 dkc100.1.0.13.0 $55$DKC100 HSZ70 XB26 dkc104.1.0.13.0 $55$DKC104 HSZ70 XB26 dkc1300.13.0.13.0 $55$DKC1300 HSZ70 XB26 dkc1307.13.0.13.0 $55$DKC1307 HSZ70 XB26 dkc1400.14.0.13.0 $55$DKC1400 HSZ70 XB26 dkc1500.15.0.13.0 $55$DKC1500 HSZ70 XB26 dkc200.2.0.13.0 $55$DKC200 HSZ70 XB26 dkc205.2.0.13.0 $55$DKC205 HSZ70 XB26 dkc300.3.0.13.0 $55$DKC300 HSZ70 XB26 dkc400.4.0.13.0 $55$DKC400 HSZ70 XB26 dva0.0.0.1000.0 DVA0 ewa0.0.0.11.0 EWA0 08-00-2B-E4-CF-0B pka0.7.0.6.0 PKA0 SCSI Bus ID 7 pkb0.7.0.12.0 PKB0 SCSI Bus ID 7 5.54 pkc0.7.0.13.0 PKC0 SCSI Bus ID 7 5.54

The console does not automatically attempt to use an alternate path to a device if I/O fails on the current path. For many console commands, however, it is possible to specify a list of devices that the console will attempt to access in order. In a multipath configuration, you can specify a list of console device names that correspond to the multiple paths of a device. For example, a boot command, such as the following, will cause the console to attempt to boot the multipath device through the DKB100 path first, and if that fails, it will attempt to boot through the DKC100 path:

BOOT DKB100, DKC100

	privacy and legal statement
6318PRO_007.HTML