Document revision date: 15 July 2002

Starting with OpenVMS Alpha Version 7.3, qualifiers to the DCL command SET DEVICE are provided for specifying management attributes for shadow set members located at different sites. By using these qualifiers, system managers can override the default volume shadowing actions that can occur when the systems at one site of a disaster-tolerant OpenVMS Cluster configuration fail. These qualifiers are designed primarily for use in a configuration that uses Fibre Channel for a site-to-site storage interconnect. They can be used in other configurations as well.

Similarly, the DCL command DISMOUNT has been enhanced by the addition of the qualifier /FORCE_REMOVAL ddcu:. This qualifier has been added for the same purpose---to give system managers greater control of shadow set members located at different sites. For more information about this qualifier, see Section 4.8.1.

Table 4-3 SET DEVICE Command Qualifiers for Multiple-Site Shadow Set Members

Qualifier	Function
/FORCE_REMOVAL ddcu:	If connectivity to a device has been lost and the shadow set is in mount verification, this qualifier causes the member to be expelled from the shadow set immediately. If the shadow set is not currently in mount verification, no immediate action is taken. If connectivity to a device has been lost but the shadow set is not in mount verification, this qualifier lets you flag the member to be expelled from the shadow set, as soon as it does enter mount verification. The device specified must be a member of a shadow set that is mounted on the node where the command is issued.
/MEMBER_TIMEOUT = xxxxxx ddcu:	Specifies the timeout value to be used for a member of a shadow set. The value supplied by this qualifier overrides the SYSGEN parameter SHADOW_MBR_TMO for this specific device. Each member of a shadow set can be assigned a different MEMBER_TIMEOUT value. The valid range for xxxxxx is 1 to 16,777,215 seconds. The device specified must be a member of a shadow set that is mounted on the node where the command is issued. After you have applied this qualifier to a member, the setting remains in effect as long as the member is part of the shadow set. If the member is removed from the shadow set and later returned, this qualifier must be specified again.
/MVTIMEOUT = yyyyyy DSA nnnn:	Specifies the mount verification timeout value to be used for this shadow set, specified by its virtual unit name, DSA nnnn. The value supplied by this qualifier overrides the SYSGEN parameter MVTIMEOUT for this specific shadow set. The valid range for yyyyyy is 1 to 16,777,215 seconds. The device specified must be a shadow set that is mounted on the node where the command is issued. After you have applied this qualifier, the setting remains in effect as long as the shadow set is mounted. If the shadow set is dismounted and later remounted, this qualifier must be specified again
/READ_COST = zzz ddcu:	The valid range for zzz is 1 to 65,535 units. The device specified must be a member of a shadow set that is mounted on the node where the command is issued. This qualifier enables you to modify the default "cost" assigned to each member of a shadow set. By modifying the assignments, you can bias the reads in favor of one member of a two-member shadow set, or, in the case of three-member shadow sets, in favor of one or two members of the set over the remaining members. The shadowing driver assigns default READ_COST values to shadow set members when each member is initially mounted. The default value depends on the device type and its configuration relative to the system mounting it. There are default values for a DECRAM device; a directly connected device in the same physical location; a directly connected device in a remote location; a DECram served device; and a default value for other served devices. The value supplied by this qualifier overrides the default assignment. The shadowing driver adds the value of the current queue depth of the shadow set member to the READ_COST value and then reads from the member with the lowest value. After you have applied this qualifier to a member, the setting remains in effect as long as the member is part of the shadow set. If the member is removed from the shadow set and later returned, this qualifier must be specified again. Different systems in the cluster can assign different costs to each shadow set member. If the /SITE command qualifier has been specified, the shadowing driver will take site values into account when it assigns default READ_COST values. Note that in order for the shadowing software to determine whether a device is in the category of "directly connected device in a remote location," the /SITE command qualifier must have been applied to both the shadow set and the DSA device. Reads requested for a shadow set from a system at site 1 are performed from a shadow set member that is also at site 1. Reads requested for the same shadow set from site 2 can read from the member located at site 2.
/READ_COST = y DSA nnnn	The valid range for y is any number. The value supplied has no inherent meaning. The purpose of this qualifier is to switch the read cost setting for all shadow set members back to the default read cost settings established automatically by the shadowing software. DSA nnnn must be a shadow set that is mounted on the node from which this command is issued.
/SITE = ( nnn, logical_name) ( ddcu:, DSA nnnn:)	This qualifier indicates to the shadowing driver the site location of the shadow set member or of the shadow set (represented by its virtual unit name). Prior to using this qualifier, you can define the site location in the SYLOGICALS.COM command procedure to simplify its use. The valid range for nnn is 1 through 255. After you have applied this qualifier, the setting remains in effect as long as the member is part of the shadow set. If the member is removed from the shadow set and later returned, this qualifier must be specified again The following example shows the site locations defined, followed by the use of the /SITE qualifier: $ DEFINE/SYSTEM/EXEC ZKO 1 $ DEFINE/SYSTEM/EXEC LKG 2 $! $! At the ZKO site ... $ MOUNT/SYSTEM DSA0/SHAD=($1$DGA0:,$1$DGA1:) TEST $ SET DEVICE/SITE=ZKO DSA0: $! $! At the LKG site ... $ MOUNT/SYSTEM DSA0/SHAD=($1$DGA0,$1$DGA1) TEST $ SET DEVICE/SITE=LKG DSA0: $! $! At both sites, the following would be used: $ SET DEVICE/SITE=ZKO $1$DGA0: $ SET DEVICE/SITE=LKG $1$DGA1:
/COPY_SOURCE ( ddcu:, DSA nnnn:)	Controls whether one or both source members of a shadow set are used as the source for read data during full copy operations, when a third member is added to the shadow set. This only affects copy operations that do not use DCD operations. Some storage controllers, such as the HSG80, have a read-ahead cache, which significantly improves single-disk read performance. Copy operations normally alternate reads between the two source members, which effectively nullifies the benefits of the read-ahead cache. This qualifier lets you force all reads from a single source member for a copy operation. If the shadow set is specified, then all reads for full copy operations will be performed from whichever disk is the current "master" member, regardless of physical location of the disk. If a member of the shadow set is specified, then that member will be used as the source of all copy operations. This allows you to choose a local source member, rather than a remote master member.
/ABORT_VIRTUAL_UNIT DSA nnnn:	To use this qualifier, the shadow set must be in mount verification. When you specify this qualifier, the shadow set aborts mount verification immediately on the node from which the qualifier is issued. This qualifier is intended to be used when it is known that the unit cannot be recovered. Note that after this command completes, the shadow set must still be dismounted. Use the following command to dismount the shadow set: $ DISMOUNT/ABORT DSA nnnn

4.7.1 How to Use the Multiple-Site SET DEVICE and DISMOUNT Command Qualifiers

Figure 4-1 depicts a typical multiple-site cluster using Fibre Channel. The figure illustrates the steps required to recover one site manually when the site-to-site storage interconnect fails. These steps must be taken for multiple-site OpenVMS Cluster systems that are running:

• Earlier versions of OpenVMS (prior to OpenVMS Alpha Version 7.3-1) that do not support MSCP failover to a served path
• Versions of OpenVMS Alpha that support MSCP failover to a served path (Version 7.3-1 onward) but serve only a subset of their disks.
  If you have chosen to serve only a subset of the disks in your configuration, you must use this configuration method for site recovery for the disks that are not served. One reason to serve only a subset of your disks is that failover of the served disks, from a Fibre Channel interconnect to the LAN interconnects and the inter-site link, can put a very heavy load on these interconnects, which can seriously degrade performance.

Figure 4-1 Multiple-Site OpenVMS Cluster System With FC and LAN Interconnects

To prevent the shadowing driver from automatically recovering shadow sets from connection-related failures, three steps must be taken prior to any failure:

1. Every device that is a member of a multiple-site shadow set must have its MEMBER_TIMEOUT setting raised to a high value, using the following command:

   $ SET DEVICE /MEMBER_TIMEOUT= x ddcu:

   
   This command will override the SHADOW_MBR_TMO value, which would normally be used for a shadow set member. A value for x of 259200 would be a 72-hour wait time.

2. Every shadow set that spans multiple sites must have its mount verification timeout setting raised to a very high value, higher than the MEMBER_TIMEOUT settings for each member of the shadow set.
   Use the following command to increase the mount verification timeout setting for the shadow set:

   $ SET DEVICE /MVTIMEOUT = y DSAnnnn

   
   The y value of this command should always be greater than the x value of the SET DEVICE/MEMBER_TIMEOUT= x ddcu: command.
   The SET DEVICE /MVTIMEOUT = y command will override the MVTIMEOUT value, which would normally be used for the shadow set. A value for y of 262800 would be a 73 hour wait.

3. Every shadow set and every shadow set member must have a site qualifier. As already noted, a site qualifier will ensure that the read cost is correctly set. The other critical factor is three-member shadow sets. When they are being used, the site qualifier will ensure that the master member of the shadow set will be properly maintained.

Figure 4-1 shows a shadow set DSA42, whose members are devices $1$DGA1000 and $1$DGA2000. Systems at Site A or Site B have direct access to all devices at both sites via Fibre Channel connections. XYZZY is a theoretical point between the two sites. If the Fibre Channel connection were to break at this point, each site could access different "local" members of DSA42 without error.

For the purpose of this example, Site A will be the sole site chosen to retain access to the shadow set.

The following steps must be taken to recover the shadow set at Site A.

1. On Site A, issue the following command:

   $ DISMOUNT /FORCE_REMOVAL= $1$DGA2000:

   
   Once the command has completed, the shadow set will be available for use only at site A.

2. On Site B, issue the following command:

   $ SET DEVICE /ABORT_VIRTUAL_UNIT DSA42:

   
   Once the command has completed, the shadow set status will be MntVerifyTimeout .

3. Next, issue the following command to free up the shadow set:

   $ DISMOUNT/ABORT DSA42:

   
   These steps must be taken for all affected multiple-site shadow sets.

You can remove shadow set members and dissolve shadow sets with the DCL command DISMOUNT. You must have GRPNAM and SYSNAM user privileges to dismount group and system volumes. The DISMOUNT command has the following format:

DISMOUNT {device-name[:] virtual-unit-name}

The action taken differs depending on whether you specify an individual shadow set member or the shadow set (by its virtual unit name) on the DISMOUNT command:

• If you specify the device name of a shadow set member, only that member is dismounted, and the remaining shadow set members continue servicing I/O requests.
• If you specify a shadow set virtual unit, all shadow set members are dismounted and the shadow set is dissolved.

To dismount a shadow set that is mounted across an OpenVMS Cluster system, include the /CLUSTER qualifier with the DISMOUNT command. If you dismount a shadow set without including the /CLUSTER qualifier, only the node from which you issued the command dismounts the shadow set. The shadow set remains operational on the other OpenVMS Cluster nodes that have the shadow set mounted.

If the disks on your system are neither SCSI nor Fibre Channel disks, you can use the /NOUNLOAD qualifier on the DISMOUNT command to prevent the disk volume or volumes from spinning down. The devices remain in a ready state. If you specify the /UNLOAD qualifier when dismounting a virtual unit, the disk volumes are physically spun down after the shadow set is dissolved. See the OpenVMS DCL Dictionary for more information about using the DISMOUNT command and its qualifiers.

4.8.1 Removing Members from Shadow Sets

To remove an individual member from a shadow set, specify the name of the physical device with the DISMOUNT command. For example:

$ DISMOUNT $5$DUA7:

When you dismount an individual shadow set member, all outstanding I/O operations are completed and the member is removed from the set.

Starting with OpenVMS Alpha Version 7.3, the /FORCE_REMOVAL ddcu: qualifier is available. If connectivity to a device has been lost and the shadow set is in mount verification, /FORCE_REMOVAL ddcu: can be used to immediately expel a named shadow set member (ddcu:) from the shadow set. If you omit this qualifier, the device is not dismounted until mount verification completes. Note that this qualifier cannot be used in conjunction with the /POLICY=MINICOPY (=OPTIONAL) qualifier.

The device specified must be a member of a shadow set that is mounted on the node where the command is issued.

This qualifier gives system managers greater control of shadow sets whose members are located at different sites in an OpenVMS Cluster configuration. SET DEVICE command qualifiers are also available for specifying disaster-tolerant management attributes for shadow set members, as described in Section 4.7.

You dissolve a shadow set by specifying its virtual unit name with the DISMOUNT command. Specifying the virtual unit name causes all members in the shadow set to be dismounted, thereby dissolving the shadow set on the node that issues the DISMOUNT command. The command in Example 4-9 includes the /CLUSTER qualifier to dissolve the DSA36 shadow set across an OpenVMS Cluster system.

Example 4-9 Dissolving a Shadow Set

$ DISMOUNT /CLUSTER DSA36:

Dismounting the virtual unit can be done only after all files are closed, thereby ensuring that the dismounted disks are fully consistent from a file system perspective. The dismount operation marks the shadow set members as being properly dismounted so that a rebuild is not required the next time the disks are mounted. However, if a merge operation was either pending or in progress, then the dismount operation marks the shadow set members as being improperly dismounted and requires a merge operation.

As discussed in Section 4.8.2, the virtual unit can be dismounted on the system or across an OpenVMS Cluster system. To ensure that the virtual unit has been dismounted correctly, the following steps are recommended:

1. Issue the MOUNT/NOWRITE command, followed by the SHOW DEVICE command, for example:

   $ MOUNT/NOWRITE DSA42: /SHADOW=($4$DUA3,$4$DUA4,$4$DUA5) volume-label $ SHOW DEVICE DSA42:

2. Observe that the virtual unit is in a steady state; that is, all members are consistent and no copy or merge operation is in progress. If a copy or merge operation is in progress, you must wait for the operation to complete.
3. When the virtual unit is in a steady state, remove a member from the shadow set with the DISMOUNT command, as shown in the following example:

   $ DISMOUNT $4$DUA5

4. Dismount the virtual unit and then remount it with one less member, as shown by the following command:

   $ MOUNT/SYS DSA42: /SHADOW=($4$DUA3,$4$DUA4) volume-label

   
   The shadow set member that was removed can now be used for a backup operation of the virtual unit.

You can use the DCL command SHOW DEVICE or the F$GETDVI lexical function to get information about a shadow set virtual unit and the physical volumes that make up the members. You can also use the System Dump Analyzer (SDA) to get more information about shadow sets.

The following sections describe how to use these tools to examine volume shadowing virtual units and shadow set members. See also the OpenVMS DCL Dictionary for a full description of how to use the SHOW DEVICE command and the F$GETDVI lexical function. See the OpenVMS Alpha System Analysis Tools Manual and the OpenVMS VAX System Dump Analyzer Utility Manual for more information about how to use SDA on OpenVMS Alpha and OpenVMS VAX systems, respectively.

You can use any of the SHOW DEVICE qualifiers when you examine shadow sets (by specifying a shadow set's virtual unit name) or shadow set members.

	privacy and legal statement
5423PRO_005.HTML