Document revision date: 19 July 1999

When the bus is not very busy, and bus contention is uncommon, the simple arbitration scheme is adequate to perform I/O requests for all devices on the system. However, as initiators make more and more frequent I/O requests, contention for the bus becomes more and more common. Consequently, targets with lower ID numbers begin to perform poorly, because they are frequently blocked from completing their I/O requests by other users of the bus (in particular, targets with the highest ID numbers). If the bus is sufficiently busy, low-numbered targets may never complete their requests. This situation is most likely to occur on systems with more than one initiator because more commands can be outstanding at the same time.

The OpenVMS system attempts to prevent low-numbered targets from being completely blocked by monitoring the amount of time an I/O request takes. If the request is not completed within a certain period, the OpenVMS system stops sending new requests until the tardy I/Os complete. While this algorithm does not ensure that all targets get equal access to the bus, it does prevent low-numbered targets from being totally blocked.

A.7.5.2 Solutions for Resolving Arbitration Problems

If you find that some of your disks are not being serviced quickly enough during periods of heavy I/O, try some or all of the following, as appropriate for your site:

• Obtain the DWZZH--05 SCSI hub and enable its fair arbitration feature.
• Assign the highest ID numbers to those disks that require the fastest response time.
• Spread disks across more SCSI buses.
• Keep disks that need to be accessed only by a single host (for example, page and swap disks) on a nonshared SCSI bus.

Another method that might provide for more equal servicing of lower and higher ID disks is to set the host IDs to the lowest numbers (0 and 1) rather than the highest. When you use this method, the host cannot gain control of the bus to send new commands as long as any disk, including those with the lowest IDs, need the bus. Although this option is available to improve fairness under some circumstances, this configuration is less desirable in most instances, for the following reasons:

• It can result in lower total throughput.
• It can result in timeout conditions if a command cannot be sent within a few seconds.
• It can cause physical configuration difficulties. For example, StorageWorks shelves such as the BA350 have no slot to hold a disk with ID 7, but they do have a slot for a disk with ID 0. If you change the host to ID 0, you must remove a disk from slot 0 in the BA350, but you cannot move the disk to ID 7. If you have two hosts with IDs 0 and 1, you cannot use slot 0 or 1 in the BA350. (Note, however, that you can have a disk with ID 7 in a BA353.)

Any active device, such as a DWZZx, that connects bus segments introduces small delays as signals pass through the device from one segment to another. Under some circumstances, these delays can be another cause of unfair arbitration. For example, consider the following configuration, which could result in disk servicing problems (starvation) under heavy work loads:

Although disk 5 has the highest ID number, there are some circumstances under which disk 5 has the lowest access to the bus. This can occur after one of the lower-numbered disks has gained control of the bus and then completed the operation for which control of the bus was needed. At this point, disk 5 does not recognize that the bus is free and might wait before trying to arbitrate for control of the bus. As a result, one of the lower-numbered disks, having become aware of the free bus and then submitting a request for the bus, will gain control of the bus.

If you see this type of problem, the following suggestions can help you reduce its severity:

• Try to place all disks on the same bus segment.
• If placing all disks on the same bus segment is not possible (for example if you have both some RZ28 disks by themselves and an HSZxx, try to use a configuration that has only one isolator between any pair of disks.
• If your configuration requires two isolators between a pair of disks (for example, to meet distance requirements), try to balance the number of disks on each bus segment.
• Follow the suggestions in Section A.7.5.2 to reduce the total traffic on the logical bus.

With proper procedures, certain SCSI devices can be removed from or inserted onto an active SCSI bus without disrupting the ongoing operation of the bus. This capability is referred to as hot plugging. Hot plugging can allow a suitably configured OpenVMS Cluster system to continue to run while a failed component is replaced. Without hot plugging, it is necessary to make the SCSI bus inactive and remove power from all the devices on the SCSI bus before any device is removed from it or inserted onto it.

In a SCSI OpenVMS Cluster system, hot plugging requires that all devices on the bus have certain electrical characteristics and be configured appropriately on the SCSI bus. Successful hot plugging also depends on strict adherence to the procedures described in this section. These procedures ensure that the hot-plugged device is inactive and that active bus signals are not disturbed.

The terms shown in bold in this section are used in the discussion of hot plugging rules and procedures.

• A SCSI bus segment consists of two terminators, the electrical path forming continuity between them, and possibly, some attached stubs. Bus segments can be connected together by bus isolators (for example, DWZZx), to form a logical SCSI bus or just a SCSI bus.
• There are two types of connections on a segment: bussing connections, which break the path between two terminators, and stubbing connections, which disconnect all or part of a stub.
• A device is active on the SCSI bus when it is asserting one or more of the bus signals. A device is inactive when it is not asserting any bus signals.
  The segment attached to a bus isolator is inactive when all devices on that segment, except possibly the bus isolator, are inactive.
• A port on a bus isolator has proper termination when it is attached to a segment that is terminated at both ends and has TERMPWR in compliance with SCSI--2 requirements.

Follow these rules when planning for and performing hot plugging:

• The device to be hot plugged, and all other devices on the same segment, must meet the electrical requirements described in Annex A, Section A.4, of the SCSI-3 Parallel Interface (SPI) Standard, working draft X3T10/855D.¹ The SPI document places requirements on the receivers and terminators on the segment where the hot plugging is being performed, and on the transceivers, TERMPWR, termination, and power/ground/signal sequencing, of the device that is being hot plugged.
• Hot plugging must occur only at a stubbing connection.
  This implies that a hot-plugged device can make only one connection to the SCSI bus, the device must not provide termination for the SCSI bus, and the device's connection must not exceed the maximum stub length, as shown in Figure A-3. An example of a SCSI bus topology showing the valid hot plugging connections is illustrated in Figure A-13.

  Figure A-13 SCSI Bus Topology

  ---

  
• Take precautions to ensure that electrostatic discharge (ESD) does not damage devices or disrupt active signals on the SCSI bus. You should take such precautions during the process of disconnecting and connecting, as well as during the time that SCSI bus conductors are exposed.
• Take precaution to ensure that ground offset voltages do not pose a safety hazard and will not interfere with SCSI bus signaling, especially in single-ended configurations. The procedures for measuring and eliminating ground offset voltages are described in Section A.7.8.
• The device that is hot plugged must be inactive during the disconnection and connection operations. Otherwise, the SCSI bus may hang.²

  Note Ideally, a device will also be inactive whenever its power is removed, for the same reason.

  
  The procedures for ensuring that a device is inactive are described in Section A.7.6.3.
• A quorum disk must not be hot plugged. This is because there is no mechanism for stopping the I/O to a quorum disk, and because the replacement disk will not contain the correct quorum file.
  The OpenVMS Cluster system must be reconfigured to remove a device as a quorum disk before that device is removed from the bus. The procedure for accomplishing this is described in OpenVMS Cluster Systems.
  An alternate method for increasing the availability of the quorum disk is to use an HSZxx mirror set as the quorum disk. This would allow a failed member to be replaced while maintaining the quorum disk functionality.
• Disks must be dismounted logically before removing or replacing them in a hot-plugging operation. This is required to ensure that the disk is inactive and to ensure the integrity of the file system.
• The DWZZx must be powered up when it is inserted into an active SCSI bus and should remain powered up at all times while it is attached to the active SCSI bus. This is because the DWZZx can disrupt the operation of the attached segments when it is powering up or down.
• The segment attached to a bus isolator must be maintained in the inactive state whenever the other port on the bus isolator is terminated improperly. This is required because an improperly terminated bus isolator port may pass erroneous signals to the other port.
  Thus, for a particular hot-plugging operation, one of the segments attached to a bus isolator must be designated as the (potentially) active segment, and the other must be maintained in the inactive state, as illustrated in Figure A-14. The procedures for ensuring that a segment is inactive are described in Section A.7.6.3.

  Figure A-14 Hot Plugging a Bus Isolator

  ---

  
  Note that, although a bus isolator may have more than one stubbing connection and thus be capable of hot plugging on each of them, only one segment can be the active segment for any particular hot-plugging operation.
• Take precautions to ensure that the only electrical conductor that contacts a connector pin is its mate. These precautions must be taken during the process of disconnecting and connecting as well as during the time the connector is disconnected.
• Devices must be replaced with devices of the same type. That is, if any system in the OpenVMS Cluster configures a SCSI ID as a DK or MK device, then that SCSI ID must contain only DK or MK devices, respectively, for as long as that OpenVMS Cluster member is running.
  Different implementations of the same device type can be substituted (for example, an RZ26L can be replaced with an RZ28B). Note that the system will not recognize the change in device type until an attempt is made to mount the new device. Also, note that host-based shadowing continues to require that all members of a shadow set be the same device type.
• SCSI IDs that are empty when a system boots must remain empty as long as that system is running. This rule applies only if there are multiple processors on the SCSI bus and the MSCP server is loaded on any of them. (The MSCP server is loaded when the MSCP_LOAD system parameter is set to 1).
  This is required to ensure that nodes on the SCSI bus use their direct path to the disk rather than the served path. When the new device is configured on a system (using SYSMAN IO commands), that system serves it to the second system on the shared SCSI bus. The second system automatically configures the new device by way of the MSCP served path. Once this occurs, the second system will be unable to use its direct SCSI path to the new device because failover from an MSCP served path to a direct SCSI path is not implemented.

Use the following procedures to ensure that a device or a segment is inactive:

• To ensure that a disk is inactive:
  1. Dismount the disk on all members of the OpenVMS Cluster system.
  2. Ensure that any I/O that can occur to a dismounted disk is stopped, for example:
     • Disable the disk as a quorum disk.
     • Allocate the disk (using the DCL command ALLOCATE) to block further mount or initialization attempts.
     • Disable console polling by all halted hosts on the logical SCSI bus (by setting the console variable SCSI_POLL to OFF and entering the INIT command).
     • Ensure that no host on the logical SCSI bus is executing power-up or initialization self-tests, booting, or configuring the SCSI bus (using SYSMAN IO commands).
• To ensure that an HSZxx controller is inactive:
  1. Dismount all of the HSZxx virtual disks on all members of the OpenVMS Cluster system.
  2. Shut down the controller, following the procedures in the HS Family of Array Controllers User's Guide.
  3. Power down the HSZxx (optional).
• To ensure that a host adapter is inactive:
  1. Halt the system.
  2. Power down the system, or set the console variable SCSI_POLL to OFF and then enter the INIT command on the halted system. This ensures that the system will not poll or respond to polls.
• To ensure that a segment is inactive, follow the preceding procedures for every device on the segment.

To remove an SBB (storage building block) disk from an active SCSI bus, use the following procedure:

1. Use an ESD grounding strap that is attached either to a grounding stud or to unpainted metal on one of the cabinets in the system. Refer to the system installation procedures for guidance.
2. Follow the procedure in Section A.7.6.3 to make the disk inactive.
3. Squeeze the clips on the side of the SBB, and slide the disk out of the StorageWorks shelf.

To plug an SBB disk into an active SCSI bus, use the following procedure:

1. Use an ESD grounding strap that is attached either to a grounding stud or to unpainted metal on one of the cabinets in the system. Refer to the system installation procedures for guidance.
2. Ensure that the SCSI ID associated with the device (either by jumpers or by the slot in the StorageWorks shelf) conforms to the following:
   • The SCSI ID is unique for the logical SCSI bus.
   • The SCSI ID is already configured as a DK device on all of the following:
     • Any member of the OpenVMS Cluster system that already has that ID configured
     • Any OpenVMS processor on the same SCSI bus that is running the MSCP server
3. Slide the SBB into the StorageWorks shelf.
4. Configure the disk on OpenVMS Cluster members, if required, using SYSMAN IO commands.

To remove an HSZxx controller from an active SCSI bus:

1. Use an ESD grounding strap that is attached either to a grounding stud or to unpainted metal on one of the cabinets in the system. Refer to the system installation procedures for guidance.
2. Follow the procedure in Section A.7.6.3 to make the HSZxx inactive.
3. The HSZxx can be powered down, but it must remain plugged in to the power distribution system to maintain grounding.
4. Unscrew and remove the differential triconnector from the HSZxx.
5. Protect all exposed connector pins from ESD and from contacting any electrical conductor while they are disconnected.

To plug an HSZxx controller into an active SCSI bus:

1. Use an ESD grounding strap that is attached either to a grounding stud or to unpainted metal on one of the cabinets in the system. Refer to the system installation procedures for guidance. Also, ensure that the ground offset voltages between the HSZxx and all components that will be attached to it are within the limits specified in Section A.7.8.
2. Protect all exposed connector pins from ESD and from contacting any electrical conductor while they are disconnected.
3. Power up the HSZxx and ensure that the disk units associated with the HSZxx conform to the following:
   • The disk units are unique for the logical SCSI bus.
   • The disk units are already configured as DK devices on the following:
     • Any member of the OpenVMS Cluster system that already has that ID configured
     • Any OpenVMS processor on the same SCSI bus that is running the MSCP server
4. Ensure that the HSZxx will make a legal stubbing connection to the active segment. (The connection is legal when the triconnector is attached directly to the HSZxx controller module, with no intervening cable.)
5. Attach the differential triconnector to the HSZxx, using care to ensure that it is properly aligned. Tighten the screws.
6. Configure the HSZxx virtual disks on OpenVMS Cluster members, as required, using SYSMAN IO commands.

To remove a host adapter from an active SCSI bus:

1. Use an ESD grounding strap that is attached either to a grounding stud or to unpainted metal on one of the cabinets in the system. Refer to the system installation procedures for guidance.
2. Verify that the connection to be broken is a stubbing connection. If it is not, then do not perform the hot plugging procedure.
3. Follow the procedure in Section A.7.6.3 to make the host adapter inactive.
4. The system can be powered down, but it must remain plugged in to the power distribution system to maintain grounding.
5. Remove the "Y" cable from the host adapter's single-ended connector.
6. Protect all exposed connector pins from ESD and from contacting any electrical conductor while they are disconnected.
7. Do not unplug the adapter from the host's internal bus while the host remains powered up.
   At this point, the adapter has disconnected from the SCSI bus. To remove the adapter from the host, first power down the host, then remove the adapter from the host's internal bus.

To plug a host adapter into an active SCSI bus:

1. Use an ESD grounding strap that is attached either to a grounding stud or to unpainted metal on one of the cabinets in the system. Refer to the system installation procedures for guidance. Also, ensure that the ground offset voltages between the host and all components that will be attached to it are within the limits specified in Section A.7.8.
2. Protect all exposed connector pins from ESD and from contacting any electrical conductor while they are disconnected.
3. Ensure that the host adapter will make a legal stubbing connection to the active segment (the stub length must be within allowed limits, and the host adapter must not provide termination to the active segment).
4. Plug the adapter into the host (if it is unplugged).
5. Plug the system into the power distribution system to ensure proper grounding. Power up, if desired.
6. Attach the "Y" cable to the host adapter, using care to ensure that it is properly aligned.

Use the following procedure to remove a DWZZx from an active SCSI bus:

1. Use an ESD grounding strap that is attached either to a grounding stud or to unpainted metal on one of the cabinets in the system. Refer to the system installation procedures for guidance.
2. Verify that the connection to be broken is a stubbing connection. If it is not, then do not perform the hot plugging procedure.
3. Do not power down the DWZZx. This can disrupt the operation of the attached SCSI bus segments.
4. Determine which SCSI bus segment will remain active after the disconnection. Follow the procedure in Section A.7.6.3 to make the other segment inactive.
   When the DWZZx is removed from the active segment, the inactive segment must remain inactive until the DWZZx is also removed from the inactive segment, or until proper termination is restored to the DWZZx port that was disconnected from the active segment.
5. The next step depends on the type of DWZZx and the segment that is being hot plugged, as follows:

   DWZZx Type	Condition	Action
   SBB 1	Single-ended segment will remain active.	Squeeze the clips on the side of the SBB, and slide the DWZZ x out of the StorageWorks shelf.
   SBB 1	Differential segment will remain active.	Unscrew and remove the differential triconnector from the DWZZ x.
   Table top	Single-ended segment will remain active.	Remove the "Y" cable from the DWZZ x's single-ended connector.
   Table top	Differential segment will remain active.	Unscrew and remove the differential triconnector from the DWZZ x.

   ---

   ¹SSB is the StorageWorks abbreviation for storage building block.
   

   ---

6. Protect all exposed connector pins from ESD and from contacting any electrical conductor while they are disconnected.

To plug a DWZZx into an active SCSI bus:

1. Use an ESD grounding strap that is attached either to a grounding stud or to unpainted metal on one of the cabinets in the system. Refer to the system installation procedures for guidance. Also, ensure that the ground offset voltages between the DWZZx and all components that will be attached to it are within the limits specified in Section A.7.8.
2. Protect all exposed connector pins from ESD and from contacting any electrical conductor while they are disconnected.
3. Ensure that the DWZZx will make a legal stubbing connection to the active segment (the stub length must be within allowed limits, and the DWZZx must not provide termination to the active segment).
4. The DWZZx must be powered up. The SCSI segment that is being added must be attached and properly terminated. All devices on this segment must be inactive.
5. The next step depends on the type of DWZZx, and which segment is being hot plugged, as follows:

   DWZZx Type	Condition	Action
   SBB 1	Single-ended segment is being hot plugged.	Slide the DWZZ x into the StorageWorks shelf.
   SBB 1	Differential segment is being hot plugged.	Attach the differential triconnector to the DWZZ x, using care to ensure that it is properly aligned. Tighten the screws.
   Table top	Single-ended segment is being hot plugged.	Attach the "Y" cable to the DWZZ x, using care to ensure that it is properly aligned.
   Table top	Differential segment is being hot plugged.	Attach the differential triconnector to the DWZZ x, using care to ensure that it is properly aligned. Tighten the screws.

   ---

   ¹SSB is the StorageWorks abbreviation for storage building block.
   

   ---

6. If the newly attached segment has storage devices on it, then configure them on OpenVMS Cluster members, if required, using SYSMAN IO commands.

	privacy and legal statement
6318PRO_021.HTML