Order Number: AA-PWQ3P-TE

This manual contains information and guidelines for operation of HSM and Media and Device management Services (MDMS).

Compaq Computer Corporation
Houston, Texas

1 Introduction to HSM

1.1 Storage Management in the OpenVMS Environment 1

1.1.1 Data Categories 1

1.1.2 Device Capacity, Cost, and Performance 1

1.1.3 Storage Management Planning 2

1.2 Storage Management with HSM 3

1.2.1 File Headers and Location 3

1.2.2 Controlling File Movement 3

1.3 HSM Storage Management Concepts 4

1.3.1 Shelf 4

1.3.2 Archive Class 4

1.3.3 HSM Policies 4

1.4 The Shelving Process 4

1.4.1 Starting the Shelving Process 4

1.4.2 File Selection for Explicit Shelving 5

1.4.3 File Selection for Implicit Shelving 5

1.4.4 Modifying File Attributes of a Shelved File 6

1.5 The Unshelving Process 6

1.5.1 Starting the Unshelving Process 6

1.5.2 Process Default Unshelving Action 6

1.5.3 The Results of Unshelving a File 7

1.5.4 Handling Duplicate Requests to Unshelve a File 7

1.6 The Preshelving Process 7

1.7 The Unpreshelving Process 7

1.8 File Headers and Access Security 7

1.9 HSM File State Diagram 8

1.10 HSM Cache 8

1.10.1 HSM Operations with Cache 9

1.10.2 Cache in the Shelving and Preshelving Processes 9

1.10.3 Unshelving from Cache 9

1.10.4 Exceeding Cache Capacity 9

1.10.5 Flushing Cache 9

1.11 HSM Catalogs 10

1.12 HSM Archive Repacking 10

1.13 HSM Software Modes 10

1.13.1 HSM Basic Mode Functions 11

1.13.2 HSM Plus Mode Functions 11

1.14 Media Types for HSM Basic Mode 12

1.15 Device Support 14

1.16 Online Devices Not Supported for HSM Operations 14

1.17 HSM Support for Remote Operations 15

2 Understanding HSM Concepts

2.1 The HSM Environment 2-1

2.2 The HSM Facility 2-2

2.2.1 HSM Mode 2-2

2.2.2 HSM Operations 2-3

2.2.3 Shelf Servers 2-3

2.2.4 Event Logging 2-4

2.3 The Shelf 2-5

2.3.1 Using Multiple Shelf Copies 2-5

2.3.2 Defining Shelf Copies 2-6

2.3.2.1 Archive Lists and Restore Archive Lists 2-6

2.3.2.2 Primary and Secondary Archive Classes 2-7

2.3.2.3 Multiple Shelf Copies 2-7

2.3.3 Shelving Operations 2-7

2.3.4 Shelf Catalog 2-8

2.3.5 Save Time 2-8

2.3.6 Number of Updates for Retention 2-8

2.4 HSM Basic Mode Archive Class 2-8

2.5 HSM Plus Mode Archive Class 2-9

2.6 Device 2-10

2.6.1 Sharing and Dedicating Devices 2-10

2.6.2 Device Operations 2-11

2.6.3 Devices and Archive Classes 2-11

2.6.4 Magazine Loaders for HSM Basic Mode 2-13

2.6.5 Compatible Media for HSM Basic Mode 2-14

2.6.6 Automated Loaders and HSM Plus Mode 2-15

2.7 Volume 2-15

2.7.1 Shelf 2-15

2.7.2 Shelving Operations 2-15

2.7.3 Volume Policy 2-16

2.7.4 High Water Mark 2-16

2.7.5 Files Excluded from Shelving 2-16

2.8 Cache Usage 2-17

2.8.1 Advantages and Disadvantages of Using a Cache 2-17

2.8.2 Cache Flushing 2-17

2.8.3 Cache Attributes 2-17

2.8.3.1 Timing of Shelf Copies 2-17

2.8.3.2 Cache Block Size 2-18

2.8.3.3 High Water Mark 2-18

2.8.3.4 Cache Flush Interval 2-18

2.8.3.5 Cache Flush Delay 2-18

2.8.3.6 Delete and Modify File Action 2-18

2.8.4 Optimizing Cache Usage 2-18

2.8.5 Using Magneto-Optical Devices 2-19

2.9 Policy 2-19

2.9.1 HSM Policy Options 2-19

2.9.2 Trigger Events 2-19

2.9.2.1 Scheduled Trigger 2-20

2.9.2.2 User Disk Quota Exceeded Trigger 2-21

2.9.2.3 High Water Mark Trigger 2-21

2.9.2.4 Volume Full Trigger 2-21

2.9.3 File Selection Criteria 2-21

2.9.4 Policy Goal 2-23

2.9.5 Make Space Requests and Policy 2-23

2.10 Schedule 2-24

2.10.1 Online Volumes 2-24

2.10.2 Execution Timing and Interval 2-24

2.10.3 Server Node 2-25

2.11 HSM System Files and Logical Names 2-25

2.11.1 HSM$MANAGER 2-25

2.11.2 HSM$LOG 2-26

2.11.3 HSM$REPACK 2-26

3 Customizing the HSM Environment

3.1 Configuring a Customized HSM Environment 3-1

3.1.1 Customizing the HSM Facility 3-1

3.1.2 Creating Shelf Definitions 3-1

3.1.3 Enabling and Disabling a Shelf Definition 3-2

3.1.4 Modifying Archive Classes 3-2

3.1.5 Creating Device Definitions 3-3

3.1.6 Modifying Device Definitions 3-3

3.1.7 Enabling and Disabling a Volume Definition 3-3

3.1.8 Working with Caches 3-3

3.1.9 Enabling and Disabling a Policy Definition 3-4

3.1.10 Scheduling Policy Executions 3-4

3.2 Implementing Shelving Policies 3-4

3.2.1 Determining the Disk Volumes 3-5

3.2.2 Creating Volume Definitions 3-5

3.2.3 Determining File Selection Criteria 3-6

3.2.4 Creating Policy Definitions 3-6

3.2.5 Using Expiration Dates 3-6

3.2.6 Creating Schedule Definitions 3-7

3.2.7 Enabling Preventive Policy 3-8

4 Using HSM

4.1 What the User Sees in an HSM Environment 4-1

4.1.1 Identifying Shelved Data using the DIRECTORY Command 4-1

4.1.1.1 DIRECTORY/FULL 4-1

4.1.1.2 DIRECTORY/FULL for Unpopulated Index Files 4-2

4.1.1.3 DIRECTORY/FULL for Populated Indexed Files 4-3

4.1.1.4 DIRECTORY/SHELVED_STATE 4-4

4.1.1.5 DIRECTORY/SIZE 4-4

4.1.2 Accessing Files 4-5

4.1.3 Decreasing Volume Full and Disk Quota Exceeded Errors 4-5

4.1.4 Viewing Messages 4-5

4.2 Controlling Shelving and Unshelving 4-6

4.2.1 Automatic Shelving Operations 4-6

4.2.2 User-Controlled Shelving Operations 4-6

4.2.3 Unshelving Files 4-7

4.3 Finding Lost Data 4-8

4.4 Working with Remote Files 4-8

4.5 Resolving Duplicate Operations on the Same File 4-8

4.5.1 Resolving Conflicting Operations on the Same File 4-8

4.6 Controlling Other HSM Functions 4-9

5 Managing the HSM Environment

5.1 Dismounting Disks 5-2

5.2 Copying Shelved Files 5-2

5.3 Renaming Disks 5-3

5.4 Restoring Files to a Different Disk 5-4

5.5 Protecting System Files from Shelving 5-4

5.5.1 Critical HSM Product Files 5-4

5.5.2 OpenVMS System Files and System Disks 5-5

5.5.3 Files Not Shelved 5-5

5.6 DFS, NFS and PATHWORKS Access Support 5-6

5.6.1 DFS Access 5-6

5.6.2 NFS Access 5-6

5.6.3 PATHWORKS 5-7

5.6.4 Logical Names for NFS and PATHWORKS Access 5-8

5.7 Ensuring Data Safety with HSM 5-8

5.7.1 Access Control Lists for Shelved Files 5-8

5.7.2 Handling Contiguous and Placed Files 5-8

5.8 Using Backup Strategies with HSM 5-9

5.8.1 Backing up Critical HSM Files 5-9

5.8.1.1 Defining a Backup Strategy 5-9

5.8.1.2 Using OpenVMS BACKUP to Save the Files 5-9

5.8.1.3 Maintaining a Manual Copy of the Files 5-9

5.8.2 Backing Up Shelved Data 5-10

5.8.2.1 Considerations for OpenVMS BACKUP and Shelving 5-10

5.8.2.2 Using Multiple HSM Archive Classes for Backup 5-10

5.8.2.3 Storing HSM Archive Classes Offsite 5-10

5.8.3 Backing Up Data Stored in an Online Cache 5-10

5.8.3.1 Flushing the Cache 5-11

5.9 Finding Lost User Data 5-11

5.10 Disaster Recovery 5-12

5.10.1 Recovering Data Shelved Through HSM 5-12

5.10.2 Recovering Critical HSM Files 5-12

5.10.3 Recovering Boot-Up Files 5-13

5.10.4 Reshelving an Archive Class 5-13

5.11 Maintaining Shelving Policies 5-13

5.11.1 The HSM Policy Model 5-13

5.11.1.1 Concepts of HSM Policy 5-13

5.11.1.2 Policy Governs the Shelving Process 5-14

5.11.1.3 The Balance to Achieve When Implementing Policy 5-16

5.11.2 HSM Policy Situations and Resolutions 5-16

5.11.2.1 Situation : Volume Occupancy Full Event 5-16

5.11.2.2 Situation : Shelving Goal Not Reached 5-17

5.11.2.3 Situation : Frequent Reactive Shelving Requests 5-18

5.11.2.4 Situation : Application and User Performance Impeded 5-19

5.11.3 Ranking Policy Execution 5-20

5.12 Managing HSM Catalogs 5-21

5.13 Repacking Archive Classes 5-23

5.13.1 Repack Performance 5-24

5.14 Replacing and Creating Archive Classes 5-25

5.15 Replacing A Lost or Damaged Shelf Volume 5-25

5.16 Catalog Analysis and Repair 5-25

5.17 Consolidated Backup with HSM 5-28

5.17.1 Setting up SLS 5-28

5.17.2 Preshelving Files 5-29

5.17.3 Nightly Backups 5-30

5.17.4 Restoring Volumes 5-30

5.17.5 Restoring Files 5-31

5.17.6 Repacking 5-31

5.17.7 Other Recommendations 5-31

5.18 Determining Cache Usage 5-31

5.19 Maintaining File Headers 5-31

5.19.1 Determining File Header Limit 5-32

5.19.2 Specifying a Volume's File Headers 5-32

5.19.3 Extending the Index File 5-32

5.19.4 Maintaining the Number of File Headers 5-33

5.20 Event Logging 5-33

5.20.1 Accessing the Logs 5-33

5.20.2 Shelf Handler Log Entries 5-34

5.21 Activity Logging 5-35

5.22 Converting from Basic Mode to Plus Mode 5-37

5.22.1 Shutting Down the Shelf Handler 5-37

5.22.2 Disabling the Shelving Facility 5-37

5.22.3 Entering Information for MDMS 5-37

5.22.4 Changing from Basic Mode to Plus Mode 5-38

5.22.5 Restarting the Shelf Handler 5-38

5.22.6 Using the Same Archive Classes 5-38

5.22.7 Enabling the Facility 5-38

5.22.8 Example Mode Conversion 5-38

6 Operator Activities in the HSM
Environment

6.1 Enabling the Operator Interface 6-1

6.2 Loading and Unloading Single Tapes for HSM Basic Mode 6-1

6.2.1 Load Volume, No Reply Needed 6-1

6.2.2 Load Volume 6-2

6.2.3 Reinitialize Volume 6-2

6.2.4 Volume Initialization Confirmation 6-2

6.2.5 Unload Label Request 6-2

6.3 Responding to BACKUP Requests for HSM Basic Mode 6-3

6.4 Working with Magazine Loaders for HSM Basic Mode 6-3

6.4.1 Load Magazine 6-3

6.4.2 Illegal Magazine 6-3

6.4.3 Unload Magazine 6-3

6.5 Working with Automated Loaders for HSM Plus Mode 6-3

6.5.1 Providing the Correct Magazine 6-3

6.5.2 Providing the Correct Volume for a TL820 6-4

6.6 Other MDMS Messages 6-5

6.7 Drive Selection and Reservation Messages for Both Modes 6-6

6.7.1 Unavailable Drive 6-6

6.7.2 Reservation Stalled 6-6

6.7.3 Wrong Tape Label 6-6

6.8 Informational Operator Messages 6-6

7 Solving Problems with HSM

7.1 Introduction to Troubleshooting 7-1

7.2 Troubleshooting Tools 7-2

7.2.1 Startup Logs 7-2

7.2.2 After a problem occurs, the first things you should check are the event logs: 7-3

7.2.3 Activity Log 7-3

7.2.4 SMU LOCATE 7-3

7.2.5 UNSHELVE/OVERRIDE 7-4

7.2.6 SMU RANK 7-4

7.2.7 SMU SET and SHOW Commands 7-4

7.2.8 MDMS Tools for HSM Plus Mode 7-5

7.3 Installation Problems 7-5

7.4 HSM Startup Problems 7-6

7.4.1 SMU Does Not Run 7-6

7.4.2 The Shelf Handler Does Not Start Up 7-6

7.4.3 Policy Execution Process Does Not Start Up 7-7

7.4.4 HSM Does Not Shut Down 7-8

7.4.5 Shelving and SMU Commands Do Not Work 7-8

7.5 Mass Shelving 7-9

7.6 Shelving on System Disks 7-10

7.6.1 HSM Plus Mode (MDMS) Problems 7-10

7.7 HSM VMScluster Problems 7-11

7.8 Online Disk Problems 7-12

7.9 Cache Problems 7-14

7.10 Magneto-Optical Device Problems 7-15

7.11 Offline Device Problems 7-16

7.12 Magazine and Robotic Loader Problems 7-17

7.13 Shelving Problems 7-18

7.14 Unshelving Problems 7-19

7.15 Policy Problems 7-20

7.16 HSM System File Problems 7-21

7.17 HSM Limitations 7-22

7.17.1 OpenVMS Limit on File Headers 7-22

7.17.2 Attempting to Cancel Execution of a Shelved File 7-23

7.17.3 Automatic Unshelving of Files across a Network 7-23

7.17.4 Opening and Deleting RMS Indexed Files 7-23

8 What is MDMS?

8.1 MDMS Objects 8-1

8.2 MDMS Interfaces 8-2

9 Basic MDMS Operations

9.1 MDMS User Interfaces 9-1

9.1.1 Command Line Interface 9-1

9.1.1.1 Command Structure 9-1

9.1.1.2 Process Symbols and Logical Names for DCL Programming 9-1

9.1.1.3 Creating, Changing, and Deleting Object Records With the CLI 9-1

9.1.1.4 Add and Remove Attribute List Values With the CLI 9-2

9.1.1.5 Operational CLI Commands 9-2

9.1.1.6 Asynchronous Requests 9-3

9.1.2 Graphic User Interface 9-3

9.1.2.1 Object Oriented Tasks 9-3

9.1.2.2 Combined Tasks 9-4

9.2 Access Rights for MDMS Operations 9-5

9.2.1 Description of MDMS Rights 9-5

9.2.1.1 Low Level Rights 9-5

9.2.1.2 High Level Rights 9-5

9.2.2 Granting MDMS Rights 9-6

9.3 Creating, Modifying, and Deleting Object Records 9-8

9.3.1 Creating Object Records 9-8

9.3.1.1 Naming Objects 9-8

9.3.1.2 Differences Between the CLI and GUI for Naming Object Records 9-8

9.3.2 Inheritance on Creation 9-9

9.3.3 Referring to Non-Existent Objects 9-9

9.3.4 Rights for Creating Objects 9-9

9.3.5 Modifying Object Records 9-9

9.3.6 Protected Attributes 9-9

9.3.7 Rights for Modifying Objects 9-10

9.3.8 Deleting Object Records 9-10

9.3.9 Reviewing Managed Objects for References to Deleted Objects 9-10

9.3.10 Reviewing DCL Command Procedures for References to Deleted Objects 9-11

9.3.11 Rights for Deleting Objects 9-13

10 MDMS Configuration

10.1 The MDMS Management Domain 10-1

10.1.1 The MDMS Database 10-2

10.1.1.1 Database Performance 10-3

10.1.1.2 Database Safety 10-3

10.1.1.3 Moving the MDMS Database 10-5

10.1.2 The MDMS Process 10-6

10.1.2.1 Server Availability 10-6

10.1.2.2 The MDMS Account 10-7

10.1.3 The MDMS Start Up File 10-7

10.1.3.1 MDMS$DATABASE_SERVERS - Identifies Domain Database Servers 10-8

10.1.3.2 MDMS$LOGFILE_LOCATION 10-9

10.1.3.3 MDMS Shut Down and Start Up 10-9

10.1.4 Managing an MDMS Node 10-9

10.1.4.1 Defining a Node's Network Connection 10-9

10.1.4.2 Defining How the Node Functions in the Domain 10-10

10.1.4.3 Enabling Interprocess Communication 10-10

10.1.4.4 Describing the Node 10-11

10.1.4.5 Communicating with Operators 10-11

10.1.5 Managing Groups of MDMS Nodes 10-11

10.1.6 Managing the MDMS Domain 10-12

10.1.6.1 Domain Configuration Parameters 10-12

10.1.6.2 Domain Options for Controlling Rights to Use MDMS 10-12

10.1.6.3 Domain Default Volume Management Parameters 10-13

10.1.7 MDMS Domain Configuration Issues 10-13

10.1.7.1 Adding a Node to an Existing Configuration 10-13

10.1.7.2 Removing a node from an existing configuration 10-14

10.2 Configuring MDMS Drives, Jukeboxes and Locations 10-14

10.2.1 Configuring MDMS Drives 10-14

10.2.1.1 How to Describe an MDMS Drive 10-15

10.2.1.2 How to Control Access to an MDMS Drive 10-15

10.2.1.3 How to Configure an MDMS Drive for Operations 10-15

10.2.1.4 Determining Drive State 10-15

10.2.1.5 Adding and Removing Managed Drives 10-16

10.2.2 Configuring MDMS Jukeboxes 10-16

10.2.2.1 How to Describe an MDMS Jukebox 10-16

10.2.2.2 How to Control Access to an MDMS Jukebox 10-16

10.2.2.3 How to Configure an MDMS Jukebox for Operations. 10-16

10.2.2.4 Attribute for DCSC Jukeboxes 10-16

10.2.2.5 Attributes for MRD Jukeboxes 10-16

10.2.2.6 Determining Jukebox State 10-17

10.2.2.7 Magazines and Jukebox Topology 10-17

10.2.3 Summary of Drive and Jukebox Issues 10-18

10.2.3.1 Enabling MDMS to Automatically Respond to Drive and Jukebox Requests 10-18

10.2.3.2 Creating a Remote Drive and Jukebox Connection 10-19

10.2.3.3 How to Add a Drive to a Managed Jukebox 10-19

10.2.3.4 Temporarily Taking a Managed Device From Service 10-19

10.2.3.5 Changing the Names of Managed Devices 10-19

10.2.4 Locations for Volume Storage 10-20

10.3 Sample MDMS Configurations 10-21

11 Connecting and Managing Remote Devices

11.1 The RDF Installation 11-1

11.2 Configuring RDF 11-1

11.3 Using RDF with MDMS 11-2

11.3.1 Starting Up and Shutting Down RDF Software 11-2

11.3.2 The RDSHOW Procedure 11-2

11.3.3 Command Overview 11-2

11.3.4 Showing Your Allocated Remote Devices 11-2

11.3.5 Showing Available Remote Devices on the Server Node 11-2

11.3.6 Showing All Remote Devices Allocated on the RDF Client Node 11-3

11.4 Monitoring and Tuning Network Performance 11-3

11.4.1 DECnet Phase IV 11-3

11.4.2 DECnet-Plus (Phase V) 11-4

11.4.3 Changing Network Parameters 11-4

11.4.4 Changing Network Parameters for DECnet (Phase IV) 11-4

11.4.5 Changing Network Parameters for DECnet-Plus(Phase V) 11-5

11.4.6 Resource Considerations 11-6

11.4.7 Controlling RDF's Effect on the Network 11-7

11.4.8 Surviving Network Failures 11-8

11.5 Controlling Access to RDF Resources 11-9

11.5.1 Allow Specific RDF Clients Access to All Remote Devices 11-9

11.5.2 Allow Specific RDF Clients Access to a Specific Remote Device 11-9

11.5.3 Deny Specific RDF Clients Access to All Remote Devices 11-9

11.5.4 Deny Specific RDF Clients Access to a Specific Remote Device 11-10

11.6 RDserver Inactivity Timer 11-10

11.7 RDF Error Messages 11-10

12 MDMS Management Operations

12.1 Managing Volumes 12-1

12.1.1 Volume Life Cycle 12-1

12.1.2 Volume States by Manual and Automatic Operations 12-2

12.1.2.1 Creating Volume Object Records 12-2

12.1.2.2 Initializing a Volume 12-3

12.1.2.3 Allocating a Volume 12-3

12.1.2.4 Holding a Volume 12-4

12.1.2.5 Freeing a Volume 12-4

12.1.2.6 Making a Volume Unavailable 12-4

12.1.3 Matching Volumes with Drives 12-4

12.1.4 Magazines for Volumes 12-4

12.1.5 Symbols for Volume Attributes 12-5

12.2 Managing Operations 12-5

12.2.1 Setting Up Operator Communication 12-6

12.2.1.1 Set OPCOM Classes by Node 12-6

12.2.1.2 Identify Operator Terminals 12-6

12.2.1.3 Enable Terminals for Communication 12-6

12.2.2 Activities Requiring Operator Support 12-6

12.3 Serving Clients of Managed Media 12-7

12.3.1 Maintaining a Supply of Volumes 12-7

12.3.1.1 Preparing Managed Volumes 12-7

12.3.2 Servicing a Stand Alone Drive 12-9

12.3.3 Servicing Jukeboxes 12-9

12.3.3.1 Inventory Operations 12-9

12.3.4 Managing Volume Pools 12-10

12.3.4.1 Volume Pool Authorization 12-11

12.3.4.2 Adding Volumes to a Volume Pool 12-11

12.3.4.3 Removing Volumes from a Volume Pool 12-11

12.3.4.4 Changing User Access to a Volume Pool 12-12

12.3.4.5 Deleting Volume Pools 12-12

12.3.5 Taking Volumes Out of Service 12-12

12.3.5.1 Temporary Volume Removal 12-12

12.3.5.2 Permanent Volume Removal 12-12

12.4 Rotating Volumes from Site to Site 12-13

12.4.1 Required Preparations for Volume Rotation 12-13

12.4.2 Sequence of Volume Rotation Events 12-13

12.5 Scheduled Activities 12-14

12.5.1 Logical Controlling Scheduled Activities 12-15

12.5.2 Job Names of Scheduled Activities 12-15

12.5.3 Log Files for Scheduled Activities 12-15

12.5.4 Notify Users When Volumes are Deallocated 12-16

13 MDMS High Level Tasks

13.1 Creating Jukeboxes, Drives, and Volumes 13-1

13.2 Deleting Jukeboxes, Drives, and Volumes 13-4

13.3 Rotating Volumes Between Sites 13-5

13.4 Servicing Jukeboxes Used for Backup Operations 13-7

A HSM Error Messages

A.1 OpenVMS Messages A-1

A.2 Shelf Handler Messages A-1

A.3 Shelf Management Utility Messages A-14

B MDMS Error Messages

C Differences Between MDMS Version 2 and MDMS Version 3

C.1 Comparing STORAGE and MDMS Commands C-1

C.2 MDMS V2 Forms Interface Options C-2

C.3 TAPESTART.COM Command Procedure C-4

D Sample Configuration of MDMS

D.1 Configuration Order D-1

D.1.1 Configuration Step 1 Example - Defining Locations D-2

D.1.2 Configuration Step 2 Example - Defining Media Type D-2

D.1.3 Configuration Step 3 Example - Defining Domain Attributes D-2

D.1.4 Configuration Step 4 Example - Defining MDMS Database Nodes D-4

D.1.5 Configuration Step 5 Example - Defining a Client Node D-5

D.1.6 Configuration Step 6 Example - Creating a Jukebox D-5

D.1.7 Configuration Step 7 Example - Defining a Drive D-6

D.1.8 Configuration Step 8 Example - Defining Pools D-7

D.1.9 Configuration Step 9 Example - Defining Volumes using the /VISION qualifier D-7

E Converting SLS/MDMS V2.X to MDMS V3

E.1 Operational Differences Between SLS/MDMS V2 & MDMS V3 E-1

E.1.1 Architecture E-1

E.1.2 MDMS Interfaces E-2

E.1.3 Rights and Privileges E-3

E.1.4 The MDMS Domain E-3

E.1.5 Drives E-4

E.1.6 Jukeboxes E-5

E.1.7 Locations E-6

E.1.8 Media Types E-6

E.1.9 Magazines E-7

E.1.10 Nodes E-7

E.1.11 Groups E-8

E.1.12 Pools E-8

E.1.13 Volumes E-8

E.1.14 Remote Devices E-10

E.2 Converting SLS/MDMS V2.X Symbols and Database E-10

E.2.1 Executing the Conversion Command Procedure E-11

E.2.2 Resolving Conflicts During the Conversion E-11

E.3 Things to Look for After the Conversion E-14

E.4 Using SLS/MDMS V2.x Clients With the MDMS V3 Database E-18

E.4.1 Limited Support for SLS/MDMS V2 during Rolling Upgrade E-18

E.4.2 Upgrading the Domain to MDMS V3 E-18

E.4.3 Reverting to SLS/MDMS V2 E-19

E.4.4 Restrictions E-20

E.5 Convert from MDMS Version 3 to a V2.X Volume Database E-20

Glossary

Preface

Introduction to HSM

This chapter provides an introduction to the general concepts of storage management in the OpenVMS? environment and defines the role of Compaq's Hierarchical Storage Management (HSM) for OpenVMS? software. Henceforth in this book, the term HSM is used as a replacement for Hierarchical Storage Management.

1.1.1 Data Categories

Typically, data exists in one of three categories:

• Active-Data that you access frequently. You want virtually immediate access to this data.
• Dormant-Data that you access less frequently and are willing to wait a short time to access.
• Inactive-Data that you do not expect to access again but must keep. Generally, this type of data is kept in an archive for legal or business purposes.

On most systems, 80 percent of the I/O requests access only 20 percent of stored data. The remaining 80 percent of your data occupies expensive media (magnetic disks), but is used infrequently.

1.1.2 Device Capacity, Cost, and Performance

There are many different devices on which your data can be stored, and the selection of which device best meets your storage needs depends on three factors:

• Performance
• Capacity
• Cost

The relationship among these three factors is illustrated in Figure 1-1. In general, high-performance devices have a lower capacity and higher cost than high-capacity devices. High-capacity devices trade performance for the ability to store large amounts of data.

Figure 1-1

 

1.1.3 Storage Management Planning

Your storage management plan should allow you to cost effectively place your data on those devices best suited to meet your cost and access requirements. This plan should include placing your active data on the most responsive devices in your system, placing your dormant data on less responsive devices, and placing your inactive data on the highest capacity devices. File activity and associated data storage are summarized in Table 1-1.

1.2.1 File Headers and Location

While a file is shelved, the file's header information is maintained in primary storage. When you display the header of a shelved file, the allocated file size is shown as zero blocks, indicating that the data contents are located in shelf storage.

The directory information and file headers for your shelved data are maintained in directories on your primary storage devices. The data itself is located in shelf storage. When access is requested for the shelved data, HSM automatically returns it to primary storage. Introduction to HSM

Information on your files always can be found in your active directories, even though the actual data resides in shelf storage.

1.2.2 Controlling File Movement

You can control shelving in the following ways:

• You can specify which files are to be moved between primary storage and shelf storage.
• You can specify which files are not to be moved from primary storage.
• You can set a high water mark on primary storage to automatically trigger shelving of your dormant data to shelf storage. A high water mark is a defined percentage of disk space used that, when exceeded, causes shelving to begin.
• You can set a low water mark as a space-recovered goal to limit the number of files that are moved to shelf storage. A low water mark is a defined percentage of disk space used that, when reached, causes policy-defined shelving to stop.

To implement shelving control, you use HSM policies. For additional information about HSM policies, see See HSM Policies

1.3.1 Shelf

An HSM shelf is a logical software object that relates the data in a set of online disk volumes, on which shelving is enabled, to a set of archive classes that contain the shelved data for those volumes.

1.3.2 Archive Class

An archive class is a logical software object that represents a single copy of shelved data. Identical copies are written to one or more archive classes when a file is shelved. For each shelf, you can specify the number of archive classes (data copies) to have to ensure reliability of the data. Because shelved data is not backed up automatically, multiple shelf copies provide the only means of recovery if the primary copy of the shelf data is lost or destroyed. Compaq recommends you have at least two archive classes for each shelf.

1.3.3 HSM Policies

An HSM policy is a defined set of parameters that controls when shelving begins and ends.
HSM implements data management through HSM policies that specify responses to events. HSM policies contain HSM-specific commands to shelve or unshelve data in response to a scheduled or situational trigger event. Trigger events, used in conjunction with appropriately designed file selection criteria, work to provide enough online disk space to satisfy users' needs. For detailed information about HSM policies, see See File Selection Criteria.

1.4.1 Starting the Shelving Process

Your control over the start of the shelving process is either explicit or implicit.

Explicit shelving is a process that starts in response to the DCL SHELVE command. You can issue the SHELVE command directly to the OpenVMS? operating system, or you can execute it in an OpenVMS? command procedure.

Implicit shelving is a process that occurs in response to one of the following triggers:

• Volume full, user disk quota exceeded, or high water mark exceeded requests initiated by OpenVMS?. These conditions trigger reactive policy.
• Scheduled policy execution initiated by HSM. This condition triggers preventive policy.

1.4.2 File Selection for Explicit Shelving

The DCL SHELVE command accepts file specifications, including wildcards, for files to process. Qualifiers to this command allow flexibility in selecting files for explicit shelving. Refer to HSM Command Reference Guide for complete information about using the SHELVE command.

1.4.3 File Selection for Implicit Shelving

File selection for implicit shelving is specified through HSM policy. Once you understand the file selection process, you can use Shelf Management Utility (SMU) commands to specify file selection criteria and achieve efficient use of primary storage.

1.4.4 Modifying File Attributes of a Shelved File

After a file has been shelved, its header remains on the disk. You still see the file in directories, and you may view and modify the file's attributes without having to access the data in shelf storage. Any modifications you make to the shelved file's header will be in effect when the file is unshelved.

1.5.1 Starting the Unshelving Process

Your control over the start of the unshelving process is either explicit or implicit.

Explicit unshelving is a process that starts in response to the DCL UNSHELVE command.
You can issue the UNSHELVE command directly to the OpenVMS? operating system, or you can execute it in an OpenVMS? command procedure. The UNSHELVE command accepts one or more file specifications, including wildcard file specifications.

Implicit unshelving is a process that HSM starts in response to a file fault. A file fault is a high-priority request that occurs when a shelved file is accessed for a read, write, extend, or truncate operation.

Table 1-3 shows the process for unshelving a file.

1.5.2 Process Default Unshelving Action

For each user process, you can specify a default unshelving action that controls implicit unshelving initiated by DCL commands and applications. By default, access to a shelved file causes a file fault.

However, you can specify instead that an error be returned on such access by issuing a SET PROCESS/NOAUTO_UNSHELVE command. This is especially useful

for commands such as wildcard searches when you do not need to unshelve files to examine them for the matching string.

1.5.3 The Results of Unshelving a File

When a file is unshelved, its data contents are moved into the location defined by its current directory entry in the(online) file header. If you renamed the file header while the file was shelved, the file will be unshelved into its new location or its new name. After a file has been unshelved from nearline/offline media, the copy remains on the nearline/offline media. Once unshelved, the file can be shelved again. If the file has been modified, a new shelf copy is made and the old copy is invalidated. If a file has not been modified since it was shelved originally, the previously shelved file copy remains valid and a new copy is not made.

1.5.4 Handling Duplicate Requests to Unshelve a File

Subsequent requests to unshelve a given file while the file is undergoing the unshelving process are treated as duplicate requests. HSM signals that both requests have completed after the first request (the one that initiated the unshelving process) completes.

Benefits of Preshelving Files

Preshelving files allows the system to respond rapidly to make space requests. Because preshelved files already are copied to shelf storage, HSM only needs to truncate files to respond to make space requests.

Figure 1-2

Using a Cache Has Significant Advantages

Using a cache greatly improves shelving performance, because the time needed to complete the operation is only as long as it takes to copy a file to another disk. The cache then can be flushed to a nearline or offline device at a later time when the shelving operation will have less impact on system performance.

Using Magneto-Optical Devices as Cache

Magneto-optical (MO) devices make an ideal repository for shelved data because they cost less than magnetic disks but still provide excellent response time. HSM supports MO devices as cache devices, rather than nearline devices, because the OpenVMS? system sees them as system-mounted, Files-11 devices. This means you can define an MO device as temporary cache or as permanent (nonflushing) cache that functions as shelf storage.

1.10.1 HSM Operations with Cache

There are four HSM operations that involve the cache:

• Shelving
• Preshelving
• Unshelving
• Flushing

1.10.2 Cache in the Shelving and Preshelving Processes

Because cache is an alternate location for temporarily storing shelved files, the shelving and preshelving processes differ only slightly when cache is enabled.

The file selection process does not function differently when cache is used. Table 1-4 describes both the shelving and preshelving processes in which cache is used.

1.10.3 Unshelving from Cache

The time taken to unshelve a file from cache is almost the same as that for copying the file from one disk to another.

1.10.4 Exceeding Cache Capacity

Files that exceed the capacity of the cache are moved directly to the nearline/offline media. You can limit the amount of storage the cache can use on each online volume you designate as a cache, or you can use the entire volume for the cache.

1.10.5 Flushing Cache

Flushing the cache is the process used to reclaim cache space. Any of the following events can start the cache flushing process:

• The used cache capacity meets or exceeds the defined high water mark for the cache volume.
• The cache disk experiences a volume full event.
• A periodic cache flush trigger event occurs.

Depending on how you defined the cache, the following events occur when the cache is flushed:

1.13.1 HSM Basic Mode Functions

HSM Basic mode provides the following functionality and features:

• Complete HSM functionality for small to medium customer environments that can use smaller capacity tape loaders (for example, DLT loaders), standalone tape devices, and magneto-optical devices
• A simple, integrated user interface provided completely by HSM
• Limited media management that is not integrated with other storage management products
• Support of up to 36 archive classes for data reliability
• An HSM naming convention for tape volume labels
• Local tape device support within the VMScluster ? environment: the shelf server nodes must have visibility to all tape devices (this can include TMSCP-served devices)

1.13.2 HSM Plus Mode Functions

HSM Plus mode provides the following functionality and features:

• Complete HSM functionality for medium to large customer environments that use large tape jukeboxes and for locations that already have the MDMS or SLS software installed
• Support for large capacity nearline devices that support multiple terabytes of data, such as the TL820 and StorageTek? silos
• Common media management with other OpenVMS? storage management products through the MDMS software
• Device and media management support provided through the MDMS command line and menu interfaces; this requires a more complex configuration process than for HSM Basic mode
• Support of up to 9999 archive classes for data reliability
• No fixed naming conventions for HSM tape volumes; the Storage Administrator controls volume names through MDMS
• Tape device support within the cluster: the shelf server nodes do not require direct visibility to all tape devices within the cluster
• Support for remote tape devices, those that are not directly connected within the cluster, through the Remote Device Facility? (RDF) portion of MDMS HSM Mode Comparison Table 1-5 identifies the functionality HSM for OpenVMS? provides and which mode provides it.

Table 1-5 HSM Basic and Plus Functionality

Function	Basic	Plus
OpenVMS? Versions	6.1, 6.2	6.1, 6.2
Supported hardware platforms	VAX, Alpha	VAX, Alpha
Common media and device management with other Compaq storage products	No	Yes, through MDMS
Support for Digital Linear Tape (DLT) magazine loaders as robotically-controlled devices	Yes	Yes
Support for TL81x, TL82x	No	Yes
Maximum number of archive classes	36	9999
Requires specific HSM volume names	Yes	No
Provides support for remote devices	No	Yes
Uses a single, integrated interface for configuration and use	Yes	No

All other functions, including HSM policies and cache, are provided in both modes.

Understanding HSM Concepts

Before running HSM in your production environment, you need to understand various definitions and concepts. For each concept, HSM provides a configuration option that you use to manage the HSM environment. This chapter presents an explanation of the HSM concepts and configuration options, structured around the following managed entities in the system:

• Facility
• Shelf
• Archive class
• Device
• Volume
• Cache
• Policy
• Schedule

This chapter also defines the relationships among the managed entities, and provides guidelines for their definition to create an optimal HSM environment. Once you understand the configuration options, you can proceed with the required configuration tasks, as described in the Getting Started with HSM Chapter of the HSM Installation Guide.

For additional information and guidelines for migrating to a more specialized environment that best meets your system requirements, see Chapter 3.

Figure 2-1

2.2.1 HSM Mode

You can specify whether HSM operates in Basic or Plus mode.

• Basic mode - Provides shelving, preshelving, and unshelving functionality using simple
  devices, Digital Linear Tape (DLT) magazine loaders, and 4mm DAT load
  ers. All interaction occurs through SMU commands.
• Plus mode - Provides shelving, preshelving, and unshelving functionality using the full
  suite of devices supported through Media and Device Management Services
  for OpenVMS(MDMS), including robotically-controlled devices like
  TL820s. Interaction requires a combination of SMU commands, STORAGE
  commands, and forms-driven menus.

2.2.2 HSM Operations

You can specify whether shelving or unshelving operations are enabled across the cluster as a whole. This includes operations initiated as a result of policy triggers, cache flush operations, and manually initiated HSM commands.

The shelving parameter controls shelving, preshelving and cache flush operations. The unshelving parameter controls unshelving and automatically-generated file faults.

Under normal circumstances, you should enable both shelving and unshelving across your cluster. This allows HSM to maintain desired disk usage through automatic policy operations and also allows users access to shelved data at all times.

2.2.3 Shelf Servers

A shelf server is a single HSM node in a cluster that performs all operations to nearline and offline devices on behalf of all nodes in the cluster. It also coordinates clusterwide operations such as checkpointing archive classes and resetting event logs.

If the facility option Catalog_Server is enabled, all cache operations and catalog updates are also performed by the shelf server. By default, cache operations are performed by the requesting client node for performance reasons. Such operations are passed from other (client) nodes to the shelf server for processing. The shelf server consolidates requests from all nodes and optimizes operations to minimize tape loading and positioning, as well as to support dedicated device access.

2.2.4 Event Logging

HSM provides four event log files that enable you to monitor and tune the HSM environment, as well as to detect errors in HSM operation:

• HSM$LOG:HSM$SHP_AUDIT.LOG: The shelf handler audit log, containing information on the parameters and final status of all requests
• HSM$LOG:HSM$PEP_AUDIT.LOG: The policy audit log, containing information on the parameters, number of files processed, and final status of all policy executions
• HSM$LOG:HSM$SHP_ERROR.LOG: The shelf handler error log, containing detailed information about any serious errors encountered during request processing, including exception information
• HSM$LOG:HSM$PEP_ERROR.LOG: The policy error log, containing detailed information about any serious errors encountered during policy execution, including exception information

Event logging can be enabled and disabled within the following categories:

• Audit log: Records all HSM requests
• Error log: Provides information on important errors
• Exception log: Provides error information that is useful to Compaq in the error logs

Compaq recommends that you enable all logging at all times to keep track of all activity. This is especially important when you have to report a problem.

The Default Shelf (HSM$DEFAULT_SHELF)

HSM provides a default shelf, named HSM$DEFAULT_SHELF, to which all volumes are associated if no other associations are defined.

If your data reliability requirements are the same across all disk volumes, you can simply use the default shelf and specify the desired number of copies to use on that shelf. All volumes acquire the data reliability specified by the default shelf.

If your data reliability requirements differ from volume to volume, you can define multiple shelves, each of which can contain different numbers of copies for data reliability purposes. You can then relate each volume to the shelf that has the appropriate number of copies.

Compaq recommends that you specify at least two copies for each volume.

If you have a very large number of online disk volumes, Compaq recommends that you define multiple shelves, each with a separate catalog. This prevents any particular catalog from becoming so large that catalog access performance degrades. Compaq recommends that you associate between 10 and 50 online disk volumes with each shelf, depending on the amount of shelving you plan to do.

The shelf entity does not define the volumes associated with the shelf. Instead, you associate individual volume entities (see See Volume) with the shelf. You can associate a particular volume with exactly one shelf. If you do not define volumes explicitly, all volumes implicitly use the default shelf.

2.3.1 Using Multiple Shelf Copies

This section explains why you need multiple shelf copies and how to define them.

One of the most important decisions that you need to make concerns the number of copies of shelved file data that you need for data safety purposes.

Shelved data is not normally backed up in the normal backup regimen because the OpenVMS BACKUP utility (and layered products like Storage Library System software that use BACKUP) work in the following way:

• An image backup saves only the headers of shelved files.
• An incremental backup does save the entire file, but the files that are selected for backup are those that have been recently modified and are not the files that usually are shelved.

In other words, after a file is shelved, it is likely that its data will not be backed up again. A typical backup strategy recycles the backup tapes when a certain number of more recent copies have been made. This cycle may be anywhere from a few days to several years.

However, there eventually will come a time when all of the backup tapes contain only the headers of shelved files.

Unless the tapes are never recycled, the shelved file data on the backup media will eventually be lost. As such, the easy way to enhance reliability of shelved file data is to make duplicate copies of the data by using multiple shelf copies.

2.3.2 Defining Shelf Copies

Shelf copies are defined using a concept called an archive class.

An archive class is a named entity that represents a single copy of shelf data. Identical copies of the data are written to each archive class when a file is shelved.

For each shelf, you can specify the archive classes to be used for shelf copies for all volumes associated with the shelf.

The minimum recommended number of copies (archive classes) for each shelf is two.

Archive classes are represented by both an archive name and an archive identifier. Archive identifiers are used in Shelf Management Utility (SMU) commands for ease of use. HSM Basic mode supports 36 archive classes named HSM$ARCHIVE01 to HSM$ARCHIVE36, with associated archive identifiers of 1 to 36 respectively. HSM Plus mode supports up to 9999 archive classes, named HSM$ARCHIVE01 through HSM$ARCHIVE9999, with associated archive identifiers of 1 to 9999.

2.3.2.3 Multiple Shelf Copies

You need to determine the appropriate number of shelf copies for your shelved file data, depending on the importance of the data being shelved.

Compaq recommends a minimum of at least two shelf copies of all data, because media can be lost or destroyed. If the data is especially critical, you can make additional copies, some of which might be taken offsite and stored in a vault. HSM provides a mechanism called checkpointing to synchronize your shelved data media and backup media so that they can be removed to an offline location together (see HSM Command Reference Guide).

See illustrates the relationship between volumes and archive classes. Each disk volume has an associated archive class and restore archive class, as shown in the archive and restore archive lists. In this example, as with most cases, the archive and restore lists are identical.

Figure 2-2

 

2.3.3 Shelving Operations

You can control the same operations for a shelf as you can for the facility, except that the operations defined for the shelf affect only the volumes associated with the shelf.

This gives you a finer level of shelving control, which might be useful if certain classes of volumes are not regularly accessed at certain times, and you want to disable shelving activity. However, as with the facility control, it is expected that shelving and unshelving operations usually are enabled.

2.3.4 Shelf Catalog

The shelf catalog contains information regarding the location of near-line and off-line data for all volumes associated with the shelf. Compaq recommends that you define a separate catalog for each shelf, but it is possible for several shelves to share a catalog, or for all shelves to use the default catalog.

Defining a separate catalog for each shelf has the following advantages:

• It restricts the impact of a temporary loss of a catalog to a known set of volumes associated with the shelf
• It reduces the size of the catalog file, allowing more flexible placement in your storage subsystem
• It increases catalog access performance, since the catalog is smaller and there are fewer records to scan
• It reduces the time for a restoration of a catalog from BACKUP tapes

As a guideline, Compaq recommends that each shelf be associated with between 10 and 50 volumes, and that each shelf has its own catalog. A shelf catalog needs to be protected with a similar level of protection as the default catalog, namely:

• The catalog should be in a shadow-set or RAID-set
• The catalog should be backed up on a regular basis.

It is also recommended that the catalog for a shelf be placed on a disk volume other than one associated with the shelf itself. In very large environments, it might be appropriate to dedicate one or more shadowed disk sets for HSM catalogs, and to disable shelving on those disks. When defining a new catalog for a shelf, or a new shelf for a volume, HSM automatically splits all associated shelving data from the old catalog, and merges it into the new catalog. See See Managing HSM Catalogs for more information on this process.

2.3.5 Save Time

You can specify a delete save option for shelved files that have been deleted. This option allows the specification of a delta time which keeps a file's shelved data in the HSM subsystem for this period after the file is deleted. The actual purging of deleted files (after the specified delay) is performed by the REPACK function.

2.3.6 Number of Updates for Retention

This option allows the specification of a number of updates to a shelved file that will be kept in the HSM subsystem.

This option applies to files that have been updated in place, not new versions of files that have been created after an update. New versions are controlled by online disk maintenance outside the scope of HSM. The actual purging of obsolete shelf data is performed by the REPACK function.

Default Device (HSM$DEFAULT_DEVICE)

HSM provides a default device record that has the following attributes:

• Device is shared
• Device is enabled for HSM use
• No archive class is associated with the device

These defaults are applied if you specify a device for HSM without identifying these attributes. Once the device is defined, you can modify the attributes for that device. You also can modify the default device record attributes if you find that you are typically using a different set of attributes for your devices.

2.6.1 Sharing and Dedicating Devices

For HSM use, you can specify a nearline or offline device to be used for dedicated or shared usage.

When a device is dedicated, HSM does not release it to other applications and keeps the current volume mounted until the drive is needed for another HSM volume.

When a device is shared, HSM releases the device, and dismounts and unloads the associated media within one minute of inactivity on the device. The media is unloaded for security reasons.

When thinking about devices, you should consider the trade-offs involved in dedicating devices to HSM.

2.6.2 Device Operations

For each device, you can specify which operations are enabled. The choices are shelving and unshelving. By default, both operations are enabled when a device is specified.

When operating in Plus mode it is recommended that all devices are defined for both shelving and unshelving as MDMS, not HSM, actually chooses the optimal device. Restricting operations sometimes leads to conflicts between HSM and MDMS.

When you are using multiple devices in Basic mode, you can optimize operations by specifying that only shelving or only unshelving is enabled on the device. This will effectively guide those operations to the enabled device rather than allowing many load/unload operations as the requests come in. For example, if you are using two devices, you might specify that one is used for shelving and the other is used for unshelving. A special override allows unshelving on a shelving device if the currently mounted media contains the requested file, which is common if the file is unshelved shortly after it is shelved.

If you specify only a single device for HSM, it must support both operations for correct usage.

2.6.3 Devices and Archive Classes

Nearline and offline devices are associated with archive classes that relate to shelves. When specifying archive classes for shelf copies, you must consider the media type on which you want these copies to reside. Each archive class uses exactly one media type, so that all data written to a specific archive class uses compatible media. Be aware that multiple archive classes can use the same media type.

You establish the relationship between archive classes, devices, and media type by using the SMU SET DEVICE command and specifying an archive list. Remember that for HSM Plus mode, you also use the media type definitions in the MDMS TAPESTART.COM file to encapsulate the media type and drives relationship. Regardless of how archive classes and shelves relate, the relationship between archive classes and devices is not one-to-one. This means that:

• A single device (for example a TA90 tape drive) can support multiple archive classes of the same media type.
• Operations on different shelves and archive classes can share devices.
• An archive class can use one or more compatible tape drives.
• Different volumes from within an archive class can be mounted simultaneously on separate compatible drives.

See shows the archive class/media type/device relationship for three archive classes and the associated TA90 and TK50 tape devices. As shown in the figure, the two TA90 devices can each archive data belonging to their common archive classes, but the TK50 device can only operate with a single archive class.

Figure 2-3

 

Ideally, an HSM configuration uses identical media types for all archive classes, allowing the maximum sharing of devices, because each device could support all archive classes. However, this is not always possible or desirable. For example, you may want to define a primary archive class that uses a robot-controlled nearline device and some secondary archive classes that use human-operated 9-track magnetic tape devices.

2.6.4 Magazine Loaders for HSM Basic Mode

HSM Basic mode supports certain tape magazine loaders as nearline devices that can be associated with archive classes. A magazine is a stacker containing one or more tape volumes that can be loaded into a single drive. The following magazine loaders are fully supported with random-access loading and unloading of tape volumes:

• TA857, TF857, TZ857
• TA867, TF867, TZ867
• TLZ6L, TLZ7L
• TZ875, TZ877
• TZ885, TZ887

HSM Basic mode supports multiple magazines, with multiple volumes per magazine. In addition, volumes for multiple archive classes may reside in a single magazine. However, there are a few restrictions that must be observed for HSM:

• All volumes placed in magazines must be initialized prior to use with the OpenVMS INITIALIZE command. Volume labels are the same as for nonmagazine loaders and must conform to the conventions shown in Table 2-1. In addition, it is vital that all tape volumes have unique labels. HSM does not support multiple volumes with the same label, which can result in the loss of access to shelved data.
• An archive class must be specified for loader operations, or nonloader operations exclusively, and must be assigned to appropriate devices. You cannot mix loader and nonloader tape operations for the same archive class at the same time. However, you can migrate an archive class from nonloader to loader (or vice versa) as long as it has the same media type.

Robot Name (Basic Mode only)

When defining a device as a magazine loader, it is necessary to specify a robot name to be associated with the device. The robot name depends on the controller to which the tape device is connected, as follows:

• A directly-connected SCSI device- such a device will have a name in the format alloc$MKxnn0:. The associated robot name is alloc$GKxnn1: or alloc$MKxnn1:. For example, for device $1$MKB100:, the associated robot name is $1$GKB101: or $1$MKB101:.
• A directly-connected DSA device, such as a TF867-in this case, the robot name is identical to the device name, but must still be specified.
• A device connected to an MSCP-controller, such as an HSC, HSJ or HSD - in this case, the robot name is the name of the controller's command disk. An example might be $1$DUA812:.

The robot name should include the allocation class if there is one.

Upgrading from HSM V1.x
If you are upgrading from HSM V1.x, please note that the robot name replaces the HSM$device_name logical defined for MSCP-controllers. The robot name must be specified for all Basic mode magazine loaders after installing this version before robotic operations will occur. This applies to devices connected to all types of controller.

2.6.5 Compatible Media for HSM Basic Mode

HSM Basic mode makes a first-level attempt to ensure that tape device configurations and loading are directed to compatible media. For this level, HSM ensures that the media type is physically capable of being loaded into the specified device, and that the media can support the operation. HSM also verifies that media contained in magazine loaders are not requested for nonloader drives and vice versa.

Table 2-2 lists the compatible media types HSM supports. HSM also supports unknown media types, but does not check them for compatibility. It is therefore possible to specify different types of tape devices with "Unknown" media type into an impractical configuration. If using such drives and media, you must ensure that the configuration is practical.

2.6.6 Automated Loaders and HSM Plus Mode

HSM Plus mode supports automated loaders according to the MDMS functionality and requirements. In general, MDMS recognizes automated loaders and the volumes contained therein only by process of how you configure the information in TAPESTART.COM and through the STORAGE commands. For more information, see the Getting Started with HSM Chapter of the HSM Installation and Configuration Guide.

Default Volume Attributes

The default volume is preconfigured with a default set of attributes. You can modify any or all of the attributes on the default volume, which are then applied to all volumes associated with the default volume. The attributes of the default volume also are used as a template for specific volume entities.

With the volume entity, you can specify the following attributes:

• Shelf
• Shelving operations
• Volume policy
• High water mark
• Files excluded from shelving

2.7.1 Shelf

The shelf attribute relates the disk volume definition to a single shelf definition. The shelf must be set up before associating a volume with it. For information on setting up the shelf, see See The Shelf By default, all volumes use the default shelf HSM$DEFAULT_ SHELF.

2.7.2 Shelving Operations

HSM provides volume definition options that allow you to control shelving operations on the online disk volume for which the volume definition applies. If no volume definition is found, HSM uses the HSM$DEFAULT_VOLUME definition.

The following operations can be enabled on a per-volume basis:

• High water mark-The ability to trigger the specified occupancy policy if disk usage exceeds the specified high water mark.
• Occupancy-The ability to trigger the specified occupancy policy if an application attempts to exceed the volume capacity.
• Exceeded quota-The ability to trigger the specified occupancy policy if an application attempts to exceed a file owner's quota.
• Shelving-Any shelving or preshelving operation, including those initiated by policy and manual operations.
• Unshelving-Any unshelving operation, including those initiated by file access and manual operations.

By default, implicit operations (high water mark, occupancy, and quota) are disabled and explicit operations (shelve and unshelve) are enabled on the volume.

2.7.3 Volume Policy

The volume policy parameters identify the policy definitions used to shelve files when a critical need for space on the disk is encountered. This policy implementation reacts to critical situations in which additional primary storage space is needed.

A reactive policy is implemented with a disk volume definition. Reactive policy determines how to react to high water mark, volume occupancy exceeded, and user disk quota exceeded events. In these instances, some event takes place that requires primary storage space be made available.

HSM takes action to make the space available only when the event takes place. A reactive policy execution can be disabled by specifying that no policy is desired for the specified event.

2.7.4 High Water Mark

You can specify a percentage of the volume's capacity that will be used as a trigger for running the occupancy policy on the volume. See See Policy for more details.

2.7.5 Files Excluded from Shelving

There are two types of files that you should give special attention to when considering their disposition in an HSM environment:

• Files marked contiguous
• Files placed at specific logical block numbers

These files have special attributes when they are created that may not be possible to recreate when the files are shelved and later unshelved.

2.8.1 Advantages and Disadvantages of Using a Cache

By using a cache, you gain speed for shelving operations by dedicating additional online storage for the HSM system. With online cache, a shelving operation can complete in the time it takes for the files to be copied to another disk.
The archive/backup system is not needed immediately. However, you lose online storage capacity otherwise dedicated to applications and users. This is the trade-off to consider when using online cache. If your system includes some older, slower online drives, then online cache provides multilevel hierarchical storage management.

All cache devices must be system-mounted and accessible to all nodes in the cluster except when the Catalog Server facility option is enabled. In this case, the cache devices need only be system-mounted and accessible to all designated shelf server nodes.

2.8.2 Cache Flushing

Another major advantage to using online cache is that flush operations to
nearline/offline storage can be performed at regular intervals. These flush operations are optimized to reduce the amount of tape reloading and positioning compared to individual shelve operations directly to tape. This is especially true when multiple archive classes are specified, and the archive classes are sharing devices.

2.8.3 Cache Attributes

You can specify the following attributes for each online disk volume supporting the cache:

• Timing of shelf copies
• Block size
• High water mark
• Flush interval
• Flush delay
• Delete and modify file action

2.8.4 Optimizing Cache Usage

The following guidelines on configuring the cache will provide optimal HSM performance for all users on the cluster:

• Set your cache size to be between 100 percent and 150 percent of the typical amount of data shelved within the flush interval. For example, if about 100,000 blocks of data are shelved daily, and your flush interval is 24 hours, then set your cache size at 150,000 blocks. You can then expect that no shelving operations to the shelf archive classes will be needed until the cache is flushed.
• Distribute the total cache size across several online volumes, with different sizes for each volume. This enables you to use `low usage' disk volumes effectively and provides an effective way to equalize the usage across all your disk volumes.
• Schedule the cache flush so that it happens when your nearline and offline devices are idle. HSM optimizes all operations during the cache flush to minimize tape loading and positioning.

By using a cache effectively, you are using HSM in the most efficient way and providing the best overall service to the system users.

2.8.5 Using Magneto-Optical Devices

Magneto-optical (MO) devices make an ideal repository for shelved file data, because their cost is significantly lower than magnetic disks but their response time is good. HSM supports magneto-optical as cache devices only; they cannot be defined like tape devices to support archive classes.

To configure a magneto-optical device, you should define a label and mount the volume as a normal Files-11 disk. The volume label should not be an HSM label in the HSxxxx format, but should be of the system administrator's choosing. If you are using a magneto-optical robot loader with multiple platters, each platter that you want HSM to use should:

• Be system-mounted as a Files-11 device with a specific `device name'
• Be defined in an SMU SET CACHE command

You can define the magneto-optical devices as either a cache staging area, or as a permanent shelf for fast response time using the /BACKUP attribute of the SET CACHE command. For more information and an example, see the SMU SET CACHE command in HSM Command Reference Guide.

2.9.1 HSM Policy Options

HSM policies are implemented through the available file selection options. These options allow you to define how HSM will implement storage management on your system. The HSM policy file selection options which may be set are:

• Trigger events
• File selection criteria
• Goal

Figure 2-4 shows the general sequence of HSM policy operations. Once a reactive or preventative policy is established, system operations continue normally until a trigger event occurs. The trigger event activates HSM policy and files are shelved in accordance with the file selection criteria until the policy goal is reached.

2.9.2 Trigger Events

The trigger is an event that causes the shelving process to begin moving files to shelf storage. These events activate HSM policies that fall into two general categories, based on the kind of trigger used:

• Preventive-Preventive policy criteria include scheduled movement of files between primary and shelf storage using such determinants as file event dates and file size.
• Reactive-Reactive policy criteria include reactions to system events, such as exceeding the amount of space available on a disk volume (volume full), exceeding a high water mark (the amount of space defined for use on a disk volume), or exceeding a user's disk quota.

When you install HSM, you get a set of default policy definitions. You can obtain the most value from HSM by modifying the default preventive and reactive policies according to the exact types and usage of data in your installation and the specific archive storage devices that are installed.

Figure 2-4

 

2.9.3 File Selection Criteria

The file selection criteria determine the best files to be shelved in response to the need for shelving. You define the file selection criteria depending on your need to create and access data.

Examples of file selection criteria include:

• Least recently used (LRU)-Files are moved to shelf storage based on the time that has elapsed since they were accessed, created, modified, or backed up.
• Space time working set (STWS)-Files are moved to shelf storage based on an algorithm that takes into account the file's size and the defined LRU criteria.
• By running a script-The file is shelved during execution of a user-defined OpenVMS command procedure.

2.9.4 Policy Goal

The goal is the condition that causes the shelving process to stop. There are two ways to reach the shelving goal:

• Shelve enough files to recover the specified percentage of the disk volume as defined by the low water mark. Recovering sufficient space to reach the low water mark is adequate to continue using the disk volume.
• Shelve all files that meet the policy but do not reach the specified percentage of recovered capacity. This condition could indicate your file selection criteria is not broad enough.

The low water mark is checked at the completion of, but not during, a script execution. The secondary policy is run if the primary policy did not reach the low water mark.

2.9.5 Make Space Requests and Policy

2.10.1 Online Volumes

When you schedule a policy execution, you specify the online volumes on which to apply the policy. When setting up a schedule, a separate entry is created for the policy execution for each volume. The volume selection should be based on the goal of maintaining volume capacity between the low water mark and the high water mark at all times. Thus, you need to schedule policies to execute more often on those volumes on which files are frequently created or modified and less often on those volumes on which files are infrequently created or modified.

2.10.2 Execution Timing and Interval

Policies can be scheduled to execute at a certain time of day, and at regular intervals. Compaq recommends you run nightly scheduled policy runs at an hour that does not conflict with high system activity or system backups. Ideally, the frequency of policy runs should coincide with the rate of new data creation on the specified volumes. The preventive policy should be run prior to the volume reaching its high water mark capacity, so that all shelving operations can be controlled to occur at certain times of day. This not only reduces overhead of reactive policy execution during the period of high system activity, but also minimizes the use of nearline/offline resources for HSM purposes.

2.10.3 Server Node

You can specify the node on which you want the policy to run. Although policies can run on any node that has access to the online volume, cache devices, and nearline/offline devices, it is more efficient if it runs on a shelf server node. If the shelf server node changes, you can use HSM's requeue feature to requeue any and all policy entries to run on an alternative shelf server node.

2.11.1 HSM$MANAGER

The files stored in the location referenced by HSM$MANAGER are important in HSM operations, but can usually be recovered. These files include:

• All SMU databases
• The shelf handler request log
• The magazine loader database, for HSM Basic mode

Loss of these files will result in a temporarily unusable HSM system, until SMU commands are entered to restore the environment. However, as long as the catalog is available, user data can be recovered. Although the critical level of files in HSM$MANAGER is not as high as HSM$CATALOG, the same protection mechanisms are recommended, if possible. At a minimum, a backup of the current SMU database should always be available. The size of the files in HSM$MANAGER is relatively fixed, but depends on the number of nodes in the cluster. You should allocate 5000 blocks plus 2049 blocks for each node in the cluster.

2.11.2 HSM$LOG

HSM uses the HSM$LOG location for storing event logs. These logs are written during HSM operation, but their content is designed for the use of the system administrator to monitor HSM activity. As such, their existence is not critical. The size of the event log files can grow rather large if not maintained. However, once the logs have been analyzed by the system administrator, they can be RESET and then deleted.

2.11.3 HSM$REPACK

HSM uses the optional HSM$REPACK logical name to point to a staging area used while repacking archive classes. If the logical name is not defined, the repack function uses HSM$MANAGER instead. Repack needs a staging area in order to repack files into multi-file savesets. The staging area must be at least 100,000 blocks for repack to function. The staging area is cleaned up after a repack operation.

Repack can be a time consuming process if the catalogs are huge. Repack can be performed in 2 phases which is facilitated by the use of the following qualifiers:

• /REPORT

If REPORT option is specified, Repack will only perform the analysis phase of a repack and not actual repacking. This feature would be extremely useful for a system manager to:

• analyze repacking requirements/benefits
• select the most useful threshold values and
• schedule repacks at convenient times.

If used with the /SAVE option, the resultant candidates file will be saved and can be used in subsequent repack/s if the system manager wants the entire repack, as analyzed, to proceed.

• /RESTART

Since repacks can take several hours/days to complete, it would be useful to allow the continuation of a repack that had been interrupted for any reason. The /RESTART qualifier would help do this along with /SAVE which would preserve the current candidates file. The repack can be started later from where it left off, without a further analysis or repacking files/volumes that had already been repacked.

Customizing the HSM Environment

This chapter provides a task-oriented description for changing the HSM environment to better suit your operating environment. It contains the following sections:

• Configuring a customized environment
• Implementing shelving policies

For a complete example of a custom configuration for HSM Basic mode or PLUS mode, see the Appendix in the HSM Installation Guide.

3.1.1 Customizing the HSM Facility

Commands submitted to the HSM facility control operations across the entire cluster.

3.1.2 Creating Shelf Definitions

Create shelf definitions that include the archive classes for shelving and unshelving data.

3.1.3 Enabling and Disabling a Shelf Definition

There are three options for enabling and disabling shelving operations that use a particular shelf. The following table lists the options that may be used with the SET SHELF /ENABLE or SET SHELF/DISABLE command.

3.1.4 Modifying Archive Classes

HSM provides multiple archive classes for you to use. You cannot modify the archive class names. You can, however, determine the devices to which an archive class is written and reassign volumes to allow you to move archive class to offsite storage.

In HSM Plus mode, you can modify the media type and density only if the archive class has not been used and no devices or shelves reference the archive class. You can add or remove volume pools as desired.

3.1.5 Creating Device Definitions

Create device definitions to identify the devices you will use for shelving operations. Also decide whether to dedicate the devices for the sole use by HSM or to share them with other applications.

3.1.6 Modifying Device Definitions

The device definitions let HSM know which devices to use for a given archive class and whether to dedicate or share the devices.

3.1.7 Enabling and Disabling a Volume Definition

The volume definition allows you to enable and disable specific reactive policy operations or control operations on the entire volume.

3.1.8 Working with Caches

HSM allows you to defines temporary caches or permanent caches. If you want to use magneto-optical devices with HSM, you must define them as a cache.

Define a magneto-optical device as a permanent cache !/SMU SET CACHE/BLOCK=0/BACKUP /NOINTERVAL/HIGHWATER_MARK=100

3.1.9 Enabling and Disabling a Policy Definition

You can enable or disable specific policy definitions.

Disabling a policy definition affects both primary and secondary policy as follows:

• Disabling a preventive policy causes the files on any disk volume scheduled for shelving with that policy to not be shelved with that policy. If other policies scheduled for the disk volume are enabled, they remain operable.
• The files on any disk volume subject to a make space request-high water mark reached, volume occupancy full, or disk quota exceeded event-are not shelved when the named policy is disabled.

3.1.10 Scheduling Policy Executions

Once you have defined and enabled preventive policies, you may want to ensure they run only at particular times or according to some specific interval.

3.2.1 Determining the Disk Volumes

Determine the disk volumes on which you want to manage storage capacity. The following example commands are used to perform this task.

3.2.2 Creating Volume Definitions

Create volume definitions for the disk volumes. Use the SMU SET VOLUME command to create a volume definition and consider the capabilities offered by the volume definitions.

3.2.3 Determining File Selection Criteria

Determine how files should be selected for shelving on a regular basis. The following list gives you some planning considerations:

• Do all disk volumes require the same or different file selection criteria?
• How many users are storing data?
• How much data is being stored by each user?
• What is the purpose of the data stored on the defined disk volumes?
• How often are files created or extended?
• What are the sizes of the files created or extended?
• How would applications or users having to unshelve files be affected?
• How many files are temporary or are expected to be deleted shortly after they are created?

3.2.4 Creating Policy Definitions

Create policy definitions that specify the file selection criteria anticipated to be most useful. Use the SMU to create a policy definition considering the capabilities offered.

3.2.5 Using Expiration Dates

If you plan on using a fileís expiration date as an event for file selection, you must make sure the OpenVMS file system is processing them. Follow the procedure in See Procedure for Setting File Expiration Dates to establish file expiration dates for the files on the disk volumes.

3.2.6 Creating Schedule Definitions

Use the SMU SET SCHEDULE command to create the schedule definitions that apply the policy definitions to the volume definitions.

3.2.7 Enabling Preventive Policy

Enable preventive policy on the system by enabling and disabling operations as follows:

Using HSM

This chapter contains information about what a user, not the storage administrator or operator, sees in an HSM environment and explains HSM functions the user can control. It includes the following topics:

• A general description of the user's HSM environment
• Control of file shelving and unshelving

4.1.1 Identifying Shelved Data using the DIRECTORY Command

As described in Chapter 1, HSM shelves file data but retains the file header information in online storage. You can use the DCL DIRECTORY command, with specific qualifiers, to determine if a file is shelved.

To find out which, if any, files have been shelved, use one of the following qualifiers on the DCL DIRECTORY command:

• /FULL
• /SHELVED_STATE
• /SIZE=ALL

4.1.2 Accessing Files

You use the same DCL commands and application programs to access shelved files as you would online data files. If you are working on a system that is running HSM, you will notice some differences in file access time. When shelving is occurring, file access time may be temporarily lengthened while the shelving process completes.

When you access a currently shelved file through a read, write, extend, or truncate operation, it may take longer for that file to be accessed than you would expect. You may see a message indicating that unshelving is occurring.

Depending on the storage device being used to shelve and unshelve the data, you may experience a large or small increase in the access time. See Typical File Access Time by Storage Deviceshows how various storage devices relate to file access time in an HSM environment.

These access times depend on the type of storage device used, rather than on the working time of HSM. In other words, if you already use various storage devices to access your data, using HSM will not significantly increase your access time.

4.1.3 Decreasing Volume Full and Disk Quota Exceeded Errors

Well-defined shelving policies will decrease the number of volume full and user disk quota exceeded conditions on your system. However, if the volume should become full or if you exceed your OpenVMS-defined disk quota, HSM may shelve files according to policies defined by the storage administrator.

4.1.4 Viewing Messages

When you access a currently shelved file through a read, write, extend, or truncate operation, you might see a message like this:

%HSM-I-UNSHLVPRG, unshelving file $1$DUA0:[MY_DIR]AARDVARKS.TXT

If you attempt to create or extend a file and there is not enough space available to do so, you might see this message:

%HSM-I-SHLVPRG, shelving files to free disk space

You see these messages only if you have enabled /BROADCAST on your terminal.

4.2.1 Automatic Shelving Operations

If the storage administrator defines policies to shelve and unshelve files, you do not need to specifically request files be shelved and unshelved. In this case, the storage administrator decides when data ought to be shelved based on various criteria discussed in Chapter 2.

You may not notice when the files are shelved and may only notice when a file is unshelved if the file access time is significantly longer than expected. You can find out if you have shelved files using the qualifiers discussed above for the DIRECTORY command.

4.2.2 User-Controlled Shelving Operations

To specifically shelve a file (or files), use the DCL SHELVE command or the DCL PRESHELVE command.

Using the SHELVE command frees up disk space by shelving files you do not expect to need soon and by minimizing the possibility that files you do intend to use are not shelved automatically.

Using the PRESHELVE command copies the file to shelf storage. The data in the file remains in your work area. Preshelving files allows the system to respond more rapidly when it needs to free up disk space for use.

4.2.3 Unshelving Files

You can cause a shelved file to be returned to primary storage through one of the following methods:

• Enter a DCL command to read, write, extend, or truncate a shelved file. This causes a file fault that initiates an implicit HSM unshelving operation.
• Use the UNSHELVE command to explicitly unshelve a file. This operation requires that you have read access to the file.

When you access the data of a shelved file through a file fault, you will receive the following message as the file is being routinely unshelved:

$ EDIT AARDVARKS.TXT
%HSM-I-UNSHLVPRG, unshelving file $1$DUA0:[MY_DIR]AARDVARKS.TXT

4.5.1 Resolving Conflicting Operations on the Same File

If two users simultaneously enter conflicting commands on the same file, the action taken by HSM is dependent upon the nature of the conflicting commands. A summary of the actions taken by HSM is given in See How HSM Resolves Conflicting Requests.

• Unshelve means either an explicit UNSHELVE or a file fault.
• Shelve means either an explicit SHELVE or a make space request.

Managing the HSM Environment

This chapter provides information on managing and maintaining your systems in an HSM environment. Storage administrators will find this information especially useful. This chapter is divided into two main parts:

5.5.1 Critical HSM Product Files

The HSM product files listed in Table 5-1 must not be deleted or shelved. During installation, these files are protected from deletion and marked /NOSHELVABLE, but care must be taken to prevent inadvertent deletion or shelving.

Compaq strongly recommends that the disks on which these files reside be shadowed and backed up on a regular basis (both image and incremental).1

5.5.2 OpenVMS System Files and System Disks

Compaq recommends that shelving be disabled on system disks. If shelving is allowed on system disks, critical files may be shelved when a policy is triggered. Serious performance degradation or a deadlock during boot operations may result when these files are accessed. You can disable shelving on system disks with the following command:

SMU> SET VOLUME/DISABLE=ALL SYS$SYSDEVICE:

If shelving is allowed on system disks, care should be taken to avoid shelving system-critical files by using SET FILE/NOSHELVABLE for each system file. The HSM installation process will perform this operation on OpenVMS system files but not on layered product files. Certain files on the system disk have the /NOSHELVABLE flag set by default. These flags should not be reset.

5.5.3 Files Not Shelved

HSM does not shelve or preshelve the following files:

• Directory files
• Open files
• Deleted and corrupted files
• Empty files
• Files whose names begin with HSM$
• Files with defined logical block placements (optional)
• Files marked by the SET FILE /NOMOVE command
• Files marked by the SET FILE /NOSHELVABLE command
• Files on volumes marked by the SMU SET VOLUME volume_name/DISABLE=SHELVE command
• Contiguous files (optional, see Section 5.7.2)
• Files that are larger than 45 percent of the total online volume capacity

5.6.1 DFS Access

File faulting (and therefore file events) work as expected, with the exception of Ctrl/Y. Typing Ctrl/Y during a file fault has no effect. The client process waits until the file fault completes and the file fault is not canceled.

In addition, with DFS one can determine the shelved state of a file just as if the disk were local (i.e. DIRECTORY /SHELVED and DIRECTORY/SELECT both work correctly).

The SHELVE and UNSHELVE commands do not work on files on DFS-served devices. The commands do work on the cluster that has local access to the devices, however.

5.6.2 NFS Access

The normal default faulting mechanism (fault on data access), does not work for NFS-served files. The behavior is as if the file is a zero-block sequential file. Performing "cat", for example, (or similar commands) results in no output.

However, at installation time, HSM Version 3.0A enables such access by defining a logical name that causes file faults on an OPEN of a file by the NFS server process. By default, the following logical name is defined:

$ DEFINE/SYSTEM HSM$FAULT_ON_OPEN "NFS$SERVER"

This definition supports access to NFS-served files upon an OPEN of a file. If you do not want NFS access to shelved files, simply de-assign the logical name as follows:

$ DEASSIGN/SYSTEM HSM$FAULT_ON_OPEN

For a permanent change, this command should be placed in:

SYS$STARTUP:HSM$LOGICALS.COM

For NFS-served files, file events (device full and user quota exceeded) occur normally with the triggering process being the NFS$SERVER process. The quota exceeded event occurs normally because any files extended by the client are charged to the client's proxy not NFS$SERVER.

If the new logical is defined for the NFS$SERVER, the fault will occur on OPEN and will appear transparent to the client, with the possible exception of messages as follows:

% cat /usr/bubble/shelve_test.txt.2
NFS2 server bubble not responding still trying
NFS2 server bubble ok

The first message appears when the open doesn't complete immediately. The second message (ok) occurs when the open completes. The file contents, in the above example, are then displayed.

Typing Ctrl/C during the file fault returns the user to the shell. Since the NFS server does not issue an IO$_CANCEL against the faulting I/O, the file fault is not canceled and the file will be unshelved eventually.

It is not possible to determine whether a given file is shelved from the NFS client. Further, like DFS devices, the SHELVE and UNSHELVE commands are not available to NFS clients.

5.6.3 PATHWORKS

Normal attempts to access a shelved file from a PATHWORKS client initiate a file fault on the server node. If the file is unshelved quickly enough (e.g. from cache), the user sees only the delay in accessing the file. If the unshelve is not quick enough, an application-defined timeout may occur and a message window pops up indicating the served disk is not responding. The timeout value depends on the application. No retry is attempted. However, this behavior can be modified by changing HSM's behavior to a file open by returning a file access conflict error, upon which most PC applications retry (or allow the user to retry) the operation after a delay. After a few retries, the file fault will succeed and the file can be accessed normally. To enable PATHWORKS access to shelved files using the retry mechanism, HSM defines the following logical name on installation:

$ DEFINE/SYSTEM HSM$FAULT_AFTER_OPEN "PCFS_SERVER, PWRK$LMSRV"

This definition supports access to PATHWORKS files upon an OPEN of a file. If you do not want PATHWORKS to access shelved files via retries, simply de-assign the logical name as follows:

$ DEASSIGN/SYSTEM HSM$FAULT_AFTER_OPEN

For a permanent change, this command should be placed in:

SYS$STARTUP:HSM$LOGICALS.COM

The decision on which access method to use depends upon the typical response time to access shelved files in your environment.

If the logical name is defined, HSM imposes a 3-second delay in responding to the OPEN request for PATHWORKS accesses only. During this time, the file may be unshelved: otherwise, a "background" unshelve is initiated which will result in a successful open after a delay and retries.

At this point, the file fault on the server node is under way and cannot be canceled.

The affect of the access on the PC environment varies according to the PC operating system. For windows 3.1 and DOS, the computer waits until the file is unshelved. For Windows NT and Windows-95, only the windows application itself waits.

File events (device full and user quota exceeded) occur normally with the triggering process being the PATHWORKS server process. The quota exceeded event occurs normally because any files extended by the client are charged to the client's proxy not the PATHWORKS server.

It is not possible from a PATHWORKS client to determine whether a file is shelved. In addition, there is no way to shelve or unshelve files explicitly (via shelve or unshelve commands). There is also no way to cancel a file fault once it has been initiated.

Most PC applications are designed to handle "file sharing" conflicts. Thus, when HSM detects the PATHWORKS server has made an access request, it can initiate unshelving action, but return "file busy". The typical PC application will continue to retry the original open, or prompt the user to retry or cancel. Once the file is unshelved, the next OPEN succeeds without shelving interaction.

5.6.4 Logical Names for NFS and PATHWORKS Access

As just discussed, HSM supports two logical names that alter the behavior of opening a shelved file for NFS and PATHWORKS access support. These are:

• HSM$FAULT_ON_OPEN - This logical name forces a file fault on an Open of a file for the processes listed in the equivalence name, and the open waits until the file fault is complete. Designed for use with the NFS server.
• HSM$FAULT_AFTER_OPEN - This logical name forces a deferred file fault on the file when it is opened. If the fault completes within three seconds, the open completes successfully, otherwise it fails with "file busy", but initiates a "background" file fault. Repeated attempts to open the file will eventually succeed. Designed for use with the PATHWORKS server.

The default behavior is to perform no file fault on Open; rather the file fault occurs upon a read or write to the file.

Each logical name can take a list of process names to alter the behavior of file faults on open. For example:

$ DEFINE/SYSTEM HSM$FAULT_ON_OPEN "NFS$SERVER, USER_SERVER, SMITH"

The HSM$FAULT_ON_OPEN can also be assigned to "HSM$ALL", which will cause a file fault on open for all processes. This option is not allowed for HSM$FAULT_AFTER_OPEN.

As these logicals are defined to allow NFS and PATHWORKS access, they are not recommended for use with other processes, since they will cause many more file faults than are actually needed in a normal OpenVMS environment. When used, the logicals must be system-wide, and should be defined identically on all nodes in the VMScluster environment.

These logical name assignments or lack thereof take effect immediately without the need to restart HSM.

5.7.1 Access Control Lists for Shelved Files

Access control lists (ACLs) for shelved files should not be deleted. In particular, the following commands should not be entered for shelved or preshelved files:

$ SET ACL /DELETE=ALL
$ SET FILE /ACL /DELETE=ALL

If the ACLs for shelved files are deleted, data is usually recovered automatically because a full catalog scan is performed. This causes a degradation of HSM performance. If the catalog scan fails, the data usually can be recovered manually using the SMU LOCATE command.

You may modify or delete ACE entries not used by HSM, for example, file protection ACEs.

5.7.2 Handling Contiguous and Placed Files

By default, HSM does not shelve files marked contiguous, files that must occupy sequential blocks of disk space. If these files are shelved, HSM will not unshelve them to noncontiguous disk space. If HSM cannot unshelve the file to contiguous space, it aborts the operation and displays an error message. When this happens, defragment the disk to restore contiguous space and retry the operation.

Placed files are files that are placed on specific blocks of disk space by an application. By default, HSM shelves these files, but does not necessarily unshelve placed files to their original location on the disk volume.

Usually, this change is not critical to the operation of an application. If a problem arises with a placed file after unshelving, the file should be set to NOSHELVABLE, or you can use the SMU SET VOLUME/NOPLACEMENT command to cause these files to not be shelved for a specified volume.

5.8.1 Backing up Critical HSM Files

As explained in Section 5.5.1, HSM requires certain files to operate. To facilitate HSM recovery in the event a disaster occurs, Compaq strongly recommends you backup these critical files using one of the methods described in this section. This is a preventive situation; if you do not use one of these methods to backup the critical files, you may not be able to easily recover shelved data after a disaster.

5.8.2 Backing Up Shelved Data

Once data is shelved, there are several mechanisms you can use to ensure there is a backup copy of that data:

• Use multiple archive classes for each shelf
• Move archive classes to offsite locations
• Use OpenVMS BACKUP to back up the shelved file headers for disaster recovery purposes (image backup only)

5.8.3 Backing Up Data Stored in an Online Cache

Because an online cache is part of online storage, it is backed up as part of your defined backup strategy. If, however, you use the online cache as a staging area to a shelf, there are some additional considerations for ensuring the information in the cache is backed up.

Using SMU LOCATE

The SMU LOCATE command retrieves full information about a file's data locations from the shelving catalog. SMU LOCATE reads the HSM catalog(s) directly to find a shelved file's data locations.

You should note that SMU LOCATE does not work quite the same way as a typical OpenVMS utility when it comes to look-up and wildcard processing. The file-descriptor you supply as input (including any wildcards) applies to the file as stored in the HSM catalog at the time of shelving. Thus, for example:

• You may locate a shelved file by name, even if it has been deleted from the online system, unless the file was shelved to a cache defined with /NOHOLD.
• If the file has been renamed on the online system, you should use the old name (current at the time of shelving) to locate it.
• You also can identify a file by file identifier (FID), which together with a device name, uniquely identify a file, regardless of any renaming that may have been done after shelving.
• If you do not specify a device and/or directory in the SMU LOCATE command, it uses a default of *:[000000...]*.*;*.

When you retrieve information using SMU LOCATE, several instances or groups of stored locations may be displayed. These reflect the locations of the file when it was shelved at various stages of its life. You should carefully review the shelving time and revision time of the file to determine which, if any, is the appropriate copy to restore.

Recovering Data from a Lost Shelved File

Although HSM tries to restore data from all known locations automatically, even when some of the file's metadata is missing, there may be occasions when this fails. In these situations, you should use SMU LOCATE to locate the file's data, then attempt to restore the data through BACKUP (from tape) or COPY (from cache).

If the user is certain file data was shelved, but is unable to simply retrieve that data through either an implicit or explicit unshelving operation, use the following procedure to find and retrieve the missing data:

5.10.1 Recovering Data Shelved Through HSM

If you have a site disaster in which your onsite data is unavailable, you may be able to recover that data from BACKUP files and tapes dismounted using the SMU CHECKPOINT command.

Once onsite, the following sequence is recommended:

5.10.2 Recovering Critical HSM Files

If you lose any of the following HSM data, you must recover it before HSM will function correctly:

• HSM database
• HSM catalogs
• HSM$UID files

Recovering Critical Files

If any or all of the critical HSM product files are deleted and you have backed up this information through a mechanism such as the OpenVMS BACKUP utility, you should restore them from the latest backup sets (including incremental backups) as soon as possible. Then, you should restart HSM.

You may lose data shelved since the last image (or incremental) backup.

5.10.3 Recovering Boot-Up Files

If you inadvertently shelved your boot-up files, you can only recover them if you have an alternate system disk you can use to boot the system and then unshelve the files.

5.10.4 Reshelving an Archive Class

The most efficient way to recover an archive class is to use the SMU REPACK command, and specify a /FROM_ARCHIVE and one or more volumes with the /VOLUME qualifier. This command uses the /FROM_ARCHIVE to retrieve shelved data and copy it to the archive class containing the lost shelf media. See Section 5.15 for more details.

An alternative but much slower way to reclaim lost shelf media is to reshelve files. Use the following command:

$ SHELVE/SELECT=NOONLINE

This variation of the SHELVE command shelves only data whose status is SHELVED, not ONLINE. It transparently unshelves the data from its current archive class and reshelves the data to any new archive classes. Data in an archive class is reshelved also if the online ACL is deleted.

5.11.1 The HSM Policy Model

This section presents a model-related concepts-that explains how shelving works. Understanding the model will help you define and manage an effective shelving policy.

By implementing HSM, you can maintain a reasonable amount of available online storage capacity, and reduce the cost of storing large amounts of data.

Your particular disk configurations and their usage dictate specific values to consider when you create the various definitions. The policies you implement with HSM determine how you meet your storage management needs.

5.11.1.1 Concepts of HSM Policy

To apply these concepts, first think of each of your online disk volumes in terms of its total online storage capacity. Then, consider how much space should always remain available.

The central element of policy is the latitude of available online storage capacity you maintain.

Figure 5-1 shows the HSM policy model. Table 5-2 provides definitions for each of the concepts shown in the figure.

Figure 5-1

 

.

Table 5-2 HSM Policy Model Concept Definitions

Concepts	Definitions
Maximum capacity	The total online storage capacity you are allowed to occupy on an online disk volume. This is a threshold for reactive shelving determined by the capacity of the online disk volume.
High water	A value you define to automatically trigger mark the shelving process. This is a threshold for reactive shelving that you determine.
Low water	The shelving goal expressed in terms of a mark percentage of disk capacity.
Capacity latitude	The capacity latitude is the range you create, monitor, and manage to make sure you are efficiently using your online storage resources. Adjusting the limits of this latitude determines the operating efficiency of your system.

5.11.1.2 Policy Governs the Shelving Process

The policies you implement by creating and modifying the various HSM definitions govern the shelving process. This example of reactive policy shows you how the HSM system reacts to a high water mark event, returning the available capacity to the low water mark.

Figure 5-2 shows the policy model in the stages of the shelving process. Table 5-3 lists the stages of the shelving process as they occur in response to reactive policy

Figure 5-2

 

.

Table 5-3 Process for Shelving to Reactive Policy

Stage	Event
1	The system is in use. Online storage capacity is within the limits that define the capacity latitude. Implications: When the amount of online storage capacity lies within the capacity latitude, it implies the following: The files on the disk are frequently accessed, meeting the demands of their applications and users for immediate access. Enough space is available to accommodate new files or extensions to files on the disk volume. Enough space is available to accommodate unshelved files if they need to be accessed. Average access latency is acceptable for the users and applications whose files are shelved from the disk volume
2	Used storage capacity exceeds the defined high water mark. This condition is caused by a user or application requiring more capacity than is allowed by definition on the online disk volume. Any of the following require more online storage: Creating a new file Extending an existing file Unshelving a file
3	Available capacity increases in response to the event. HSM automatically moves those files meeting the selection criteria in the policy definition to shelf storage. Implications: Having completed the shelving process implies the following: The likelihood that shelved files will be accessed soon is small because the use and access patterns were matched to the file selection criteria. Adequate disk space has been made available to satisfy additional requests for storage for an acceptable of time.

5.11.2 HSM Policy Situations and Resolutions

The model described in Section 5.11.1 has practical application. This section demonstrates how the model can be applied to help monitor the effectiveness of policy in various situations.

5.11.2.1 Situation : Volume Occupancy Full Event

One of the benefits of HSM is the ability to implement a preventive policy that helps avoid volume occupancy full events. Figure 5-3 shows the policy model as it applies during a volume occupancy full event.

Figure 5-3

 

Resolution

If volume occupancy full events occur while your preventive policy is in effect, you can do either or both of the following actions to avoid them:

• Decrease the high water mark
• Increase the frequency of scheduled policy

5.11.2.2 Situation : Shelving Goal Not Reached

The goal is an important part of policy as it is the result of the shelving process controlled through file selection criteria in the policy definition. Figure 5-4 shows the policy model when a shelving policy fails to reach its defined goal.

Figure 5-4

 

Resolution

If shelving operations fail to reclaim the defined capacity, you can do either or both of the following actions to make sure your shelving goal is reached:

• Change the file selection criteria to include more files
• Increase the low water mark value

5.11.2.3 Situation : Frequent Reactive Shelving Requests

Your reactive policy should be planned and implemented as a contingency. As such, shelving in response to reactive policy should occur infrequently. The policy model in Figure 5-5 shows the policy that creates frequent requests for reactive policy.

Figure 5-5

 

Resolution

If your system experiences frequent reactive shelving requests, you can take the following actions:

WHEN the Trigger Is . . .	THEN You Should . ..
High water mark reached	Increase the high water mark value, decrease the low water mark value or both.
User disk quota exceeded	Decrease the low water mark.

5.11.2.4 Situation : Application and User Performance Impeded

With HSM, you design and implement policy that allows you to maintain available online capacity and retain data on less expensive media. The trade-off with implementing HSM is that when shelved files are needed, applications and users trying to access them must wait until the files are restored. Figure 5-6 shows the policy model in a situation when available storage is maintained at the expense of application and user performance.

Figure 5-6

 

Resolution

If your applications or users experience delays in their work, or if productivity drops because files must frequently be unshelved to be accessed, you can do any or all of the following actions:

• Implement online cache
• Increase the high water mark value
• Increase the low water mark value

5.11.3 Ranking Policy Execution

HSM provides the means to determine what a policy execution would do before the policy is run. This process is called ranking a policy on a volume, and is initiated by the SMU RANK command.

This feature helps you determine the effectiveness of your policies by letting you see exactly what files would be shelved if the policy were run. The files are listed in the order that they would be shelved. Ranking applies only to policies that use the automatic algorithms STWS and LRU. HSM cannot rank policies based on user script files.

Compaq recommends that you rank all your policies before putting them into a production environment.

The following example shows how to rank a policy:

$ SMU RANK DISK$USER1: HSM$DEFAULT_OCCUPANCY
Policy HSM$DEFAULT_OCCUPANCY is enabled for shelving
Policy History:
Created: 20-OCT-1999 10:36:36.45
Revised: 20-OCT-1999 11:26:21.09
Selection Criteria:
State: Enabled
Action: Shelving
File Event: Expiration date
Elapsed time: 180 00:00:00
Before time: <none>
Since time: <none>
Low Water Mark: 80 %
Primary Policy: Space Time Working Set(STWS)
Secondary Policy: Least Recently Used(LRU)
Verification:
Mail notification: <none>
Output file: <none>
Volume capacity: 2271640 blocks
Current utilization: 1818245 blocks
Volume lowwater mark: 1817312 blocks
Blocks to be reclaimed: 933
Executing primary policy definition
DISK$USER1:[SMITH]WATCH_BATCH.COM;5
date: 21-OCT-1999 size: 462
DISK$USER1:[SMITH]LOCAL_DB.COM;1
date: 20-OCT-1999 size: 279
DISK$USER1:[SMITH]PERSONAL.LGP;1
DISK$USER1:[SMITH]REMOTE.MEM;1
date: 20-OCT-1999 size: 57
Total of 4 files ranked which will recover 951 blocks
Volume lowwater mark can be reached

5.13.1 Repack Performance

The repack operation, especially on tape volumes created under HSM V1.x, is likely to take several days to complete. While repacking is being performed, certain tape-oriented operations are suspended and queued to avoid conflicts. However, when HSM detects that a conflicting tape operation is pending, the repack operation is suspended temporarily, usually within 10 minutes, to allow the other operations to proceed. Therefore, despite the duration of the repack operation, other HSM operations will only suffer minor delays, and the long duration of repack should not be a concern.

Example of the ANALYZE Command With No /REPAIR

$ SMU ANALYZE DKB500
%SMU-I-PROCESSING, processing input device DKB500
%SMU-I- scanning for shelved files on disk volume _$1$DKB500:
File (14,1,0) "$1$DKB500:[ANALYZE_TEST]STATUS.RPT;1"
Stored in catalog as:
FID (13,1,0) "BOGUS$DEVICE1:[ANALYZE_TEST]STATUS.RPT;1"
Invalid HSM metadata found for
File (15,1,0) "$1$DKB500:[ANALYZE_TEST]LOGIN.COM;1"
Stored in catalog as:
FID (12,1,0) "$1$DKB500:[ANALYZE_TEST]LOGIN.COM;1"
Invalid HSM metadata found for
File (16,1,0) "$1$DKB500:[ANALYZE_TEST]Q4_RESULTS.TXT;1"
No catalog entry found - file not repairable
Invalid HSM metadata found for
File (17,1,0) "$1$DKB500:[ANALYZE_TEST]ANALYSIS.DAT;1"
File (18,1,0) "$1$DKB500:[ANALYZE_TEST]RECIPE.MEM;1"
Revision date mismatch -
Current: 9-JUL-1999 16:45:39.37
Catalog: 10-JUL-1999 15:54:21.74
File (19,1,0) "$1$DKB500:[ANALYZE_TEST]MAIL.SAV;1"
Stored in catalog as:
FID (19,1,0) "BOGUS$DEVICE1:[ANALYZE_TEST]MAIL.SAV;1"
%SMU- completed scan for shelved files on disk volume _
%SMU-I-ERRORS, 6 error(s) detected, 0 error(s) repaired

Example of the ANALYZE Command with Default Confirmation

$ SMU ANALYZE/REPAIR DKB500
%SMU-I-PROCESSING, processing input device DKB500
%SMU-I-scanning for shelved files on disk volume _$1$DKB500:
File (14,1,0) "$1$DKB500:[ANALYZE_TEST]STATUS.RPT;1"
Stored in catalog as:
FID (13,1,0) "BOGUS$DEVICE1:[ANALYZE_TEST]STATUS.RPT;1"
File entry repaired - 1 repairs made.
Invalid HSM metadata found for
File (15,1,0) "$1$DKB500:[ANALYZE_TEST]LOGIN.COM;1"
Stored in catalog as:
FID (12,1,0) "$1$DKB500:[ANALYZE_TEST]LOGIN.COM;1"
File entry not repaired.
Invalid HSM metadata found for
File (16,1,0) "$1$DKB500:[ANALYZE_TEST]Q4_RESULTS.TXT;1"
No catalog entry found - file not repairable
Invalid HSM metadata found for
File (17,1,0) "$1$DKB500:[ANALYZE_TEST]ANALYSIS.DAT;1"
File entry repaired - 1 repairs made.
File (18,1,0) "$1$DKB500:[ANALYZE_TEST]RECIPE.MEM;1"
Revision date mismatch -
Current: 9-JUL-1999 16:45:39.37
Catalog: 10-JUL-1999 15:54:21.74
File entry repaired - 1 repairs made.
File (19,1,0) "$1$DKB500:[ANALYZE_TEST]MAIL.SAV;1"
Stored in catalog as:
FID (19,1,0) "BOGUS$DEVICE1:[ANALYZE_TEST]MAIL.SAV;1"
File entry repaired - 1 repairs made.
%SMU- completed scan for shelved files on disk volume _
%SMU-I-ERRORS, 6 error(s) detected, 4 error(s) repaired

Example of ANALYZE/REPAIR/CONFIRM

$ SMU ANALYZE/REPAIR/CONFIRM DKB500
%SMU-I-PROCESSING, processing input device DKB500
%SMU-I- scanning for shelved files on disk volume _$1$DKB500:
File (14,1,0) "$1$DKB500:[ANALYZE_TEST]STATUS.RPT;1" Stored in catalog as:
FID (13,1,0) "BOGUS$DEVICE1:[ANALYZE_TEST]STATUS.RPT;1"
** Repair catalog entry to reset volume, FID to _ (14,1,0)? [Y]: N
File entry not repaired.
Invalid HSM metadata found for
File (15,1,0) "$1$DKB500:[ANALYZE_TEST]LOGIN.COM;1"
Stored in catalog as:
FID (12,1,0) "$1$DKB500:[ANALYZE_TEST]LOGIN.COM;1"
** Repair catalog entry to reset FID to (15,1,0) ?
** WARNING: Repair may affect the wrong file - with caution [N]: Y
File entry repaired - 1 repairs made.
Invalid HSM metadata found for
File (16,1,0) "$1$DKB500:[ANALYZE_TEST]Q4_RESULTS.TXT;1"
No catalog entry found - file not repairable
Invalid HSM metadata found for
File (17,1,0) "$1$DKB500:[ANALYZE_TEST]ANALYSIS.DAT;1"
** Repair by adding HSM metadata for file (17,1,0) ? [Y]:
File entry repaired - 1 repairs made.
File (18,1,0) "$1$DKB500:[ANALYZE_TEST]RECIPE.MEM;1"
Revision date mismatch -
Current: 9-JUL-1999 18:29:09.96
Catalog: 10-JUL-1999 17:37:52.33
** Repair by deleting HSM metadata for file (18,1,0) ? [Y]: Y
File entry repaired - 1 repairs made.
File (19,1,0) "$1$DKB500:[ANALYZE_TEST]MAIL.SAV;1"
Stored in catalog as:
FID (19,1,0) "BOGUS$DEVICE1:[ANALYZE_TEST]MAIL.SAV;1"
** Repair catalog entry to reset volume to _ ? [Y]: Y
File entry repaired - 1 repairs made.
%SMU- completed scan for shelved files on disk volume _
%SMU-I-ERRORS, 6 error(s) detected, 4 error(s) repaired

Example of ANALYZE/REPAIR/CONFIRM/OUTPUT

$ SMU ANALYZE/REPAIR/CONFIRM/OUTPUT=ANALYZE.OUT DKB500
File (14,1,0) "$1$DKB500:[ANALYZE_TEST]STATUS.RPT;1"
Stored in catalog as:
FID (13,1,0) "BOGUS$DEVICE1:[ANALYZE_TEST]STATUS.RPT;1"
** Repair catalog entry to reset volume, FID to _ (14,1,0) ? [Y]: Y
File entry repaired - 1 repairs made.
Invalid HSM metadata found for File (15,1,0) "$1$DKB500:[ANALYZE_TEST]LOGIN.COM;1"
Stored in catalog as:
FID (12,1,0) "$1$DKB500:[ANALYZE_TEST]LOGIN.COM;1"
** Repair catalog entry to reset FID to (15,1,0) ?
** WARNING: Repair may affect the wrong file - with caution [N]: Y
File entry repaired - 1 repairs made.
Invalid HSM metadata found for
File (16,1,0) "$1$DKB500:[ANALYZE_TEST]Q4_RESULTS.TXT;1"
No catalog entry found - file not repairable
Invalid HSM metadata found for
File (17,1,0) "$1$DKB500:[ANALYZE_TEST]ANALYSIS.DAT;1"
** Repair by adding HSM metadata for file (17,1,0) ? [Y]:
File entry repaired - 1 repairs made.
File (18,1,0) "$1$DKB500:[ANALYZE_TEST]RECIPE.MEM;1"
Revision date mismatch - Current:9-JUL-1999 18:38:58.06
Catalog: 10-JUL-1999 17:47:40.42
** Repair by deleting HSM metadata for file (18,1,0) ? [Y]: Y
File entry repaired - 1 repairs made.
File (19,1,0) "$1$DKB500:[ANALYZE_TEST]MAIL.SAV;1"
Stored in catalog as:
FID (19,1,0) "BOGUS$DEVICE1:[ANALYZE_TEST]MAIL.SAV;1"
** Repair catalog entry to reset volume to _ ? [Y]: Y
File entry repaired - 1 repairs made.
$
$ TYPE ANALYZE.OUT
%SMU-I-PROCESSING, processing input device DKB500
%SMU-I- scanning for shelved files on disk volume _$1$DKB500:
File (14,1,0) "$1$DKB500:[ANALYZE_TEST]STATUS.RPT;1"
Stored in catalog as:
FID (13,1,0) "BOGUS$DEVICE1:[ANALYZE_TEST]STATUS.RPT;1"
File entry repaired - 1 repairs made.
Invalid HSM metadata found for File (15,1,0)
"$1$DKB500:[ANALYZE_TEST]LOGIN.COM;1"
Stored in catalog as:
FID (12,1,0) "$1$DKB500:[ANALYZE_TEST]LOGIN.COM;1"
File entry repaired - 1 repairs made.
Invalid HSM metadata found for
File (16,1,0) "$1$DKB500:[ANALYZE_TEST]Q4_RESULTS.TXT;1"
No catalog entry found - file not repairable
Invalid HSM metadata found for
File (17,1,0) "$1$DKB500:[ANALYZE_TEST]ANALYSIS.DAT;1"
File entry repaired - 1 repairs made.
File (18,1,0) "$1$DKB500:[ANALYZE_TEST]RECIPE.MEM;1"
Revision date mismatch - Current: 9-JUL-1999 18:38:58.06
Catalog: 10-JUL-1999 17:47:40.42
File entry repaired - 1 repairs made.
File (19,1,0) "$1$DKB500:[ANALYZE_TEST]MAIL.SAV;1"
Stored in catalog as:
FID (19,1,0) "BOGUS$DEVICE1:[ANALYZE_TEST]MAIL.SAV;1"
File entry repaired - 1 repairs made.
%SMU- completed scan for shelved files on disk volume -
%SMU-I-ERRORS, 6 error(s) detected, 5 error(s) repaired

5.17.1 Setting up SLS

If you are using SLS as your regular BACKUP product, you need to configure SLS to use the new HSM$BACKUP image for your regular backups. This feature is supported only with SLS V2.8 or later.

The steps you need to take are:

• Defining a logical name to reference HSM$BACKUP
• Changing the SBK files to handle shelved and preshelved files
• Optionally adding qualifiers to manual system backups
• Optionally adding qualifiers to user backups

You set up SLS to use HSM$BACKUP by defining the following logical name:

$ DEFINE/TABLE=LNM$SLS$VALUES SLS$HSM_BACKUP 1

Depending on the type of backups or restores you are performing, you may want to include the new /[NO]SHELVED and /[NO]PRESHELVED qualifiers (as described in Section 5.17.3) in the following cases:

• In the SBK files for regular system backups
• In the Manual System Backup screen (from the SLSOPER menu) for special system backups
• In the /QUALIFIERS qualifier in the STORAGE SAVE command

This paradigm is not yet supported for Archive/Backup System (ABS).

5.17.2 Preshelving Files

The key to this paradigm is preshelving most files on the system. From HSM V2.0, preshelved files have a unique state, and are flagged as preshelved in the file header. Since the data of a preshelved file remains online, the file can be modified at any time. If a preshelved file is modified, extended, or truncated, a new HSM function changes the file from preshelved to unshelved. Also, in V2.0 and later, the eligibility for preshelving files is the same as shelved file, and the following types of files cannot be preshelved:

• Open files
• Directory files
• Files beginning with HSM$
• Files marked /NOSHELVABLE
• Files marked /NOMOVE
• Files on disks with shelving disabled
• Bad files
• Empty files

However, all other files (except those on system disks) can and should be preshelved to utilize this paradigm. This can be done in two ways:

$!
$! This sets up a preshelve policy to regularly execute on all
$! affected volumes on a regular basis:
$!
$ SMU SET POLICY policy_name /PRESHELVE /NOELAPSED /LOWWATER_MARK=0
$ SMU SET SCHEDULE volume_list policy_name/AFTER=time
$!
$! This manually preshelves all files on a volume; this command may
$! be placed in an HSM policy script file.
$! $ PRESHELVE volume:[000000...]*.*;*

HSM will not preshelve files that are already preshelved or shelved, so these commands affect only files that have been created or modified since the last preshelve operation. Since HSM does not preshelve open files, you can perform the preshelving during the day.

When starting this paradigm up for the first time, however, thousands of files per volume will be preshelved, so it is recommended that only one volume at a time is processed during this startup phase.

5.17.3 Nightly Backups

While using this paradigm, it is still necessary to perform regularly (for example, nightly) backups using your regular backup regimen. This is required to restore a disk's index file and directory structure following a disk failure.
For this paradigm to work, you must use "HSM$BACKUP" as provided with the HSM kit as your backup engine. This backup engine can be supported by SLS. The paradigm substantially reduces the backup window because only the 512-byte header for each preshelved file is backed up: the data is stored in the HSM subsystem.

The recommended paradigm for regular backups is:

• A regular (for example, weekly/monthly) image backup
• A more regular (for example, nightly) incremental backup

Two new qualifiers are provided to HSM$BACKUP to implement this paradigm:

• /NOPRESHELVED - NOPRESHELVED backs up only the headers of preshelved files; should be applied to image and incremental backup commands. /PRESHELVED (the default) backs up the data of preshelved files.
• /NOSHELVED - NOSHELVED backs up only the headers of shelved files; should be applied to incremental backups only. SHELVED causes a file fault and backs up the data of formerly shelved, now unshelved files. IMAGE backup always backs up only the headers of shelved files.

The following examples contain the recommended options for performing image and incremental backups using this paradigm:

$!
$! Image BACKUP
$!
$ HSM$BACKUP/IMAGE/IGNORE=INTERLOCK/RECORD/LOG/NOPRESHELVED -
_$ volume: device:saveset/SAVESET
$!
$! Incremental BACKUP
$!
$ HSM$BACKUP/RECORD/SINCE=BACKUP/NOPRESHELVED/NOSHELVED/LOG/ IGNORE=INTERLOCK
_$ volume: device:saveset/SAVESET
$!

Each of these commands backs up only the headers of shelved and preshelved files, and they are copied to the backup saveset in the shelved state. The online state remains unchanged.

5.17.4 Restoring Volumes

If there becomes a need to restore a disk volume because it has become damaged, the normal restoration process is follows, namely:

• Restore the last image backup
• Use BACKUP/INCREMENTAL to restore incremental BACKUP in reverse chronological order

After applying the image and incremental backups, you have restored all the metadata and directory structure for the volume, and also have restored most of the files to the shelved state (that is, all files that were preshelved and shelved during the backup are restored to the shelved state). You can use either HSM$BACKUP or normal OpenVMS Backup for the restore process.

Before making the volume available to users, it is necessary to repair the HSM catalog, since the file identifiers (FIDs) of shelved and preshelved files may have changed. You can repair them with the following command, which will take several minutes to run:

$ SMU ANALYZE/REPAIR volume:

Note that this operation completes successfully if you restore the files to the same volume (device name) or to a different device.

Once this command completes, the disk volume is ready for use. Note, however, that most files are still shelved. If you wish to avoid file faults on first file access on recently-accessed files,

you may want to initiate an unshelve procedure such as the following:

$ UNSHELVE volume:[000000...]*.*;*/SINCE=10-OCT-1999/EXPIRED

This command unshelves all files that have been accessed since 10-OCT-1999 (assuming you have enabled volume retention as recommended). The use of this command is optional.

5.17.5 Restoring Files

You restore individual files by locating the volume that has the latest (or desired) copy of the file and restoring the file in the usual way. If, however, the file is restored in the shelved state, you should run the SMU ANALYZE/REPAIR command to reset its file identifier in the catalog.

5.17.6 Repacking

Since you are using HSM as the repository of virtually all files on your system, the number of HSM media is liable to become very large. In order to keep this under control, it is recommended that you repack your archive classes regularly. Once every three-six months is recommended in such an environment. See section 7.14 for information on repacking archive classes.

5.17.7 Other Recommendations

You should not use consolidated backup with HSM on system disks. Preshelving files on system disks (and having them restored in the shelved state) will likely result in an inability to reboot your system. This is highly unrecommended.

Also, you should define multiple shelves and multiple catalogs for this environment. The catalogs should be stored on shadowed disks with preshelving disabled. You should not preshelve any HSM-internal files, otherwise unshelving may not be possible after a restore.

5.19.1 Determining File Header Limit

OpenVMS limits the number of file headers you can have on a volume by calculating a value

for MAXIMUM_FILES, using the following equation:

MAXIMUM_FILES = maxblock / (cluster_factor + 1)

Where maxblock is the value for "total blocks" from SHOW DEVICE/FULL and cluster_factor must be between:

Min value: maxblock / (255 * 4096) (or 1 whichever is greater)
Max value: maxblock / 100

Many systems use the default value for cluster_factor, which is 3 for disks whose capacity is greater than or equal to 50,000 blocks. Occasionally, you may have a problem with very large disks when the default value of three does not work and you need to calculate the minimum value using the equation. For additional information, see the INITIALIZE command in the OpenVMS DCL Dictionary.

By default, MAXIMUM_FILES is (maxblock / ((cluster_factor + 1) * 2)), which is half of the actual maximum.

5.19.2 Specifying a Volume's File Headers

To initialize a volume with the greatest number of file headers possible, use the following DCL command:

$ INITIALIZE {device} {label}/CLUSTER = (maxblock/(255*4096)) -
/MAXIMUM_FILES = (maxblock/(cluster + 1)) -
/HEADERS = (maxblock/(cluster + 1))

If you initialize a volume with the largest number of file headers, the index file will be very large, and none of that space can be used for anything but file headers. This is not necessary nor desirable, because you end up using approximately 25 percent of your disk space for file metadata. In reality, you probably want to set aside about 1 percent of your disk space for file metadata.

Note in the INITIALIZE command that /MAXIMUM_FILES reserves space for the index file while /HEADERS allocates space for the index file. Using the /HEADERs qualifier is the only way to guarantee you can create that many files. Once initialized, you cannot ever have more files on the disk than the value given with the MAXIMUM_FILES qualifier.

5.19.3 Extending the Index File

If you do not initialize your volumes using the /HEADERS qualifier, the file system will extend INDEXF.SYS for you as it needs file headers. The file system will not allow INDEXF.SYS to become multiheadered, which means you can have a maximum of approximately 77 extents in the header before you will get an error saying the index file is full.

You can tell how close you are to the index file limit using DUMP/HEADER/BLOCK=COUNT=0 [000000]INDEXF.SYS. The display contains a field called "Map area words in use." This field has a maximum of 155 for INDEXF.SYS. If the number of mapping words in use is around 120 to 130, you should schedule an image backup/restore cycle for the volume.

5.19.4 Maintaining the Number of File Headers

To prevent your system from reaching its file header limit, make sure you delete file headers as appropriate. What this means is, when you no longer need a file, do not leave it shelved with the file header on disk. Use another strategy to archive the file, just in case you need it someday. Then, delete the file from the disk.