Updated: 11 December 1998

OpenVMS Programming Concepts Manual

Write an exit handler as a subroutine, because no function value can be returned. The dummy arguments of the exit subroutine should agree in number, order, and data type with the arguments you specified in the call to SYS$DCLEXH.

In the following example, assume that two or more programs are cooperating with each other. To keep track of which programs are executing, each has been assigned a common event flag (the common event flag cluster is named ALIVE). When a program begins, it sets its flag; when the program terminates, it clears its flag. Because it is important that each program clear its flag before exiting, you create an exit handler to perform the action. The exit handler accepts two arguments, the final status of the program and the number of the event flag to be cleared. In this example, since the cleanup operation is to be performed regardless of whether the program completes successfully, the final status is not examined in the exit routine. (This subroutine would not be used with the exit handler declaration in the previous example.)

CLEAR_FLAG.FOR

SUBROUTINE CLEAR_FLAG (EXIT_STATUS, 2 FLAG) ! Exit handler clears the event flag ! Declare dummy argument INTEGER EXIT_STATUS, 2 FLAG ! Declare status variable and system routine INTEGER STATUS, 2 SYS$ASCEFC, 2 SYS$CLREF ! Associate with the common event flag ! cluster and clear the flag STATUS = SYS$ASCEFC (%VAL(FLAG), 2 'ALIVE',,) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) STATUS = SYS$CLREF (%VAL(FLAG)) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL (STATUS)) END

If for any reason you must perform terminal I/O from an exit handler, use appropriate RTL routines. Trying to access the terminal from an exit handler using language I/O statements may cause a redundant I/O error.

15.15.3 Debugging an Exit Handler

To debug an exit handler, you must set a breakpoint in the handler and wait for the operating system to invoke that handler; you cannot use the DEBUG command STEP/INTO to enter an exit handler. In addition, when the debugger is invoked, it establishes an exit handler that exits using the SYS$EXIT system service. If you invoke the debugger when you invoke your image, the debugger's exit handler does not affect your program's handlers because the debugger's handler is established first and so executes last. However, if you invoke the debugger after your program begins executing (the user presses Ctrl/Y and then types DEBUG), the debugger's handler may affect the execution of your program's exit handlers, because one or more of your handlers may have been established before the debugger's handler and so is not executed.

15.15.4 Example of an Exit Handler

As in the example in Section 15.15.2, write the exit handler as a subroutine because no function value can be returned. The dummy arguments of the exit subroutine should agree in number, order, and data type with the arguments you specify in the call to SYS$DCLEXH.

In the following example, assume that two or more programs are cooperating. To keep track of which programs are executing, each has been assigned a common event flag (the common event flag cluster is named ALIVE). When a program begins, it sets its flag; when the program terminates, it clears its flag. Because each program must clear its flag before exiting, you create an exit handler to perform the action. The exit handler accepts two arguments: the final status of the program and the number of the event flag to be cleared.

In the following example, because the cleanup operation is to be performed regardless of whether the program completes successfully, the final status is not examined in the exit routine.

! Arguments for exit handler INTEGER*4 EXIT_STATUS ! Status INTEGER*4 FLAG /64/ ! Setup for exit handler STRUCTURE /EXIT_DESCRIPTOR/ INTEGER LINK, 2 ADDR, 2 ARGS /2/, 2 STATUS_ADDR, 2 FLAG_ADDR END STRUCTURE RECORD /EXIT_DESCRIPTOR/ HANDLER ! Exit handler EXTERNAL CLEAR_FLAG INTEGER*4 STATUS, 2 SYS$ASCEFC, 2 SYS$SETEF ! Associate with the common event flag ! cluster and set the flag. STATUS = SYS$ASCEFC (%VAL(FLAG), 2 'ALIVE',,) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) STATUS = SYS$SETEF (%VAL(FLAG)) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL (STATUS)) ! Do not exit until cooperating program has a chance to ! associate with the common event flag cluster. ! Enter the handler and argument addresses ! into the exit handler description. HANDLER.ADDR = %LOC(CLEAR_FLAG) HANDLER.STATUS_ADDR = %LOC(EXIT_STATUS) HANDLER.FLAG_ADDR = %LOC(FLAG) ! Establish the exit handler. CALL SYS$DCLEXH (HANDLER) ! Continue with program . . . END ! Exit Subroutine SUBROUTINE CLEAR_FLAG (EXIT_STATUS, 2 FLAG) ! Exit handler clears the event flag ! Declare dummy argument INTEGER EXIT_STATUS, 2 FLAG ! Declare status variable and system routine INTEGER STATUS, 2 SYS$ASCEFC, 2 SYS$CLREF ! Associate with the common event flag ! cluster and clear the flag STATUS = SYS$ASCEFC (%VAL(FLAG), 2 'ALIVE',,) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) STATUS = SYS$CLREF (%VAL(FLAG)) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL (STATUS))

This chapter describes the operating system's synchronization features. It focuses on how the operating system synchronizes the sequencing of events to perform memory operations. Memory access synchronization techniques are based on synchronization techniques that other types of storage use, whether to share hardware resources or data in files.

This chapter contains the following sections:

Section 16.1 describes synchronization, execution of threads, and atomicity.

Section 16.2 describes alignment, granularity, ordering of read and write operations, and performing memory read and write operations on VAX and Alpha systems in uniprocessor and multiprocessor environments.

Section 16.3 describes how to perform memory read-modify-write operations on VAX and Alpha systems in uniprocessor and multiprocessor environments.

Section 16.4 describes hardware-level synchronization methods, such as interrupt priority level, load-locked/store-conditional and interlocked instructions, memory barriers, and PALcode routines.

Section 16.5 describes software-level synchronization methods, such as process-private synchronization techniques, process priority, and spin locks. It also describes how to write applications for a multiprocessor environment using higher-level synchronization methods and how to write to global sections.

Section 16.6 describes how to use local and common event flags for synchronization.

Section 16.7 describes how to use the PPL$ routines for synchronization in a multiprocessor configuration.

Section 16.8 describes how to use SYS$SYNCH system service for synchronization.

16.1 Overview of Synchronization

Software synchronization refers to the coordination of events in such a way that only one event happens at a time. This kind of synchronization is a serialization or sequencing of events. Serialized events are assigned an order and processed one at a time in that order. While a serialized event is being processed, no other event in the series is allowed to disrupt it.

By imposing order on events, synchronization allows reading and writing of several data items indivisibly, or atomically, in order to obtain a consistent set of data. For example, all of process A's writes to shared data must happen before or after process B's writes or reads, but not during process B's writes or reads. In this case, all of process A's writes must happen indivisibly for the operation to be correct. This includes process A's updates---reading of a data item, modifying it, and writing it back (read-modify-write sequence). Other synchronization techniques are used to ensure the completion of an asynchronous system service before the caller tries to use the results of the service.

16.1.1 Threads of Execution

Code threads that can execute within a process include the following:

• Mainline code in an image being executed by a kernel thread, or multiple threads
• Asynchronous system traps (ASTs) that interrupt the image
• Condition handlers established by the image and that run after exceptions occur
• Inner access-mode threads of execution that run as a result of system service, OpenVMS Record Management Services (RMS), and command language interpreter (CLI) callback requests

Process-based threads of execution can share any data in the P0 and P1 address space and must synchronize access to any data they share. A thread of execution can incur an exception, which results in passing of control to a condition handler. Alternatively, the thread can receive an AST, which results in passing of control to an AST procedure. Further, an AST procedure can incur an exception, and a condition handler's execution can be interrupted by an AST delivery. If a thread of execution requests a system service or RMS service, control passes to an inner access-mode thread of execution. Code that executes in the inner mode can also incur exceptions, receive ASTs, and request services.

Multiple processes, each with its own set of threads of execution, can execute concurrently. Although each process has private P0 and P1 address space, processes can share data in a global section mapped into each process's address spaces. You need to synchronize access to global section data because a thread of execution accessing the data in one process can be rescheduled, allowing a thread of execution in another process to access the same data before the first process completes its work. Although processes access the same system address space, the protection on system space pages usually prevents outer mode access. However, process-based code threads running in inner access modes can access data concurrently in system space and must synchronize access to it.

Interrupt service routines access only system space. They must synchronize access to shared system space data among themselves and with process-based threads of execution.

A CPU-based thread of execution and an I/O processor must synchronize access to shared data structures, such as structures that contain descriptions of I/O operations to be performed.

Multiprocessor execution increases synchronization requirements when the threads that must synchronize can run concurrently on different processors. Because a process executes on only one processor at a time, synchronization of threads of execution within a process is unaffected by whether the process runs on a uniprocessor or on a symmetric multiprocessing (SMP) system. However, multiple processes can execute simultaneously on different processors. Because of this, processes sharing data in a global section can require additional synchronization for SMP system execution. Further, process-based inner mode and interrupt-based threads can execute simultaneously on different processors and can require synchronization of access to system space beyond what is sufficient on a uniprocessor.

16.1.2 Atomicity

Atomicity is a type of serialization that refers to the indivisibility of a small number of actions, such as those occurring during the execution of a single instruction or a small number of instructions. With more than one action, no single action can occur by itself. If one action occurs, then all the actions occur. Atomicity must be qualified by the viewpoint from which the actions appear indivisible: an operation that is atomic for threads running on the same processor can appear as multiple actions to a thread of execution running on a different processor.

An atomic memory reference results in one indivisible read or write of a data item in memory. No other access to any part of that data can occur during the course of the atomic reference. Atomic memory references are important for synchronizing access to a data item that is shared by multiple writers or by one writer and multiple readers. References need not be atomic to a data item that is not shared or to one that is shared but is only read.

16.2 Memory Read and Memory Write Operations

This section presents the important concepts of alignment and granularity and how they affect the access of shared data on VAX and Alpha systems. It also discusses the importance of the order of reads and writes are completed on VAX and Alpha systems, and how VAX and Alpha system perform memory reads and writes.

16.2.1 Alignment

The term alignment refers to the placement of a data item in memory. For a data item to be naturally aligned, its lowest-addressed byte must reside at an address that is a multiple of the size of the data item in bytes. For example, a naturally aligned longword has an address that is a multiple of 4. The term naturally aligned is usually shortened to "aligned."

On VAX systems, a thread on a VAX uniprocessor or multiprocessor can read and write aligned byte, word, and longword data atomically with respect to other threads of execution accessing the same data.

On Alpha systems, in contrast to the variety of memory accesses allowed on VAX systems, an Alpha processor allows atomic access only to an aligned longword or an aligned quadword. Reading or writing an aligned longword or quadword of memory is atomic with respect to any other thread of execution on the same processor or on other processors.

16.2.2 Granularity

VAX and Alpha systems differ in granularity of data access. The phrase granularity of data access refers to the size of neighboring units of memory that can be written independently and atomically by multiple processors. Regardless of the order in which the two units are written, the results must be identical.

VAX systems have byte granularity: individual adjacent or neighboring bytes within the same longword can be written by multiple threads of execution on one or more processors, as can aligned words and longwords.

VAX systems provide instructions that can manipulate byte-sized and aligned word-sized memory data in a single, noninterruptible operation. On VAX systems, a byte-sized or word-sized data item that is shared can be manipulated individually.

Alpha systems have longword and quadword granularity. That is, only adjacent aligned longwords or quadwords can be written independently. Because Alpha systems support only instructions that load or store longword-sized and quadword-sized memory data, the manipulation of byte-sized and word-sized data on Alpha systems requires that the entire longword or quadword that contain the byte- or word-sized item to be manipulated. Thus, simply because of its proximity to an explicitly shared data item, neighboring data might become shared unintentionally. Manipulation of byte-sized and word-sized data on Alpha systems requires multiple instructions that:

1. Fetch the longword or quadword that contains the byte or word
2. Mask the nontargeted bytes
3. Manipulate the target byte or word
4. Store the entire longword or quadword

On Alpha systems, because this sequence is interruptible, operations on byte and word data are not atomic on Alpha sytems. Also, this change in the granularity of memory access can affect the determination of which data is actually shared when a byte or word is accessed.

On Alpha systems, the absence of byte and word granularity on Alpha systems has important implications for access to shared data. In effect, any memory write of a data item other than an aligned longword or quadword must be done as a multiple-instruction read-modify-write sequence. Also, because the amount of data read and written is an entire longword or quadword, programmers must ensure that all accesses to fields within the longword or quadword are synchronized with each other.

16.2.3 Ordering of Read and Write Operations

On VAX uniprocessor and multiprocessor systems, write operations and read operations appear to occur in the same order in which you specify them from the viewpoint of all types of external threads of execution. Alpha uniprocessor systems also guarantee that read and write operations appear ordered for multiple threads of execution running within a single process or within multiple processes running on a uniprocessor.

On Alpha multiprocessor systems, you must order reads and writes explicitly to ensure that they occur in a specific order from the viewpoint of threads of execution on other processors. To provide the necessary operating system primitives and compatibility with VAX systems, Alpha systems support instructions that impose an order on read and write operations.

16.2.4 Memory Reads and Memory Writes

On VAX systems, most instructions that read or write memory are noninterruptible. A memory write done with a noninterruptible instruction is atomic from the viewpoint of other threads on the same CPU.

On VAX systems, on a uniprocessor system, reads and writes of bytes, words, longwords, and quadwords are atomic with respect to any thread on the processor. On a multiprocessor, not all of those accesses are atomic with respect to any thread on any processor; only reads and writes of bytes, aligned words, and aligned longwords are atomic. Accessing unaligned data can require multiple operations. As a result, even though an unaligned longword is written with a noninterruptible instruction, if it requires multiple memory accesses, a thread on another CPU might see memory in an intermediate state. VAX systems do not guarantee multiprocessor atomic access to quadwords.

On Alpha systems, there is no instruction that performs multiple memory accesses. Each load or store instruction performs a maximum of one load from or one store to memory. A load can occur only from an aligned longword or quadword. A store can occur only to an aligned longword or quadword.

On Alpha systems, although reads and writes from one thread appear to occur ordered from the viewpoint of other threads on the same processor, there is no implicit ordering of reads and writes as seen by threads on other processors.

16.3 Memory Read-Modify-Write Operations

A fundamental synchronization primitive for accessing shared data is an atomic read-modify-write operation. This operation consists of reading the contents of a memory location and replacing them with new contents based on the old. Any intermediate memory state is not visible to other threads. Both VAX systems and Alpha systems provide this synchronization primitive, but they implement it in significantly different ways.

16.3.1 Uniprocessor Operations

On VAX systems, many instructions are capable of performing a read-modify-write operation in a single, noninterruptible (atomic) sequence from the viewpoint of multiple application threads executing on a single processor. VAX systems provide synchronization among multiple threads of execution running on a uniprocessor system.

On VAX systems, the implicit dependence on the atomicity of VAX instructions is not recommended. Because of the optimizations they perform, the VAX compilers do not guarantee that a certain type of program statement, such as an increment operation (x=x+1), is implemented using a VAX atomic instruction, even if one exists.

On Alpha systems, there is no single instruction that performs an atomic read-modify-write operation. As a result, even uniprocessing applications in which processes access shared data must provide explicit synchronization of these accesses, usually through compiler semantics.

On Alpha systems, read-modify-write operations that can be performed atomically on VAX systems require a sequence of instructions on Alpha systems. Because this sequence can be interrupted, the data may be left in an unstable state. For example, the VAX increment long (INCL) instruction fetches the contents of a specified longword, increments its value, and stores the value back in the longword, performing the operations without interruption. On Alpha systems, each step---fetching, incrementing, storing---must be explicitly performed by a separate instruction. Therefore, another thread in the process (for example, an AST routine) could execute before the sequence completes. However, because atomic updates are the basis of synchronization, and to provide compatibility with VAX systems, Alpha systems provide the following mechanisms to enable atomic read-modify-write updates:

• Privileged architecture library (PALcode) routines perform queue insertions and removals.
• Load-locked and store-conditional instructions create a sequence of instructions that implement an atomic update.

On multiprocessor systems, you must use special methods to ensure that a read-modify-write sequence is atomic. On VAX systems, interlocked instructions provide synchronization; on Alpha systems, load-locked and store-conditional instructions provide synchronization.

On VAX systems, a number of uninterruptible instructions are provided that both read and write memory with one instruction. When used with an operand type that is accessible in a single memory operation, each instruction provides an atomic read-modify-write sequence. The sequence is atomic with respect to threads of execution on the same VAX processor, but it is not atomic to threads on other processors. For instance, when a VAX CPU executes the instruction INCL x, it issues two separate commands to memory: a read, followed by a write of the incremented value. Another thread of execution running concurrently on another processor could issue a command to memory that reads or writes location x between the INCL's read and write. Section 16.4.3 describes read-modify-write sequences that are atomic with respect to threads on all VAX CPUs in an SMP system.

On a VAX multiprocessor system, an atomic update requires an interlock at the level of the memory subsystem. To perform that interlock, the VAX architecture provides a set of interlocked instructions that include Add Aligned Word Interlocked (ADAWI), Remove from Queue Head Interlocked (REMQHI), and Branch on Bit Set and Set Interlocked (BBSSI).

If you code in VAX MACRO, you use the assembler to generate whatever instructions you tell it. If you code in a high-level language, you cannot assume that the compiler will compile a particular language statement into a specific code sequence. That is, you must tell the compiler explicitly to generate an atomic update. For further information, see the documentation for your high-level language.

On Alpha systems, there is no single instruction that performs an atomic read-modify-write operation. An atomic read-modify-write operation is only possible through a sequence that includes load-locked and store-conditional instructions, (see Section 16.4.2). Use of these instructions provides a read-modify-write operation on data within one aligned longword or quadword that is atomic with respect to threads on all Alpha CPUs in an SMP system.

Copyright © Compaq Computer Corporation 1998. All rights reserved. Legal

5841PRO_046.HTML