Updated: 11 December 1998

OpenVMS Programming Concepts Manual

Use immediate I/O to read or write to another image without waiting for a response from that image. To ensure that the other process receives the information that you write, either do not exit until the other process has a channel to the mailbox, or use a permanent mailbox.

Queueing an Immmediate I/O Request

To queue an immediate I/O request, invoke the SYS$QIOW system service. See the OpenVMS System Services Reference Manual for more information.

Reading Data from the Mailbox

Since immediate I/O is asynchronous, a mailbox may contain more than one message or no message when it is read. If the mailbox contains more than one message, the read operation retrieves the messages one at a time in the order in which they were written. If the mailbox contains no message, the read operation generates an end-of-file error.

To allow a cooperating program to differentiate between an empty mailbox and the end of the data being transferred, the process writing the messages should use the IO$_WRITEOF function code to write an end-of-file message to the mailbox as the last piece of data. When the cooperating program reads an empty mailbox, the end-of-file message is returned and the second longword of the I/O status block is 0. When the cooperating program reads an end-of-file message explicitly written to the mailbox, the end-of-file message is returned and the second longword of the I/O status block contains the process identification number of the process that wrote the message to the mailbox.

In Example 2-4, the first program creates a mailbox named MAIL_BOX, writes data to it, and then indicates the end of the data by writing an end-of-file message. The second program creates a mailbox with the same logical name, reads the messages from the mailbox into an array, and stops the read operations when a read operation generates an end-of-file message and the second longword of the I/O status block is nonzero, confirming that the writing process sent the end-of-file message. The processes use common event flag 64 to ensure that SEND.FOR does not exit until RECEIVE.FOR has established a channel to the mailbox. (If RECEIVE.FOR executes first, an error occurs because SYS$ASSIGN cannot find the mailbox.)

Example 2-4 Immediate I/O Using a Mailbox

!SEND.FOR . . . INTEGER*4 STATUS ! Name and channel number for mailbox CHARACTER*(*) MBX_NAME PARAMETER (MBX_NAME = 'MAIL_BOX') INTEGER*2 MBX_CHAN ! Mailbox message CHARACTER*80 MBX_MESSAGE INTEGER LEN CHARACTER*80 MESSAGES (255) INTEGER MESSAGE_LEN (255) INTEGER MAX_MESSAGE PARAMETER (MAX_MESSAGE = 255) ! I/O function codes and status block INCLUDE '($IODEF)' INTEGER*4 WRITE_CODE STRUCTURE /STATUS_BLOCK/ INTEGER*2 IOSTAT, 2 MSG_LEN INTEGER*4 READER_PID END STRUCTURE RECORD /STATUS_BLOCK/ IOSTATUS ! System routines INTEGER SYS$CREMBX, 2 SYS$ASCEFC, 2 SYS$WAITFR, 2 SYS$QIOW ! Create the mailbox STATUS = SYS$CREMBX (, 2 MBX_CHAN, 2 ,,,, 2 MBX_NAME) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) ! Fill MESSAGES array . . . ! Write the messages DO I = 1, MAX_MESSAGE WRITE_CODE = IO$_WRITEVBLK .OR. IO$M_NOW MBX_MESSAGE = MESSAGES(I) LEN = MESSAGE_LEN(I) STATUS = SYS$QIOW (, 2 %VAL(MBX_CHAN), ! Channel 2 %VAL(WRITE_CODE), ! I/O code 2 IOSTATUS, ! Status block 2 ,, 2 %REF(MBX_MESSAGE), ! P1 2 %VAL(LEN),,,,) ! P2 IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) IF (.NOT. IOSTATUS.IOSTAT) 2 CALL LIB$SIGNAL (%VAL(IOSTATUS.STATUS)) END DO ! Write end-of-file WRITE_CODE = IO$_WRITEOF .OR. IO$M_NOW STATUS = SYS$QIOW (, 2 %VAL(MBX_CHAN), ! Channel 2 %VAL(WRITE_CODE), ! End-of-file code 2 IOSTATUS, ! Status block 2 ,,,,,,,) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) IF (.NOT. IOSTATUS.IOSTAT) 2 CALL LIB$SIGNAL (%VAL(IOSTATUS.IOSTAT)) . . . ! Make sure cooperating process can read the information ! by waiting for it to assign a channel to the mailbox STATUS = SYS$ASCEFC (%VAL(64), 2 'CLUSTER',,) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) STATUS = SYS$WAITFR (%VAL(64)) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) END

RECEIVE.FOR

INTEGER STATUS INCLUDE '($IODEF)' INCLUDE '($SSDEF)' ! Name and channel number for mailbox CHARACTER*(*) MBX_NAME PARAMETER (MBX_NAME = 'MAIL_BOX') INTEGER*2 MBX_CHAN ! QIO function code INTEGER READ_CODE ! Mailbox message CHARACTER*80 MBX_MESSAGE INTEGER*4 LEN ! Message arrays CHARACTER*80 MESSAGES (255) INTEGER*4 MESSAGE_LEN (255) ! I/O status block STRUCTURE /STATUS_BLOCK/ INTEGER*2 IOSTAT, 2 MSG_LEN INTEGER*4 READER_PID END STRUCTURE RECORD /STATUS_BLOCK/ IOSTATUS ! System routines INTEGER SYS$ASSIGN, 2 SYS$ASCEFC, 2 SYS$SETEF, 2 SYS$QIOW ! Create the mailbox and let the other process know STATUS = SYS$ASSIGN (MBX_NAME, 2 MBX_CHAN,,,) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) STATUS = SYS$ASCEFC (%VAL(64), 2 'CLUSTER',,) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) STATUS = SYS$SETEF (%VAL(64)) IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) ! Read first message READ_CODE = IO$_READVBLK .OR. IO$M_NOW LEN = 80 STATUS = SYS$QIOW (, 2 %VAL(MBX_CHAN), ! Channel 2 %VAL(READ_CODE), ! Function code 2 IOSTATUS, ! Status block 2 ,, 2 %REF(MBX_MESSAGE), ! P1 2 %VAL(LEN),,,,) ! P2 IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) IF ((.NOT. IOSTATUS.IOSTAT) .AND. 2 (IOSTATUS.IOSTAT .NE. SS$_ENDOFFILE)) THEN CALL LIB$SIGNAL (%VAL(IOSTATUS.IOSTAT)) ELSE IF (IOSTATUS.IOSTAT .NE. SS$_ENDOFFILE) THEN I = 1 MESSAGES(I) = MBX_MESSAGE MESSAGE_LEN(I) = IOSTATUS.MSG_LEN END IF ! Read messages until cooperating process writes end-of-file DO WHILE (.NOT. ((IOSTATUS.IOSTAT .EQ. SS$_ENDOFFILE) .AND. 2 (IOSTATUS.READER_PID .NE. 0))) STATUS = SYS$QIOW (, 2 %VAL(MBX_CHAN), ! Channel 2 %VAL(READ_CODE), ! Function code 2 IOSTATUS, ! Status block 2 ,, 2 %REF(MBX_MESSAGE), ! P1 2 %VAL(LEN),,,,) ! P2 IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) IF ((.NOT. IOSTATUS.IOSTAT) .AND. 2 (IOSTATUS.IOSTAT .NE. SS$_ENDOFFILE)) THEN CALL LIB$SIGNAL (%VAL(IOSTATUS.IOSTAT)) ELSE IF (IOSTATUS.IOSTAT .NE. SS$_ENDOFFILE) THEN I = I + 1 MESSAGES(I) = MBX_MESSAGE MESSAGE_LEN(I) = IOSTATUS.MSG_LEN END IF END DO . . .

2.2.1.7 Using Asynchronous Mailbox I/O

Use asynchronous I/O to queue a read or write request to a mailbox. To ensure that the other process receives the information you write, either do not exit the other process until the other process has a channel to the mailbox, or use a permanent mailbox.

To queue an asynchronous I/O request, invoke the SYS$QIO system service; however, when specifying the function codes, do not specify the IO$M_NOW modifier. The SYS$QIO system service allows you to specify an AST to be executed or an event flag to be set when the I/O operation completes.

Example 2-5 calculates gross income and taxes and then uses the results to calculate net income. INCOME.FOR uses SYS$CREPRC, specifying a termination mailbox, to create a subprocess to calculate taxes (CALC_TAXES) while INCOME calculates gross income. INCOME issues an asynchronous read to the termination mailbox, specifying an event flag to be set when the read completes. (The read completes when CALC_TAXES completes, terminating the created process and causing the system to write to the termination mailbox.) After finishing its own gross income calculations, INCOME.FOR waits for the flag that indicates CALC_TAXES has completed and then figures net income.

CALC_TAXES.FOR passes the tax information to INCOME.FOR using the installed common block created from INSTALLED.FOR.

Example 2-5 Asynchronous I/O Using a Mailbox

!INSTALLED.FOR ! Installed common block to be linked with INCOME.FOR and ! CALC_TAXES.FOR. ! Unless the shareable image created from this file is ! in SYS$SHARE, you must define a group logical name ! INSTALLED and equivalence it to the full file specification ! of the shareable image. INTEGER*4 INCOME (200), 2 TAXES (200), 2 NET (200) COMMON /CALC/ INCOME, 2 TAXES, 2 NET END

!INCOME.FOR ! Status and system routines . . . INCLUDE '($SSDEF)' INCLUDE '($IODEF)' INTEGER STATUS, 2 LIB$GET_LUN, 2 LIB$GET_EF, 2 SYS$CLREF, 2 SYS$CREMBX, 2 SYS$CREPRC, 2 SYS$GETDVIW, 2 SYS$QIO, 2 SYS$WAITFR ! Set up for SYS$GETDVI ! Define itmlst structure STRUCTURE /ITMLST/ UNION MAP INTEGER*2 BUFLEN INTEGER*2 CODE INTEGER*4 BUFADR INTEGER*4 RETLENADR END MAP MAP INTEGER*4 END_LIST END MAP END UNION END STRUCTURE ! Declare itmlst RECORD /ITMLST/ DVILIST (2) INTEGER*4 UNIT_BUF, 2 UNIT_LEN EXTERNAL DVI$_UNIT ! Name and I/O channel for mailbox CHARACTER*(*) MBX_NAME PARAMETER (MBX_NAME = 'MAIL_BOX') INTEGER*2 MBX_CHAN INTEGER*4 MBX_LUN ! Logical unit number for I/O CHARACTER*84 MBX_MESSAGE ! Mailbox message INTEGER*4 READ_CODE, 2 LENGTH ! I/O status block STRUCTURE /STATUS_BLOCK/ INTEGER*2 IOSTAT, 2 MSG_LEN INTEGER*4 READER_PID END STRUCTURE RECORD /STATUS_BLOCK/ IOSTATUS ! Declare calculation variables in installed common INTEGER*4 INCOME (200), 2 TAXES (200), 2 NET (200) COMMON /CALC/ INCOME, 2 TAXES, 2 NET ! Flag to indicate taxes calculated INTEGER*4 TAX_DONE ! Get and clear an event flag STATUS = LIB$GET_EF (TAX_DONE) IF (.NOT. STATUS) CALL LIB$SIGNAL(%VAL(STATUS)) STATUS = SYS$CLREF (%VAL(TAX_DONE)) IF (.NOT. STATUS) CALL LIB$SIGNAL(%VAL(STATUS)) ! Create the mailbox STATUS = SYS$CREMBX (, 2 MBX_CHAN, 2 ,,,, 2 MBX_NAME) IF (.NOT. STATUS) CALL LIB$SIGNAL(%VAL(STATUS)) ! Get unit number of the mailbox DVILIST(1).BUFLEN = 4 DVILIST(1).CODE = %LOC(DVI$_UNIT) DVILIST(1).BUFADR = %LOC(UNIT_BUF) DVILIST(1).RETLENADR = %LOC(UNIT_LEN) DVILIST(2).END_LIST = 0 STATUS = SYS$GETDVIW (, 2 %VAL(MBX_CHAN), ! Channel 2 MBX_NAME, ! Device 2 DVILIST, ! Item list 2 ,,,) IF (.NOT. STATUS) CALL LIB$SIGNAL(%VAL(STATUS)) ! Create subprocess to calculate taxes STATUS = SYS$CREPRC (, 2 'CALC_TAXES', ! Image 2 ,,,,, 2 'CALC_TAXES', ! Process name 2 %VAL(4), ! Priority 2 , 2 %VAL(UNIT_BUF),) IF (.NOT. STATUS) CALL LIB$SIGNAL(%VAL(STATUS)) ! Asynchronous read to termination mailbox ! sets flag when tax calculations complete READ_CODE = IO$_READVBLK LENGTH = 84 STATUS = SYS$QIO (%VAL(TAX_DONE), ! Indicates read complete 2 %VAL(MBX_CHAN), ! Channel 2 %VAL(READ_CODE), ! Function code 2 IOSTATUS,,, ! Status block 2 %REF(MBX_MESSAGE),! P1 2 %VAL(LENGTH),,,,) ! P2 IF (.NOT. STATUS) CALL LIB$SIGNAL (%VAL(STATUS)) ! Calculate incomes . . . ! Wait until taxes are calculated STATUS = SYS$WAITFR (%VAL(TAX_DONE)) IF (.NOT. STATUS) CALL LIB$SIGNAL(%VAL(STATUS)) ! Check mailbox I/O IF (.NOT. IOSTATUS.IOSTAT) 2 CALL LIB$SIGNAL (%VAL(IOSTATUS.IOSTAT)) ! Calculate net income after taxes . . . END

CALC_TAXES.FOR

! Declare calculation variables in installed common INTEGER*4 INCOME (200), 2 TAXES (200), 2 NET (200) COMMON /CALC/ INCOME, 2 TAXES, 2 NET ! Calculate taxes . . . END

2.3 Intra-Cluster Communication

Intra-cluster communication (ICC), available through ICC system services, form an application program interface (API) for process-to-process communications. For large data transfers, intra-cluster communication is the highest performance OpenVMS application communication mechanism, better than standard network transports and mailboxes.

Intra-cluster communication enables application program developers to create distributed applications with connections between different processes on a single system or between processes on different systems within a single OpenVMS Cluster system. Intra-cluster communication does not require a network product. It uses memory or System Communication Services (SCS).

The intra-cluster system services used to implement intra-cluster communication do the following:

• Allow the creation of both client and server processes
• Maintain a simple registry of servers and services
• Manage security of the server process namespace and access to server processes
• Establish connections between these processes and transmit data between these processes
• Provide 64-bit buffer and address support

Intra-cluster system services provide the following benefits when implementing ICC:

• An easy-to-use conventional system service interface for interprocess communications within a cluster
• An interface usable for communications between processes within a single, nonclustered node
• An interface callable from all modes and from execlets as well as from images
• An easy-to-use interface giving access to high-speed interconnects such as Memory Channel
• An interface independent of the installation of any networking product
• An interface usable for nonprivileged clients and authorized (but not necessarily privileged) servers

The intra-cluster communication system services are as follows:

• Open Association: SYS$ICC_OPEN_ASSOC
• Close Association: SYS$ICC_CLOSE_ASSOC
• Connect: SYS$ICC_CONNECT and SYS$ICC_CONNECTTW
• Accept: SYS$ICC_ACCEPT
• Reject: SYS$ICC_REJECT
• Disconnect: SYS$ICC_DISCONNECT and SYS$ICC_DISCONNECTW
• Transmit Data: SYS$ICC_TRANSMIT and SYS$ICC_TRANSMITW
• Receive Data: SYS$ICC_RECEIVE and SYS$ICC_RECEIVEW
• Transceive Data: SYS$ICC_TRANSCEIVE and SYS$ICC_TRANSCEIVEW
• Reply: SYS$ICC_REPLY and SYS$ICC_REPLYW

See the OpenVMS System Services Reference Manual: GETQUI--Z for additional information about the ICC system services.

2.3.1 Programming with Intra-Cluster Communications

The following sections provide information on how to program with intra-cluster communictions (ICC) using ICC system services.

2.3.1.1 ICC Concepts

The following terms and their definitions are central to creating and using intra-cluster communication.

An ASSOCIATION is a named link between an application and ICC. The association name, combined with the node name of the system on which the application is running, is also used to identify this application to other applications that want to communicate within a node or cluster.

An ASSOCIATION NAME is a string used to identify an ASSOCIATION. Association Names are 1 to 31 characters in length and must be unique within a node. Association Names are case-sensitive. Associations are created by calling the SYS$ICC_OPEN_ASSOC service (or by the first call to SYS$ICC_CONNECT if OPEN was not previously called) and are identified by an Association Handle.

An ASSOCIATION HANDLE is an opaque identifier used to represent an association to ICC. Association Handles are passed as unsigned longwords.

A NODE is either a standalone system or a member of an OpenVMS Cluster. It is identified by a one-to-six-character case-blind name that matches the SYSGEN parameter SCSNODE.

A CONNECTION is a link between two associations created by calling the SYS$ICC_CONNECT or SYS$ICC_ACCEPT service. An association may have multiple simultaneous connections at any given time. A connection is identified by a Connection Handle. A given application may choose to either only initiate connections, or initiate and accept connections as well. Connections support both synchronous and asynchronous communications.

A CONNECTION HANDLE is an opaque identifier used to represent a connection to ICC. Connection Handles are passed as unsigned longwords.

In ICC terminology, a SERVER is an application that has declared to ICC that it is willing to accept connections. A server must call SYS$ICC_OPEN_ASSOC and supply the address of a connection routine. Upon receipt of a connection request AST, a server completes its side of the connection by calling the SYS$ICC_ACCEPT service. It may also choose not to accept the connection request by calling SYS$ICC_REJECT.

A CLIENT, in the ICC model, is an application that calls SYS$ICC_CONNECT to request a connection to a server. Note that a server may also be a client. A client need not call SYS$ICC_OPEN_ASSOC (although there are certain benefits to doing so) but may instead call SYS$ICC_CONNECT using the supplied constant Default Association Handle. An association opened in this manner is called the Default Association. Only one default association is permitted per process; however, any number of connections may be established even when there is only one association.

The ICC SIMPLE CLUSTERWIDE REGISTRY provides a method for servers to register their node and association names under a single logical name, thus eliminating the need for clients to determine on which node the server process is running. Multiple servers can register under the same logical name, and ICC will randomly select one from the list. The selection algorithm leads to load sharing, but not necessarily to load balancing.

2.3.1.2 Design Considerations

This section contains information about ICC design considerations.

2.3.1.2.1 Naming

An ICC server may have two visible names---the association name and, if the ICC registry is used, a registry name. The registry name may be the same as the association name.

If, however, you wish to run multiple copies of the server on a single node, the registry name should be the more general name, while the association name should reference a particular instance of the server.

2.3.1.2.2 Message Ordering

ICC guarantees that messages will be delivered in the order in which they were transmitted. There is no provision for out-of-order delivery.

2.3.1.2.3 Flow Control

ICC implements flow control on a per-association basis. This value may be changed by specifying a value for the MAXFLOWBUFCNT parameter to the SYS$ICC_OPEN_ASSOC service.

In addition to flow control, ICC uses the value of MAXFLOWBUFCNT to determine pre-allocation of certain resources cached by the services for performance reasons. Increasing this value will result in a larger charge against process quotas. The default value of MAXFLOWBUFCNT is 5.

Because all ICC connections within the association are subject to flow control, failure to receive data in a timely manner will eventually cause all connections in the association to stall.

2.3.1.2.4 Transfer Sizes and Receiving Data

ICC supports two models for receiving data. In the simple receive model, a SYS$ICC_RECEIVE call will either complete immediately if data is present, or else the receive buffer will be queued up to be filled when data arrives from the sender. SYS$ICC_RECEIVEW will not return to the caller until either data is present, or the connection has disconnected.

The major disadvantage to this method is that the buffer you supply to SYS$ICC_RECEIVE must be large enough to receive the largest message the sender might transmit. Receipt of a message larger than the queued buffer will cause ICC to disconnect the link to preserve order. There is no provision within ICC to break a single message over multiple receive calls.

The second model is data event driven. In this method, the application provides SYS$ICC_OPEN_ASSOC (parameter recv_rtn) the address of a routine to be called whenever a connection established over this particular association is the target of a TRANSMIT. The data routine is called at AST level in the original mode of the caller and supplied with the size of the incoming transfer, the connection handle for this transfer, and a user-supplied context value. Once notified of incoming data, the application must then allocate a sufficiently large buffer and issue a SYS$ICC_RECEIVE call to obtain the data. The maximum transfer size using this method is 1 Mb.

It is not required that the SYS$ICC_RECEIVE call be made from the data routine, only that a receive request cannot be made before receipt of a data event. Therefore, receive operations are never queued within ICC when using a data event.

Since there is a single data routine per association, all connections on the association share the same data routine. In order to have some connections use both methods, at least two associations must be opened.

2.3.1.2.5 Transfer Sizes and Transceive

SYS$ICC_TRANSCEIVE is a single service that sends a message and completes when the other side responds with a call to SYS$ICC_REPLY. The maximum size of the return message is fixed by the size of the buffer the caller provides to the transceive service at the time the call is made. The maximum transmission side for both SYS$ICC_TRANSCEIVE and SYS$ICC_REPLY is 1 Mb. The minimum length of the reply transmission is zero bytes.

2.3.1.2.6 Disconnection

In a properly coded communications protocol, responsibility for disconnecting the connection should be left to the side of the protocol that last received data. This rule assures that there is no data in transit for which the state may be uncertain at the time the disconnect event occurs.

An ICC connection may be disconnected under any of the following circumstances:

• One side of the connection calls either SYS$ICC_DISCONNECT or SYS$ICC_CLOSE_ASSOC.
• The client or server process in the connection is deleted.
• The node on which one side of the connection is running is either shut down, crashes, or loses communication with the rest of the cluster.
• An unrecoverable error within the ICC services occurs, and the only possible recovery by the service is to disconnect the link.
• A pending receive buffer is too small to receive all the incoming data. In order to preserve message order, ICC will disconnect the link.

No matter what the cause of the disconnection, the application should be prepared to deal gracefully with disconnection other than that as an expected part of the application communication protocol.

2.3.1.2.7 Error Recovery

Whenever possible, ICC services attempt to return an appropriate status value to the caller, either as the routine status value or in the status fields of the IOSB or IOS_ICC structure. There are, however, situations in which the only recovery method available to ICC is to disconnect the connection. This action is taken only when the ICC services lack sufficient context to return status to the calling application.

Copyright © Compaq Computer Corporation 1998. All rights reserved. Legal

5841PRO_004.HTML