Updated: 11 December 1998

OpenVMS RTL Library (LIB$) Manual

If the transition calls a subexpression to determine a match, the symbol type syntax includes the state label of the subexpression to be called. It is indicated with the MACRO expression !label. See Section 3.5 for information about subexpressions.

label

LIB$T[ABLE_]PARSE defines two expressions you can also specify as the target state in the label argument:

• TPA$_EXIT --- The parsing operation in progress terminates with a success status.
• TPA$_FAIL --- The parsing operation stops with a failure status, as if a syntax error had occurred.

action

Because the action routine address is self-relative, it cannot be in a shared image separate from the state table.

mask

When LIB$T[ABLE_]PARSE executes the transition, it performs an inclusive OR operation using the mask value and the contents of msk-adr and stores the result in msk-adr.

You can associate one or more bits in mask with a particular transition and set those bits. When LIB$T[ABLE_]PARSE returns, you can check the bits in msk-adr to determine which transitions were executed. You can also use an action routine to check the bit and ensure that a transition is executed only once.

If the mask argument is present, the msk-adr argument must also be present.

msk-adr

• If mask is present, msk-adr is the address of a longword associated with the preceding mask argument. LIB$T[ABLE_]PARSE performs the inclusive OR operation on the contents of this address and the mask argument and stores the result in msk-adr.
  Initialize the contents of msk-adr to zero before calling LIB$T[ABLE_]PARSE.
• If mask is not present, you can use msk-adr to specify the address of a location where LIB$T[ABLE_]PARSE stores information about the matching token. No OR operation is performed. This capability lets a program extract the most commonly needed information from the input string without using action routines.
  The kind of information that LIB$T[ABLE_]PARSE stores in the location you specify as the msk-adr argument depends on the symbol type specified for the type argument and on the argument block, as follows:
  • If the symbol type is TPA$_DECIMAL, TPA$_OCTAL, or TPA$_HEX, LIB$T[ABLE_]PARSE stores the binary representation of the matching number as an unsigned longword for a 32-bit argument block and as an unsigned quadword for a 64-bit argument block.
  • If the symbol type is TPA$_DECIMAL_64, TPA$_OCTAL_64, or TPA$_HEX_64, LIB$T[ABLE_]PARSE stores the binary representation of the matching number as an unsigned quadword for both 32-bit and 64-bit argument blocks.
  • If the symbol type is 'x', TPA$_ANY, TPA$_ALPHA, or TPA$_DIGIT, LIB$T[ABLE_]PARSE stores the 8-bit matching character as an unsigned byte.
  • If the symbol is of any other type, you must specify msk-adr as the address of a 32-bit or 64-bit string descriptor, as appropriate, that you allocate in your program. LIB$T[ABLE_]PARSE assumes a 32-bit or 64-bit descriptor if the argument block with which you called it is 32-bit or 64-bit, respectively.
    For a 32-bit descriptor, LIB$T[ABLE_]PARSE stores the length of the token in the first 32 bits (longword) of the descriptor. It stores a pointer to the token in the second longword. This pointer is the address of the token in the input string.
    For a 64-bit descriptor, LIB$T[ABLE_]PARSE stores the length of the token in the second quadword of the descriptor and stores the address of the token in the input string in the third quadword. On entry, LIB$T[ABLE_]PARSE writes the fields of the first quadword as follows:
    • DSC64$B_CLASS = DSC64$K_CLASS_S
    • DSC64$B_DTYPE = DSC64$K_DTYPE_T
    • DSC64$L_MBMO = --1
    • DSC64$W_MBO = +1

Using msk-adr makes your parsing program nonmodular. The resulting program, which contains this state table, includes code that is not position independent.

Because the address specified by msk-adr is self-relative, it cannot be in a shared image separate from the state table.

argument

Because LIB$T[ABLE_]PARSE does not know the form or meaning of argument the value is stored in its absolute form. If you use argument to pass an address, you must store the address in its absolute form rather than as a self-relative pointer. In this case the resulting program, which contains this state table, is nonmodular. That is, it includes code that is not position independent.

1.3.1.4 $END_STATE---Ends the State Table
The $END_STATE macro declares the end of the state table. It is mandatory, in order to permit the orderly cleanup of the LIB$T[ABLE_]PARSE macro system. The $END_STATE macro has no arguments. You code it as follows:

$END_STATE

1.3.2 BLISS State Table Generation Macro Calls
The SYS$LIBRARY:TPAMAC.L32 and SYS$LIBRARY:TPAMAC.L64 files each contain a set of BLISS macros that allow convenient and readable coding of LIB$T[ABLE_]PARSE state tables in BLISS.

Use one of the following BLISS state table generation macros:

• $INIT_STATE---Initializes the macros (see Section 1.3.2.1 )
• $STATE---Defines a state and its transitions (see Section 1.3.2.2 )

To make the macros available to the program, include the following declaration in the module containing the state tables:

LIBRARY 'SYS$LIBRARY:TPAMAC';

The BLISS compiler you use, BLISS-32 or BLISS-64, chooses the corresponding SYS$LIBRARY:TPAMAC file.

The BLISS table generation macros contain no BEGIN or END statements. This allows $STATE macros to refer to each other. They generate all storage with OWN declarations. This means that the macros modify PSECT declarations for OWN and GLOBAL storage. Thus if other data declarations follow the state table declarations, they may not have the correct attributes. You cannot simply surround the state table with BEGIN/END, because this constitutes an expression. No declarations of any kind, including ROUTINE declarations, can follow an expression.

Use one of the following techniques to include LIB$T[ABLE_]PARSE a state table in a BLISS module:

• Follow the state table with explicit redeclarations of the OWN and GLOBAL PSECTs. Example 3 illustrates this technique.
• Place the state table in a separate module. The high-level language examples in the next section use this technique.
• Place the state table between BEGIN and END statements after the declarations within a routine body.
• Place the state table between BEGIN and END statements at the end of a module.

In all cases you must define all action routines, masks, addresses, and arguments with suitable declarations (which can be FORWARD or EXTERNAL). The LIB$T[ABLE_]PARSE macros handle the necessary FORWARD declarations for forward references to labels within the state table.

1.3.2.1 $INIT_STATE---Initializes the LIB$T[ABLE_]PARSE Macros
The $INIT_STATE macro initializes the LIB$T[ABLE_]PARSE macro system in the same manner it does for MACRO.

$INIT_STATE (state-table, key-table);

state-table

The name assigned to the state table. LIB$T[ABLE_]PARSE equates this label to the start of the first state in the state table.

key-table

The name assigned to the keyword table. LIB$T[ABLE_]PARSE equates this label to the start of the keyword table.

Both names are declared as global vectors of length zero. As with the MACRO state table generation macros, you can invoke $INIT_STATE more than once to declare several state tables within a single module.

1.3.2.2 $STATE---Declares a State and Its Transitions
In BLISS, you use the $STATE macro to declare a state in its entirety, including its transitions.

$STATE ([label], ( transition ), ( transition ), ( transition ) . . . );

label

Optional address of the start of the state. The compiler declares label as a local vector of length zero. Note that the comma following the optional label is mandatory.

transition

Each transition appears within parentheses in the same form as the transition argument list for the MACRO $TRAN macro.

type [,label] [,action] [,mask] [,msk-adr] [,argument]

The arguments of each transition are expressed in exactly the same format as in the MACRO macros, with the exception of the subexpression symbol type. In BLISS, this symbol type has the form (label).

Note that the transitions are not specified as keyword macros. Therefore, you must use commas to indicate arguments you have skipped.

1.4 LIB$T[ABLE_]PARSE Argument Block
LIB$T[ABLE_]PARSE finds the input string through the argument block. This argument block is the impure database upon which LIB$T[ABLE_]PARSE operates. That is, it is a set of variable data that can be written as well as read. It contains information about the string to be parsed, option flags for LIB$T[ABLE_]PARSE, and data about the current token. If LIB$T[ABLE_]PARSE calls an action routine, it passes the argument block to the action routine. This permits the action routine efficient reference to relevant data.

1.4.1 Choosing an Argument Block
LIB$T[ABLE_]PARSE provides an argument block for 32-bit operations on both VAX and Alpha systems. It also provides an argument block for 64-bit operations on Alpha systems.

1.4.1.1 32-Bit Argument Block
The 32-bit LIB$T[ABLE_]PARSE argument block accommodates longword addresses and values as well as input tokens whose binary representations require no more than 32 bits.

On Alpha systems, the LIB$T[ABLE_]PARSE 32-bit argument block can also accommodate a numeric input token whose binary representation requires up to 64 bits.

LIB$T[ABLE_]PARSE defines the first 9 longwords of the 32-bit argument block as shown in Figure lib-20. You must pass an argument block of at least this length as the first argument to LIB$T[ABLE_]PARSE. You can add fields to the end of the argument block as a means of passing user-defined data to action routines.

The TPA$K_LENGTH0 symbol represents the number of bytes (36) in the basic 32-bit argument block. You can use this symbol to determine the start of any user-defined fields you add to the argument block.

Table lib-10 describes the argument block fields.

Figure lib-20 LIB$T [ABLE_]PARSE 32-Bit Argument Block

The 64-bit LIB$T[ABLE_]PARSE argument block accommodates quadword addresses and values as well as input tokens whose binary representations require no more than 64 bits.

LIB$T[ABLE_]PARSE defines the first 10 words of the 64-bit argument block as shown in Figure lib-21. You can add fields to the end of the argument block as a means of passing data to action routines.

The TPA64$K_LENGTH0 symbol represents the number of bytes (80) in the basic 64-bit argument block. You can use this symbol to determine the start of any user-defined fields you add to the argument block.

Table lib-10 describes the argument block fields.

Figure lib-21 LIB$T [ABLE_]PARSE 64-Bit Argument Block (Alpha Only)

1.4.2 Symbolic Names for Argument Block Fields
The fields in each type of argument block have symbolic names. Figure lib-20 and Figure lib-21 show some of these symbolic names. This section tells you how to access these names in some of the most commonly used languages:

• MACRO assembly language --- MACRO language programs can define both the 32-bit and 64-bit argument block names by invoking the macro $TPADEF (automatically loaded from the system macro library). The field names define the byte offset of the field from the start of the argument block. This includes the bit fields ($V_names). In addition, bit mask values ($M_names) are available for the bit fields.
• BLISS --- The field names are also available to BLISS programs from the system macro SYS$LIBRARY:STARLET.L32 and SYS$LIBRARY:STARLET.L64 libraries. Each name (except for the $M_names) is defined as a fixed-reference macro that operates on a byte-based block. The $M_names are defined as literals.
• C --- The same field names are available to C programs from the tpadef.h file. For the 32-bit and 64-bit argument blocks, the names are defined as elements of the tpadef and tpa64def structures, respectively.

See Section 2.2 for an example of an argument block declaration.

1.4.3 32-Bit and 64-Bit Argument Block Fields
Table lib-10 describes the fields of the 32-bit and 64-bit argument blocks.

Note that most fields have two symbols and one description. The symbol that begins with the prefix TPA$ is used with a 32-bit argument block, while the symbol that begins with the prefix TPA64$ is used with a 64-bit argument block. To prevent cumbersome explanations, Table lib-10 uses only the main part of a field name, without the prefix used in the actual code, when referring to a field for both the 32-bit and 64-bit argument blocks. For example, the options field is referred to as OPTIONS rather than specifying both TPA$L_OPTIONS and TPA64$L_OPTIONS. The complete field name is used only when referring to a field for one particular form of argument block.

Table lib-10 LIB$T [ABLE_]PARSE Argument Block Fields

Symbol	Description
TPA$L_COUNT TPA64$L_COUNT	A longword containing the value of TPA$K_COUNT0 for 32-bit argument blocks or TPA64$K_COUNT0 for 64-bit argument blocks. TPA$K_COUNT0 is defined to be 8. TPA64$K_COUNT0 is defined to be --1. If the value contained in this longword is greater than or equal to 8, LIB$T[ABLE_]PARSE treats the argument block as a 32-bit argument block. If the value is --1, LIB$T[ABLE_]PARSE treats the argument block as a 64-bit argument block. For LIB$TPARSE (VAX only), a longword containing the number of longwords that make up the rest of the argument block. This longword functions as the argument count when the argument block becomes the argument list to an action routine. This field must contain a value that is greater than or equal to the value of TPA$K_COUNT0, whose numeric value is 8.
TPA$L_OPTIONS TPA64$L_OPTIONS	Contains various flag bits and other options. The defined flags are as follows: TPA$V_BLANKS, TPA64$V_BLANKS 1 --- Setting this bit causes LIB$T[ABLE_]PARSE to process blanks and tabs explicitly, rather than treating them as separators. See Section 3.2 for information about processing blanks. TPA$V_ABBRFM, TPA64$V_ABBRFM 1 --- Setting this bit allows keywords to be abbreviated to any length. If an abbreviated keyword string is ambiguous, the first eligible transition listed in the state matches it. TPA$V_ABBREV, TPA64$V_ABBREV 1 --- Setting this bit allows keywords to be abbreviated to the shortest length that is unambiguous in that state. See the Abbreviating Keywords section. TPA$V_AMBIG, TPA64$V_AMBIG 1 --- LIB$T[ABLE_]PARSE sets this bit when it has detected an ambiguous keyword string in the current state. The OPTIONS field also contains the following option: TPA$B_MCOUNT, TPA64$B_MCOUNT --- This byte contains the minimum number of characters allowed for the abbreviation of a keyword. If its value is zero, abbreviations are not allowed. Preventing ambiguity is the responsibility of the state table designer. If the ABBRFM or ABBREV flag is set, LIB$T[ABLE_]PARSE ignores MCOUNT. MCOUNT is the high byte of the OPTIONS field.
TPA64$Q_STRINGDESC	For a 64-bit argument block, the three quadwords starting with TPA64$Q_STRINGDESC form an embedded 64-bit descriptor for the input string. 2 On entry, LIB$T[ABLE_]PARSE writes the fields of TPA64$Q_STRINGDESC as follows: DSC64$B_CLASS = DSC64$K_CLASS_S DSC64$B_DTYPE = DSC64$K_DTYPE_T DSC64$L_MBMO = --1 DSC64$W_MBO = +1
TPA$L_STRINGCNT TPA64$Q_STRINGCNT	Contains the number of characters remaining in the input string. For a 32-bit argument block, TPA$L_STRINGCNT and TPA$L_STRINGPTR form an embedded 32-bit descriptor for the input string. 2
	For both 32-bit and 64-bit argument blocks: You must initialize the STRINGCNT and STRINGPTR fields to describe the input string. Use LIB$ANALYZE_SDESC or LIB$ANALYZE_SDESC_64 to read the string length and address from the string's descriptor and write them in STRINGCNT and STRINGPTR, respectively. Before LIB$T[ABLE_]PARSE calls an action routine, it modifies STRINGCNT and STRINGPTR to describe the remainder of the input string. When LIB$T[ABLE_]PARSE returns, STRINGCNT and STRINGPTR describe the portion of the input string that LIB$T[ABLE_]PARSE did not process. This occurs whether LIB$T[ABLE_]PARSE returns success or failure.
TPA$L_STRINGPTR TPA64$Q_STRINGPTR	Contains the address of the remainder of the string being parsed.
TPA64$Q_TOKENDESC	For a 64-bit argument block, the three quadwords starting with TPA64$Q_TOKENDESC form an embedded 64-bit descriptor for the current token. 2 On entry, LIB$T[ABLE_]PARSE writes the fields of TPA64$Q_TOKENDESC as follows: DSC64$B_CLASS = DSC64$K_CLASS_S DSC64$B_DTYPE = DSC64$K_DTYPE_T DSC64$L_MBMO = --1 DSC64$W_MBO = +1
TPA$L_TOKENCNT TPA64$Q_TOKENCNT	Contains the number of characters in the current token. For a 32-bit argument block, TPA$L_TOKENCNT and TPA$L_TOKENPTR form an embedded 32-bit descriptor for the input token. 2 For both 32-bit and 64-bit argument blocks, LIB$T[ABLE_]PARSE updates TOKENCNT and TOKENPTR, to reflect the current token.
TPA$L_TOKENPTR TPA64$Q_TOKENPTR	Contains the address of the current token.
TPA$B_CHAR 3 TPA64$B_CHAR 3	Contains the character matched by one of the single-character symbol types: 'x', TPA$_ANY, TPA$_ALPHA, or TPA$_DIGIT.
TPA$L_NUMBER 3 TPA64$Q_NUMBER 3	Contains the binary representation of a numeric token that matches TPA$_OCTAL, TPA$_DECIMAL, TPA$_HEX, TPA$_UIC, or TPA$_IDENT. For a 64-bit argument block, it can also contain the binary representation of a numeric token that matches TPA$_DECIMAL_64, TPA$_OCTAL_64, or TPA$_HEX_64.
(Alpha specific) TPA$Q_NUMBER 3	For a 32-bit argument block on an Alpha system, contains the binary representation of a numeric token that matches TPA$_DECIMAL_64, TPA$_OCTAL_64, or TPA$_HEX_64. LIB$T[ABLE_]PARSE coverts the numeric token in the appropriate radix before storing it in the TPA$Q_NUMBER field. In the 32-bit argument block, TPA$Q_NUMBER overlays TPA$L_NUMBER and the longword in which TPA$B_CHAR resides.
TPA$L_PARAM TPA64$Q_PARAM	Contains the optional 32-bit argument supplied by the state transition in its argument argument. For a 64-bit argument block, LIB$T[ABLE_]PARSE sign-extends the argument value before storing it in TPA64$Q_PARAM.

2 Coding and Using a Simple State Table
LIB$T[ABLE_]PARSE can parse programming languages, command languages, or any other grammar for which a deterministic parser is the best choice.

To code a program to use LIB$T[ABLE_]PARSE, perform the following steps:

1. Set up state tables to implement the language's grammar (See Section 2.1 )
2. Define the argument block and other common variables (See Section 2.2 )
3. Include the call to LIB$T[ABLE_]PARSE in the main program (See Section 2.3 )

This section provides examples that demonstrate the use of LIB$T[ABLE_]PARSE to perform these three steps. The examples parse the command language of a simple report management utility. This hypothetical utility allows a user to perform the following activities:

• Obtain a list of available reports (SHOW command).
• Read reports on the terminal (READ command).
• Print reports (PRINT command).
• Store new reports (FILE command).

The examples use the BASIC programming language for everything except the state and keyword tables, which are coded in BLISS.

This simple state table program does not use any action routines or other arguments. See Section 3 for information about how to use these features of LIB$T[ABLE_]PARSE.

2.1 Setting Up a State Table
A state table associates the parser's alphabet with a set of possible transitions.

It is often helpful to create a graphical representation of a state table before attempting to code it. The following section illustrates two possible approaches.

2.1.1 Diagramming the Transitions
One way to set up these tables is to start from a transition diagram of the language you want to parse. (If you do not know how to construct a transition diagram, you might find it helpful to read an introductory text about compiler design and construction before you start.) Each circle represents a state in the state table. Each arrow, labeled with an input option, represents a transition out of one state to another state or within the same state.

Figure lib-22 shows a transition diagram for the hypothetical utility described in this section.

Figure lib-22 Transition Diagram for a Hypothetical Utility

Another technique for developing a state table starts with a tabular diagram in which the first column is the starting state, the second column identifies the input token, or keyword, and the third gives the resultant state.

Figure lib-23 is a tabular diagram of the utility that appears in Figure lib-22.

Figure lib-23 Tabular Diagram of a Hypothetical Utility

In this case, each unique entry in the Starting State or Resulting State column represents a state in the state table. Each entry in the Input column represents a possible transition out of the state in the Starting State column to a state in the Resulting State column.

2.1.2 Coding a State Table
For both MACRO and BLISS, you begin the state table with an $INIT_STATE macro. If you use MACRO to define your state table, then:

• Use the $STATE macro to define each state.
• Follow each $STATE macro with one instance of the $TRAN macro for each transition from this state to another state or within the same state.

• Use the $STATE macro to define each state and its associated transitions.

The following MACRO and BLISS examples code the state table for the hypothetical utility diagrammed in Figure lib-22 and Figure lib-23. Note that neither of these state tables includes the error state, because LIB$T[ABLE_]PARSE automatically generates an error if the input token does not match a transition in the current state. To provide a transition to your own error state, code the last transition in the state with the TPA$_LAMBDA symbol type and specify a transition to your error state. The TPA$_LAMBDA symbol type matches any input token.

The state table, coded using MACRO, for this simple language looks like this:

.TITLE simplelang .ident 'v1' ;+ ; Define the LIB$TABLE_PARSE control symbols ;- $TPADEF $INIT_STATE SIMPLE_LANGUAGE_TABLE, SIMPLE_KEYWORD_TABLE $STATE START $TRAN 'PRINT', STATE1 $TRAN 'READ', STATE1 $TRAN 'FILE', STATE1 $TRAN 'SHOW', STATE1 $STATE STATE1 $TRAN TPA$_STRING, STATE1 $TRAN TPA$_EOS, TPA$_EXIT $END_STATE .END

Using the BLISS macros yields the following state table definition:

MODULE simple_statetable = BEGIN !+ ! These libraries contain the macros and other definitions ! needed to generate the state tables. !- LIBRARY 'SYS$LIBRARY:STARLET'; LIBRARY 'SYS$LIBRARY:TPAMAC'; !+ ! UFD_STATE is the name you are giving the state table. ! UFD_KEY names the keyword table. ! Be sure to use the same name in the call to LIB$T[ABLE_]PARSE. !- $INIT_STATE (UFD_STATE, UFD_KEY); !+ ! Read the command name (to the first blank in the command). ! Each string is a keyword; you are limited to 220 keywords ! per state table. !- $STATE (START, !Be careful of your punctuation here. ('CREATE',STATE1), ! Each transition is surrounded by ('FILE',STATE1), ! parentheses; each entry except the ('PRINT',STATE1), ! last is followed by a comma. ('READ',STATE1) ); $STATE (STATE1, (TPA$_STRING, STATE1), ! If there is more than one report name (TPA$_EOS, TPA$_EXIT) ! specified, go back and process it. ); ! exit when done. END ELUDOM ! End of module CREATE_TABLE

Copyright © Compaq Computer Corporation 1998. All rights reserved. Legal

5932PRO_045.HTML