Reliable Transaction Router
Application Design Guide


Previous Contents Index

Broadcast Messages

Broadcast messaging lets client and server applications send non- transactional information to recipients. Recipients must be declared with the opposite role; that is, a client can send a broadcast message to servers, and a server can send a broadcast message to clients. Broadcasts are delivered on a "best try" basis; they are not guaranteed to reach every potential recipient. A broadcast message can be directed to a specific client or server, or be sent to all reachable recipients.

This point-to-point messaging using broadcast semantics is a feature to use instead of transactions when the information being sent is not recorded as a transaction in the database, and when you need to send information to several clients (or servers) simultaneously. For example, in a stock trading environment, when a trade has been completed and the stock price has changed, the application can use a broadcast message to send the new stock price to all trading stations. Another use for such messages is to inform the applications about a state change in the environment (for example, the fact that the exchange is now closed for business).

Other considerations when using broadcast messages include:

Flow Control

Broadcast messages are subject to flow control. A broadcaster may be blocked and unable to send messages when traffic is high and recipients are unable to process the broadcasts. The broadcaster sends at the minimum rate (MINIMUM_BROADCAST_RATE) which can be set to send "no matter what" for a given node. However, if an application does this, the application may in practice hold up broadcasts for others, and application design must take this into account. For example, no client application should be able to issue a Control S (^ S) to hold up all broadcasts. If an application doing broadcasts works with transactions that might get held up, it may be time to consider using multiple channels on multiple threads.

Sequencing of Broadcasts

RTR guarantees that broadcasts are received in the same order as sent by a specific sender. However, if there is more than one sender in an application, different recipients can receive broadcasts in different orders. For example, Sender A could send broadcasts ABC and Sender B, broadcasts XYZ. These could be received by two different recipients as ABCXYZ or XYZABC. If this is important in your application, correct application design is to use one sender that takes in all input needed for such broadcasts.

Sequencing Relative to Transaction Delivery

Consider a shadowed trading environment that initiates 5PM processing with a broadcast for closing of the exchange. Application design should send broadcasts and transactions through different pipes. Because RTR does not guarantee receipt of a broadcast at all servers, but does guarantee receipt of transactions, this critical "broadcast" could be most effectively handled by being sent in a transaction as an event through the transaction pipe.

Recovery of Broadcasts

There is no replay or recovery for broadcasts.

Lost Broadcasts

A broadcast can sometimes be lost. This can be caused by link loss or perhaps when there is excessively high volume.

Coping with Broadcast Loss

There is overhead associated with managing and correcting for loss of broadcasts. Thus Compaq recommends that applications do not use broadcasts for critical information. If, however, an application decides to use broadcasts and wants to ensure that all broadcasts are accounted for, one approach is to add a tracking sequence number to each broadcast that is sent out. All recipients can then check for missing sequence numbers and request a resend of any missing broadcasts.

Broadcast Messaging Processes

A client or server application may need to send unsolicited messages to one or more participants. Applications tell RTR which broadcast classes they want to receive.

The sender sends one message received by several recipients. Recipients subscribe to a specific type of message. Delivery is not guaranteed. Broadcast messages can be:

Clients cannot broadcast to other clients, and servers cannot broadcast to other servers. To enable communication between two applications of the same type, open a second instance of the application of the other type. Messaging destination names can include wildcards, enabling flexible definition of the subset of recipients for a particular broadcast.

Broadcast types include user event s and RTR events; both are numbered.

User Events

Event numbers are provided as a list beginning with RTR_EVTNUM_USERDEF and ending with RTR_EVTNUM_ENDLIST . To subscribe to all user events, an application can use the range indicators RTR_EVTNUM_USERBASE and RTR_EVTNUM_USERMAX , separated by RTR_EVTNUM_UP_TO , to specify all possible user event numbers.

A user broadcast is named or unnamed. An unnamed broadcast does a match on user event number; the event number completely describes the event. A named broadcast does a match on both user event number and recipient name. The recipient name is a user-defined string. Named broadcasts provide greater control over who receives a particular broadcast.

Named events specify an event number and a textual recipient name. The name can include wildcards (% and *).

For all unnamed events specify the evtnum field and RTR_NO_RCPSPC as the recipient name.

RTR Events

RTR delivers status information to which client and server applications can subscribe. Status information is delivered as messages, where the type of each message is an RTR event.

RTR events are numbered. The base value for RTR events is defined by the symbol RTR_EVTNUM_RTRBASE ; its maximum value is defined by the symbol RTR_EVTNUM_RTRMAX . RTR events and event numbers are listed in the Reliable Transaction Router API manuals and in the RTR header files rtr.h and rtrapi.h.

An application can subscribe to RTR events to receive notification of external events that are of interest to the application. For example, a shadow server may need to know if it is a primary or a secondary server to perform certain work, such as uploading information to a central database, that is done at only one site.

To subscribe to all RTR events, use the range indicators RTR_EVTNUM_RTRBASE and RTR_EVTNUM_RTRMAX . RTR events are delivered as messages of type rtr_mt_rtr_event .

In application design, consider creating separate facilities for sending broadcasts. By separating broadcast notification from transactional traffic, performance improvements can be substantial. Facilities can further be reconfigured to place the RTR routers strategically to minimize wide-area traffic.

A server application can expect to see a primary or secondary event delivered only in certain transaction states. For more detail, see the state diagrams in Appendix C, Server States.

Location Transparency

With location transparency, applications do not need to be modified when the hardware configuration is altered, whether changes are made to systems running RTR services or to the network topology. Client and server applications do not know the location of one another so services can be started anywhere in the network. Actual configuration binding is a system management operation at run time, through the assignment of roles (frontend/backend/router) within a given facility to the participant nodes.

For RTR to automatically take care of failover, server applications need to specify certain availability attributes for the partition.

Because RTR automatically takes care of failover, applications need not be concerned with specifying the location of server resources.

Handling Error Conditions

RTR can provide information to an application with the RTRMessage and RTREvent classes (for the C++ API). Certain inherited methods within these classes translate RTR internal error message values to informational text meaningful to the reader. For the C API, this is done with the rtr_error_text call .

If an application encounters an error, it should log the error message received. Error messages are more fully described in rtrapi.h for the C++ API and in rtr.h for the C API, where each error code is explained.

For example, the following short program uses the standard C library output function to display the text of an error status code.


Program "prog": 
#include "rtr.h" 

or


#include <rtr.h> 
main() { 
 printf("%s", 
  rtr_error_text(RTR_STS_NOLICENSE)); 
} 

When this program is run, it produces the following output:


$run prog 
 No license installed 

The several hundred error or status codes reside in the rtr.h header file; status codes can come from any RTR subsystem. A few codes that an application is likely to encounter are described in Table 3-3.

Table 3-3 RTR Error Codes
Status Code Meaning
RTR_STS_COMSTAUNO Commitment status unobtainable. The fate of the transaction currently being committed is unobtainable; this may be due to a hardware failure.
RTR_STS_DLKTXRES The transaction being processed was aborted due to deadlock with other transactions using the same servers. RTR will replay the transaction after the deadlock has been resolved and cleared.
RTR_STS_FELINLOS Frontend link lost; probably due to a network failure.
RTR_STS_INVFLAGS Invalid flags.
RTR_STS_NODSTFND No destination found; no server had declared itself to handle the key value specified in the sent message. Probably a server down or disconnected.
RTR_STS_REPLYDIFF Two servers respond with different information during a replay; transaction aborted.
RTR_STS_TIMOUT Timeout expired; transaction aborted.
RTR_STS_SRVDIED Probably a server image exited, for example because a node is down.
RTR_STS_SRVDIEDVOT A server exited before committing a transaction.
RTR_STS_SRVDIEDCOM A server exited after being told to commit a transaction.

RTR can abort a transaction at any time, so the application must be prepared to deal with such aborted transactions. Server applications are expected to roll back transactions as the need arises, and must be built to take the correct action, and subsequently carry on to deal with new transactions that are received.

A client application can also get a reject and must also be built to deal with the likely cases it will encounter. The application must be built to decide on the correct course of action in the event of a transaction abort.

Using Locks

When using a database system with RTR, an application designer must be aware of how the database system works and how it handles database locks. Because Oracle is a frequently used database system, this section provides a short summary of Oracle locking methods. The application designer must use Oracle documentation to supplement this brief description. This material is fully discussed in the Oracle8 and Oracle8i Application Developer's Guides, specifically in the chapters on Processing SQL Statements, Explicit Data Locking, Explicitly Acquiring Row Locks, Serializable and Row Locking Parameters, User Locks, Non-Default Locking, and Concurrency Control Using Serializable Transactions. Oracle database operations are performed using Structured Query Language (SQL).

Oracle Locking

Privileges Required

In its own schema, an application can automatically acquire any type of table locks. However, to acquire a table lock on a table in another schema, the application must have the LOCK ANY TABLE system privilege or an object privilege such as SELECT or UPDATE for the table.

Overriding Default Locking

By default, Oracle locks data structures automatically. However, an application can request specific data locks on rows or tables when it needs to override default locking. Explicit locking lets an application share or deny access to a table for the duration of a transaction.

An application can explicitly lock entire tables using the LOCK TABLE statement, but locking a table means that no other transaction, user, or application can access it. This can cause performance problems.

With the SELECT FOR UPDATE statement, an application explicitly locks specific rows of a table to ensure the rows do not change before an update or a delete. Oracle automatically obtains row-level locks at update or delete time, so use the FOR UPDATE clause only to lock the rows before the update or delete.

A SELECT statement with Oracle does not acquire any locks, but a SELECT ... FOR UPDATE does. For example, the following is a typical SELECT ... FOR UPDATE statement:


 
 SELECT partno FROM parts FOR UPDATE OF price 

This statement starts a transaction to update the parts table with a price change for a specific part.

Oracle Explicit Data Locking

To ensure data concurrency, integrity, and statement-level read consistency, Oracle always performs necessary locks. However, an application can override default locks. This can be useful when:

Overrides to Oracle locks can be done at two levels:

At transaction level: Transactions override Oracle default locks with the following SQL statements:

At system level : Oracle can start an instance with non-default locking by adjusting the following initialization parameters:

If an application overrides any Oracle default locks, the application itself must:

Table Locks

When a LOCK TABLE statement executes, it overrides default locking, and a transaction explicitly acquires the specified table locks. A LOCK TABLE statement on a view locks the underlying base tables (see Table 3-4).

Table 3-4 LOCK TABLE Statements
Statement Meaning
LOCK TABLE tablename IN EXCLUSIVE MODE [NOWAIT]; Acquires exclusive table locks. Locks all rows of the table. No other user can modify the table. With NOWAIT, the application acquires the table lock only if the lock is immediately available, and Oracle issues an error if not. Without NOWAIT, the transaction does not proceed until the requested table lock is acquired. If the wait for a table lock reaches the limit set by the initialization parameter DISTRIBUTED_LOCK_TIMEOUT, a distributed transaction can time out. As no data will have been modified due to the timeout, the application can proceed as if it has encountered a deadlock.
LOCK TABLE tablename IN ROW SHARE MODE;
LOCK TABLE tablename IN ROW EXCLUSIVE MODE;
These offer the highest degree of concurrency. Consider if the transaction must prevent another transaction from acquiring an intervening share, share row, or exclusive table lock for a table before the table can be updated in the transaction. If another transaction acquires an intervening share, share row, or exclusive table lock, no other transactions can update the table until the locking transaction commits or rolls back.
LOCK TABLE tablename IN SHARE MODE; Consider this share table lock if:
  • Your transaction only queries the table and requires a consistent set of table data for the duration of the transaction (requires transaction-level read consistency for the locked table).
  • Other transactions to update the locked table concurrently can be made to wait until all transactions with the share table locks commit or roll back.
  • Other transactions can acquire concurrent share table locks on the same table, providing them transaction-level read consistency.

Note: If multiple transactions concurrently hold share table locks for the same table, NO transaction can update the table. Thus if share table locks on the same table are common, deadlocks will be frequent and updates will not proceed. For such a case, use share row exclusive or exclusive table locks.
LOCK TABLE tablename IN SHARE ROW EXCLUSIVE MODE; Acquire a share row exclusive table lock when:
  • Your transaction requires both transaction-level read consistency for the specified table, and the ability to update the locked table.
  • Other transactions can obtain explicit row locks.
  • The application needs only a single transaction to have this behavior.

Acquiring Row Locks

The SELECT ... FOR UPDATE statement acquires exclusive row locks of selected rows. The statement can be used to lock a row without changing the row. Acquiring row locks can also be used to ensure that only a single interactive application user updates rows at a given time. For information on using this statement with cursors or triggers, see the Oracle8 or Oracle8i documentation. To acquire a row lock only when it is immediately available, include NOWAIT in the statement.

Each row in the return set of a SELECT ... FOR UPDATE statement is individually locked. The statement waits until a previous transaction releases the lock. If a SELECT ... FOR UPDATE statement locks many rows in a table, and the table is subject to moderately frequent updates, it may improve performance to acquire an exclusive table lock rather than using row locks.

If the wait for a row lock reaches the limit set by the initialization parameter DISTRIBUTED_LOCK_TIMEOUT, a distributed transaction can time out. As no data will have been modified, the application can proceed as if it has encountered a deadlock.

Setting SERIALIZABLE and ROW_LOCKING Parameters

How an instance handles locking is determined by the SERIALIZABLE option on the SET TRANSACTION or ALTER SESSION command, and the initialization parameter ROW_LOCKING. By default, SERIALIZABLE is set to false and ROW_LOCKING is set to always.

Normally these parameters should never be changed. However they may be used for compatibility with applications that run with earlier versions of Oracle, or for sites that must run in ANSI/ISO-compatible mode. Performance will usually suffer with non-default locking.

Using the LOCK TABLE Statement

The application uses the LOCK TABLE statement to lock entire database tables in a specified lock mode to share or deny access to them. For example, the statement below locks the parts table in row-share mode. Row-share locks allow concurrent access to a table; they prevent other users from locking the entire table for exclusive use. Table locks are released when your transaction issues a commit or rollback.


    LOCK TABLE parts IN ROW SHARE MODE NOWAIT; 

The lock mode determines which other locks can be placed on the table. For example, many users can acquire row-share locks on a table at the same time, but only one user at a time can acquire an exclusive lock. While one user has an exclusive lock on a table, no other users can insert, delete, or update rows in that table. For more information about lock modes, see Oracle8 or Oracle8i Server Application Developer's Guide.

A table lock never keeps other users from querying a table, and a query never acquires a table lock. Only if two different transactions try to modify the same row will one transaction wait for the other to complete.

If your program includes SQL locking statements, make sure the Oracle users requesting the locks have the privileges needed to obtain the locks.

Table 3-5 Summary of Locking Options
Case Description SERIALIZABLE ROW_LOCKING
0 Default settings FALSE ALWAYS
1 As Oracle Version 5 and earlier (no concurrent inserts, updates or deletes in a table) FALSE (disabled) INTENT
2 ANSI compatible Enabled ALWAYS
3 ASNI compatible with table-level locking (no concurrent inserts, updates or deletes in a table) Enabled INTENT

Table 3-6 Non-Default Locking Behavior
Statement   Case 0 Case 1   Case 2   Case 3  
SERIALIZABLE FALSE (disabled) Disabled   Enabled   Enabled  
ROW_LOCKING ALWAYS INTENT   ALWAYS   INTENT  
      Row Table Row Table Row Table
SELECT   - - - S - S
INSERT   X SRX X RX X SRX
UPDATE   X SRX X SRX X SRX
DELETE   X SRX X SRX X SRX
SELECT ... FOR UPDATE   X RS X S X S
LOCK TABLE ... IN..            
 
  ROW SHARE MODE RS RS RS RS RS RS
  ROW EXCLUSIVE MODE RX RX RX RX RX RX
  SHARE MODE S S S S S S
  SHARE ROW EXCLUSIVE MODE SRX SRX SRX SRX SRX SRX
  EXCLUSIVE MODE X X X X X X
DDL Statements   - X - X - X
Modes: X = EXLUSIVE
  RS = ROW SHARE
  RX = ROW EXCLUSIVE
  S = SHARE
  SRX = SHARE ROW EXCLUSIVE


Previous Next Contents Index