The software described in this guide is furnished under a license agreement or nondisclosure agreement. The software may be used or copied only in accordance with the terms of the agreement.

Compaq and the Compaq logo are registered in United States Patent and Trademark Office.

The following are trademarks of Compaq Computer Corporation:
AlphaServer, DEC, DECnet, DIGITAL, OpenVMS, PATHWORKS, Reliable Transaction Router, TruCluster, VAX, VMScluster, and the DIGITAL Logo.

The following are third-party trademarks:

AIX and IBM are registered trademarks of International Business Machines, Corp.
Hewlett-Packard and HP-UX are registered trademarks of Hewlett-Packard Corp.
Informix is a registered trademark of Informix Software, Inc.
Intel is a registered trademark of Intel Corporation.
Memory Channel is a trademark of Encore Computer Corporation.
Microsoft, Microsoft Access, Microsoft SQL Server, Internet Explorer, and Windows NT are trademarks or registered trademarks of Microsoft Corporation.
Netscape and Netscape Navigator are registered trademarks of Netscape Communications Corporation.
Oracle is a registered trademark of Oracle Corporation.
Sun and Solaris are registered trademarks of Sun Microsystems, Inc.
Sybase is a registered trademark of Sybase, Inc.
Tuxedo is a registered trademark of Novell, Inc.
UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company, Ltd.

All other trademarks and registered trademarks are the property of their respective holders.

This document was prepared using Microsoft Word 8.

System Design

The RTR Runtime Environment

The RTR System Management Environment

Conclusions

rtr_open_channel(&(&Channel,
RTR_F_OPE_SERVER,
FACILITY_NAME,
NULL,
RTR_NO_PEVTNUM,
NULL,
Key.GetKeySize(),
Key.GetKey()!= ()!= RTR_STS_OK);

rtr_open_channel(&(&Channel,
RTR_F_OPE_SERVER|RTR_F_OPE_NOCONCURRENT,
FACILITY_NAME,
NULL,
RTR_NO_PEVTNUM,
NULL,
Key.GetKeySize(),
Key.GetKey()!= ()!= RTR_STS_OK);

rtr_status_t
rtr_open_channel ((
…
rtr_ope_flag_t

Serialization Issues

Journal Sizing

Replay Anomalies

Making Broadcasts Reliable

case rtr_mt_prepare:

/* if (txn is independent)…*/
status = rtr_accept_tx (&channel,
RTR_F_ACC_INDEPENDENT,
RTR_NO_REASON);

if (status != RTR_STS_OK)

To help make your RTR system as manageable and operable as possible, you can consider several tradeoffs in establishing your RTR configuration. Consider these before creating your RTR facilities and deploying an application. Make these considerations part of your design and validation process.

Define your facilities with an eye to the number and placement of frontends, routers, and backends.

To avoid problems with quorum resolution, design your configuration with an odd number of routers to ensure that quorum can be achieved.

Place your routers separate from your backends to improve failover, so that failure of one node does not take out both the router and the backend.

If your application requires frontend failover when a router fails, frontends must be on separate nodes from the routers, but frontends and routers must of course be in the same facility. For frontend failover, a frontend must be in a facility with multiple routers.

To identify a node used only for quorum resolution, define the node as a router or as a router and frontend. Define all backends in the facility, but no other frontends.

With a widely dispersed set of nodes, for example, nodes distributed across an entire country, use local routers to deal with local front ends. This can be more efficient than having many dispersed frontends connecting to a small number of distant routers.

In many configurations, it may be more effective to place routers near backends.

For security purposes, your application transactions may need to use pass through firewalls in the path from the client to the server application. RTR provides this capability within the CREATE FACILITY syntax. (See the RTR Reliable Transaction Router System Manager’s Manual, Network Transports, for specifics on how to specify a node to be used as a firewall, and how to set up your application to tunnel through the firewall.)

Nodes in your configuration are often specified with names and IP or DECnet addresses fielded by a name server. When the name server goes down or becomes unavailable, the name service is not available and certain requests may fail. To minimize such outages, declare the referenced node name entries in a local host names file that is available even when the name server is not. Using a host names file can also improve performance for name lookups.

Operations staff often create batch or command procedures to take snapshots of system status to assist in monitoring applications. RTR’s The character cell displays (ASCII output) of RTR can provide input to such procedures. Be aware that system responses from RTR can change with each release, which can cause such command procedures to fail. If possible, plan for such changes when bringing up new versions of the product.

In addition to understanding the RTR run-time and system management environments, you must also understand the RTR applications environment and the implications of that environment in your implementation. This section provides you with information on requirements that transaction processing applications must take into account and deal with effectively and gracefully, and rules you should follow so that your application does not violate any of the rules for ensuring that your transactions are ACID compliant. The requirements and rules complement each other and sometimes repeat a similar concept. Your application must take both into account.

Applications written to operate in the RTR environment should adhere to the following rules:

• Be transaction aware
• Avoid server-specific data
• Optionally, have independent transactions
• Optionally, use two identical databases for transactional shadow servers
• Make transactions self-contained
• Lock shared resources

RTR expects server applications to be transaction aware; an application must be able to roll back an appropriate amount of work when asked. Furthermore, to preserve transaction integrity, rollback must be all or nothing. Each transaction incurs some overhead, and the application must be prepared to deal with failures and concomitant rollback gracefully.

When designing your client and server applications, note the outcome of transactions. Transactional applications often store data in variables outside the control of RTR that pertain to the operation taking place. Depending on the outcome of the RTR transaction, the values of these variables may need to be adjusted. RTR guarantees delivery of messages (usually to a database), but RTR does not know about any data not passed through RTR.

The rule is: Code your application to preserve transaction integrity through failures.

The client and server applications must not exchange any data that makes sense on only one node in the configuration. Such data can include, for example, a memory reference pointer, whose purpose is to allow the client to reference this context in a later transaction, indexes into files, node names, or database record numbers. These values would only make sense on the machine on which they were generated. If your application sends data to another machine, that machine will not be able to interpret the data correctly. Furthermore, data cannot be shared across servers or across channels.

The rule is: The means you adopt to track state must be meaningful on all nodes where your application runs.

Transactions are assumed to contain all the context information required to be successfully executed. An RTR transaction is assumed to be independent of time of processing. For example, in a shadow environment, if the secondary server cannot credit an account because it is past midnight, but the transaction has already been successfully committed on the primary server, this would cause an inconsistency between the primary and secondary databases. Or, in another example, Transaction B cannot rely on the fact that Transaction A performed some operation before it.

Make no assumptions about the amount of time that will occur between transactions, and avoid using a transaction to establish a session with a server application that can time out. Such a timeout might occur in a client application that logs into a server application that sets a timer to determine when to log the client off. If a crash occurs after a successful logon, subsequent transactions may fail because the logged-on session is no longer valid.

The rule is: If you have operations that must not be shadowed, identify them and exclude them from your application. Furthermore, do not keep a state that can become stale over time.

In your application, you can define transactions as independent, using the independent transaction flag in your rtr_accept_tx or rtr_reply_to_client calls. For more information on the independent transaction flag and the different uses of these calls, see the RTR Application Programmer’s Reference Manual.

Shadow server use is aimed at keeping two identical copies of the database synchronized. For example, Figure 30, Transactional Shadow Servers, illustrates a configuration with a router serving two backends to two shadow databases. The second router is for router failover.

If an update of a copy triggers the update of a third common database, then the application must determine whether it is running as a primary or a secondary, and only carry out an update if it is the primary. Otherwise, there can be complex failure scenarios where duplication can occur.

For example, RTR has no way to determine if a transaction being shadowed is a one-time only transaction, such as a bookstore debiting your credit card for the purchase of a book. If this transaction is processed on the primary node and the processed data fed to a third common database, and the transaction is later processed on the secondary node, your account would incorrectly be double charged. The application must handle this situation correctly.

The rule is: Design your application to deal correctly with transactions, such as debiting a credit card or bank account, that must never be done more than once.

Figure 31, Shadow Servers and Third Common Database (not recommended) shows a configuration with two shadow servers and a third independent server for a third common database. This is not a configuration recommended for use with RTR without application software that deals with the kind of failure situation described above. Another method is to decouple the shadow message from the other branch.

The recommended method to use when updating a single resource through multiple paths is to use the RTR standby functionality.

All information required to process a transaction from the perspective of the server application should be contained within the transaction message. For example, if the application required a user-id established earlier to successfully execute the transaction, then the user-id should be included in the transaction message.

The rule is: Construct complete transaction messages within your application.

While a server application is processing a transaction, and particularly before it "accepts" the transaction, it must assume ensure that all shared resources accessed by that transaction are locked. Failure to do so can cause unpredictable results in shadowing or recovery.

The rule is: Lock shared resources while processing each transaction.

To ensure that your application deals with transactions correctly, its transactions must be:

• Atomic
• Consistent
• Isolated
• Durable

For the atomic attribute, the result of a transaction is all or nothing, that is, either totally committed or totally rolled back. To ensure atomicity, do not use a data manager that cannot roll back its updates on request. All standard data managers or database management systems have the atomicity attribute, but in some cases when interfacing to an external legacy system, a flat-file system, or an in-memory database, a transaction may not be atomic.

For example, a client application may believe that a transaction has been rejected, but the database server does not. With a database manager that can make this mistake, the application itself must be able to generate a compensating transaction to roll back the update.

Data managers that do not use XA/DTC or DECdtm and Microsoft DTC to integrate with RTR using XA or DECdtm must be programmed to handle rtr_mt_msg1_uncertain messages.

For example, to illustrate the atomicity rules, Figure 32, Uncertain Interval for Transactions, shows the uncertain interval in a transaction sequence that the application program must be aware of and take into account, with by performing appropriate rollback.

If there is a crash before the rtr_accept statement is executed, on recovery, the transaction is replayed as rtr_mt_msg1 because the database will have rolled back the prior transaction instance. However, if there is a crash after the rtr_accept statement is executed, on recovery, the transaction is replayed as rtr_mt_msg1_uncertain because RTR does not know the status of the prior transaction instance. Your application must understand the implications of such failures and deal with them appropriately.

Several rules must be considered to ensure consistency. They are:

• Shadowed applications must lock records being updated and hold these locks while RTR does commit processing. Note that shadowed transactions may be executed at different times and in a different order between two sites because RTR assumes that transactions are independent.
• Client applications that rely on system-dependent information sent from the server application should use the REPLYDIFF option in RTR to ensure that a single transaction does not span multiple nodes. For example, if data is returned to the client application before the server crashes, the recovered transaction may produce results that conflict with the message previously returned to the client.
• Applications should never access shadowed data sets outside the RTR environment. This could, for example, be the case if your application relies on batch processing that does not use RTR or database maintenance. Updating between the two sites will not be serialized if all updates do not pass through RTR. Updating two sites only after quiescing shadow systems is impractical in most mission-critical applications.
• Partition your data so that all access is done through a single RTR partition (key range). Note that it is not usual to partition by function or use shared tables. Although partitioning is recommended for improving concurrency, RTR serializes transactions only on a partition basis. If two partitions have overlapping data records, these records might be updated in differing order across the two sites, unless the application deals with them correctly. Do not overlap partitions; make them discrete.
• Do all data-constraint checking before committing the database. This can be an issue with certain database optimizations.

The changes to shared resources that a transaction effects do not become visible outside the transaction until the transaction commits. This makes transactions serializable. To ensure isolation:

Hold record locks throughout the RTR commit cycle. If the server crashes, the transaction could be recovered after changes to the data by a dependent transaction, generating results that are different from those already sent to the client. Also, new transactions can overtake the completion of a previous transaction from the same client if shared records are not locked.

Do not use RTR concurrent servers if your data manager does not support record locking. This can be important, for example, with in-memory databases. Concurrency relies on the independence of two operations that may affect common data records. Record locking ensures that a concurrent transaction cannot affect the consistency of data being operated on by another transaction.

Avoid certain site-dependent actions when running RTR shadow servers. For example, using transaction sequence numbers or time and date comparisons, can introduce problems. Shadowed transactions are serialized based on commit groups. If your application requires absolute transaction serialization, you cannot run concurrent servers. For example, Figure 33, Concurrent Server Commit Grouping, illustrates the serialization of commit groups. The first commit group is A1, processed on the primary backend; it is followed by commit group A2, followed by A3. Commit to the database, however, is in the order A3, A1, A2, as shown in the diagram. One the shadow site, commit to the database will be in the order A3, A2, A1, due to use of concurrent servers. If absolute serialization of CSNs is important in your application, you cannot use concurrent servers.

RTR commit grouping allows independent transactions to be scheduled together on the shadow secondary.

For a transaction to be durable, the changes that are the result of transaction commitment, must survive subsequent system and media failure. Thus transactions are both persistent and stable.

For example, your bank deposit is durable if, once the transaction is complete, your account balance reflects what you have deposited.

The durability rule is:

• Standby servers that update the database must have access to each other’s RTR journal, and use cluster-aware data managers such as Oracle Parallel Servers. If a node running as a standby server crashes, in-progress transactions will be recovered from the failed node’s journal files.

RTR provides an application programming interface (API) that features transaction semantics and intelligent routing in the client/server environment. It provides software fault tolerance using shadow servers, standby servers, and concurrent server processing. It can provide authentication with callout servers. RTR makes single-point failures transparent to the application, guaranteeing that, within the limits of reliability of the basic infrastructure and the physical hardware used, a transaction will arrive at its destination.

The RTR API contains sixteen 16 calls that address 4 groups of activity:

• Initialization/termination calls
• Messaging calls
• Transactional calls
• Informational calls

The initialization call signals RTR that a client or server application wants to use RTR services and the termination call releases the connection once the requested work is done.

Messaging calls enable client and server applications to send and receive messages and broadcasts.

Transactional calls collect groups of messages as transactions (tx).

Informational calls enable an application to set RTR options or interrogate RTR data structures.

To execute these calls using the RTR CLI, precede each with the keyword CALL. For example,

The following table provides additional information on RTR API calls, which are listed in alphabetical order.

The rtr.h file included with the product defines all RTR data, status, and message types, including text that can be returned in error messages from an application. You must use it in application compilation.

To support the multioperating system environment, error codes are processed by RTR using data values in rtr.h, and translated into text messages. Status codes are described in the RTR Reliable Transaction Router Application Programmer’s Reference Manual.

The command line interface (CLI) to the RTR API enables the programmer to write short RTR applications from the RTR command line. This can be useful for testing short program segments and exploring how RTR works. For example, the following sequence of commands starts RTR and exchanges a message between a client and a server. To use these examples, you execute RTR commands simulating your RTR client application on the frontend and commands simulating your server application on the backend.

The following example shows communication between a client and a server created by entering commands at a terminal keyboard. The client application is executing on the frontend and the server on the backend:.

The user is called user, the facility being defined is called DESIGN, a client and a server are established, and test messages containing the words "Kathy's text today" is sent from the client to the server. After the server receives this text, the user on the server enters the words "And this is my response." System responses begin with the characters % RTR-. Notes on the procedure are enclosed in square brackets [ ]. For clarity, commands you enter are shown in bold. You can view the status of a transaction with the SHOW TRANSACTION command.

RTR> RTR_RECEIVE_MESSAGE

Commands the user issues on the server application where RTR is running on the backend:

The exchange of messages you observe in executing these commands illustrates RTR activity. You must will need to retain a similar sequence in your own designs for starting up RTR and initiating your own application.

You can use RTR SHOW and MONITOR commands to display status and examine system state at any time from the CLI. For more information on RTR commands, see the Reliable Transaction Router System Manager’s Manual.

Snippets from similar client and server programs using the RTR API follow and are more fully shown in the appendix to this document.

Example of an open channel call in an RTR client program:

Example of a receive message call in an RTR server program:

A client can have one or multiple channels, and a server can have one or multiple channels. A server can use concurrent servers, each with one channel. How you create your design depends on whether you have a single CPU or a multiple CPU machine, and on your overall design goals and implementation requirements.

The design of an RTR client/server application should capitalize on the RTR multilayer model, separating client activity in the application from server and database activity. The RTR client software layers passes messages transparently to the router layer, and the router layer sends messages and transactions reliably and transparently, based on message content, to appropriate server processes. Server application software typically interacts with database software to update and query the database, and respond back to the client.

Most RTR calls complete asynchronously. Subsequent completion events are returned as messages to the application. However, the following calls can complete synchronously:

• rtr_receive_message()
• rtr_get_tid()
• rtr_error_text()
• rtr_set_user_handle()
• rtr_start_tx()

The client/server environment has both plusses and minuses. Performing processing on the client that does not need to be handled by the server is a plus, as it enables the client to perform tasks that the server need have no knowledge of, and need expend no resources to support. And with RTR as the medium for moving transactions from client to server, the application need not be concerned in detail about how the transactions are sent, or what to do in the event of node or site failures. RTR takes care of all required journaling and recovery without direct intervention by the application. The application needs no code to deal with server and link failures.

However, in a client/server environment, application design becomes more complex because the designer must consider what tasks the clients are to perform, and what the servers must do. Typically the client application will capture information entered by the user, while the server interacts with the database to update the database with transactions passed to it by the router.

The RTR journal is always in play, recording transaction activity to persistent storage. The journal thus provides the capability of recovery from any single hardware failure. When a server no longer provides service, for example, when it goes offline, goes down, or is taken out of service temporarily, RTR continues to record transaction information in a journal. Then when the offline server returns to service, the appropriate journal entries are resent to the server and recorded in the server's database. Similarly, when doing transactional shadowing, transactions that must be recorded in two shadowed databases are kept in synchronization. With RTR Version 3.2, journaling on frontends is required to support nested transactions.

If transactions do not update the database, specify them as read-only by using the RTR_F_SEN_READONLY flag on the rtr_send_to_server call. This will minimize journaling activity in a shadowed environment. RTR journaling assumes that writes are atomic at the 512-byte block level.

With RTR, client/server messaging enables the application to send:

• Transactional messages
• Broadcast messages

With RTR, client and server applications communicate by exchanging messages in a transaction dialog. Transactional messages are grouped in a unit of work called a transaction. RTR takes ownership of a message when called by the application.

A transaction is a group of logically connected messages exchanged in a transaction dialog. Each dialog forms a transaction in which all participants have the opportunity to accept or reject the transaction. A transaction either succeeds or fails. When the transaction is complete, all participants are informed of the transaction’s completion status. The transaction succeeds if all participants accept it, but fails if even one participant rejects it.

In the context of a transaction, an RTR client application sends one or more messages to the server application, which responds with zero or more replies to the client application. Client messages can be grouped to form a transaction. All work within a transaction is either fully completed or all work is undone. This ensures transaction integrity from client initiation to database commit.

For example, say you want to take $20 from your checking account and add it to your savings account. With an application using RTR you are assured that this entire transaction is completed; you will not be left at the point where you have taken $20 from your checking account but it has not yet been deposited in your savings account. This feature of RTR is transactional integrity, illustrated in Figure 34, Transactional Messaging.

The transactional message is either all or nothing for each bracket (enclosed in brackets [ ] in the figure).

An RTR client application sends one or more messages to one or more server applications and receives zero or more responses from one or more server applications. For example:

• The client application: initiates the transaction, sends the messages, receives the replies, and ends the transaction. The client application must be on the RTR frontend, and can be accessed by an applet, for example on a PC.
• The server application: waits to receive the transaction, receives messages from the client, votes its acceptance of the transaction, commits the transaction. The server application is deployed on the RTR backends.

RTR generates a unique number, the transaction ID or tid, for each transaction.

Figure 35, Transactional Messaging Interaction, illustrates frontend/backend interaction with pseudo-code for transactions and shows transaction brackets. The transaction brackets show the steps in completing all parts of a transaction, working from left to right and top to bottom. The transaction (txn) is initiated at "Start txn" at the frontend, and completed after the "Commit txn" step on the backend. The transaction ID is encoded to ensure its uniqueness throughout the entire distributed system. In the prepare phase on the server, the application should lock the relevant database (db) records. The commit of a transaction hardens the commit to the database. The rtr_start_tx message specifies the characteristics of the transaction.

Figure 36, RTR API Calls for Transactional Messaging, shows the RTR API calls you use to achieve transactional messaging in your application.

Broadcast messaging lets client and server applications send non-transactional information to recipients. Recipients must be declared with the opposite role; that is, a client can send a broadcast message to servers, and a server can send a broadcast message to clients. Broadcasts are delivered on a ""best try"" basis; they are not guaranteed to reach every potential recipient. A broadcast message can be directed to a specific client or server, or be sent to all reachable recipients.

This point-to-point messaging using broadcast semantics is a feature to use rather than transactions when the information being sent is not recorded as a transaction in the database, and when you need to send information to several clients (or servers) simultaneously. For example, in a stock trading environment, when a trade has been completed and the stock price has changed, the application can use a broadcast message to send the new stock price to all trading stations.

With location transparency, applications do not need to be modified when the hardware configuration is altered, whether changes are made to systems running RTR services or to the network topology. Client and server applications do not know the location of one another so services can be started anywhere in the network. Actual configuration binding is a system management operation at run time.

Because RTR automatically takes care of failover, applications need not be concerned with specifying the location of server resources.

With RTR, applications can use data content routing to partition separate transactions to different database segments. An application achieves data content or data partition routing using key ranges.

RTR enables an application to route transactions by partition or key range, rather than sending all transactions processed by an application to a single database disk.

To plan for future expansion, consider using compound keys rather than single field keys. For example, using a bank example, with a bank that has multiple branches, an application that routes data to each bank can use a BankID key field or partition. Over time, the application may also need to further subdivide transactions not only by bank but also by customer ID. If the application is initially written with a compound key providing both a BankID and a CustomerID key, it can be simpler to make such a change in future.

An application can be multithreaded. Check the RTR Reliable Transaction Router Release Notes and SPD for the current extent of support for multithreaded programming.

For a client, an application typically uses the following calls in this order:

• rtr_open_channel
• rtr_receive_message() one per channel
• rtr_start_tx()
• rtr_send_to_server() [one or more,
  zero or more receive messages for the per transaction
  expected number of replies]
• rtr_accept_tx()
• rtr_receive_message()

For a server, an application typically uses the following calls in this order:

• rtr_open_channel()
• rtr_receive_message() one per channel
• rtr_receive_message()
  [one or more]
• rtr_reply_to_client() [zero or more]
• rtr_accept_tx() per transaction
• rtr_receive_message()

The rtr_receive_message call returns a message and a message status block (MsgStatusBlock). For example,

The message status block contains the message type, the user handle, if any, the message length, the tid, and the event number, if the message type is rtr_mt_rtr_event or rtr_mt_user_event. For more information on the message status block, see the descriptions of rtr_receive_message and rtr_set_user_handle in the Reliable Transaction Router Application Programmer’s Reference Manual.

With the RTR SET TRANSACTION command and the rtr_set_info API call, users can change a transaction state in their application servers.

Consider the following scenario: upon receiving an mt_accepted message indicating that a transaction branch is committed, the server normally performs an SQL commit to write all changes to the underlying database. However, the server that is to detect the SQL commit would crash the server or the underlying database is not available temporarily.

In V3.2, users can now change transaction's state to "EXCEPTION" using rtr_set_info to temporarily put this transaction in the exception queue. When things get better, users can call rtr_set_info to change the transaction state back to "Commit" or "DONE".

The following example shows how a transaction state can be changed using rtr_set_info. Also refer to rtr_set_info documentation in the Reliable Transaction Router Application Programmer's Reference Manual for more details.

Normally at the RTR command level, users need to provide at least facility name and partition name qualifiers to help RTR select a desired set of transactions to be processed. Because a transaction's tid (typed rtr_tid_t) is unambiguously unique, a user needs only to specify the transaction's current state and its tid.

Note that if a transaction has multiple branches running on different partitions simultaneously on this node, RTR will reject this set transaction request with an error. RTR SET TRANSACTION can only change state for a transaction running in one partition.

RTR calls and responses to them contain RTR message types (mt) such as rtr_mt_reply or rtr_mt_rejected. There are four groups of message types:

• Transactional
• Transactional status
• Event-related
• Informational

The following table lists all RTR message types.

Applications should include code for all RTR expected return message types. Message types are returned to the application in the message status block. For more detail on message types see the Reliable Transaction Router Application Programmer's Reference Manual.

To work in a mixed-operating system environment, an application can specify a message format definition on the following calls:

• rtr_send_to_server
• rtr_reply_to_client
• rtr_broadcast_event

RTR does data marshalling, evaluating and converting data appropriately as directed by the message format descriptions provided in the application.

The following example shows an RTR application using a message format declaration; first the RTR call specifying that TXN_MSG_FMT contains the actual format declaration, then the definition used by the call.

RTR application call:

Data definition:

#define TXN_MSG_FMT "%1C%UL%7UL%31C"

This data structure accommodates an 8-bit signed character field ( C ) for the key, a 32-bit unsigned field (UL) for the server number, a 224-bit (7x32) field (7UL) for the transaction ID, and a 31-bit byte (248-bit) field (31C) for character text. For details of defining message format for a mixed-endian environment, see the Reliable Transaction Router Application Programmer’s Reference Manual.

You use the XA protocol supported by RTR to communicate with database managers. This eliminates the need for an application program to process rtr_mt_uncertain messages.

} while(...)

EXEC SQL COMMIT WORK RELEASE;
exit(0);
}

The XA software architecture of RTR provides interoperability with the Distributed Transaction Controller of Microsoft, MS DTC. Thus RTR users can develop application programs that update MS SQL databases, MS MQ, or other Microsoft resource managers under the control of a true distributed transaction. RTR as a distributed transaction manager communicates directly with MS DTC to manage a transaction or perform recovery using the XA protocol. For each standard XA call received from RTR, MS DTC translates it into a corresponding OLE transaction call that SQL Server or MS MQ expects to perform database updates. This is shown in Figure 40, MS DTC and RTR.

For example, using XA and DTC (DIGITAL Compaq Tru64 UNIX and Microsoft Windows NT only) eliminates the need to process uncertain messages (rtr_mt_msg1_uncertain). To use the XA protocol with RTR, you do the following:

• Use the RTR REGISTER RM (register resource manager) command to register the resource manager, specifying the appropriate libraries. For Oracle 7RDBMS, the V$XATRANS$ view must exist on the database.
• Create a facility with the referenced resource manager by executing the CREATE FACILITY/RM=xxx command, where xxx is the same as that used in the previous REGISTER RM command. For more details on the REGISTER RM command and other commands for working with other database managers, see the Reliable Transaction Router System Manager’s Manual.
• In the rtr_open_channel call in the server application, specify which resource manager the application is using. The application must submit a resource manager instance name with [OPEN_STRING] to the rtr_open_channel call before accessing the desired database.
• Remove unnecessary SQL calls from existing server code such as commit or rollback, to avoid vendor-specific warnings.

Both the resource manager instance name and the database (RM) name in [OPEN-STRING] must be identical to that in the previously executed REGISTER RM command. The information is stored in the RTR key segment structure, and the RTR_F_OPE_XA_MANAGED flag associates the channel with the XA interface.

Only one transaction at a time is processed on an RTR channel; thus a server process or thread of control can only open one channel to handle a single XA request. Better throughput may be achieved by using a multithreaded application.

For example, the following snippets from a sample server applications show (in boldface type) use of the RM key, the XA flag, and commenting out commits and rollbacks for the Oracle and DTC environments.

The following XA/DTC server application example is for a Windows NT environment only.

} while(...)

EXEC SQL COMMIT WORK RELEASE;
exit(0);
}

The following XA/DTC server application example is for a Windows NT environment only.

*/
case rtr_mt_msgn:
switch(msg.txn_type)
{
case
...
dbfcmd(dbproc, "...");
dbsqlexec(dbproc);
while(1) {
dbresults(dbproc);
...
break;
}

You can use the DECdtm protocol to communicate with OpenVMS Rdb. This provides a two-phase commit capability. For additional information on using this protocol, see the OpenVMS documentation, for example, Managing DECdtm Services in the OpenVMS System Manager's Manual, and the Oracle Rdb Guide to Distributed Transactions available from Oracle.

An RTR transaction can be part of a transaction that is coordinated by a parent transaction manager such as RTR itself, or Tuxedo, or MS DTC. To implement nested transactions, the application uses the following RTR API calls:

• rtr_open_channel with the RTR_F_OPE_FOREIGN_TM flag set on the client opens a client channel. With this call, a parent transaction manager such as RTR, DTC, or Tuxedo establishes a session with RTR, when it wishes to start an RTR nested transaction as part of a transaction controlled by the parent transaction manager.
• rtr_start_tx using the following:
• jointxid = RTR TXID, if the parent transaction manager is RTR,
• jointxid=DTC TID, if the parent transaction manager is DTC,
• jointxid=XA tid , if the parent transaction manager is Tuxedo or another transaction manager using the XA TM/RM interface.

• rtr_send_to_server to send the actual transaction data
• rtr_prepare_tx to start the first phase of the two-phase commit process. If the first phase is successful, then a message type of rtr_mt_prepared is delivered by RTR on the client channel. Otherwise, an rtr_mt_rejected message type is delivered.
• rtr_accept_tx or rtr_reject_tx to commit or roll back the transaction after the rtr_mt_prepared message has been received from RTR on the parent (server) channel. , which implicitly This completes the commit phase of the nested transaction.

rtr_open_channel using the RTR_F_OPE_FOREIGN_TM flag on the client

For additional details on the syntax and use of these calls, see the RTR Reliable Transaction Router Application Programmer’s Reference Manual.

When using nested transactions, a journal is required on the frontend node, if that node does not already support a frontend.

RTR lets transactions be embedded within other transactions; such transactions are called nested transactions or subtransactions. A nested transaction is considered indivisible within its enclosing transaction, typically coordinated by a parent transaction manager.

A transaction that is not nested is called a top-level transaction. A nested transaction is a child of its parent (enclosing) transaction. A parent may have several children who are siblings; ancestor and descendent relationships apply. A top-level transaction and its descendants are collectively known as a transaction family or a family.

A nested transaction must be strictly nested within its enclosing transaction; it must be completed (committed or aborted) before the enclosing transaction can complete. If the enclosing transaction aborts, all effects of the nested transaction are also undone.

A transaction can create several child transactions; the parent transaction performs no work until all child transactions are complete. A transaction cannot, however, observe the effects of a sibling transaction until that sibling completes.

Nested transactions isolate the effect of failures from the enclosing transaction and from other concurrent transactions. A nested transaction that has not completed can abort without causing its parent transaction to abort.

Committed nested transactions are durable (permanent) only with respect to certain other transactions: a committed child is permanent with respect to its parent. To stop a committed child, the parent transaction is stopped. The child is said to be committed with respect to its parent, or with respect to its siblings. Every transaction is committed with respect to itself and its descendants. To abort a committed nested transaction, all of its committed-with-respect-to transactions must be aborted.

To pass transactions from client to server, RTR uses channels as identifiers. Each application communicates with RTR on a particular channel. In a multithreaded application, when multiple transactions are outstanding, the channel is the means through which the application informs RTR which transaction a command is for.

With RTR, the client or server application can :

• Open a channel
• Send a transaction to a channel
• Bind a transaction to a channel
• Receive a transaction on a channel
• Specify a receive programming style (blocked, polled, or signaled)
• Close a channel

To open a channel, the application uses the rtr_open_channel call. This opens a channel for communication with a client or server application on a specific facility. Each application process can open up to 255 channels.

For example, the rtr_open_channel call in this client application opens a single channel for the facility called DESIGN:

The application uses parameters on the rtr_open_channel call to define the application environment. Typically, the application defines the following:

• The role of the application (client or server)
• The name of the application facility [1]
• The facility access key, a password; in this case no password is used [2]
• The key range or key segment for data content routing [3]

For a server application, the rtr_open_channel call additionally supplies the number of key segments, numseg, and the partition name, in pkeyseg.

The syntax of the rtr_open_channel call is as follows:

You can set up a variable section in your client program to define the required parameters and then set up your rtr_open_channel call to pass those parameters. For example, the variables definition would contain something like the following:

The rtr_open_channel call would contain:

You will find more complete samples of client and server code in the appendix of this document and on the RTR software kit in the [EXAMPLES] directory.

To specify the location to return the channel identifier, use the channel argument in the rtr_open_channel call. For example,

or
rtr_channel_t
*p_channel = &channel;

This parameter points to a valid channel identifier when the application receives an rtr_mt_opened message.

To define the application role type (client or server), use the flags parameter. For example,

or

The facility name is a required string supplied by the application. It identifies the RTR facility used by the application. The default facility name for the RTR CLI only is RTR$DEFAULT_FACILITY; there is no default facility name for an RTR application. You must supply one.

To define the facility name, use the facnam parameter. For example,

To specify a recipient name, use the rcpnam parameter, which is case sensitive. For example,
rtr_rcpnam_t
rcpnam = "* Rogers";

To specify user event numbers, use the evtnum parameter. For example,

rtr_evtnum_t all user_events[]={
RTR_EVTNUM_USERDEF,
RTR_EVTNUM_USERBASE,
RTR_EVTNUM_UP_TO,
RTR_EVTNUM_USERMAX,
RTR_EVTNUM_ENDLIST
};
There are both RTR events and user events. For more information on employing events, see Broadcast Messaging Processes, page *, and the section on RTR Events in the Reliable Transaction Router Application Programmer's Reference Manual.

You can use the facility access key to restrict client or server access to a facility. The key acts as a password to restrict access to the specific facility for which it is declared.

To define the facility access key, use the access parameter. For example,

The facility access key is a string supplied by the application. The first server channel in an RTR facility defines the access key; all subsequent server and client open channel requests must specify the same access value. To use no access key, use RTR_NO_ACCESS or NULL for the access argument.

To specify the number of key segments defined for a server application, use the numseg parameter. For example,

To specify the key range for a partition to do data content routing, the server application defines the routing key when it opens a channel on a facility with the rtr_open_channel call. All servers in a facility must specify the same offset, length, and data type for the key segments in the rtr_open_channel call; only high and low bounds (*ks_lo_bound, *ks_hi_bound) can be unique to each server key segment. By application convention, the client places key data in the message at the offset, length, and data type defined by the server.

The channel-open operation completes asynchronously. Call completion status is returned in a subsequent message. RTR sends a message to the application indicating successful or unsuccessful completion; the application receives the status message using an rtr_receive_message call. If status is rtr_mt_opened, the open operation is successful. If status is rtr_mt_closed, the open operation is unsuccessful, and the application must examine the failure and respond accordingly. The channel is closed.

Data returned in the user buffer with rtr_mt_opened and rtr_mt_closed include both the status and a reason. For example:

Use the call rtr_error_text to find the meaning of returned status. A client channel will receive no message at all if a facility is configured but no server is available. Once a server becomes available, RTR sends the rtr_mt_accepted message.

To close a channel, the application uses the rtr_close_channel call. A channel can be closed at any time after it has been opened. Once closed, no further operations can be performed on a channel, and no further messages for the channel are received.

To receive on a channel and obtain status information from RTR, use the rtr_receive_message call. To receive on any open channel, use the RTR_ANYCHAN value for the p_rcvchan parameter in the rtr_receive_message call. To receive from a list of channels, use the p_rcvchan parameter as a pointer to a list of channels, ending the list with RTR_CHAN_ENDLIST. An application can receive on one or more opened channels. RTR returns the channel identifier. A pointer to a channel is supplied on the rtr_open_channel call, and RTR returns the channel id identification (ID) by filling in the contents of that pointer.

To simplify matching an RTR channel ID with an application thread, an application can associate a user handle with a channel. The handle is returned in the message status block with the rtr_receive_message call. The application can use the message status block (MsgStatusBlock) to identify the message type of a received message. For example,

RTR delivers both RTR and application messages when the rtr_receive_message call completes. The application can use the message type indicator in the message status block to determine relevant buffer format. For further details on using message types and interpreting the contents of the user buffer, see the Reliable Transaction Router Application Programmer's Reference Manual.

An application can specify one of three reception styles for the rtr_receive_message call. These are:

• Blocked
• Polled
• Signaled

An application can use a blocking receive to wait until a message arrives. To use a blocking receive, include RTR__NO_TIMOUTMS in the rtr_receive_message call. The call completes only when a message is available on one of the specified channels. For example,

An application can use a polled receive to poll RTR with a specified timeout. To use a polled receive, the application can set a value in milliseconds on the timoutms parameter.

The timeout can be:

• Immediate: timoutms = 0 (Do not wait for messages; RTR notifies the application if there are any.)
• Infinite: RTR_NO_TIMOUTMS
• A user-specified value. Current clock granularity is one 1 second (1000 ms).

The call completes after the specified timeout or when a message is available on one of the specified channels.

For example, the following declaration sets polling at 1 second (1000 milliseconds).

An application can use a signaled receive to be alerted by RTR when a message is received. The application establishes a signal handler using the rtr_set_wakeup call, informing RTR where to call it back when the message is ready.

To use a signaled receive, the application uses the rtr_set_wakeup call and provides the address of a routine to be called by RTR when a message is available. When the wakeup routine is called, the application can use the rtr_receive_message call to get the message. For example,

When using rtr_set_wakeup in a multithreaded application, be careful not to call any non-re-entrant functions or tie up system resources unnecessarily inside the callback routine.

The rtr_open_channel parameters are further described in the Reliable Transaction Router Application Programmer's Reference Manual.

There are two ways to start a transaction:

• Explicitly, using rtr_start_tx
• Implicitly, using rtr_send_to_server

Use the rtr_start_tx call when the application must set a client-side transaction timeout to ensure that both client and server do not wait too long for a message. When a transaction is started with rtr_send_to_server, no timeout is specified.

For example:

The rtr_send_to_server() call sends a message as part of a transaction from a client. If there is no transaction currently active on the channel, a new one is started. The transaction accept can be bundled with the last message. A client has two options for message delivery after a failure:

• Return to sender

• Accept transaction

The rtr_reply_to_client() call sends a reply message from a server to the client. The reply message is part of the transaction initiated by the client. For example:,

The reply message format can be of any form as designed by the application. For example:

When an application receives a message with the rtr_receive_message() call, the message status block (MsgStatusBlock) contains the transaction identifier. For example:

The pointer &MsgStatusBlock points to the message status block that describes the received message. For example:,

Use the rtr_get_tid call to obtain the RTR transaction identifier for the current transaction. (The tid (transaction identifier) is a unique number generated by RTR for each transaction.) The application can use the tid if the client needs to know the tid to take some action before receiving a response.

Use the rtr_set_user_handle call to set a user handle on a transaction rather than by channel. A client application with multiple transactions outstanding can match a reply or completion status with the appropriate transaction by establishing a new user handle each time a transaction is started.

A server application ends a transaction by accepting or rejecting it. A transaction is accepted explicitly with the rtr_accept_tx call, and rejected explicitly with the rtr_reject_tx call. RTR can reject a transaction at any time once the transaction is started but before it is committed. If RTR cannot deliver a transaction to its destination, it rejects the transaction explicitly and delivers the reject completion status to all participants.

A transaction participant can specify a reason for an accept or reject on the rtr_accept_tx and rtr_reject_tx call. If more than one transaction participant specifies a reason, RTR uses the OR operator to combine the reason values together. For example, with two servers A and B, each providing a reason code of 1 and 2, respectively, the client receives the result of the OR operation, reason code 3, in its message buffer.

The client receives the results of the OR operation in its message buffer.

A transaction is done once a client or server application receives a completion message, either an rtr_mt_closed, rtr_mt_accepted, or rtr_mt_rejected message from RTR. An application no longer receives messages related to a transaction after receiving a completion message or if the application calls rtr_reject_tx. A client or server can also specify RTR_F_ACC_FORGET on the rtr_accept_tx call to signal its acceptance and end its involvement in a transaction early. RTR returns no more messages (including completion messages) associated with the transaction; any such messages received will be returned to the caller.

When issuing the rtr_accept_tx call with RTR_NO_FLAGS on the call, the caller expresses its request for successful completion of the transaction, and may give an accept ""reason"" that is passed on to all participants in the transaction. The accept is final: the caller cannot reject the transaction later. The caller cannot send any more messages for this transaction.

A client can accept a transaction in one of two ways: with the rtr_accept_tx call or by using the RTR_F_SEN_ACCEPT flag on the rtr_send_to_server call.

When the client sets RTR_F_SEN_ACCEPT on the rtr_send_to_server call, this removes the need to issue an rtr_accept_tx call and can help optimization of client traffic. Merging the data and accept messages in one call puts them in a single network packet. This can make better use of network resources and improve throughput.

The rtr_reject_tx call rejects a transaction. Any participant in a transaction can call rtr_reject_tx. The reject is final; the caller cannot accept the transaction later. The caller can specify a reject "reason" that is passed to all accepting participants of the transaction. Once the transaction has been rejected, the caller receives no more messages for this transaction.

The server can set the retry flag RTR_F_REJ_RETRY to have RTR redeliver the transaction beginning with msg1 without aborting the transaction for other participants. Issuing an rtr_reject_tx call with this flag can let another transaction proceed if locks held by this transaction cause a database deadlock.

RTR provides client applications the option to specify a transaction timeout, but has no provision for server applications to specify a timeout on transaction duration. If there is a scarcity of server application processes, then all other client transactions remain queued. If these transactions have also specified timeouts, then these would be aborted by RTR (assuming that the timeout value is less than 2 minutes).

To get around this problem, the application designer has two options:

• Use concurrent server processes
• Have the application administer its own timeout

The first (and easier) option is to use concurrent server processes. This allows transaction requests to be serviced by other free servers, even if one server is occupied by such a transaction which is taking a long time to disappear. The second option is to design the server application so that it can abort the transaction independently.

There are three cases where this use of concurrent servers is not ideal. First, there is an implicit assumption about how many such lingering transactions might remain on the system. In the worst case, this could exceed or equal the number of client processes. But having so many concurrent server processes to cater to this contingency is wasteful of system resources. Second, use of concurrent servers is beneficial when the servers do not need to access a common resource. For instance, if all these servers needed to update the same record in the database, they would simply be waiting on a lock taken by the first server. Additional servers do not resolve this issue. Third, it must make business sense to have additional servers. For example, if transactions must be executed in the exact order in which they entered the system, concurrent servers may introduce sequencing problems.

Take the example of the order matcher in a stock trading application. Business rules may dictate that orders be matched on a first-come first-matched basis; using concurrent servers would negate this rule.

The second option is to let the server application process administer its own timeout and abort the transaction when it sees no activity on its input stream. RTR provides a relatively simple way to do this in the server. Use of timeout values on the rtr_receive_message function lets an server application specify how long it is prepared to wait for the next message. (Of course, the server should be prepared to wait forever to get a new transaction or for the result of an already-voted transaction.)

One way to achieve this would be to have a channel-specific global variable, say, called SERVER_INACTIVITY_TIMEOUT, which is set to the desired value (in milliseconds—that is, use a value of 5000 to set a 5 second timeout). Note that this timeout value should be used AFTER receiving the first message of the transaction. The value should be reset to RTR_NO_TIMOUTMS after receiving the rtr_mt_prepare message. Whenever the rtr_receive_message completes with a RTR_STS_TIMOUT, the server calls the rtr_reject_tx function on that channel to abort the partially-processed transaction. This would prevent transactions from occupying the server process beyond a reasonable time.

RTR uses the two-phase commit process for committing a transaction, with a prepare phase and a commit phase. Transactions must reach the commit phase before they are hardened in the database.

Transactions are prepared before being committed by accept processing. Transaction preparation consists of several steps represented by transaction states as seen by an RTR backend:

The RTR frontend sees several transaction states during accept processing:

To initiate the prepare phase, the application specifies the RTR_F_OPE_EXPLICIT_PREPARE flag when opening the channel, and can use the message rtr_mt_prepare to check commit status. The message indicates to the server application that it is time to prepare any updates for a later commit or rollback operation. RTR lets the server application explicitly accept a transaction using the RTR_F_OPE_EXPLICIT_ACCEPT flag on the rtr_open_channel call. Alternatively, RTR implicitly accepts the transaction after the rtr_mt_accepted message is received when the server issues its next rtr_receive_message call.

The two-phase commit process is initiated by the client application when it issues a call to RTR indicating that the client "accepts" the transaction. This does not mean that the transaction is fully accepted, only that the client is prepared to accept it. RTR then asks the server applications participating in the transaction if they are prepared to accept the transaction. A server application that is prepared to accept the transaction votes its intention by issuing the rtr_accept_tx call, an " accept" vote. A server application that is not prepared to accept the transaction issues the rtr_reject_tx call, a "not accept" vote. Issuing all votes concludes the prepare phase.

When RTR has collected all votes from all participating server applications, it determines if the transaction is to be committed. If all collected votes are "accept," the transaction is committed; RTR informs all participating channels. If any vote is "not accept," the transaction is not committed. A server application can expose the prepare phase of two-phase commit by using the rtr_mt_prepare message type with the RTR_F_OPE_EXPLICIT_PREPARE flag. If the application’s rtr_open_channel call sets neither RTR_F_OPE_EXPLICIT_ACCEPT nor RTR_F_OPE_EXPLICIT_PREPARE flags, then both prepare and accept processing are implicit.

The server application can participate in the two-phase commit process fully, somewhat, a little, or not at all. To participate fully, the server does an explicit prepare and an explicit accept of the transaction; to participate somewhat, the server does an explicit prepare and an implicit accept of the transaction; to participate a little, the server does an explicit accept of the transaction; to participate not at all, the server does an implicit accept of the transaction. The following table summarizes level of server participation:

Your application can use the level of participation that makes the most sense for your business and operations needs.

To request an explicit accept and explicit prepare of transactions, the server channel is opened with the RTR_F_OPE_EXPLICIT_PREPARE and RTR_F_OPE_EXPLICIT_ACCEPT flags. These specify that the channel will receive both prepare and accept messages. The server then explicitly accepts or rejects a transaction when it receives the prepare message. The transaction sequence for an explicit prepare and explicit accept is as follows:

Client RTR Server
rtr_start_tx
rtr_send_to_server à rtr_mt_msg1 à rtr_receive_message
rtr_accept_tx à rtr_mt_prepare à rtr_receive_message
ß rtr_accept_tx
rtr_receive_message ß rtr_mt_accepted ß rtr_receive_message

With explicit transaction handling, the following steps occur:

The server application waits for a message from the client application.

The server application receives the rtr_mt_prepare request message from RTR.

The server application issues the accept or reject.

A participant can reject the transaction up to the time RTR has sent the rtr_mt_prepare message type to the server in the rtr_accept_tx call. A participant can reject the transaction up to the time the rtr_accept_tx call is executed. Once the client application has called rtr_accept_tx, the result cannot be changed.

The sequence for an implicit prepare and explicit accept is as follows:

Client RTR Server
rtr_start_tx
rtr_send_to_server à rtr_mt_msg1 à rtr_receive_message
rtr_accept_tx à ß rtr_accept_tx
rtr_receive_message ß rtr_mt_accepted ß rtr_receive_message

In certain database applications, where the database manager does not let an application explicitly prepare the database, transactions can simply be accepted or rejected. Server applications that do not specify the RTR_F_EXPLICIT_ACCEPT flag in their rtr_open_channel call implicitly accept the in-progress transaction when an rtr_receive_message call is issued after the last message has been received for the transaction. This call returns the final status for the transaction, rtr_mt_accepted or rtr_mt_rejected. If neither the RTR_F_OPE_EXPLICIT_ACCEPT nor the RTR_F_OPE_EXPLICT_PREPARE flags are set in the rtr_open_channel call, then both prepare and accept processing will be implicit.

For server optimization, the server can signal its acceptance of a transaction with either rtr_reply_to_client, using the RTR_F_REP_ACCEPT flag, or with the client issuing the rtr_send_to_server call, using the RTR_F_SEN_ACCEPT flag. This helps to minimize network traffic for transactions by increasing the likelihood that the data message and the RTR accept message will be sent in the same network packet.

When a transaction fails in progress, RTR provides recovery support using RTR replay technology. RTR, as a distributed transaction manager, communicates with a database resource manager in directing the two-phase commit process. When using the XA protocol, RTR does not need to process rtr_mt_uncertain messages (see the section Using XA, page *, for more details on using XA).

The typical server application transaction sequence for committing a transaction to the database is as follows:

This sequence is also illustrated in Figure 26, CSN Vote Window.

A failure can occur at any step in this sequence; the impact of a failure depends on when (at which step) it occurs, and on the server configuration.

If a failure occurs before the rtr_accept_tx call is issued, RTR causes the following to occur:

• With no database replication: the transaction is aborted and the database rolled back.
• With a concurrent server: the database is rolled back and the transaction replayed to another server instance.
• With a standby server: the database is rolled back and the transaction replayed to the standby.
• With a shadow server: the shadow server completes the transaction as a lone member. When the failed server returns, the database is rolled back and the transaction is replayed.

If a failure occurs after the rtr_accept_tx call is issued but before the rtr_receive_message, the transaction is replayed. The type of the first message is then rtr_mt_uncertain when the server is restarted. Servers should check to see if the transaction has already been executed in a previous presentation. If not, it is safe to re-execute the transaction since because the database operation never occurred. After the failure, the following occurs:

• With no database replication: the database is rolled back and the transaction is replayed as uncertain. The server should re-execute the transaction.
• With a concurrent server: the database is rolled back and the transaction is replayed as uncertain to another concurrent server. The server should re-execute the transaction.
• With a standby server: the database is rolled back and the transaction is replayed as uncertain to the standby server. The server should re-execute the transaction.
• With a shadow server: the shadow server completes the transaction as a lone member. When the failed server returns, the database is rolled back and the transaction is replayed as uncertain. The server should re-execute the transaction.

If a failure occurs after the SQL commit but before receipt of a message starting the next transaction, RTR does not know the difference.

If a failure occurs after an rtr_receive_message() call is made to begin a new transaction, RTR assumes a successful commit if a server calls rtr_receive_message after receiving the rtr_mt_accepted message and will forget the transaction. There is no replay following these events.

RTR keeps track of how many times a transaction is presented to a server application before it is VOTED. The rule is: three strikes and you’re out! After the third strike, RTR rejects the transaction with the reason RTR_STS_SRVDIED. The server application has committed the transaction and the client believes that the transaction is committed. The transaction is flagged as an exception and the database is not committed. Such an exception transaction can be manually committed if necessary. This process eliminates the possibility that a single rogue transaction can crash all available copies of a server application at both primary and secondary sites.

Application design can change this behavior. The application can specify the retry count to use when in recovery using the /recovery_retry_count qualifier in the rtr_set_info call, or the system administrator can set the retry count from the RTR CLI with the SET PARTITION command. If no recovery retry count is specified, RTR retries replay three time. For recovery, retries are infinite. For more information on the SET PARTITION command, see the Reliable Transaction Router System Manager's Manual; for more information on the rtr_set_info call, see the Reliable Transaction Router Application Programmer's Reference Manual.

When a node is down, the operator can select a different backend to use for the server restart. To complete any incomplete transactions, RTR searches the journals of all backend nodes of a facility for any transactions for the key range specified in the server’s rtr_open_channel call.

A client or server application may need to send unsolicited messages to one or more participants. Applications tell RTR which broadcast classes they want to receive.

The sender sends one message received by several recipients. Recipients subscribe to a specific type of message. Delivery is not guaranteed.

Broadcast messages can be:

• Application messages from client to server applications
• Application messages from server to client applications
• One-to-many or one-to-one

Client channels cannot broadcast to other client channels, and server channels cannot broadcast to other server channels. To enable communication between two applications of the same type, open a second channel of the other type. Messaging destination names can include wildcards, enabling flexible definition of the subset of recipients for a particular broadcast.

Use the rtr_broadcast_event call to broadcast a user event message.

Broadcast types include user events and RTR events; both are numbered.

Event numbers are provided as a list beginning with RTR_EVTNUM_USERDEF and ending with RTR_EVTNUM_ENDLIST. To subscribe to all user events, an application can use the range indicators RTR_EVTNUM_USERBASE and RTR_EVTNUM_USERMAX, separated by RTR_EVTNUM_UP_TO, to specify all possible user event numbers.

A user broadcast is named or unnamed. An unnamed broadcast does a match on user event number; the event number completely describes the event. A named broadcast does a match on user event number and recipient name. The recipient name is a user-defined string. Named broadcasts provide greater control over who receives a particular broadcast.

Named events specify an event number and a textual recipient name. The name can include wildcards (% and *).

For all unnamed events specify the evtnum field and RTR_NO_RCPSPC as the recipient name.

For example, the following snippet specifies named events for all subscribers:

rtr_status_t
rtr_open_channel {
…
rtr_rcpnam_t rcpnam = "*";
rtr_evtnum_t evtnum = {
RTR_EVTNUM_USERDEF,
RTR_EVTNUM_USERBASE,
RTR_EVTNUM_UP_TO,
RTR_EVTNUM_USRMAX,
RTR_EVTNUM_ENDLIST
};
rtr_evtnum_t *p_evtnum = &evtnum;

For a broadcast to be received by an application, the recipient name specified by the subscribing application on its rtr_open_channel call must match the recipient specifier used by the broadcast sender on the rtr_broadcast_event call.

An application receives broadcasts with the rtr_receive_message call. A message type returned in the message status block informs the application of the type of broadcast received. For example:,

The user event would be in msgsb.msgtype == rtr_mt_user_event. User broadcasts can also contain a broadcast message. This message is returned in the message buffer provided by the application. The size of the user’s buffer is determined by the maxlen field. The number of bytes actually received is returned by RTR in the msglen field of the message status block.

RTR delivers status information to which client and server applications can subscribe. Status information is delivered as messages, where the type of each message is an RTR event.

RTR events are numbered. The base value for RTR events is defined by the symbol RTR_EVTNUM_RTRBASE; its maximum value is defined by the symbol RTR_EVTNUM_RTRMAX. RTR events and event numbers are listed in the Reliable Transaction Router Application Programmer's Reference Manual and in the RTR header file rtr.h.

An application can subscribe to RTR events to receive notification of external events that are of interest to the application. For example, a shadow server may need to know if it is a primary or a secondary server to perform certain work, such as uploading information to a central database, that is done at only one site. An application subscribes to an RTR event with the rtr_open_channel call. For example:

To subscribe to all RTR events, use the range indicators RTR_EVTNUM_RTRBASE and RTR_EVTNUM_RTRMAX.

RTR events are delivered as messages of type _mt.rtr.event. You read the message type to determine what RTR has delivered. For example:,

Use a data structure of the following form to receive the message:

The event number is returned in the message status block in the evtnum field. The following RTR events return key range data back to the client application:

• RTR_EVTNUM_KEYRANGEGAIN
• RTR_EVTNUMKEYRANGELOSS

These data are appended to the end of the rtr_msgsb_t structure; size is msglen_sizeof(rtr_msgsb_t). Other events do not have additional data.

In application design, you may wish to consider creating separate facilities for sending broadcasts. By separating broadcast notification from transactional traffic, performance improvements can be substantial. Facilities can further be reconfigured to place the RTR routers strategically to minimize wide area traffic.

RTR can provide information to an application with the rtr_error_text call. The rtr_error_text call translates RTR internal error message values to informational text that is more meaningful to the reader.

If an application encounters an error, it should log the error message received. Error messages are more fully described in rtr.h, where each error code is explained.

For example, the following short program uses the standard C library output function to output the text of an error status code.

Program "prog":

#include "rtr.h" or #include <rtr.h>

When this program is run, it produces the following output:

The several hundred error or status codes reside in the rtr.h header file; status codes can come from any RTR subsystem. A few codes that an application is likely to encounter are described in the following table:.

RTR can abort a transaction at any time, so the application must be prepared to deal with such aborted transactions. Server applications are expected to roll back transactions as the need arises, and must be built to take the correct action, and subsequently carry on to deal with new transactions that are received.

A client application can also get a reject and must also be built to deal with the likely cases it will encounter. The application must be built to decide on the correct course of action in the event of a transaction abort.

RTR callout servers enable security checks to be carried out on all requests using a facility. Callout servers can run on backend or router nodes. They receive a copy of every transaction delivered to or passing through the node, and they vote on every transaction. To enable a callout server, use the /CALLOUT qualifier when issuing the RTR CREATE FACILITY command. Callout servers are facility based, not key-range or partition based.

An application enables a callout server by setting a flag on the rtr_open_channel call.

For a router callout server, the application sets the following flag on the rtr_open_channel call:

For a backend callout server, the application sets the following flag on the rtr_open_channel call:

A deadlock or deadly embrace can occur when transactions lock data items in a database. The typical scenario is with two transactions txn1 and txn2 executing concurrently, with the following sequence of events:

1. txn1 write-locks data item A
2. txn2 write-locks data item B
3. txn1 requests a lock on data item B but must wait because txn2 still has a lock on data item B
4. txn2 requests a lock on data item A but must wait because txn1 still has a lock on data item A

Neither txn1 nor txn2 can proceed; they are in a deadly embrace. Figure 36 illustrates a deadly embrace.

With RTR, to avoid such deadlocks, follow these guidelines:

1. Always engage servers in the same order, and wait for the reply before each send.
2. Provide several concurrent servers to minimize contention. Estimate the number of concurrent servers needed by determining the volume of transactions the servers must support, considering periods of maximum activity, and allowing for growth. The larger the volume on your servers, the more likely it is that your application will benefit from using concurrent servers.

RTR attempts to resolve deadlocks by aborting one deadlocked transaction path with error code RTR_STS_DLKTXRES and replaying the transaction. Other paths are not affected. Server applications should be written to handle this status appropriately.

One method for improving response time is to send multiple messages from clients without waiting for a reply. The messages can be sent to different partitions to provide parallel processing of transactions.

For a standard GUI development tool that has no knowledge of RTR calls, RTR becomes the transport for ODBC calls transparent to the application. The backend must be running OpenVMS with Oracle7.

For certain read-only applications, RTR can be used without shadowing to establish sites to protect against site failure. The method is to define multiple non-overlapping facilities with the same facility name across a network. In the facility, define a failover list of routers; then a client is automatically reconnected to an alternate site when the local router fails.

Another method is to define a partition on a standby server for read-only transactions. This minimizes network traffic to the standby.

A read-only partition on a standby server can reduce node-to-node transaction locking.

Generally, databases and applications built to work with them are required to be idempotent. That is, given a specific state of the database, the same transaction applied many times would always produce the same result. Because RTR relies on replays and rollbacks, if there is a server failure before a transaction is committed, then RTR assumes the database will automatically roll back to the previous state, and the replayed transaction will produce results identical to the previous presentation of the transaction. RTR assumes that the database manager and server application provides idempotency.

For example, consider an internet transaction where you log into your bank account and transfer money from one account to another, perhaps from savings to checking. If you interrupt the transfer, and replay it two hours later, the transfer may not succeed because it would be required to have been done within a certain time interval after the login. Such a transaction is not idempotent.

Under some circumstances, client applications need to be blocked from accessing a server. For example, multiple client applications may receive a common feed, but the transactional update to the database should only be done only by one. The application can establish a partition lock or "deadman" using a second key range that only one client can access at a time. With the deadman, if two clients attempt to start a transaction on their shared key range, only one is started. The first client can then use the data channel and begin processing the feed, updating the database. The second client takes over only if the first client’s deadman is terminated.

In a heterogeneous environment, you can use RTR with several hardware architectures, both little endian and big endian. RTR provides the capability to do data marshalling in your application so that you can take advantage of such a mixed environment.

If you are constructing an application for a heterogeneous environment:

• Use RTR data marshalling for smooth transfer from one architecture to another.
• Do not use binary data, if at all possible (although you could adopt a convention for passing binary data between your machines).
• Make your applications as portable as possible, for example, adopting ANSI C as your programming language.
• Use C-style arguments.
• Check the network byte-order of your systems, and prepare your application accordingly.
• Check compiler settings and switches to ensure they produce consistent results (compilers may change spacing of messages based on how their switches are set).

With RTR, applications can run on systems from more than one vendor. You can mix operating systems with RTR, and all supported operating systems and hardware architectures can interoperate in the RTR environment. For example, you can have some nodes in your RTR configuration running OpenVMS and others running Windows NT.

To develop your applications in a multivendor environment:

Develop your application on one system, for example, on Windows NT using Microsoft Visual C++ following strict ANSI C implementation.

When both the server and client code are debugged, move them to the non-NTother system.

Build and debug it on the non-NTother system, that is not a Windows NT system.

If you have an existing application written using RTR Version 2 with OpenVMS, it will still operate with RTR Version 3. See the Reliable Transaction Router Migration Guide for pointers on using RTR Version 2 applications with RTR Version 3, and moving RTR Version 2 applications to RTR Version 3.

All client and application programs must be written using C, C++, or a language that can user RTR API calls. Include the RTR data types and error messages file rtr.h in your compilation so that it will be appropriately referenced by your application. For each client and server application, your compilation/link process is as follows:

1. Write your application code using RTR calls.
2. Use RTR data and status types for cross-platform interoperability.
3. Compile your application code calling in rtr.h using ANSI C include rules. For example, if rtr.h is in the same directory as your C code, use with the following statement: #include "rtr.h".
4. Link your object code with the RTR library to produce your application executable.

This process is illustrated in the following Figure 38, RTR Compile Sequence: