| Previous | Contents | Index |
A Word about RTR Data types
You may have noticed that your client, server and router can be on any one of many different operating systems. And you've probably written code for more than one operating system and noticed that each has a number of data types that the other doesn't have. If you send data between a Solaris UNIX machine and an OpenVMS or Windows NT machine, you'll also have to worry about the order different operating system stores bytes in their data types (called "endian" order). And what happens to the data when you send it from a 16 bit Intel 486 Windows machine to a 64 bit Alpha UNIX machine?
Thanks to RTR, you don't need to worry about it. RTR will handle everything for you. Just write standard C++ code that will compile on the machines you choose, and the run-time problems won't complicate your design. When you do this, you need to use RTR data types to describe your data. RTR translates everything necessary when your data gets to a new machine by converting the data to the native data types on the operating system with which it happens to be communicating at the time.
To illustrate this, the example code evaluates your input parameters and places them into an RTRData-derived RTRApplicationMessage object, ABCOrder. One sample application data class is ABCBook, which derives from ABCOrder. This subclass defines the data that is passed from client to server for a book order. This data class is defined in ABCBook.h and implemented in ABCBook.cpp
You'll notice that the data types which make up this object aren't your standard data types - they are RTR data types. And they are generic enough to be able to be used on any operating system: 8 bit unsigned, 32 bit unsigned, and a string.
UINT m_uiPrice;
UINT m_uiISBN;
CString m_csTitle;
CString m_csAuthor;)
unsigned int m_uiISBN;
unsigned int m_uiPrice;
char m_szTitle[ABCMAX_STRING_LEN];
char m_szAuthor[ABCMAX_STRING_LEN];
|
Send/Receive Message Loop
As mentioned earlier, an RTR client application typically contains a message loop that sends messages to the server via the RTR router, and handles messages that come from the server via the router, or from RTR itself.
This code illustrates how an application can retrieve and use the message from an RTRData derived object.
while (OrderBeingProcessed == eTxnResult)
{
sStatus = Receive(&pResult);
print_sStatus_on_failure(sStatus);
if ( true == pResult->IsRTRMessage())
{
// Check to see if we have a sStatus for the txn.
// rtr_mt_accepted = Server successfully processed our request.
// rtr_mt_rejected = Server could not process our request.
sStatus = ((RTRMessage*)pResult)->GetMessageType(mtMessageType);
print_sStatus_on_failure(sStatus);
if (rtr_mt_accepted == mtMessageType)
return eTxnResult = OrderSucceeded;
if (rtr_mt_rejected == mtMessageType)
return eTxnResult = OrderFailed;
}
sStatus = SendApplicationMessage( *pRTRApplicationMessage,
bReadonly = false,
bReturnToSender = false,
mfMessageFormat=RTR_NO_MSGFMT);
}
assert(RTR_STS_OK == sStatus);
|
The first message is sent to the server in the first parameter of the SendApplicationMessage call. As you will see, this is part of the flexibility and power of RTR. The parameter pRTRApplicationMessage is a pointer to a block of memory containing your data. RTR doesn't know what it's a pointer to, but it doesn't need to know this. You, as the programmer, are the only one who cares what it is. It's your own data object that carries any and all of the information your server will need in order to do your bidding. We'll see this in detail when we look at the server code.
You do not need to tell RTR how big the piece of memory being pointed to pRTRApplicationMessage is. The data object automatically lets RTR know how many bytes to move from your client machine to your server machine, so that your server application has access to the data being sent by the client.
And now, the client waits for a response from the server.
The client receives the server's reply or an rtr_mt_rejected and calls the client message handler method, OnRejected
sStatus = Receive( *pRTRData);
assert(RTR_STS_OK == sStatus);
|
Again you see the pRTRData parameter is a pointer to a data object created by you as the programmer, and can carry any information you need your server to be able to communicate back to the your client.
The RTRData object contains a code that tells you what kind of a message you are now receiving on your transaction controller. If the RTR message type contains the value rtr_mt_reply, then you are receiving a reply to a message you already sent, and your receive message object has been written to with information from your server.
sStatus = ((RTRMessage*)pResult)->GetMessageType(mtMessageType);
print_sStatus_on_failure(sStatus);
if (rtr_mt_accepted == mtMessageType)
return eTxnResult = OrderSucceeded;
if (rtr_mt_rejected == mtMessageType)
return eTxnResult = OrderFailed;
|
If GetMessageType contains the value rtr_mt_rejected, then something has happened that caused your transaction to fail after you sent it to the router. You can find out what that `something' is by looking at the sStatus returned by the Receive call. You will recall that making the rtr_error_text call and passing the sStatus value will return a human readable null terminated ASCII string containing the error message.
This is where you'll need to make a decision about what to do with this transaction. You can abort and exit the application, issue an error message and go onto the next message, or resend the message to the server. This code re-sends a rejected transaction to the server.
When your client application receives an rtr_mt_reply message, your message has come full circle. The client has made a request of the server on behalf of the user; the server has responded to this request. If you're satisfied that the transaction has completed successfully, you must notify RTR so that it can do its own housekeeping. To this point, this transaction has been considered "in progress", and its sStatus kept track of at all times. If all parties interested in this transaction (this includes the client AND the server) notify RTR that the transaction has been completed, RTR will stop tracking it, and confirm to all parties that it has been completed. This is called `voting'.
if (msgsb.msgtype == rtr_mt_reply)
{
sStatus = AcceptTransaction(RTR_NO_REASON)
assert (RTR_STS_OK == sStatus);
|
And now the client waits to find out what the result of the voting is.
sStatus = Receive( *pRTRData, RTR_NO_TIMOUTMS);
assert(RTR_STS_OK == sStatus);
|
If everyone voted to accept the transaction, the client can move on to the next one. But if one of the voters rejected the transaction, then another decision must be made regarding what to do about this transaction. This code attempts to send the transaction to the server again.
sStatus = ((RTRMessage*)pResult)->GetMessageType(mtMessageType);
print_status_on_failure(sStatus);
if (rtr_mt_accepted == mtMessageType) return eTxnResult =
OrderSucceeded;
if (rtr_mt_rejected == mtMessageType) return eTxnResult =
OrderFailed;
|
All of the requested messages, or transactions, have been sent to the server, and responded to. The only RTR cleanup we need to do before we exit the client is to close the transaction controller. This is similar to signing off, and RTR releases all of the resources it was holding for the client application.
Now, that wasn't so bad, was it? Of course not. And what has happened so far? The client application has sent a message to the server application. The server has responded. RTR has acted as the messenger by carrying the client's message and the server's response between them.
Next, let's see how the server gets these messages, and sends a response back to the client.
Server Application
The files shipped with the RTR kit used in the server application for this tutorial are the ABCOrderProcessor, ABCSHandlers and ABCSClassFactory files , in addition to the common files . These common files, including ABCCommon, ABCBook, and ABCMagazine are used in both client and the server applications. This is for a number of reasons, but most importantly that both the client and the server use the same definitions for the data objects they pass back and forth as messages. With the exception of only two items, there will be nothing in this server that you haven't already seen in the client. It's doing much the same things as the client application is doing. It creates a server transaction controller object for connecting to the router, telling the router that it is a server application; and then registers a partition. It waits to hear that the RegisterPartition request has been successfully executed; runs a loop that receives messages from the client; carries out the client's orders; sends the response back to the client. And the server gets to vote, too, on whether each message/response loop is completed.
One of the differences between the client andserver is the types of messages a server can receive from RTR; we'll go through some of them in this section of the tutorial about the server application.
The other difference is the RegisterPartition call which is sent to RTR. We mentioned partitions while discussing the client application, but said we'd discuss them later. Well, it's later...
Initialize RTR
The server creates a transaction controller and registers a partition. In addition, the server registers message and event handlers and a class factory, causing RTR to initialize a number of resources for use by the server, as well as to gather information about the server. In the Register methods in the server application, ABCOrderProcessor.cpp, you'll find the example server calling RegisterPartition. You see that the RegisterPartition method creates a single RTR data partition for each time it is called. In the server code, there are two partitions, ABCPartition1 and ABCPartition2.
sStatus = RegisterPartition(ABCPartition1);
print_sStatus_on_failure(sStatus);
sStatus = RegisterPartition(ABCPartition2);
print_sStatus_on_failure(sStatus);
|
In order to call RegisterPartition, the sample application includes a CreateRTREnvironment method that is first called in the ABCOrderProcessor::Register method.
Data Partitions
What is data partitioning, and why would you wish to take advantage of it? It is possible to run a server application on each of multiple backend machines, and to run multiple server applications on any backend machine. When a server registers a partition to begin communicating with the RTR router, it uses the KeySegment information to tell RTR that it is available to handle certain key segments. A key segment can be "all last names that start with A to K" and "all last names that start with L to Z", or "all user identification numbers from zero to 1000" and "all user identification numbers from 1001 to 2000".
In the sample application, the implementation is as follows:
void ABCOrderProcessor::CreateRTREnvironment()
{
rtr_sStatus_t sStatus;
// If RTR is not already started then start it now.
StartRTR();
// Create a Facility if not already created.
CreateFacility();
// Create a partition that processes ISBN numbers in
// the range 0 - 99
unsigned int low = 0;
unsigned int max = 99;
RTRKeySegment KeyZeroTo99(rtr_keyseg_unsigned,
sizeof(int),
0,
&low,
&max );
RTRPartitionManager PartitionManager;
sStatus = PartitionManager.CreateBackendPartition(ABCPartition1,
ABCFacility,
KeyZeroTo99,
false,
true,
false);
print_sStatus_on_failure(sStatus);
// Create a partition that processes ISBN numbers in
// the range 100 - 199
low = 100;
max = 199;
RTRKeySegment Key100To199( rtr_keyseg_unsigned,
sizeof(int),
0,
&low,
&max );
sStatus = PartitionManager.CreateBackendPartition(ABCPartition2,
ABCFacility,
Key100To199,
false,
true,
false);
print_sStatus_on_failure(sStatus);
}
|
Each key segment describes a data partition. Data partitions allow you to use multiple servers to handle the transactions all of your clients are attempting to perform; in this way, they don't all have to wait in line to use the same server. They can get more done in less time. Data partitions can be specified through a command line interface or programmatically through the RTRPartitionManager class.
The RTR Application Design Guide and goes into more detail about data partitioning.
Again, we use the RTR data object that RTR will place information in, and the user-defined data object, ABCOrder, that the client's data will be copied into. But at this point, the server is talking with RTR only, not the client, so it is expecting an answer from RTR; all the server really wants to know is that the transaction controller is ready to receive a client request. If it isn't, the server application will write out an error message and exit with a failure sStatus. The implementation of the sample server application's Register function:
The sStatus is checked after each of these calls and if they are all successful, the server is ready to receive incoming requests from the client application.
void ABCOrderProcessor::Register()
{
rtr_sStatus_t sStatus;
// Create an environment that our server can run in.
CreateRTREnvironment();
// Register with RTR the following objects
sStatus = RegisterFacility(ABCFacility);
print_sStatus_on_failure(sStatus);
// ABC Partition
sStatus = RegisterPartition(ABCPartition1);
print_sStatus_on_failure(sStatus);
sStatus = RegisterPartition(ABCPartition2);
print_sStatus_on_failure(sStatus);
// ABC Class Factory
sStatus = RegisterClassFactory(&m_ClassFactory);
print_sStatus_on_failure(sStatus);
// ABC Handlers
sStatus = RegisterHandlers(&m_rtrHandlers,&m_rtrHandlers);
print_sStatus_on_failure(sStatus);
return;
|
The RegisterHandlers method takes two parameters; the first parameter is a pointer to an RTRServerMessageHandler object and the second parameter is a pointer to an RTRSeverEventHandler object. ABCHandlers multiply derives from both of these foundation classes.
The server message handler specifies all messages generated by RTR or the RTR application that a server application may receive. The server event handler specifies all events generated by RTR or the RTR application that a server application may receive.
And now that the transaction controller has been established, the server waits to receive messages from the client application and the RTR router.
Receive/Reply Message Loop
The server sits in a message loop receiving messages from the router, or from the client application via the router. Like the client, it must be prepared to receive various types of messages in any order and then handle and reply to each appropriately. But the list of possible messages the server can receive is different than that of the client. This example includes some of those. First, the server waits to receive a message from RTR.
The implementation in the ProcessIncomingOrders method of the ABCOrderProcessor class:
// Start processing orders
abc_sStatus sStatus;
RTRData *pOrder = NULL;
while (true)
{
// Receive an Order
sStatus = Receive(&pOrder);
print_sStatus_on_failure(sStatus);
// If we can't get an Order then stop processing.
if(ABC_STS_SUCCESS != sStatus) break;
// Dispatch the Order to be processed
sStatus = pOrder->Dispatch();
print_sStatus_on_failure(sStatus);
// Check to see if there were any problems processing the order.
// If so, let the handler know to reject this txn when asked to
// vote.
CheckOrderStatus(sStatus);
|
Upon receiving the message the server checks the RTRData object's message type field to see what kind of message it is. Some are messages directly from RTR and others are from the client. In any event, the class factory creates the appropriate data object for the server application to handle the incoming data. When the message is from the client, your application will read the data object you constructed to pass between your client and server and, based on what it contains, do the work it was written to do. In many cases, this will involve storing and retrieving information using your database.
But when the message is from RTR, how should you respond? Let's look at some of the types of messages a server gets from RTR, and what should be done about them.
The following implementation from the ABCSHandlers.cpp file is for an rtr_mt_msg1_uncertain RTR message:
void ABCSHandlers::OnUncertainTransaction( RTRApplicationMessage
*pRTRData, RTRServerTransactionController *pController )
{
return;
}
|
The rtr_mt_msg1 and rtr_mt_msg1_uncertain messages identify the beginning of a new transaction. The rtr_mt_msg1 message says that this is a message from the client, and it's the first in a transaction. When you receive this message type, you will find the client data in the object pointed to by pRTRData parameter of this call. The client and server have agreed on a common data object that the client will send to the server whenever it makes a request: this is the ABCOrder object we looked at in the client section of this tutorial. RTR has copied the data from the client's data object into the one whose memory has been supplied by the server. The server's responsibility when receiving this message is to process it. On receiving an rtr_mt_msg1, the server application calls the OnInitialize and OnApplicationMessage server message handler methods.
On receiving an RTR message rtr_mt_msg1, the server application calls the handler methods OnInitialize and OnApplicationMessage by default. Business logic processing can be done within the OnApplicationData method.
The sample server application implements the OnInitialize method and overloads the Dispatch method with an implementation in ABCOrder that deserializes the ABCOrder data object, rather than having the default Dispatch method invoking OnApplicationMessage.
From the ABCSHandlers class:
void ABCSHandlers::OnInitialize( RTRApplicationMessage *pRTRData,
RTRServerTransactionController *pController )
{
m_bVoteToAccept = true;
return;
}
|
The overloaded Dispatch method in the ABCOrder data object:
rtr_sStatus_t ABCOrder::Dispatch()
{
// Populate the derived object
ReadObject();
// Process the purchase that the derived object represents
bool bStatus = Process();
if (true == bStatus)
{
return ABC_STS_SUCCESS;
}
else
{
return ABC_STS_ORDERNOTPROCESSED;
}
}
|
Recovered Transactions:
The rtr_mt_msg1_uncertain message type tells the server that this is the first message in a recovered transaction. In this instance, the original server the application was communicating with failed, possibly leaving some of its work incomplete, and now the client is talking to the standby server. What happens to that incomplete work left by the original server? Looking back at the client you will recall that everyone got to vote as to whether the transaction was accepted or rejected, and then the client waited to see what the outcome of the vote was. While the client was waiting for the results of this vote, the original server failed, and the standby server took over. RTR uses the information it kept storing to the recovery journal, which you also created earlier, to replay to the standby server so that it can recover the incomplete work of the original server.
When a server receives the `uncertain' message, it knows that it is stepping in for a defunct server that had, to this point, been processing client requests. But it doesn't know how much of the current transaction has been processed by that server, and how much has not, even though it receives the replayed transactions from RTR. The standby server will need to check in the database or files to see if the work represented by this transaction is there and, if not, then process it. If it has already been done, the server can forget about it .
If the received message contains rtr_mt_msg1_uncertain:
replay = RTR_TRUE;
else
replay = RTR_FALSE;
if ( replay == TRUE )
// The server should use this opportunity to
// clean up the original attempt, and prepare
// to process this request again.
else
// Process the request.
|
The server then replies to the client indicating that it has received this message and handled it.
The server typically uses the SendApplicationMessage call to answer the request the client has made. In some cases, this may mean that data needs to be returned. This will be done in the data object that has been agreed upon by both the client and the server.
Prepare Transaction
The rtr_mt_prepare message tells the server to prepare to commit the transaction. All messages from the client that make up this transaction have been received, and it is now almost time to commit the transaction in the database. This message type will never be sent to a server that has not requested an explicit prepare.
After determining whether it is possible to complete the transaction based on what has occurred to this point, the server can either call RejectTransaction to reject the transaction, or set all of the required locks on the database before calling AcceptTransaction to accept the transaction.
void ABCSHandlers::OnPrepareTransaction( RTRMessage *pRTRMessage,
RTRServerTransactionController *pController )
{
// Check to see if anything has gone wrong. If so, reject
// the transaction, otherwise accept it.
if (true == m_bVoteToAccept)
{
pController->AcceptTransaction();
}
else
{
pController->RejectTransaction();
}
return;
}
|
Because this example code is not dealing with a database, nor is it bundling multiple messages into a transaction, the code here immediately votes to accept the transaction.
Transaction Rejected
The rtr_mt_rejected message is from RTR, telling the server application that a participant in the transaction voted to reject it. If one participant rejects the transaction, it fails for all. The transaction will only be successful if all participants vote to accept it. When it receives this message, the server application should take this opportunity to roll back the current transaction if it is processing database transactions.
The sample server application includes the following code to check the order if it was not processed. If the order was not processed properly, then the handler method OnABCOrderNotProcessed is called and m_bVoteToAccept is set to false. This causes OnPrepareTransaction to reject the transaction.
void ABCOrderProcessor::CheckOrderStatus (abc_sStatus sStatus)
{
// Check to see if there were any problems
// processing the order. If so, let the handler know
// to reject this txn when asked to vote.
if (sStatus == ABC_STS_ORDERNOTPROCESSED)
{
// Let the handler know that the current txn should be rejected
GetHandler()->OnABCOrderNotProcessed();
};
};
void ABCSHandlers::OnABCOrderNotProcessed()
{
m_bVoteToAccept = false;
return;
}
|
Finally, explicitly end the transaction on a reject, the handler method OnRejected is called:
void ABCSHandlers::OnRejected( RTRMessage *pRTRMessage,
RTRServerTransactionController *pController )
{
pController->AcknowledgeTransactionOutcome();
return;
}
|
Transaction Accepted
RTR is telling the server that all participants in this transaction have voted to accept it. If database transactions are being done by the server, this is the place at which the server will want to commit the transaction to the database, and release any locks it may have taken on the database.
void ABCSHandlers::OnAccepted( RTRMessage *pRTRMessage,
RTRServerTransactionController *pController )
{
pController->AcknowledgeTransactionOutcome();
return;
}
|
Note the AcknowledgeTransactionOutcome call in the server. This is an explicit method for completing a transaction.
That's it. You now know how to write a client and server application using RTR as your network communications, availability and reliability infrastructure. Congratulations!
Build and Run the Servers
Compile the ABC server and ABC shared files on the operating system that will run your server applications. If you are using two different operating systems, then compile it on each of them. To build on UNIX, issue the command:
cxx -o server server.c shared.c /usr/shlib/librtr.so -DUNIX
|
You should start the servers before you start your clients. They will register with the RTR router so that the router will know where to send client requests. Start your primary server with the appropriate `run' command for its operating system along with the two parameters `1' and `h'. To run on UNIX:
% ./server 1 h
|
Start your standby server with the parameters `2' and `h'.
| Previous | Next | Contents | Index |