Example 2-3 shows the data type that is returned in the user buffer with message types rtr_mt_accepted, rtr_mt_rejected, rtr_mt_opened and rtr_mt_closed. You can find the meaning of rtr_status_t using the call rtr_error_text .

Example 2-3 Type rtr_status_data_t

/* Type returned with rtr_mt_rejected,*/ typedef struct /* rtr_mt_accepted, rtr_mt_opened */ { /* and rtr_mt_closed messages. */ rtr_status_t status; /* RTR status */ rtr_reason_t reason; /* User-supplied reason */ } rtr_status_data_t;

2.11 RTR Events

What are events?

An event in RTR is a trigger which causes an event notification (also called a "broadcast") to be sent to the application which has subscribed to that event. RTR Events are created only by RTR and are used internally by RTR to help manage activities such as site failover. Application developers may subscribe to RTR Events to activate certain processing in their application. User Events are also available to enable application developers to send event notification or broadcast messages to other RTR applications. RTR provides the call rtr_broadcast_event to enable an application developer to trigger a User Event.

Events have special characteristics and restrictions:

• Event notification is delivered on a subscription basis using information supplied on the rtr_open_channel call.
• Events are not transactional and should not be used to transmit information which is, or will be, part of an RTR transaction.
• A user application can turn on or off the reception of any events, both RTR and user events.
• Events can only be transmitted within the RTR facility in which they are defined. Events cannot be sent between facilities or outside RTR.
• Event notification may include an optional message, which has a size limit of 64K.
• User Events can only be transmitted from frontend-to-backend or from backend-to-frontend. User Events cannot be used for peer-to-peer communication such as from frontend-to-frontend or from backend-to-backend.
• RTR Events are transmitted from RTR-to-frontend or RTR-to-backend.

The list below shows the RTR Events which are available for subscription. These events can be grouped in four basic categories:

• Shadow node activity (failover, failback, recovery complete)
• Standby node activity (become active, become standby, recovery complete)
• Changes in facility state and participants (clients/routers/servers entering or exiting the facility)
• Changes in configuration of partition key ranges (server available, server not available)

Table 2-5 lists the RTR events that can be received on a channel (associated with the rtr_mt_rtr_event message type). Events are listed in order of event number. See the description for rtr_open_channel in Chapter 3, RTR Call Reference, for further information. RTR sends events to the server either inside or outside a transactional boundary. A transaction is considered to start on receipt of an rtr_mt_msg1 message, and to end when the transaction is accepted or rejected (receipt of an rtr_mt_accepted or rtr_mt_rejected message). Events containing information about primary, secondary, or standby servers can arrive ahead of a transactional boundary. Gain and loss events arrive inside transactional boundaries.

Table 2-5 RTR Event Names and Numbers

Event Number Name	No.	Description
RTR_EVTNUM_FACREADY	96	The facility has become operational.
RTR_EVTNUM_FACDEAD	97	The facility is no longer operational.
RTR_EVTNUM_FERTRGAIN	98	Frontend link to current router established.
RTR_EVTNUM_FERTRLOSS	99	Frontend link to current router lost.
RTR_EVTNUM_RTRBEGAIN	100	Current router established link to a backend.
RTR_EVTNUM_RTRBELOSS	101	Current router lost link to a backend.
RTR_EVTNUM_KEYRANGEGAIN	102	Server(s) for new routing key range are now available.
RTR_EVTNUM_KEYRANGELOSS	103	No more servers remain for a particular routing key range.
RTR_EVTNUM_BERTRGAIN	104	Backend established link to a router.
RTR_EVTNUM_BERTRLOSS	105	Backend lost link to a router.
RTR_EVTNUM_RTRFEGAIN	106	Router established link to a frontend.
RTR_EVTNUM_RTRFELOSS	107	Router lost link to a frontend.
RTR_EVTNUM_SRPRIMARY	108	Server has become primary. 1
RTR_EVTNUM_SRSTANDBY	109	Server has become standby.
RTR_EVTNUM_SRSECONDARY	110	Server in a shadow pair has become secondary. 1
RTR_EVTNUM_SRSHADOWLOST	111	Server in a shadow pair lost its shadow partner. 2
RTR_EVTNUM_SRSHADOWGAIN	112	Server in a shadow pair acquired a shadow partner.
RTR_EVTNUM_SRRECOVERCMPL	113	Server completed recovery processing.

Subscribing to Events

RTR Events can be used for triggering special application processing based on a change in RTR system status, or for sending notification to the system operator after certain application or RTR conditions that require intervention.

User Events can be used for actions such as broadcasting stock prices to update a price table, or triggering special application processing such as handling a failed transaction. User events can be used to send a message in a one-to-one or a one-to-many method.

Event subscription is established when the rtr_open_channel call is executed. See the RTR rtr_open_channel call description for details on this call. The rtr_open_channel call is as follows:

rtr_open_channel (channel, flags, facnam, rcpnam, pevtnum, access, numseg, pkeyseg)

Two parameters on the call are used to establish event subscription: rcpnam and pevtnum.

rcpnam is a pointer to an optional channel name for receiving event messages. If a User Event is sent to a particular channel name, only those subscribers that match both name AND event number are notified. For example, a client channel named "New York" and a client channel named "Hong Kong" could both subscribe to receive User Event number 999. If event 999 was triggered by the server using the channel named "Hong Kong," the event would be received only by the "Hong Kong" client. Specify RTR_NO_RCPNAM for this parameter if a name if not used. This parameter is case sensitive.

pevtnum is a pointer to lists of RTR and User event numbers to which the channel wants to subscribe. These lists use the numeric values of the events shown in the Table 2-5. Use the special symbols in Table 2-6 to construct the event list:

Table 2-6 Symbols for Event Lists

Symbol	Description
RTR_NO_PEVTNUM	No events selected.
RTR_EVTNUM_USERDEF	Begin User Event list.
RTR_EVTNUM_RTRDEF	Begin RTR Event list.
RTR_EVTNUM_ENDLIST	End of entire list.
RTR_EVTNUM_UP_TO	Specifies an event range in the form x RTR_EVTNUM_UP_TO y.
RTR_EVTNUM_USERBASE	Smallest User Event number (0).
RTR_EVTNUM_USERMAX	Largest User Event number (250).
RTR_EVTNUM_RTRBASE	Smallest RTR Event number.
RTR_EVTNUM_RTRMAX	Largest RTR Event number.

Example 2-4 illustrates how to set up a list of all User Event numbers for the rtr_open_channel call:

Example 2-4 User Event Example

rtr_evtnum_t all_user_events[]={ RTR_EVTNUM_USERDEF, RTR_EVTNUM_USERBASE, RTR_EVTNUM_UP_TO, RTR_EVTNUM_USERMAX, RTR_EVTNUM_ENDLIST } ;

Example 2-5 illustrates how to set up a list of all RTR and User Event numbers for the rtr_open_channel call.

Example 2-5 RTR and User Event Example

rtr_evtnum_t all_events[]={ RTR_EVTNUM_USERDEF, RTR_EVTNUM_USERBASE, RTR_EVTNUM_UP_TO, RTR_EVTNUM_USERMAX, RTR_EVTNUM_RTRDEF, RTR_EVTNUM_RTRBASE, RTR_EVTNUM_UP_TO, RTR_EVTNUM_RTRMAX, RTR_EVTNUM_ENDLIST } ;

Sending Events

A broadcast event is triggered when the rtr_broadcast_event call is executed. See the rtr_broadcast_event call description for details on this call. The rtr_broadcast_event call syntax is as follows:

rtr_broadcast_event (channel, flags, pmsg, msglen, evtnum, rcpspc, msgfmt)

The significant parameters on this call are:

channel is the channel identifier returned from the rtr_open_channel call.
pmsg is a pointer to the message to be broadcast.
msglen is the length in bytes of the message.
evtnum is the User Event number that the application developer has assigned to this event.
rcpspc is the optional recipient channel name which can be specified with the rcpnam parameter on the rtr_open_channel call.

Example 2-6 shows an example of an rtr_broadcast_event call.

Example 2-6 Broadcast Event Example

if ( bServerShutdown ) { sts = rtr_broadcast_event ( /* channel */ BY_CHAN_CLIENT(cCurrentChannel,client)->chan, /* flags */ RTR_NO_FLAGS, /* pmsg */ &msgbuf, /* msglen */ cbTotalSize, /* evtnum */ USER_EVT_SHUTDOWN, /* rcpnam */ "*", /* msgfmt */ szMsgFmt); exit_if_error ( "rtr_broadcast_event", sts ); }

Receiving Events

Any RTR transaction, RTR Event, or User Event can be received when the application executes the rtr_receive_message call. See the RTR rtr_receive_message call description for details on this call. The rtr_receive_message call syntax is as follows:

rtr_receive_message (channel, flags, prcvchan, pmsg, maxlen, timoutms, pmsgsb)

The significant parameters on this call are:

channel is the channel on which the message is received.
pmsg is a pointer to an application buffer where the message is written.
maxlen is the maximum length of the application buffer in bytes.
pmsgsb is a pointer to a message status block describing the received message.

If the application has subscribed to events, any call to rtr_receive_message can return an event notification, either an RTR Event notification or a User Event notification. The results are described in Table 2-7.

Table 2-7 Event Notifications

If this notification is delivered:	the rtr_receive_message call returns a message of type:	and the user/application buffer contains the associated:
RTR Event	rtr_mt_rtr_event	event message
User Event	rtr_mt_user_event	user broadcast message

Returned Event Data

Two RTR Events return key range data to the application:

RTR_EVTNUM_KEYRANGEGAIN
RTR_EVTNUM_KEYRANGELOSS

No other RTR Events return data.

The key range data are received in the message returned to the application, with the length of the message specified in the message status block ( msgsb ). For example, the following illustrates rtr_receive_message usage.

rtr_status_t rtr_receive_message ( rtr_channel_t *pchannel, rtr_rcv_flag_t flags, rtr_channel_t *p_rcvchan, rtr_msgbuf_t pmsg, rtr_msglen_t maxlen, rtr_timout_t timoutms, rtr_msgsb_t *p_msgsb )

The message status block pointed to by *p_msgsb has the following structure:

typedef struct { rtr_msg_type_t msgtype; rtr_usrhdl_t usrhdl; rtr_msglen_t msglen; rtr_tid_t tid; rtr_evtnum_t evtnum; }rtr_msgsb_t;

When an event number is 102 or 103, RTR returns key range data (the low and high bounds) in the message, padded as required for data marshalling and interoperability. The key range data can be examined by the application. For more detail on data marshalling and formatting, see Section 2.15.

Bounds data are treated as if defined as a structure. For example, if there are two key segments defined as rtr_uns_8_t and rtr_uns_32_t , then the bounds data are copied to outbuf as if they were contained in the structure; that is, the 32-bit ints are correctly aligned in the structure and the structure size is a multiple of four. For example,

struct{ rtr_uns_8_t low_bound_1; rtr_uns_32_t low_bound_2; rtr_uns_8_t hi_bound_1; rtr_uns_32_t hi_bound_2; }

The one four-byte-alignment-fits-all is required for interoperability; no padding is allowed.

Example 2-7, which can be run manually from the RTR CLI, illustrates the return of key range data with the RTR Event RTR_EVTNUM_KEYRANGELOSS . The RTR CLI interprets the format of this message as appropriate. In Example 2-7, the format is string or ASCII data, the default.

Example 2-7 Returned Event Data Example

RTR> crea fac jws/all=sucre %RTR-S-FACCREATED, facility jws created RTR> crea part ab/fac=jws/noshadow/nostandby- /key1=(type=string,length=2,offset=0,low="AB",high="CD") %RTR-I-PRTCREATE, partition created RTR> rtr_open/chan=s/server/noshadow/nostandby/part=ab/fac=jws %RTR-S-OK, normal successful completion RTR> rtr_rec/chan=s/time=10 %RTR-S-OK, normal successful completion channel name: S msgsb msgtype: rtr_mt_opened msglen: 8 message status: normal successful completion reason: 0x00000000 RTR> rtr_open/chan=c/client/event=(102,103)/fac=jws %RTR-S-OK, normal successful completion RTR> rtr_rec/chan=c/time=10 %RTR-S-OK, normal successful completion channel name: C msgsb msgtype: rtr_mt_opened msglen: 8 message status: normal successful completion reason: 0x00000000 RTR> rtr_close/chan=s %RTR-S-OK, normal successful completion RTR> rtr_rec/chan=c/time=10 %RTR-S-OK, normal successful completion channel name: C msgsb msgtype: rtr_mt_rtr_event msglen: 4 evtnum: 103 (RTR_EVTNUM_KEYRANGELOSS) message ks_lo_bound: 0x4241 ks_hi_bound: 0x4443 RTR> reca RTR> rtr_rec/chan=c/time=10 %RTR-E-TIMOUT, call to rtr_receive_message timed out

Design consideration: When an RTR application executes an rtr_receive_message call, the programmer could incorrectly anticipate that a particular message type may be received and only write instructions to respond to the expected message. However, an RTR or User Event could be received on any instance of the rtr_receive_message call (as could other unanticipated RTR messages). Therefore, as a general application design guideline, the application developer should always program the application so that it can properly handle any type of message that could be received by the rtr_receive_message call.

Events are delivered in the order in which they are broadcast; therefore event serialization will be preserved for a particular user. However, RTR does not enforce any particular serialization across different subscribers, so different subscribers could receive event notifications in any order.

Example 2-8 shows an rtr_receive_message call in use.

Example 2-8 Receive Message Example

status = rtr_receive_message( &channel, RTR_NO_FLAGS, RTR_ANYCHAN, &receive_msg, RTR_ANYCHAN, &receive_msg, sizeof(receive_msg), receive_time_out, &msgsb); check_status( "rtr_receive_message", status );

2.11.3 Event Management by RTR

RTR manages both event routing and event delivery.

Event Routing

When an event subscription is created with the rtr_open_channel call, the event details are stored in a subscriber database on all routers. When an event is triggered, notification is delivered to all routers within that facility which are connected to that system. The routers then check their subscriber database for any systems which has subscribed to that event. If one or more subscribers are located, and the subscribers are currently attached to this router, then the router broadcasts the message to the subscribers. If no subscriber is located, then the message is discarded.

Event Delivery

RTR will reliably deliver RTR transactions and RTR events. The delivery of User Events on a properly configured system is reliable, however, delivery of User Events is subject to RTR Flow Control if the subscriber cannot process events as quickly as they are delivered. Flow Control is a message traffic governor which helps the effected system to manage a spike in message traffic. We will discuss in more detail the interaction of Flow Control and RTR event processing.

When a User Event is triggered, a broadcast including message data is routed to the subscriber system. User Events, along with RTR Events and transactions, are placed into an incoming message queue on the destination system until the subscriber application executes an rtr_receive_message call to receive the message into the application. If too many messages are sent to the destination system, then the RTR Flow Control feature will be activated.

Flow Control may then force the sending application to wait for a while in the next RTR call that sends data, or it may discard broadcasts from the message queue, until the message queue length reduces and Flow Control allows new broadcasts to be sent to the destination system. Because User Event broadcasts are usually used for streaming information such as the periodic update of a price table, RTR does not store event messages which are impacted by Flow Control for later processing. This technique would cause the application to spend time viewing stale data. Instead, RTR Flow Control may discard the message to help relieve the messaging backlog, and will rely on a future message delivery to supply the updated information.

Design issue: Because of the possibility that a User Event message could become delayed or discarded due to Flow Control, User Events should not be used for delivering information that is of a business critical nature including information that previously was, or later will be, used in a transaction. To compensate for the possibility of a discarded message, the application developer may consider adding a sequence number to the event message and providing a read-only transaction in the application to detect and request re- transmission of any discarded broadcast data from the sender.

Overhead of Using Events

Delivery of User Events is based upon the registration databases which are kept on the routers. The event is delivered from the sender to all connected routers, which means each event triggers a message traffic load of 1 (for a FE sender) or the number of routers (for a BE sender). The event is then propagated by the routers to all subscribers, creating message traffic of 0 or the number of systems with subscribers to the event.

Design Issue: Processing event messages does consume some system resources and could impact overall performance. If system resources become constrained, RTR flow control may activate which will reduce the RTR throughput on the affected systems. Care should be exercised to provide enough system resources to handle the message load.