Advanced Use of RPC

Note: DON'T EDIT with WWW yet or information will be lost. This chapter describes facilities in the RPC kernel which may be called by users wishing to perform more than transparent procedure calls. This chapter need not be read by users who are building straightforward distributed programs. Topics covered are Details of parameters of each library procedure are given in an appendix.

Varieties of server

This section describes a method of building servers which look to the client like ordinary procedures, but in fact are something more complicated. It describes how task to task communication may be performed with RPC.

Server loop

The simplest server is a program which waits in a loop, servicing each request as it comes in. The procedure RPC_Loop_Server, in the run&hyphen.time library, implements such a loop. This takes as a parameter the medium address(es) to which it is to listen. This may be used in a situation where a task can be dedicated to servicing remote calls. +---------------------->| | | | +-------+-------+ | | wait for a | | | request | | +---------------+ | | | +-------+-------+-------+-------+ | | | | | | | +---+ +---+ +---+ +---+ +---+ service the | | | | | | | | | | | particular | +---+ +---+ +---+ +---+ +---+ request | | | | | | | +-------+-------+-------+-------+ | | ------------<---------- The RPC_loop_server procedure is used in the default main program module "SERVERLOOP" provided with the VAX/VMS run time library.

Object Oriented Programming

A powerful and convenient way of designing a multitask system is to use "Object oriented programming". It means that for a data object, or physical object, you write a bunch of procedures for performing operations on that object. In a real&hyphen.time system, as well as a bunch of procedures, you probably want a controlling program which runs as an independent process, with overall responsability for the object. A smart step is to make some of the operations on the object available to other processes by RPC: +---------------+ | Controlling | | program | +---------------+ +---------------+ | remote | | | program | | +---------------+ | / | / | ______________________/ | / (RPC) | / |/ +---------------+ | Access | | procedures | +---------------+ | database &/or physical object The access procedures are linked with the main program, and share its data structures.
  • The remote program uses the access routines like any other routine
  • The control program can enable or disable access from outside for exclusive access to the object at any time.
  • The access routines may affect data structures, and synchronisation primitives, and so modify the way the main program behaves. (For instance, access routine can release the main program from a wait state).
Task to task communication may be set up between two or more processes, each one providing a set of routines which may be called by the other. +---------------+ +---------------+ | Program 1 | | Program 2 | +---------------+ +---------------+ |\ / | | \ / | | \__________ / | | \ / | | ___________\__________/ | | / rpc \ | | / \_____________ | |/ rpc \| +---------------+ +---------------+ | Access | | Access | | procedures | | procedures | +---------------+ +---------------+ | | | | Object 1 Object 2 Interprocess communication by RPC between processes responsible for different objects. In the example program "battleships" (available on demand) two instances of the same program communicate in this way, to implement a simple game of battleships. Each program manages an object which is a fleet of ships. An object manager may be implemented either by polling, or by asynchronous reponse to incoming requests. The next two sections describe these alternatives.

Polling Server

A variation of the simple server loop allows the server program to service one call at a time, and perform processing in between. It allows the object oriented approach in environments where asynchronous servers are not supported. (M6809, IBM PC/MSDOS etc) +-------------->| | | | +-------+-------+ | | wait for a |(request) | | request |-----------------> | +---------------+ | | |(timeout) | | | | | +-------+-------+-------+ | | | | | | | | +---+ +---+ +---+ +---+ service the | | | | | | | | | | particular | | +---+ +---+ +---+ +---+ request | | | | | | | | +-------+-------+-------+ | | | | |<------------------------- | | | +-------------------------------+ | | Other activities | | | | | +-------------------------------+ | | ------------<-- A call to the run&hyphen.time system RPC_Accept routine causes a wait for the next request, and the servicing of the called procedure. The procedures called in this way may share data with the user program and communicate in other ways. The library procedure RPC_Create_Server must be used to initialise the server before RPC_Accept can be called, in order to the open communication channel. It may be that the "other activities" are not related to the service procedures, if there are other things for which the task is responsible, as may be the case, for example, in a microprocessor without a multitasking kernel. It may also be that the "other activities" are very related to the service procedures. They may be fed commands by the service procedures, they may share data areas with the service procedures, and they may even call the same service procedures themselves. This method may also be appropriate if the server is regarded as a state machine, when requests from different clients must interact in some way, as in a resource manager. (This form of server is analoguous to an Ada task with "entries": examples of its use may be found in Ada tutorials.)

Asynchronous Server

This method of designing a server allows a user program on the server side to run continuously, at the same time as the client process(es). The user program simply asks the run&hyphen.time system to service any incoming requests asynchronously. The RPC_Start_Server routine is used once to set this up. From that time on, whenever a call arrives, the user task will be interrupted, the called procedure will execute as though it had been called from the user program at that point, and then the user program will continue. (Under VMS, the procedure is run at AST level; on the M68k it is run in an equivalent way, not at interrupt level). +-------------->| | | | +-------+-------+ | | | | | | | | | | | User | ------------> Interrupted | | program | by request arriving | | loop | | | | | | | | | +-------+-------+-------+ | | | | | | | | | | +---+ +---+ +---+ +---+ service the | | | | | | | | | | | particular | | | +---+ +---+ +---+ +---+ request | | | | | | | | | | +-------+-------+-------+ | | | | | | | <-------------------- reenter loop where it | | | left off | +---------------+ | | | | -----<--------- This is the most powerful form of server, and may be the easiest to use. It is only available where the environment supports asynchronous response to events. It is useful for object oriented programming, to also allow RPC to control an otherwise autonomous process, or in a single tasking system, to provide a little concurrency. When accessing common data structures, the user program may temparaily disable asynchronous calls be calling the SETAST system service provided under VMS and on the Valet&hyphen.Plus, or the equivalent.

Multi&hyphen.threaded server: Identifying the client

If there are multiple clients for the same service, we can distinguish three cases:
  1. If each service is a self&hyphen.contained operation, and no state information is held about the client, then one server task may be directly shared by many clients. ("Idempotent" operations).
  2. If some calls leave the server state changed, as, for example, when an "open" call to a file server leaves a file open, one solution is to dedicate a separate server task to each client. This is reasonable when a small number of clients will each use the server intensively.
  3. If it is not practicable to dedicate a separate process to serving each client, and some state information must be kept by the server on behalf of each client, then the server must be able to determine the identity of the client.
In the last case, the server may find, for his own information, the address of a caller using the RPC_caller_address procedure.

Example (VAX/FORTRAN)

SUBROUTINE MY_REMOTE_PROCEDURE CHARACTER*40 CALLER_ADR INTEGER STATUS ... CALL RPC_CALLER_ADDRESS(STATUS, %REF(CALLER_ADR)) ... END

Making your own service ASTs

This section is not intended for the general user. The routines mentioned allow specialised servers to be made, for instance which fork extra processes off for different clients. (This is used in the MODEL TCS system.) The RPC_Queue_Server procedure allows a user program to queue its own AST procedure to service the next request. RPC_service is a procedure taking one rpc_message parameter. It will handle one incoming call message, deallocate the message buffer passed to it, and return. It is a suitable AST routine to be passed to RPC_Queue_Server, or may be called from a specially written user routine. See the description of these routines in the appendix.

LOCAL servers

It is possible that one wants a package to be either remote or local, as selected at runtime. In this case, one has to link (bind) the server and client into the same program. +---------------------------------------+ | Application (client) | +---------------------------------------+ entry points-> | +---------------------------------------+ | Client stub | +---------------------------------------+ | +---------------------------------------+ | RPC Run Time Support | +---------------------------------------+ | +---------------------------------------+ | Server stub | +---------------------------------------+ same entry points-> | +---------------------------------------+ | Server Routines | +---------------------------------------+ A LOCAL call is made to a procedure which is in fact part of the same process. The Server routines and the client stub provide entry points with the same name. This has to be done carefully, as two sets of entry points with the same name will exist. This is because the client stub provides the same routine names as the server module. Logically, what must be done is as follows. The client must call entry points in the client stub, rather than call the server module directly. Therefore, the client and client stub should be linked togther as one part, and the server and server stub must be linked together as another. Then, these two resulting parts are linked together into one whole. The method of performing this logical operation differs from system to system, and is not discussed here in detail. Now, the system may be set up so that the client and server stubs are directly connected. In this case, when the client stub is called, the message it produces is passed straight to the server stub, which calls the procedure in question. The medium address used to set this up is LOCAL . If a different RPC address is used, and RPC_Switch called, for example, calls will be redirected to a remote process.

Calls within calls - CALLBACK

The RPC system is reentrant, so that one remote procedure may call another remote procedure. A useful special case of this is when a remote procedure calls back routines in the caller. This is known as "CALLBACK". For example, a routine to display the state of a remote moinitoring task might call back a routine in the caller to print out each line. In that way, whoever calls the procedure, the output will arrive in the correct place. Machine A Machine B +-------+ | Appl. | | prog. | +-------+ ------- call ------> +-------+ |DISPLAY| |STATE | +-------+ <----- callback ----- +-------+ |PRINT | | LINE | +-------+ Callback allows the server to call routines in the client's process. Each process is therefor a client (c) of one call and a server (s) of the other. This takes advantage of the fact that any client, while waiting for a reply, automatically behaves as a server. The package called back must be defined as a separate package, and separate client stubs (for machine B) and server stubs (machine A) generated and linked in. On machine B, the special syntax for the address of the callback package address is <package number> @ CALLBACK In this case,
  • The package number must be explicitly stated (normally 0). (This is because the address is only valid within an RPC. If you don't give the package number explicitly, the RPC system would have to query the other side at initialisation time to determine the package number. It can't do that because at initialisation time, no caller is defined.)
  • The initialisation on machine B (RPC_open or open_xxx) only happens once;
  • Callback calls may ONLY be made from within remote procedures: the address is effectively undefined outside these.
  • The RPC system will use the same connection for the call back as for the original call (over DECNET for example).
This system is not suitable if you want the server process to call the client back outside (some time after) the original call. In this case, a new channel must be opened with the "@CALLER" syntax (q.v.).

Client facilities for selecting servers

+--------+ | | | Client | | | +----|---+ | . . . . . . . . .+--------------+ . . . . . . . . . . | . +-------+ +-------+ +---|---+ +-------+ | | | | | | | | | Server| | Server| | Server| | Server| | A | | B | | C | | D | +-------+ +-------+ +-------+ +-------+ A client may wish to select one from a set of similar servers dynamically. This section applies when there may be more than one instance of the same type of server in the system. In this case, the client program may wish to select one particular server at run time. There are two ways of doing this. The first method (using RPC_Switch) allows communication to be set up with one server at a time. The second method (using RPC_Open and RPC_close) allows communication to be established with more than one server at once, the client being able to switch rapidly between them. The "binding" of the client to the server refers to the resolution of the server address, and the setting up of communication channels to the server. This is normally done by the stub itself when the open_xxxx routine in the stub is called at initialisation time. Under VMS and TurboPascal, binding is performed automatiaclly by the client stub at initialisation time. If explicit binding is to be done by the user program, the NOAUTOINIT option should be used when the client stub is compiled, to disable automatic binding. The result is a reference number called a handle which refers to the open connection. This is stored in a global variable called h_xxxx where xxxx is the name of the package.

Reconfiguring the system

The RPC_configure call allows one client to use more than one identical server in turn. The effect is for the connections to all open servers to be closed and reopened with the translation of their logical names being perfomed according to new values which may have been set up. A client process which wishes to use several identical servers may therefore repeatedly redefine the logical names which give the addresses of those servers, and call RPC_configure to cause future calls to be directed to the new servers. Note that if RPC_Configure returns a bad status, then one or all of the handles will be invailidated. In this case, as the user has no way of knowing which are valid and which are invalid, all he can do is exit. The use of RPC_Configure is not, therefore, recommended. The RPC_Switch routine performs the reconfiguration function on a single package, referenced by a handle. The package must be already open (for the handle to be valid). The value of the handle is unchanged.

Explicit binding

The RPC_Open and RPC_Close routines allow the user to perform the binding himself, and overwrite the handle used by the stub. The RPC_Open routine allows a client to open communication with a specific server. A string parameter describes the remote service required. It may be a formatted physical address (program name or number, network address, etc) or it may be a logical name which is converted by the run&hyphen time system into an appropriate address. The handle value returned is used in all future reference to the service by the client. If this value is put into the handle variable used by the stub, the stub will be directed to refer to the server in question. The handle variable used by the stub should be accessed as an external variable h_xxx where xxx is the package name. The client program may, however, perform RPC_Open serveral times, and keep a record of each of the handle values returned. It may then switch rapidly between servers by overwriting the stub's handle variable with the handle to the server it wants to access. Every server which has been opened must be closed again by calling RPC_Close.

User Error Handling

The RPC_Establish procedure may be used in a client program to handle errors occuring in remote procedure calls. By default, if an unrecoverable error occurs in the communication with the server, a message is displayed and the client program is stopped. ($EXIT occurs under VAX/VMS, a pascal halt on other systems.) If the user wishes to handle the error himself, he passes the address of his error handler to RPC_Establish. If this has been done, his handler will be called instead of the default handler. In order to revert to the default handler, RPC_Establish is called with a value of zero. Example (VAX FORTRAN): EXTERNAL HANDLER ... CALL RPC_ESTABALISH(HANDLER) CALL MY_REMOTE_PROCEDURE CALL RPC_Establish(0) ... The user error handler is a routine taking one parameter, which is the rpc_message in question. If the user is programming in pascal, he will find the offending error status in the m_status field of the rpc_message. Example (VAX Pascal): VAR global_status: [VOLATILE] rpc_status; { error flag } ... [UNBOUND, ASYNCHRONOUS] PROCEDURE handler(p: rpc_message_pointer); BEGIN global_status := p^.m_status; {record bad status } END; ... In main program: global_status := SS$_NORMAL; { Initialise error flag } RPC_Establish(handler); { Establish my error handler } my_remote_procedure; { Try calling remote module } RPC_Establish(%IMMED 0); { Reselect default handler } IF not odd(global_status) { Check error flag } THEN ... Note that [UNBOUND], [ASYNCHRONOUS], [VOLATILE] are all attributes which prevent the VAX Pascal compiler overoptimising the error handler, so that it will still work when called out of context.

Message format control - The ALIGN option.

This option controls the formatting (invisible to the user) of messages sent by the stubs. /ALIGN selects or deselects word alignment of parameter values which is less space&hyphen.efficient, but Courier compatible. It is not recommended. The ALIGN option MUST be the same for corresponding client and server stubs. For microprocessor stubs, the ALIGN choice is built into the runtime library, and cannot be selected by the stub. The microprocessor libraries provided with the RPC software are generated without ALIGN. For VAX/VMS, the ALIGN option may be selected when building the stub:
  1. If using the RPC command, NOALIGN is the default, and /ALIGN must be specified as a command line parameter if Courier compatability is required.
  2. If you are using explicit postprocessing of the stubs (discussed below), NOALIGN is the default, unless the keyword ALIGN is included in the INCLUDE keyword list.

Timeouts on remote calls

In some cases, where the communication medium and protocol used is unreliable, or where the remote software may be unreliable, one wants to specify the maximum time for execution of a remote routine. The timeout may be specified in two ways. PRAGMA TIMEOUT (q.v.) may be specified in the RPC definition file, in which case it is given separately for each procedure. Alternatively, the defualt timeout for the package may be specified on the compiler command line when generating the client stubs, using the option TIMEOUT=nnnn where nnnn is the time limit in units of 10ms. For example, the option ..bf MONO TIMEOUT=1000 ..pf selects a timeout of ten seconds. Allowance should be made for any delays which could occur in the transmission of messages, or due a machine being heavily loaded. This ensures that timeouts only occur when a real problem has arisen. If a timeout occurs on a remote call, it is treated as an error. That is, the client program will stop with an error message, unless a user error handler (discussed above) is declared, or a call_status (q.v.) parameter is given.

Stub Version Numbers

By default, the RPC compiler generates a version number from the RPC interface definition file. This 4 digit number is displayed on the standard output by the compiler, and inserted as a comment into the external definition (.ext) file produced. It is also compiled into the stub code.

At run time, the client stub sends a version number to the server stub, which checks it against its own version number. If the version numbers do not match the request is rejected, and the error message "incompatible versions" is produced at the client. This ensures that the two stubs were generated from the same definition file, and will therefore refer to compatible modules. The version number is formed from a checksum of the non&hyphen.comment characters in the definition, and so will change when a material alteration to the package definition is made.

In some cases, it is possible to make a small change to a package which will not cause serious compatabilty problems. For instance, the procedure names may be changed, another procedure is added to the end of the package definition, or a modification is made to a procedure which has in fact never yet been used. In this case it is possible to force the stubs to a have a particular version number, with the version= option. The version number must be an integer between 1 and 32767. Numbers between 1 and 999 will not be generated automatically by the compiler, and so may be used for a private version numbering scheme.

Example

: Suppose there is a requirement that the procedure names on the client machine should differ from those on the server. This may be required to prevent a clash, for example, between local and remote versions of the same procedure, so that the same program can access either. In this case, two RPCL files are made, which differ only in the procedure names. The server stub is compiled first, with an automatically generated version number, and then the client stub is compiled with the version number forced to be compatible. $ RPC MYSTUB1.RPC /SVAXVMS RPCC: Generated stub version number is 2765 $ RPC MYSTUB2.RPC /CVAXVMS /VERSION=2765 [The option version=0 causes the RPCC to generate "unsafe" stubs with no version number check. If use of this option later allows the inadvertent connection of incompatible stubs, run&hyphen.time errors (access violations etc) may occur without the cause of the problem being obvious. This may lead to wasted debugging time, and so is not recommended.]