Re: Channels in OHP

From: Peter J. Nuernberg (pnuern)
Date: Sun 08 Jun 1997 - 13:40:52 CDT

_(fwd link)_> In OHP we originally inserted the idea of a "Channel". This was an opaque
> identifier inserted by the linkservice which was unique for that
> linkservice, and which enabled the linkservice and the client to communicate
> assynchonously. i.e. one could send a message, and the other could reply at
> a later time, rather than hold the socket open until it is ready to reply.
> In otherwords, this channel held the required information to route the message!
> _(fwd link)_
> I have had some trouble with this as people keep telling me that routing
> information should not be in the protocol. However I find that this
> information *does* need to be present,since the linkservice and the session
> manager actually have to look at this information at times.
> Following a chat with Pete some time ago, Sigi and I have a proposal that
> contains a Session Id appended to all messages and this would be made up of
> the following information:
> 1. HostId (typically IP address)
> 2. SessionManagerId (typically the Port the session manager is listening on.
> The session manager is what used to be called the client side shim or
> Runtime or tool integrator, and there is one per machine per user)
> 3. ApplicationId (typically the name of application)
> 4. SubProcessId (because some applications may manage more than one doc)
> The linkservice must be able to understand 1 & 2, so they cannot be opaque.
> 3 and 4 need only be understood by the session manager which allocated them
> in the first place, so their form is unimportant.
> We also find it necessary to carry a UserId on every message for
> authentication and to be ready to deal with later extensions for collaboration.

To respond to this, I'd like to describe the way we do IPC in HOSS first. Our messages have two kinds of fields: generic and specific. Generic fields make up what one could call a "header" for the message. They have the same format in every message, so if a process gets a message, it can always find out at least a few things about it, even if it turns out it cannot process it in the expected way, whatever that may be. What one has in the header is a question that can and should be debated. As an example, we have the following generic fields in all HOSS messages:

   protocol identifier - name and version of the protocol to which

      this message belongs
   message name - NULL-terminated string containing name of message _(fwd link)_   source process id - machine name, process name, process id, and

      thread id of the sender
   destination process id - same info for intended receiver    serial - an arbitrary unsigned long assigned by the sender    <other things not very important>

Our messages are byte blocks, not ASCII streams, so there are two important points to be made about generic fields - their order (in this case, always fixed in the above order) and their "encoding" or way in which they are packed into a byte block. The toolkit we use to do IPC knows how to do encoding and decoding of the generic fields, so if you use the toolkit to encode them, you can use the toolkit to decode them. All HOSS processes use the toolkit, so any two HOSS processes can send each other arbitrary messages, and be guaranteed that at least the generic fields can be understood.

We also have specific fields in messages. These are parameters that vary not just in content but also number, type, etc. between different messages. For example, one message may send two integers in the specific fields, while another may send an array of 4 NULL-terminated strings. What programs need with respect to specific fields are functions that encode and decode these specific fields into byte blocks suitable for transport.

HOSS is a system with an open set of Sprocs, so the set of protocols is also open. It can be time-consuming to generate libraries of encoding and decoding functions, so we have built a tool called the Protocol Definition Compiler (PDC) that generates these libraries from a specification. (It also generates many other "conveience functions" for programs like "encode all of the generic and specific fields, send the message, block on the response, and decode all of the fields", but we can ignore this for the moment.)

Some Sprocs or servers in HOSS require the same kind of connection information that Hugh an Sigi are talking about. For example, the HB4 Structure Base requires this information for transaction management. If two processes send a "begin transaction" message, and then one sends an "end transaction" while the other sends an "abort transaction", we need to pair the end and abort with the correct begins. The StructBase does this by keeping a table of port-transaction pairs. This is insufficient in cases like the one Hugh and Sigi mentioned, or in general if you don't like stateful servers. The alternative is to add a "session number" to each message to the StructBase.

So, I think of three kinds of information in IPC:

   extra-IPC-systemic - Keeping of information (state) in servers that is

      completely apart from the IPC system (e.g., the StructBase port-
      transaction pairs).

   generic fields - Stuff that is the same (in type, number, etc.) for
      every message

   specific fields - Stuff that varies between messages

Also, I use the following definitions:

   process class - a kind of process defined by a semantic role (e.g.,

      "link server")

   message - an ordered list of specific fields, a list of legal

      sender classes, a list of legal recipient classes, and a name

   protocol - a list of messages, timing constraints between them

      (e.g., message B cannot be sent before message A is received),
      a list of process classes, and an identifier

I think that the info Hugh and Sigi are proposing is like header information to me, and I don't object to it being in the protocol spec. It make the most sense to me to actually define a header for this protocol, include things like message name, sender id, destination id, session id, etc., in it. We should do this up front, and then we can dispense with it in the message definitions. Defining a specific header will have several advantages. We can simplify the message definitions right away, but more importantly, if we think about having an extensible protocol, we can guarantee that any process that can understand any message can at least get the name, etc., of any message ever added to the protocol, so that it can "fail" gracefully (e.g., return a message to the sender that says "I don't know what to do with message <x>") when receiving these newer messages.

_(fwd link)_What do other people think about a standard header? What fields should be in the header? Hugh and Sigi proposed:

> 1. HostId (typically IP address)
> 2. SessionManagerId (typically the Port the session manager is listening on.
> The session manager is what used to be called the client side shim or
> Runtime or tool integrator, and there is one per machine per user)
> 3. ApplicationId (typically the name of application)
> 4. SubProcessId (because some applications may manage more than one doc)

I'm not sure why #2 is there, so maybe Hugh and/or Sigi could tell me this. I'd advocate at least adding "message name" and "serial", the first having an obvious interpretation, and the second being opaque. I've found "serial" to be useful in managing async IPC. For example, if a client sends two identical requests to a server and does not block on the replies, it may get the replies in "backwards" order. In HOSS, anyone can assign any serial to any message they send. By convention, if a process is sending a reply to a message, it uses the serial on the original message as the serial for the reply. This allows processes to "match up" replies to original requests.

I don't believe that this is too much different than the current spec, so I'm not advocating any radical redesign. I think, though, that there is some benefit to speak in the above terms (or similar ones) as opposed to keeping the same information in a "flat" format as the current spec dictates.


This archive was generated by hypermail 2.1.5 : Tue 13 Aug 2002 - 07:20:39 CDT