Messaging system¶
On this page
DISCLAIMER This document describes the messaging system. I don’t know all the internals of that, so the knowledge described here is very sparse and maybe inaccurate.
High-level view¶
Sessions¶
To use the messaging layer, you need a Session. Sessions are connected to each other and make an undirected graph that we call a galaxy of sessions. On a galaxy of sessions, there is a list of services and the service with id 1 is the ServiceDirectory which contains that list.
The ServiceDirectory runs on a session which is the entry point for other sessions to connect. When a session exposes a service, it listens on some other port so that other sessions can connect to it when they need to contact the service.
+-------------+
+---------+ Service |
| | Directory |
| +------+------+
| |
| |
| | +-------------+
| | | |
| +--------+ SomeService |
| | | |
| | +-------------+
| |
+-------+------+ | +--------------+
| Client | +--------+ OtherService |
| session +------------------+ |
+--------------+ +--------------+
There is no real difference between a client session and a service session. In this diagram, it is just that the Client session does not expose any service, but apart from that, it is a normal session. A session can call other services whether it hosts services or not.
Address system¶
There is a three-part address system in qimessaging used to call functions, trigger events, etc.
Services have a associated service ID. A service can give one or more objects (by returning them). When a service gives an object, this object gets the same service ID but a different object ID.
For example, the service directory is always service 1. If you have a service 2, that service is addressed as 2.1. If that service gives you an object, that object will have ID 2, which means its address is 2.2, the next will have ID 3. The object ID 1 is always reserved for the service itself.
There is a third part on addresses which is the function ID. Every function, event and property has an ID. User functions usually start at ID 100. IDs below 100 are hidden and reserved for qimessaging’s usage.
Exchanging objects¶
It is possible to exchange objects between a session and a service by returning it from a function call or by giving it as an argument. When doing that, the object will be wrapped by qimessaging objects.
You can see in the diagram below how an object is exposed from the server on the right to the client on the left. Keep in mind that what I call client or server in this section has no real importance as they can be swapped. You can pass objects from service to client or from client to service and you will reach the state described in this diagram.
Network
Client / Service
\
+------------------+ / +-----------------+
| | \ | |
| RemoteObject +----------+ BoundObject |
| | / | |
+---------^--------+ \ +--------+--------+
| / |
| \ |
+---------+--------+ / +--------v--------+
| | \ | |
| AnyObject | / | AnyObject |
| | \ | |
+------------------+ / +-----------------+
\
When the object is exposed from the server, it will be wrapped in a BoundObject that will own a reference to the AnyObject to keep it alive.
When the object is received by the client, you get an AnyObject which points to a DynamicObject.
If the same object is returned twice for example, two BoundObjects will be created and two RemoteObjects will be created on the remote side. They will be independent and have different addresses.
However, several calls to qi::Session::service("MyService")
on a client
session will return shared references to the same RemoteObject.
The client session communicates with the service directory only for the first call,
and then returns the cached RemoteObject, as long as the service is not replaced.
Lifetime¶
As said before, BoundObject owns a reference to the AnyObject so that the RemoteObject can still contact it later.
There are three cases which cause that reference to be dropped:
- When the RemoteObject dies, it will call the hidden method
terminate()
which will abort all pending calls and ultimately destroy the BoundObject and release the reference. - When the connection with the client is lost (legitimately or because of an error), all BoundObjects exposed to it will be destroyed, releasing their reference to the AnyObject.
- When the service is unregistered, all addresses will become invalid, thus the BoundObjects will also be destroyed, releasing their reference.
Using a RemoteObject asynchronously¶
If the RemoteObject is a service obtained from the session,
like the almotion
object in the following example, it is kept alive
by the session and the object can be used without caring about lifetime
issues.
// return robot's pose with respect to ref
qi::Future<AL::Math::Pose2D> getRobotPose(qi::SessionPtr session,
const AL::Math::Pose2D &refPose)
{
qi::AnyObject almotion = session->service("ALMotion").value();
// no need to handle `motion`'s lifetime during the async call,
// `session` keeps it alive already.
return almotion.async<void>("getRobotPosition").andThen(
[refPose](const std::vector<float> &xya)
{
refPose.inverse() * AL::Math::Pose2D(xya);
});
}
However, if the RemoteObject is obtained from a remote call,
its lifetime must be specifically handled.
This can be done using a continuation, as demonstrated in the following
example, where the robotFrame
object lifetime is extended
for the duration of the computeTransform
call.
Beware with this pattern however, if the getRobotPose
function is called
often, and the computeTransform
call is long to finish,
RemoteObjects might be created faster than they are destroyed, their
accumulation leading to loss of performance and responsiveness and an increase
in memory consumption.
qi::Future<qi::geomety::TransformTime> getRobotPose(qi::SessionPtr session,
qi::AnyObject refFrame)
{
qi::AnyObject actuation = session->service("Actuation").value();
qi::AnyObject robotFrame = actuation.call<qi::AnyObject>("robotFrame");
auto computingTf =
robotFrame.async<qi::geomety::TransformTime>("computeTransform",
refFrame);
// capture robotFrame in a no-op continuation in order to ensure it is
// kept alive until the `computeTransform` remote call is finished.
// Otherwise, the call would be aborted, and `computingTf`
// would end up in "RemoteObject destroyed" error, even though the actual
// remote AnyObject representing the robot frame is kept alive by the
// Actuation service.
computingTf.connect(
[robotFrame](qi::Future<qi::geometry::TransformTime>) {});
return computingTf;
}
Sending an object through a RemoteObject or BoundObject¶
RemoteObject
and BoundObject
both inherit from ObjectHost
which is a class
that has a list of BoundObject
. They both inherit from it because you also
need the RemoteObject
to hold BoundObject
-s when sending an object as an
argument.
Since a RemoteObject
does not have an ID to refer to (it is not a service),
when sending an object through it, it must create an address. The service ID of
the address it forges is the one of the service it sends the object to. To avoid
having two objects with the same IDs on the service, it starts counting object
IDs from 2^31.
I don’t know what happens when object IDs overflow, probably bad things that they only talk about in books.
Lower-level view¶
Messages¶
In the end, qimessaging all comes down to the exchange of qi::Message
. A
message is a structure with different fields:
- a magic number to identify qimessaging packets
- an ID and a type which will be detailed later
- a destination address with Service.Object.Function
- a variable-size signature to describe the payload of the message
- a buffer which contains the payload
The last two parts are the only variable part of the message, the rest is part of the fixed-size header.
Messages types and IDs¶
Messages have different types described in message.hpp. Message IDs are always incremented when sending a new request, for example a call (and maybe other types).
Then the response to that call may be Reply, Error or Canceled. In whichever case, the response will have the same ID as the Call message to associate them together.
Exchanging messages¶
At the lowest level, we use Boost.Asio to handle the TCP sockets. This is
wrapped in the class TcpTransportSocket
which inherits from the virtual class
TransportSocket
. It is possible to implement the messaging over something else
by specializing that class, for example to use UNIX sockets, pipes on the file
system (why not!), etc.
I don’t know the exact path of messages through the classes, but for a call,
first there is a messageReady
signal triggered in TransportSocket
. There
is a class that receives that signal (probably Server
) and forwards it to the
concerned object by calling its onMessage
method. Then the object can handle
the call and reply later by doing a send on the socket.
If the message is a Reply, I think the RemoteObject
is directly connected to
the TransportSocket
and handles the messages. There is a MessageDispatcher
class somewhere that probably does something.
Middle-level view (serialization)¶
StreamContext¶
TcpTransportSocket
inherits from StreamContext
which, as the name suggests,
is a context associated to a stream. It is used to hold various information to
know how to serialize things.
For example, when sending an object, we send the whole MetaObject
with it. But
when getting a service twice, the MetaObject
(which is a heavy structure) is
still the same and there is no need to send it again. For that, there is a
MetaObject
cache system and we keep a bit of information in the
StreamContext
which is “Have we already sent this MetaObject
on this
stream?”. That way we avoid sending it twice, and the remote side does not
expect to receive it a second time.
The other thing it is used for is capabilities. Capabilities are there to
support protocol evolutions. For example, when the MetaObject
cache system was
added, a new capability was added so that we can identify processes using the
old protocol and not use that system with them. These capabilities are
associated with a remote host, so they are stored in the StreamContext
.
Serializing¶
Serialization is done by the class BinaryEncoder
in binarycodec.cpp. That
class is a visitor that has access to the StreamContext
and visits the type to
serialize. It fills-in a buffer with its associated signature so that they can
be sent in the Message
.
Deserializing¶
Deserialization is a done in a very similar way. When a message is received, the
associated signature is used to instantiate an empty AnyValue
with the type
interfaces corresponding to it. This is done in Message::value
.
Then the BinaryDecoder
visitor visits this empty value and fills it in with
the data from the buffer.