In my last post I developed the header for my networking framework. At this point I believe that I can create a working protocol that uses this header, but that would be a misguided undertaking without thinking about how to do it efficiently.
Developing My Framework
One Connection, Many Modules?
One thing that I have slowly settled on is creating one connection from one machine to another, and utilize that for all modules in the application that need to talk to that machine. I feel that this is a better approach than letting all modules create their own connections as it creates a lot of overhead for modules and objects that are short-lived and tend to come in and out of existence quickly, as each connection may require some kind of handshake.
So then I seem to be implying the existence of some object that manages a single connection, some sort of connection manager. However, since all communication from one machine to another is handled by a single object, then all modules in the application that communicate to that machine will need a reference to that object. This starts to break down immediately: if two modules send out a message to the same machine, then both sit and wait for this connection manager to give them a response, how do we know that the right response will go to the right module?
There’s no guarantee that the computer on the other end, much less the network, will process them and return them diligently in the same order they were sent, and even worse there’s a race condition here where one thread might read first even though it sent last. In order to solve this problem, there are a number of solutions:
- Only allow a single thread to access the networking code at a time (lock on write, release on read)
- Make the connection manager responsible for differentiating between modules
- Give each message to both modules and let them differentiate which message is for them
Clearly the first option is bad from a design perspective: going with this option imposes a single-thread restriction on the application using it. It would be very rude for a nicely threaded application to experience a sudden slowdown from network traffic; even worse, a barrier like that would not be expected from something like a connection manager, so it’s an unadvertised side-effect, and a particularly nasty one at that.
The second option also has an issue when taking the mission statement of this project into consideration: putting this responsibility onto the connection manager adds overhead to all network processing. There is a concept in programming that you should not need to pay for something that you won’t use; if this were the case, a single-threaded application would be necessarily forced to pay for this constantly for all network traffic.
Thus, the tertiary option seems to be the best, but it begs another question: how do you know who should get any given message? Should you send all messages to all modules? That seems like a waste of time, as it will be white noise and incur additional processing cost on all modules, which is not going to cut it. So how do we decide who gets a given message?
Message Filtering
I will use the observer pattern to accomplish this goal. Given the fact that the header has a job specifier, I will let each module subscribe to each job specifier code, then the connection manager will be able to filter messages to the components who care to hear about them. Additionally, components will be able to unsubscribe at will from any job specifier to stop receiving updates when those messages come in.
This is a helpful mechanism because a module can subscribe to a particular job specifier, then send a message they expect a response to, then wait for their response to come in, then finally unsubscribe to stop being notified. Unfortunately, if a module gets messages that belong to other modules, how can it tell that a message is for it versus another module? The answer to this question is the transaction ID that the message is sent out with. This allows for 256 different messages per job specifier to be in flight at the same time without causing a single collision, and also allows the module to have an easy way to differentiate two messages without a lot of processing required, lowering the overhead that we have given to the modules off of the connection manager.
This also comes with another very nice feature: multicast. With this feature, two modules can receive the same update without communicating with each other about it. On the surface this seems somewhat useless, but there is a great application of it hiding in there.
Say that you wanted to incorporate some sort of networking statistics in your application. The connection manager could certainly keep track of the total network traffic (very easily, in fact). However, what must you do if you want to have more granular statistics, such as the ratio of the different jobs being communicated about? It would likely be necessary to create a module that controls the statistics, then have each of the other modules report to it when they perform a send or a receive. This, however, requires the modules to be very tightly coupled, and it also requires each other module to share the responsibility of the statistics module. Even worse, it violates the SOLID principles, as adding the statistics module necessarily requires modifying the existing modules.
This design problem is avoided completely by utilizing the multicast feature of the connection manager. Instead of requiring the modules to keep track of their own networking statistics, the statistics module can simply subscribe to the messages that it wants to measure. This means that none of the other modules ever require knowledge of the statistics module and do not share its responsibility. With multicast, we are able to isolate the responsibility of the statistics module from all of the other modules and create a more loosely coupled applications.
Message Queuing
Another problem arises with this setup: what if a thread isn’t actively reading when a response comes in? What if, for instance, a response comes in and a thread reads it, checks if it belongs to it, decides it doesn’t, then begins reading again, but in the time that they weren’t reading, the response they were waiting for arrived and they missed it? The thread would be stuck waiting forever for a message that it missed.
The solution to this is to queue all messages that a module is subscribed to. Clearly it can’t be a global queue, as a module will only need to handle a message once and it should not find a duplicate message in the queue. Thus, it is important that each module has its own message queue that the network manager will push messages into.
Further, suppose there was a nesting situation of network reads and writes, where in an outer loop a module is listening for job 0, and inside an inner loop, another write/read sequence for job 0 takes place; what would happen if the inner loop unsubscribed from job 0? The outer loop would stop receiving events!
To fix this, it becomes clear that the connection manager must have access to a different queue to operate on so that a subscription and unsubscription cycle does not imply a side-effect for any of the previous transactions.
I intend to solve this by encapsulating this behavior into a connection object that interacts with the connection manager. This simulates the behavior we wanted when we made multiple connections between two machines to separate the reads and writes, but with the added benefits of a lower overall overhead, multicast, and a built-in framework for communicating intent between two applications.
This is another sign that my design is moving in the right direction; it is beginning to work exactly how I would want it to work, and I am getting added benefit from using it versus another solution.
542