MynahSA is used to store and restore data for C++ object on streams. The streams support file I/O and network communications.
Conceptually, MynahSA is broken down into three levels of abstraction:
Conceptual Diagram
The streaming classes act as an interface between MynahSA's archiving system and the physical data stream. There are five different types of stream: TextStream, BinaryStream, SSLStream, TCPStream and BlowfishStream. These classes provide a structured input/output mechanism for the C/C++ built-in types (e.g. char, float, int ...) and std::string.
Stream classes are uni-directional, and the direction is identified by either an "I" for input or an "O" for output class name prefix. For example, MynahSA::IBlowfishStream denotes an input stream using the Blowfish algorithm for decryption.
MyanhSA's stream classes differ from the standard library streams in that they are designed for recovering data exactly as it was stored. For example, if a std::string containing spaces is written to a standard library stream the corresponding stream read (in-stream) operation will stop reading when it encounters a space or line-break. This behavior is expected because the standard library streams are designed for generating user readable text. In contrast, MynahSA's stream operators are not designed for readability. Control information is stored on the stream allowing the in-stream operator to recover the exact string that was stored.
The choice of stream type will vary depending on application - for file storage, Text, Binary or Blowfish streams may be used. For communications over the network, SSL and TCP streams are used. Naturally, the SSL and Blowfish streams provide data encryption - the other streams do not.
At the archive level, one class, MynahSA::Archive serves as the main interface between user classes and MynahSA's stream classes.
There is little design choice at the archive level: MynahSA::Archive has two descendants: IArchive for input and OArchive for output, which must be bound respectively onto an input or output stream instance.
Class MynahSA::Archive uses template meta-programming to classify types as either simple types (C/C++ built-in types), enumerations (enum), STL containers or user defined types. Types are classified at compile time and one of the following actions are chosen to handle each object:
-
C++ Built-in types and std::strings are passed directly between the stream and user class layers.
-
Enumerations in the User Class layer converted to integers at the stream layer.
-
STL Containers are recursively broken down into simple types and passed between the layers.
-
User defined types are archived by calling the user provided serialize method.
User classes interact with MynahSA::Archive by providing a method named serialize. The serialize method is symmetric - meaning it is responsible for both storing and restoring data from streams. Consider the following object definition:
class MyObject {
public:
MyObject();
~MyObject();
void serialize(MynahSA::Archive& ar) {
ar & _someInteger;
ar & _someString;
};
private:
int _someInteger;
std::string _someString;
};
The reason for forcing shared pointer usage is to prevent incomplete pointer life-cycle planning. All to often C++ projects fall foul of bad pointer programming practice leaving memory leaks, double deletion problems and segmentation faults to Valgrind and frustrated programmers to find.
With shared pointers most of the heap allocated object life cycle can be safely ignored - just throw the pointer away when it is no longer needed.
For MynahSA to store an object by shared pointer the object must derive from base class MynahSA::IoBase. Class MynahSA::IoBase acts as a pure base class requiring two members: serialize, which connects user defined types with MynahSA and ioName, which returns the name of the object on the stream. The ioName method is used by MynahSA::Archive to identify a specific object instance from a potential class hierarchy.
Restoring objects by a shared pointer requires that a constructor object be registered with the MynahSA::IArchive instance prior to attempting to restore the object. The constructor object allows the input archive to invoke a "virtual constructor" for the specific object type, instantiate the object type and invoke the correct serialize method associated with the object instance. This process ensures that storing and restoring of objects results in the correct object hierarchy.
When a shared pointer to an object is stored the entire object contents are written onto the stream. If the same pointer is stored a second time a Unique Pointer Reference (UPR) is stored in its place. UPRs allow graph structures to be stored without ending up in infinite loops. Graphs with circular links are converted into trees by replacing back-edges with a UPR.
Unlike XML, the data stored on streams using MynahSA does not contain explicit formatting information. There are no tags to identify the beginning or end of data members and there are no identifying names associated with data members. The data streams produced by MynahSA have no context on their own - the context only exists in the presence of the source code used to archive the respective objects. This approach has two key benefits:
-
Objects require significantly less storage on disk or network bandwidth.
-
Encoding and decoding requires significantly less CPU time and memory bandwidth.
The disadvantage is that the archives are not human readable, nor easily human modifiable. Furthermore, the streams are fragile - a byte inserted incorrectly will cause the stream to become corrupt and invalidate the data contained in the remainder of the stream. For most applications these are not serious concerns, however, care must be taken to ensure that the data streams are robustly stored.
Several other toolkits are available that warrant note and examination:
-
s11n - a toolkit for serializing C++ objects. Provides XML storage.
-
Boost::Serialization - Boost's serialization toolkit.
MynahSA was developed as an alternative to Boost's serialization toolkit. In fact, Boost::Serialization was in use for two internal projects at Mynah-Software prior to the decision to engineer MynahSA. Both projects communicated via SSL and used Boost::Serialization to manage object serialization. MynahSA was developed to work around three technical difficulties:
-
Inherent buffering of data: To send data, objects were first serialized onto a buffer, then the buffer contents were transferred over the network. The receiving end performed the opposite: read data from the network into a buffer, then restore the objects. Excessive copying was eliminated in the implementation of MynahSA::ISSLStream and MynahSA::OSSLStream.
-
Confusing object constructor behavior: To serialize objects by shared pointer Boost::Serialization uses a cool mechanism whereby constructors are registered through global object construction. In practice, GCC places the constructor registration code in the ELF ini/fini sections, which introduces a practical problem: ini/fini sections are executed in order determined by link order! For most programs this approach works fine, however, Mynah-Software (and its customers) were repeatedly stumped by problems related to link ordering. MynahSA uses explicit object registration to avoid this problem.
-
Template complexity and ease of learning: MynahSA was engineered for a single purpose - producing client/server applications in C++ with a minimal amount of effort. Learning, maintaining and enhancing code with complex template mechanics eats programmer time. MynahSA has been engineered to avoid this.
