Tutorial 1: Simple Object Persistence

Saving of state is a fundamental requirement of most applications. MynahSA makes it easy to store the state of your application onto a standard stream. The process involves five steps:

Adding serialize methods to all objects requiring persistence.
Instantiating a formatted stream object, and binding it onto an existing std::stream.
Instantiating an input or output archive object, and binding it onto the formatted stream.
Registering any necessary constructor objects with input archive instances.
Storing or Recovering the desired object hierarchy to/from and archive.

This example shows the creation of a simple application that acts as a trivial database storing particulars for an individual. The example source code can be found in the examples/tutorial1 directory.

The database records consist of the following structure, and may be found in personrecord.hpp:

struct PersonRecord { 
  enum EyeColor { Red, Blue, Green, Brown };

  PersonRecord();

  PersonRecord(std::string name, int age, EyeColor ec);

  std::string name;
  int age;
  EyeColor _eyeColor;
};

The first step to archiving the PersonRecord data structure is to add a serialize method. MynahSA has built in handling all C/C++ native Non-Pointer Types non-pointer type objects:

#include <mynahsa/archive.hpp>

struct PersonRecord { 
  ...
  void serialize(MynahSA::Archive& ar);

};

To avoid namespace pollution, the MynahSA:: namespace is used explicitly in header files rather than making it implicit with a "using" declaration - this is a design decision and not a requirement. The implementation is trivial, and placed in personrecord.cpp:

using namespace MynahSA;
void PersonRecord::serialize(Archive& ar) { 
  ar & _name;
  ar & _age;
  ar & _eyeColor;
}

Class Archive's operator& is overloaded and is used to both read and write data.

A simple example can be used to show the contents of the archive as it would exist on disk or on the stream. This code is from examples/tutorial1/main.cpp:

PersonRecord testRecord("Brett",32,Blue);
  
// create a text formatted output stream
OTextStream ots(cout);
  
// create an output archive on the formatted stream
OArchive<OTextStream> oa(ots);
          
// store the object onto the stream  - in this
//  case, the archive contents will be placed onto
//  cout
cout << "testRecord on stream looks like this: " << endl;
oa << testRecord;
cout << endl;

The separation of archiving and streaming is used so that components may be interchanged. If OBinaryStream were substituted for OTextStream, binary data would be written directly onto an std::ostream instead of formatted text. Direct use of ostream is not recommended as the standard i/o streams do not format object data in a robust manner. For example, when writing std::string instances OTextStream stores the length of a string followed by its contents. In comparison, std::istream::operator>> would stop reading characters when a space or carriage return character is received.

A simple examination of the output shows the contents of the stream:

5 Brett32 2

Here we see the number of characters in the following string, followed by the age and the numeric value for the enumeration PersonRecord::Blue. The archiver has simply followed the specification in PersonRecord::serialize and stored the name, age and eye-color in order.

To write a single PersonRecord onto disk an std::ofstream instance can be created and bound onto a binary archiving stream. The procedure is conceptually identical to the creation of a text stream above:

ofstream ofs("testsave.dat");
OBinaryStream obs(ofs);
OArchive<OBinaryStream> obas(obs); // output binary archive stream
obas << testRecord;

To recover the PersonRecord from the file, the process is reversed: A std::istream instance is bound a formatted input binary stream, which acts as the data source for an input archive. The input streaming operator is used to restore the object:

ifstream ifs("testsave.dat");
IBinaryStream ibs(ifs);
IArchive<IBinaryStream> ibas(ibs);  // input binary archive stream

PersonRecord testRecord2;
ibas >> testRecord2;
cout << "testRecord2 after restore from file is: " << endl << testRecord2 
     << endl;

STL container storage

MynahSA supports storage of std::list, std::map, std::multimap, std::set and std::vector containers. The template rules for C++ template instantiation are applied recursively, therefore, recursive STL container usage is supported.

To demonstrate STL container usage, this part of the tutorial uses a std::multimap to hold several PersonRecord(s). The std::multimap is archived onto a stream, and restored to demonstrate its usage:

multimap<int, PersonRecord> mm;
PersonRecord brett("Brett",32,PersonRecord::Blue);
mm.insert(pair<int, PersonRecord>(32,brett));
    
PersonRecord agnes("Agnes",35,PersonRecord::Blue);
mm.insert(pair<int, PersonRecord>(35, agnes));
   
PersonRecord john("John",32,PersonRecord::Brown);
mm.insert(pair<int, PersonRecord>(32,john));
    
// now drop mm onto a file stream 
ofstream ofs("testmultimap.dat");
OBinaryStream obs(ofs);
OArchive<OBinaryStream> obas(obs);
obas << mm;

Recovering the contents of the multimap from the file stream is identical to recovering other objects:

ifstream ifs("testmultimap.dat");
IBinaryStream ibs(ifs);
IArchive<IBinaryStream> ibas(ibs);
     
multimap<int, PersonRecord> mm;
ibas >> mm;
        
cout << "People between the age of 30 and 40: " << endl;
for (multimap<int, PersonRecord>::const_iterator cit = mm.lower_bound(30);
     cit != mm.upper_bound(40);
     ++cit) { 
  cout << (*cit).second << endl;
}

Note: Throughout the tutorial the type of objects loaded off of a stream are identical to those that were stored onto the stream. One important difference between MynahSA and XML is that MynahSA does not store object type information on the stream - the semantics of the data only exist in the context of the "serialize" methods used to load or store them. The stream is viewed as a pure data stream, and the behavior of the system is undefined when restoring type-mismatched data.

Shared Pointer types

MynahSA supports streaming and archiving of shared pointer types. A shared pointer is a reference counted pointer that automatically deletes the heap allocated object when the reference count goes to zero. MynahSA provides a reference counted pointer implementation in class MynahSA::SharedPtr, however, the shared pointer implementation may be replaced by a project specific one at compile time.

The storage of shared pointers requires a bit more work when compared with non-pointer objects. The additional work is required to manage class hierarchies, and the mechanism is derived from Stroustrap's The C++ Programming Language section 25.4.1.

The extension process starts by deriving from class MynahSA::IoBase. Class MynahSA::IoBase is a pure virtual base class that requires the implementation of two functions - the first function is:

virtual void serialize(Archive& ar);

The semantics of the serialization method are identical to the methods shown above. The only difference is that this method's implementation is now required as it is a pure virtual method.

The next method that must be overridden is:

virtual std::string ioName() const;

The ioName method is responsible for returning the class name as an std::string. Whenever a pointer object is stored on the stream its identifying name is stored prior to the data for the class - this ensures that the process of recovering an object off of the stream creates the correct object type in a class hierarchy. Therefore, the programmer must provide a constructor function and register it with each MynahSA::IArchive instance prior to attempting to restore an object by a shared pointer from an input stream.

To extend the PersonRecord the class ExtendedPersonRecord is derived from both PersonRecord and MynahSA::IoBase:

class ExtendedPersonRecord : public PersonRecord, public MynahSA::IoBase {
public:
  ExtendedPersonRecord();
  
  ExtendedPersonRecord(std::string name, int age, PersonRecord::EyeColor ec);

  ~ExtendedPersonRecord();
  
  std::string ioName() const;
  
  void serialize(MynahSA::Archive& ar);
  
  const std::list< SHARED_PTR< ExtendedPersonRecord > >& 
    getFriends() const;
  
  void addFriend(SHARED_PTR< ExtendedPersonRecord > newFriend);

private:
  std::list< SHARED_PTR< ExtendedPersonRecord > > _friends;
};

Note: The MynahSA source code uses the macro SHARED_PTR to represent the user's choice of shared pointer. When no specialization of the MynahSA library has been performed, the builtin MynahSA::SharedPtr implementation is used.

ExtendedPersonRecord can be used to create graphs of friends - the friends list can be used to connect multiple ExtendedPersonRecords in non-trivial ways. The complication with this data structure is that it may contain loops: Consider the case where Brett and Agi are mutual friends - this causes both records for Brett and Agi to point to each other, hence a circular set of pointers.

MynahSA resolves these situations by treating pointers as unique addresses. Once MynahSA has stored a pointer it assigns a Unique Pointer Reference (UPR) to the original object's address. Should that original object be transmitted a second time by the archiver, its UPR is transmitted instead. On the receiving end, every time a pointer object is received it is assigned a UPR. When a UPR is received the target pointer is assigned to the object created earlier in the streaming process with identical UPR. The process allows graph data structures with back-edges (e.g. circular loops) to be archived.

To demonstrate storage of a graph with loops we will create one using the friendship relationships:

// create the database
PersonDatabase db;
// create the individuals
SHARED_PTR<ExtendedPersonRecord> 
  brett(new ExtendedPersonRecord("Brett",32,PersonRecord::Blue));
SHARED_PTR<ExtendedPersonRecord> 
  agi(new ExtendedPersonRecord("Agi",35,PersonRecord::Blue));
SHARED_PTR<ExtendedPersonRecord> 
  jim(new ExtendedPersonRecord("Jim",65, PersonRecord::Blue));
    
// construct the friendship graph
brett->addFriend(agi);
brett->addFriend(jim);
    
agi->addFriend(brett);
agi->addFriend(jim);
    
jim->addFriend(brett);
jim->addFriend(agi);

The graph created consists of three nodes and is fully connected - everyone is a friend of each other. To store this archive onto a stream the exact same procedure is performed as in the previous parts of this tutorial:

ofstream ofs("persondb.txt");
OTextStream ots(ofs);
OArchive<OTextStream> otas(ots);
otas << db;

The next step is to recover the data from the file, and display the database. The procedure is similar to the previous parts of this tutorial; however, a constructor object is required by MynahSA::IArchive to recover shared pointer objects. To provide a constructor, two predefined macros - MYNAHSA_BUILD_CONSTRUCTOR and MYNAHSA_REGSITER_CONSTRUCTOR are used to create and register the new constructor with a MynahSA::IArchive instance. At the top of the the constructor definition is added:

#include <mynahsa/iarchiveregister.hpp>
...
#include "extendedpersonrecord.hpp"
...
MYNAHSA_BUILD_CONSTRUCTOR(ExtendedPersonRecord);

Prior to recovering an object by a shared pointer the constructor must be registered with an input archive reference:

// create the input stream, bind it to the data file
ifstream ifs("persondb.txt");
// create the formatted input text stream
ITextStream its(ifs);
// create the input archiver instance
IArchive<ITextStream> itas(its);

// Register the construction object with the IArchive instance
MYNAHSA_REGISTER_CONSTRUCTOR(itas, ExtendedPersonRecord);

The contents of the file may now be loaded from disk into a database instance and displayed:

PersonDatabase db;
itas >> db;

cout << endl << "Here is the contents of the database: " << endl << endl;
cout << db << endl;

One important thing to note about the storing and recovering of the database above: Each instance of PersonDatabase was stored directly, without using a shared pointer. If storing a shared pointer to the database was required, the registration process for class PersonDatabase would be required. In the event that an unknown object is recovered off of a stream an exception will be thrown. Catching the MynahSA::ExceptionBase exception and examining its error string would indicate that the input archive did not know the specific object type.

Summary

This tutorial has demonstrated the basics of MynahSA, showing the four primary steps that apply for all programs using MynahSA:

Addition of serialize() and ioName() methods to objects.
The constructor of a stream object.
The binding of an archiver onto the stream.
The streaming of data onto and off of archives.

With specifics aside, all MynahSA code follows these steps. The later tutorials show how these steps are used to build encrypted disk files and communicate over the network using TCP or SSL.

Notes

Non-Pointer Types

One thing that most C/C++ programmers may find odd is that the storage of standard C-sytle pointers has been omitted. MynahSA does not provide archiving for standard C/C++ pointers. This may change in the future depending on demand; however, this restriction has been a design decision. Far too often C++ programs fall foul from incorrect pointer life cycle planning - use of shared pointers greatly simplifies the design.

Archive's use of shared pointers

Class archive holds a list of shared pointers both when storing data (with class OArchive) and loading data (with IArchive). The pointer holding is necessary as there is no signal for terminating the transmission of objects. When finished transmitting data, call Archive::clearUPR(). This method will cause an Archive instance to free all of its references, potentially freeing memory in the process.