9. Extend MOSAIC

MOSAIC was designed from the start using object oriented tools, which makes it easy to extend. Meta-Classes define interfaces to five key parts of MOSAIC: time-series IO (metaTrajIO), time-series filtering (metaIOFilter), analysis output (metaMDIO), event partition and segmenting (metaEventPartition), and event processing (metaEventProcessor). Sub-classing any of these meta classes and implementing their interface functions allows one to extend MOSAIC while maintaining compatibility with other parts of the program. We highlight these capabilities via two examples. In the first example, we show how one can extend metaTrajIO to read arbitrary binary files. In the second example, we implement a new top-level class that converts files to the comma separated value (CSV) format.

9.1. Read Arbitrary Binary Data Files

In this first example, we implement a class that can read an arbitrary binary data file and make its data available via the interface functions in metaTrajIO. This allows the newly implemented binary data to be used across MOSAIC. A complete listing of the code used in this example (binTrajIO) is available in the API documentation.

The new binary IO class is implemented by sub-classing metaTrajIO as shown in the listing below.

class binTrajIO(metaTrajIO.metaTrajIO):

Next, we must fully implement the metaTrajIO interface functions (_init(), readdata() and _formatsettings()). Note that the arguments of each function must match their corresponding base-class versions. For example the _init() function only accepts keyword arguments and is defined as shown below.

def _init(self, **kwargs):

The _init() function checks the arguments passed to kwargs and raises an exception if they are not defined.

        if not hasattr(self, 'SamplingFrequency'):
                raise metaTrajIO.InsufficientArgumentsError("{0} requires the sampling rate in Hz to be defined.".format(type(self).__name__))
        if not hasattr(self, 'PythonStructCode'):
                raise metaTrajIO.InsufficientArgumentsError("{0} requires the Python struct code to be defined.".format(type(self).__name__))

Next we define the readdata() function that reads in the data and stores the results in a numpy array. This array is then passed back to the calling function.

     def readdata(self, fname):

             tempdata=np.array([])
             # Read binary data and add it to the data pipe
             for f in fname:
                     tempdata=np.hstack(( tempdata, self.readBinaryFile(f) ))

             return tempdata

Finally, we implement the _formatsettings() that returns a formatted string of the settings used to read in binary data.

def _formatsettings(self):
        """
                Return a formatted string of settings for display
        """
        fmtstr=""

        fmtstr+='\n\t\tAmplifier scale = {0} pA\n'.format(self.AmplifierScale)
        fmtstr+='\t\tAmplifier offset = {0} pA\n'.format(self.AmplifierOffset)
        fmtstr+='\t\tHeader offset = {0} bytes\n'.format(self.HeaderOffset)
        fmtstr+='\t\tData type code = \'{0}\'\n'.format(self.PythonStructCode)

        return fmtstr

The newly defined binTrajIO class can then be used as shown below and in Scripting and Advanced Features.

# Process all binary files in a directory
mosaic.SingleChannelAnalysis.SingleChannelAnalysis(
            "~/RefData/binSet1/",
            bin.binTrajIO,
            None,
            es.eventSegment,
            mosaic.adept2State.adept2State
        ).Run()

Similar to other TrajIO objects, parameters for binTrajIO are obtained from the settings file when used with SingleChannelAnalysis. Example settings for binTrajIO that read 16-bit intgers from a binary data file, assuming 50 kHz sampling, are shown below.

"binTrajIO" : {
        "filter"                : "*bin",
        "AmplifierScale"        : "1.0",
        "AmplifierOffset"       : "0.0",
        "SamplingFrequency"     : "50000",
        "HeaderOffset"          : "0",
        "PythonStructCode"      : "'h'"
}

9.2. Define Top-Level Functionality

New functionality can be added to MOSAIC by combining other parts of the code. One way of accomplishing this is by defining new top-level functionality as shown in the following example. We define a new class that converts data from one of the supported data formats to comma separated text files (CSV). A complete listing of the ConvertToCSV class in this example is available in the API documentation.

The __init__ function of ConvertToCSV class accepts two arguments: a trajIO object and the location to save the converted files. If the output directory is not specified, the data is saved in the same folder as the input data. The data conversion is performed by the Convert() function, which saves the data in blocks controlled by the blockSize parameter. Convert() saves each block to a new CSV file, named with the filename of the input data followed by an integer number (see the API documentation for _filename() for additional details).

class ConvertToCSV(object):
        def __init__(self, trajDataObj, outdir=None):
                self.trajDataObj=trajDataObj
                self.datPath=trajDataObj.datPath

                # If outdir is None, save the CSV files to the same directory as the data.
                if outdir==None:
                        self.outDir=self.datPath
                else:
                        self.outDir=outdir

                self.filePrefix=None
                self._creategenerator()

        def Convert(self, blockSize):
                data=numpy.array([], dtype=numpy.float64)

                try:
                        while(True):
                                (self.trajDataObj.popdata(blockSize)).tofile(
                                                self._filename(),
                                                sep=','
                                        )
                except EmptyDataPipeError:
                        pass

The ConvertToCSV class can now be used with any trajIO object as seen below.

ConvertToCSV( abfTrajIO(dirname="~/RefData/abfSet1/", filter="*abf") ).Convert(
                        blockSize=50000)

ConvertToCSV( qdfTrajIO(dirname="~/RefData/qdfSet1/", filter="*qdf", Rfb="2.1E+9",
                        Cfb="1.16E-12") ).Convert(blockSize=50000)

ConvertToCSV( binTrajIO(dirname="~/RefData/binSet1/", filter="*bin", AmplifierScale=1.0,
                        AmplifierOffset=0.0, SamplingFrequency=50000, HeaderOffset=0,
                        PythonStructCode='h') ).Convert(blockSize=50000)

Since ConvertToCSV accepts a trajIO object, we can apply a lowpass filter to the data before converting it to the CSV format. This is accomplished by passing the datafilter option to the trajIO object as described in the Filter Data section. In the example below, we convert ABF files to the CSV format after applying a lowpass Bessel filter to the data.

ConvertToCSV( abfTrajIO(dirname="~/RefData/abfSet1/", filter="*abf",
                datafilter=mosaic.besselFilter
        ) ).Convert(blockSize=50000)

Finally, the ConvertToCSV class can be further extended to output arbitrary binary files in place of CSV by the simple extension shown below.