14.1.2. Meta-Classes

14.1.2.1. mosaic.partition.metaEventPartition module

class mosaic.partition.metaEventPartition.metaEventPartition(trajDataObj, eventProcHnd, eventPartitionSettings, eventProcSettings, settingsString, **kwargs)[source]

Bases: object

Warning

This metaclass must be sub-classed. All abstract methods within this metaclass must be implemented.

A class to abstract partitioning individual events. Once a single molecule event is identified, it is handed off to to an event processor. If parallel processing is requested, detailed event processing will commence immediately. If not, detailed event processing is performed after the event partition has completed.

Parameters:
  • trajDataObjproperly initialized object instantiated from a sub-class

    of metaTrajIO.

  • eventProcHndhandle to a sub-class of metaEventProcessor. Objects of

    this class are initialized as necessary

  • eventPartitionSettings : settings dictionary for the partition algorithm.

  • eventProcSettings : settings dictionary for the event processing algorithm.

  • settingsString : settings dictionary in JSON format

Common algorithm parameters from settings file (.settings in the data path or current working directory)

  • writeEventTS : Write event current data to file. (default: 1, write data to file)

  • parallelProc : Process events in parallel using the pproc module. (default: 1, Yes)

  • reserveNCPU : Reserve the specified number of CPUs and exclude them from the parallel pool.

  • driftThresholdTrigger a drift warning when the mean open channel current deviates by ‘driftThreshold’*

    SD from the baseline open channel current (default: 2)

  • maxDriftRateTrigger a warning when the open channel conductance changes at a rate faster

    than that specified. (default: 2 pA/s)

  • minBaseline : Minimum value for the ionic current baseline.

  • maxBaseline : Maximum value for the ionic current baseline.

abstract _init(trajDataObj, eventProcHnd, eventPartitionSettings, eventProcSettings)[source]

Important

Abstract method: This method must be implemented by a sub-class.

This function is called at the end of the class constructor to perform additional initialization specific to the algorithm being implemented. The arguments to this function are identical to those passed to the class constructor.

abstract _stop()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Stop partitioning events froma time-series

abstract _eventsegment()[source]

Important

Abstract method: This method must be implemented by a sub-class.

An implementation of this function should separate individual events of interest from a time-series of ionic current recordings. The data pertaining to each event is then passed to an instance of metaEventProcessor for detailed analysis. The function will collect the results of this analysis.

PartitionEvents()[source]

Partition events within a time-series.

Stop()[source]

Stop processing data.

abstract formatoutputfiles()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Return a formatted string of output files.

abstract formatsettings()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Return a formatted string of settings for display

abstract formatstats()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Return a formatted string of statistics for display

14.1.2.2. mosaic.process.metaEventProcessor module

class mosaic.process.metaEventProcessor.metaEventProcessor(icurr, icurrU, Fs, **kwargs)[source]

Bases: object

Warning

This metaclass must be sub-classed. All abstract methods within this metaclass must be implemented.

Defines the interface for specific event processing algorithms. Each event processing algorithm must sub-class metaEventProcessor and implement the following abstract functions:

  • processEventprocess raw event data and populate event meta-data. Store each

    piece of processed event data in a class attribute starting with ‘md’. For example, the blockade depth meta-data can be defined as ‘mdBlockadeDepth’

  • printMetadata : print meta-data set by event processing in a human readable format.

Parameters:

  • icurr : ionic current in pA

  • icurrU: ionic current in pA with unfiltered event padding

  • Fs : sampling frequency in Hz

Keyword Args:

  • eventstart : the event start point

  • eventend : the event end point

  • baselinestats : baseline conductance statistics: a list of [mean, sd, slope] for the baseline current

  • algosettingsdict : settings for event processing algorithm as a dictionary

  • absdatidx : index of data start. This arg can allow arrival time estimation.

  • datafilehnd : reference to an metaMDIO object for meta-data IO

__mdformat(dat)

Round a float to 3 decimal places. Leave ints and strings unchanged

abstract _init(**kwargs)[source]

Important

Abstract method: This method must be implemented by a sub-class.

abstract _mdHeadingDataType()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Return a list of meta-data tags data types.

abstract _mdHeadings()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Return a list of meta-data tags for display purposes.

abstract _mdList()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Return a list of meta-data set by event processing.

abstract _processEvent()[source]

Important

Abstract method: This method must be implemented by a sub-class.

flagEvent(status)[source]

Set a warning status that starts with ‘w’ for non-critical errors. Metadata for these events is preserved and the user can be warned (e.g., in the GUI.)

abstract mdAveragePropertiesList()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Return a list of meta-data properties that will be averaged and displayed at the end of a run. This function must be overridden by sub-classes of metaEventProcessor. As a failsafe, an empty list is returned.

mdHeadingDataType()[source]

Return a list of meta-data tags data types.

mdHeadings()[source]

Return a list of meta-data tags for display purposes.

processEvent()[source]

This is the equivalent of a pure virtual function in C++.

rejectEvent(status)[source]

Set an event as rejected if it doesn’t pass tests in processing. The status is assigned to mdProcessingStatus.

writeEvent()[source]

Write event meta data to a metaMDIO object.

14.1.2.3. mosaic.filters.metaIOFilter module

class mosaic.filters.metaIOFilter.metaIOFilter(**kwargs)[source]

Bases: object

Warning

This metaclass must be sub-classed. All abstract methods within this metaclass must be implemented.

Defines the interface for specific filter implementations. Each filtering algorithm must sub-class metaIOFilter and implement the following abstract function:

  • filterData : apply a filter to self.eventData

Parameters:

  • decimate : sets the downsampling ratio of the filtered data (default:1, no decimation).

Properties:

  • filteredData : list of filtered and decimated data

  • filterFs : sampling frequency after filtering and decimation

abstract _init(**kwargs)[source]

Important

Abstract method: This method must be implemented by a sub-class.

abstract filterData(icurr, Fs)[source]

Important

Abstract method: This method must be implemented by a sub-class.

This is the equivalent of a pure virtual function in C++.

Implementations of this method MUST store (1) a ref to the raw event data in self.eventData AND (2) the sampling frequency in self.Fs.

Parameters:

  • icurr : ionic current in pA

  • Fs : original sampling frequency in Hz

filterFs

Return the sampling frequency of filtered data.

filteredData

Return filtered data

abstract formatsettings()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Return a formatted string of filter settings

14.1.2.4. mosaic.mdio.metaMDIO module

class mosaic.mdio.metaMDIO.metaMDIO[source]

Bases: object

Warning

This metaclass must be sub-classed. All abstract methods within this metaclass must be implemented.

This class provides the skeleton for storing metadata generated by algorithms. It also provides an interface to query metadata, for example in a SQL database.

Properties:

  • dbColumnNames : a list of database column names

abstract _opendb(dbname, **kwargs)[source]

Important

Abstract method: This method must be implemented by a sub-class.

abstract _initdb(**kwargs)[source]

Important

Abstract method: This method must be implemented by a sub-class.

abstract _colnames(table=None)[source]

Important

Abstract method: This method must be implemented by a sub-class.

abstract closeDB()[source]

Important

Abstract method: This method must be implemented by a sub-class.

initDB(**kwargs)[source]

Initialize a new database file.

Parameters:

The arguments passed to init change based on the method of file IO selected, in addition to the common args below:

  • dbPath : directory to store the MD database (‘<full path to data directory>’)

  • colNames : list of text names for the columns in the tables

  • colNames_t : list of data types for each column.

openDB(dbname, **kwargs)[source]

Open an existing database file.

Parameters:

  • dbname : directory to store the MD database (‘<full path to data directory>’)

See also

The arguments passed to init change based on the method of file IO selected, in addition to the common args.

abstract queryDB(query)[source]

Important

Abstract method: This method must be implemented by a sub-class.

Query a database. :Parameters:

  • query : query string

See also

See specific implementations of metaMDIO for query syntax.

abstract readAnalysisInfo()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Read analysis information from the database.

abstract readAnalysisLog()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Read the analysis log from the database.

abstract readSettings()[source]

Important

Abstract method: This method must be implemented by a sub-class.

Read JSON settings from the database.

abstract writeAnalysisInfo(infolist)[source]

Important

Abstract method: This method must be implemented by a sub-class.

Write analysis information to the database. Note that subsequent calls to this method will overwrite the analysis inoformation entry in the table.

Args:
  • infolistA list of strings in the following order [ datPath, dataType, partitionAlgorithm, processingAlgorithm, filteringAlgorithm].

    datPath : full path to the data directory

    dataType : type of data processed (e.g. ABF, QDF, etc.)

    partitionAlgorithm : name of partition algorithm (e.g. eventSegment)

    processingAlgorithm : name of event processing algorithm (e.g. multStateAnalysis)

    filteringAlgorithm : name of filtering algorithm (e.g. waveletDenoiseFilter) or None if no filtering was performed.

abstract writeAnalysisLog(analysislog)[source]

Important

Abstract method: This method must be implemented by a sub-class.

Write the analysis log string to the database. Note that subsequent calls to this method will overwrite the analysis log entry.

Args:

  • analysislog : analysis log string to save

abstract writeRecord(data, table=None)[source]

Important

Abstract method: This method must be implemented by a sub-class.

Write data to a specified table. By default table is None. In this case sub-classes should fall back to writing data to a default table.

abstract writeSettings(settingsstring)[source]

Important

Abstract method: This method must be implemented by a sub-class.

Write the settings JSON object to the database.

Args:

  • settingsstring : a JSON_ formatted settings string.

14.1.2.5. mosaic.trajio.metaTrajIO module

class mosaic.trajio.metaTrajIO.metaTrajIO(**kwargs)[source]

Bases: object

Warning

This metaclass must be sub-classed. All abstract methods within this metaclass must be implemented.

Initialize a TrajIO object. The object can load all the data in a directory, N files from a directory or from an explicit list of filenames. In addition to the arguments defined below, implementations of this meta class may require the definition of additional arguments. See the documentation of those classes for what those may be. For example, the qdfTrajIO implementation of metaTrajIO also requires the feedback resistance (Rfb) and feedback capacitance (Cfb) to be passed at initialization.

Parameters:

  • dirname : all files from a directory (‘<full path to data directory>’)

  • nfiles : if requesting N files (in addition to dirname) from a specified directory

  • fnames : explicit list of filenames ([file1, file2,…]). This argument cannot be used in conjuction with dirname/nfiles. The filter argument is ignored when used in combination with fnames.

  • filter : ‘<wildcard filter>’ (optional, filter is ‘*’ if not specified)

  • start : Data start point in seconds.

  • end : Data end point in seconds.

  • datafilter : Handle to the algorithm to use to filter the data. If no algorithm is specified, datafilter is None and no filtering is performed.

  • dcOffset : Subtract a DC offset from the ionic current data.

  • filtersettings: Dict containing low pass filter settings (optional: if not provided filter settings will be loaded from the settings file. If no settings are found, datafilter will be turned off.)

Properties:

  • FsHz : sampling frequency in Hz. If the data was decimated, this property will hold the sampling frequency after decimation.

  • LastFileProcessed : return the data file that was last processed.

  • ElapsedTimeSeconds : return the analysis time in sec.

Errors:

  • IncompatibleArgumentsError : when conflicting arguments are used.

  • EmptyDataPipeError : when out of data.

  • FileNotFoundError : when data files do not exist in the specified path.

  • InsufficientArgumentsError : when incompatible arguments are passed

abstract _init(**kwargs)[source]

Important

Abstract method: This method must be implemented by a sub-class.

This function is called at the end of the class constructor to perform additional initialization specific to the algorithm being implemented. The arguments to this function are identical to those passed to the class constructor.

abstract _formatsettings()[source]

Log settings strings

DataLengthSec

Important

Property

Return the approximate length of data that will be processed. If the data are in multiple files, this property assumes that each file contains an equal amount of data.

ElapsedTimeSeconds

Important

Property

Return the elapsed time in the time-series in seconds.

property FsHz

Important

Property

Return the sampling frequency in Hz.

LastFileProcessed

Important

Property

Return the last data file that was processed

ProcessedFiles

Important

Property

Return a list of processed data filenames.

formatsettings()[source]

Return a formatted string of settings for display

popfnames()[source]

Pop a single filename from the start of self.dataFiles. If self.dataFiles is empty, raise an EmptyDataPipeError error.

Parameters:

  • None

Returns:

A single filename if successful.

Errors:

  • EmptyDataPipeError : when the filename list is empty.

previewdata(n)[source]

Preview data points in self.currDataPipe. This function is identical in behavior to popdata, except it does not remove data point from the queue. Like popdata, it uses recursion to automatically read data files when the queue length is shorter than the requested data points. When all data files are read, an EmptyDataPipeError is thrown.

Parameters:

n : number of requested data points

Returns:

  • Numpy array with requested data

Errors:

  • EmptyDataPipeError : if the queue has fewer data points than requested.

abstract readdata(fname)[source]

Important

Abstract method: This method must be implemented by a sub-class.

Return raw data from a single data file. Set a class attribute Fs with the sampling frequency in Hz.

Parameters:

  • fname : fileame to read

Returns:

An array object that holds raw (unscaled) data from fname

Errors:

None

scaleData(data)[source]

Important

Abstract method: This optional interface method can be overridden by a sub-class to modify functionality.

Scale the raw data loaded with readdata(). Note this function will not necessarily receive the entire data array loaded with readdata(). Transformations must be able to process partial data chunks.

Parameters:

  • data : partial chunk of raw data loaded using readdata().

Returns:

  • Array containing scaled data.

Default Behavior:

  • If not implemented by a sub-class, the default behavior is to return data to the calling function without modifications.

Example:

Assuming the amplifier scale and offset values are stored in the class variables AmplifierScale and AmplifierOffset, the raw data read using readdata() can be transformed by scaleData(). We can also use this function to change the array data type.

def scaleData(self, data):
        return np.array(data*self.AmplifierScale-self.AmplifierOffset, dtype='f8')