Abstract interface for Hedgehog's Memory manager. More...

#include "abstract_memory_manager.h"

Inheritance diagram for hh::AbstractMemoryManager< ManagedMemory, class >:

Collaboration diagram for hh::AbstractMemoryManager< ManagedMemory, class >:

Public Member Functions
	AbstractMemoryManager ()=delete
	Deleted Default constructor.

	AbstractMemoryManager (size_t const &capacity)
	Only used constructor. More...

virtual	~AbstractMemoryManager ()=default
	Default destructor.

virtual std::shared_ptr< AbstractMemoryManager< ManagedMemory > >	copy ()
	Virtual copy method used for task duplication and execution pipeline. More...

void	deviceId (int deviceId)
	Device id setter. More...

void	profiler (const std::shared_ptr< NvtxProfiler > &profiler)
	NVTX profiler setter. More...

size_t	currentSize ()
	Return the current size of the inside pool. More...

std::shared_ptr< ManagedMemory >	getManagedMemory ()
	Get an available managed memory, block if none are available. More...

void	recycleMemory (std::shared_ptr< MemoryData< ManagedMemory >> managedMemory)
	Recycle memory. More...

virtual void	initialize ()
	Initialize the memory manager. More...

virtual void	initializeMemoryManager ()
	User-definable initialization step for a memory manager.

Protected Member Functions
std::unique_ptr< behavior::Pool< ManagedMemory > > const &	pool () const
	Inside pool accessor. More...

size_t	capacity () const
	Capacity accessor. More...

bool	isInitialized () const
	Initialized flag accessor. More...

std::mutex &	memoryManagerMutex ()
	User api mutex accessor. More...

void	initialized ()
	Flag the memory manager has initialized.

int	deviceId () const
	Device Id accessor. More...

Protected Attributes
std::mutex	memoryManagerMutex_ = {}
	Mutex for user interface.

Private Attributes
int	deviceId_ = 0
	Device Id of linked task.

bool	initialized_ = false
	Flag to determine if AbstractMemoryManager has been initialized.

std::unique_ptr< behavior::Pool< ManagedMemory > >	pool_ = {}
	Inside pool to store the data.

std::shared_ptr< NvtxProfiler >	profiler_ = nullptr
	NVTX profiler instance to follow memory manager state.

Detailed Description

template<class ManagedMemory, class = void>
class hh::AbstractMemoryManager< ManagedMemory, class >

Abstract interface for Hedgehog's Memory manager.

The memory managers for Hedgehog have two main goals:

Throttle the amount of memory that lives inside the graph,
Provide a mechanism for data reuse (with recycling to clean to-be-reused pieces of memory).

Attention

Data that is managed by an AbstractMemoryManager, requires:

Default constructible: can be constructed without arguments or initialization values, either cv-qualified or not. This includes scalar types, default constructible classes and arrays of such types (ref: http://www.cplusplus.com/reference/type_traits/is_default_constructible/),
Derive from MemoryData.

The amount of data available is set at construction: AbstractMemoryManager(size_t const &capacity). At construction, these data are default constructed and stored into a Pool. When constructed, a memory manager can be attached to a task with AbstractTask::connectMemoryManager().

Attention: The Type handled by the memory manager must be the same type as the AbstractTask output type.

Data can be retrieved from the memory manager from within the AbstractTask::execute method with AbstractMemoryManager::getManagedMemory(). If no data is available (i.e. the Pool is empty), the task will block on the call and wait for data to be recycled and available. To return data to the memory manager from another task, or, outside the graph, the method MemoryData::returnToMemoryManager() should be called.

Attention

When MemoryData::returnToMemoryManager() is invoked, the recycling mechanism is used which consists of calling the following virtual methods, in the following order:

MemoryData::used(): Method used to update the "state" of the MemoryData (for example in the case the MemoryData is returned to the AbstractMemoryManager multiple times before being recycled and sent back to the Pool.
MemoryData::canBeRecycled(): Boolean method to determine if the MemoryData can be recycled, and send back to the Pool.
MemoryData::recycle(): Recycle the MemoryData. The data given by the AbstractMemoryManager is default constructed the first time. If specialised class attributes are allocated, they should be deallocated in this method, to avoid memory leaks, to return the ManagedMemory to the same state as default construction.

The only pure virtual method is the copy method to duplicate a derived AbstractMemoryManager to different graphs in an ExecutionPipeline.

If an initialize step is needed to be added, AbstractMemoryManager::initializeMemoryManager() can be overload.

// Derived Memory data
template<class T>
class DynamicMemoryManageData : public MemoryData<DynamicMemoryManageData<T>> {
 private:
  T *data_ = nullptr;
  size_t ttl_ = 1; // Time to live for data reuse
 public:
  DynamicMemoryManageData() = default;
  virtual ~DynamicMemoryManageData() = default;
  void ttl(size_t ttl) { ttl_ = ttl; }
  void data(T *data) { data_ = data; }
  // Delete data allocated after default cstr by IntToDynamicMemoryManagedInt
  void recycle() override { delete[] data_; }
  void used() override { ttl_--; }
  bool canBeRecycled() override { return ttl_ == 0; }
};
// Derived AbstractMemoryManager
template<class T>
class DynamicMemoryManager : public AbstractMemoryManager<DynamicMemoryManageData<T>> {
 public:
  explicit DynamicMemoryManager(size_t const &capacity)
    : AbstractMemoryManager<DynamicMemoryManageData<T>>(capacity) {}
  std::shared_ptr<AbstractMemoryManager<DynamicMemoryManageData<T>>> copy() override {
    return std::make_shared<DynamicMemoryManager<T>>(this->capacity());
  }
};
// Task returning DynamicMemoryManagedData and getting data from a memory manager
class IntToDynamicMemoryManagedInt : public AbstractTask<DynamicMemoryManageData<int>, int> {
 public:
  explicit IntToDynamicMemoryManagedInt(size_t numberThreads) : AbstractTask("Dynamic Task", numberThreads) {}
  virtual ~IntToDynamicMemoryManagedInt() = default;
  void execute([[maybe_unused]]std::shared_ptr<int> ptr) override {
    auto mem = this->getManagedMemory(); // Get the memory from the memory manager
    mem->data(new int[30]()); // Allocate more memory
    mem->ttl(1); // Sets the time to live (number of times reused)
    addResult(mem);
  }
  std::shared_ptr<AbstractTask<DynamicMemoryManageData<int>, int>> copy() override {
    return std::make_shared<IntToDynamicMemoryManagedInt>(this->numberThreads());
  }
};
// Task accepting DynamicMemoryManagedData and return the data to the memory manager
class DynamicMemoryManagedIntToVoid :
    public AbstractTask<void, DynamicMemoryManageData<int>> {
 public:
  DynamicMemoryManagedIntToVoid() : AbstractTask("Output") {}
  virtual ~DynamicMemoryManagedIntToStaticMemoryManagedInt() = default;
  void execute(std::shared_ptr<DynamicMemoryManageData<int>> ptr) override {
    ptr->returnToMemoryManager(); // Return the data to the memory manager
  }
  std::shared_ptr<
      AbstractTask<void, DynamicMemoryManageData<int>>> copy() override {
    return std::make_shared<DynamicMemoryManagedIntToVoid>();
  }
};
// In main
auto dynamicTask = std::make_shared<IntToDynamicMemoryManagedInt>(2); // Create the task using the memory manager
// Create the task returning the data to the memory manager
auto outTask = std::make_shared<DynamicMemoryManagedIntToVoid>();
auto dynamicMM = std::make_shared<DynamicMemoryManager<int>>(2); // Create the memory manager
dynamicTask->connectMemoryManager(dynamicMM); // Associate the memory manager to the task

Virtual Functions: AbstractMemoryManager::copy
AbstractMemoryManager::initialize (advanced, should be used carefully)
AbstractMemoryManager::initializeMemoryManager (user-defined customization)

Template Parameters

ManagedMemory Type of data that will be managed by an AbstractMemoryManager

Definition at line 160 of file abstract_memory_manager.h.

Constructor & Destructor Documentation

◆ AbstractMemoryManager()

template<class ManagedMemory, class = void>

hh::AbstractMemoryManager< ManagedMemory, class >::AbstractMemoryManager ( size_t const & capacity )

inlineexplicit

Only used constructor.

Parameters

capacity Memory Manager capacity, number of elements available, set to 1 if 0

Definition at line 177 of file abstract_memory_manager.h.

Member Function Documentation

◆ capacity()

template<class ManagedMemory, class = void>

size_t hh::AbstractMemoryManager< ManagedMemory, class >::capacity ( ) const

inlineprotected

Capacity accessor.

Returns: Pool's capacity

Definition at line 259 of file abstract_memory_manager.h.

Here is the caller graph for this function:

◆ copy()

template<class ManagedMemory, class = void>

virtual std::shared_ptr<AbstractMemoryManager<ManagedMemory> > hh::AbstractMemoryManager< ManagedMemory, class >::copy ( )

inlinevirtual

Virtual copy method used for task duplication and execution pipeline.

Returns: Return a copy of this specialised AbstractMemoryManager

Reimplemented in hh::StaticMemoryManager< ManagedMemory, Args >.

Definition at line 188 of file abstract_memory_manager.h.

◆ currentSize()

template<class ManagedMemory, class = void>

size_t hh::AbstractMemoryManager< ManagedMemory, class >::currentSize ( )

inline

Return the current size of the inside pool.

Lock the api user mutex before getting the current size of the inside pool

Returns: The current number of available data

Definition at line 203 of file abstract_memory_manager.h.

Here is the caller graph for this function:

◆ deviceId() [1/2]

template<class ManagedMemory, class = void>

void hh::AbstractMemoryManager< ManagedMemory, class >::deviceId ( int deviceId )

inline

Device id setter.

Parameters

deviceId Task's device id to set

Definition at line 194 of file abstract_memory_manager.h.

Here is the caller graph for this function:

◆ deviceId() [2/2]

template<class ManagedMemory, class = void>

int hh::AbstractMemoryManager< ManagedMemory, class >::deviceId ( ) const

inlineprotected

Device Id accessor.

Returns: Device id

Definition at line 274 of file abstract_memory_manager.h.

◆ getManagedMemory()

template<class ManagedMemory, class = void>

std::shared_ptr<ManagedMemory> hh::AbstractMemoryManager< ManagedMemory, class >::getManagedMemory ( )

inline

Get an available managed memory, block if none are available.

Returns: An available managed memory

Definition at line 210 of file abstract_memory_manager.h.

◆ initialize()

template<class ManagedMemory, class = void>

virtual void hh::AbstractMemoryManager< ManagedMemory, class >::initialize ( )

inlinevirtual

Initialize the memory manager.

Lock the user api mutex, fill the pool with default constructed data, and call initializeMemoryManager()

Reimplemented in hh::StaticMemoryManager< ManagedMemory, Args >.

Definition at line 235 of file abstract_memory_manager.h.

◆ isInitialized()

template<class ManagedMemory, class = void>

bool hh::AbstractMemoryManager< ManagedMemory, class >::isInitialized ( ) const

inlineprotected

Initialized flag accessor.

Returns: True is the memory manager has been initialized, else False

Definition at line 263 of file abstract_memory_manager.h.

Here is the caller graph for this function:

◆ memoryManagerMutex()

template<class ManagedMemory, class = void>

std::mutex& hh::AbstractMemoryManager< ManagedMemory, class >::memoryManagerMutex ( )

inlineprotected

User api mutex accessor.

Returns: User api mutex

Definition at line 267 of file abstract_memory_manager.h.

◆ pool()

template<class ManagedMemory, class = void>

std::unique_ptr<behavior::Pool<ManagedMemory> > const& hh::AbstractMemoryManager< ManagedMemory, class >::pool ( ) const

inlineprotected

Inside pool accessor.

Returns: Inside pool

Definition at line 255 of file abstract_memory_manager.h.

Here is the caller graph for this function:

◆ profiler()

template<class ManagedMemory, class = void>

void hh::AbstractMemoryManager< ManagedMemory, class >::profiler ( const std::shared_ptr< NvtxProfiler > & profiler )

inline

NVTX profiler setter.

Parameters

profiler NVTX profiler to set

Definition at line 198 of file abstract_memory_manager.h.

Here is the caller graph for this function:

◆ recycleMemory()

template<class ManagedMemory, class = void>

void hh::AbstractMemoryManager< ManagedMemory, class >::recycleMemory ( std::shared_ptr< MemoryData< ManagedMemory >> managedMemory )

inline

Recycle memory.

Lock the user api mutex before call used(), canBeRecycled() and if true, recycle() and push it back into the pool

Parameters

managedMemory Data to recycle

Definition at line 223 of file abstract_memory_manager.h.

Here is the caller graph for this function:

The documentation for this class was generated from the following file:

/home/tjb3/work/hedgehog-nist/hedgehog/api/memory_manager/abstract_memory_manager.h

Public Member Functions

Protected Member Functions

Protected Attributes

Private Attributes

Detailed Description

template<class ManagedMemory, class = void> class hh::AbstractMemoryManager< ManagedMemory, class >

Constructor & Destructor Documentation

◆ AbstractMemoryManager()

Member Function Documentation

◆ capacity()

◆ copy()

◆ currentSize()

◆ deviceId() [1/2]

◆ deviceId() [2/2]

◆ getManagedMemory()

◆ initialize()

◆ isInitialized()

◆ memoryManagerMutex()

◆ pool()

◆ profiler()

◆ recycleMemory()

template<class ManagedMemory, class = void>
class hh::AbstractMemoryManager< ManagedMemory, class >