Class DeviceMemoryAllocator

Class Documentation

class mgb::cg::DeviceMemoryAllocator

allocation strategy for device storage in computing graphs

Note: all the graph params would be NULL for requests originating from ComputingGraph::prealloc_static_storage. Otherwise they are not NULL.

This base class already provides an implementation using memory management on the comp node. Sub-classes can override only the methods of interest.

Public Functions

~DeviceMemoryAllocator() = default
void alloc_static(ComputingGraph *graph, DeviceTensorStorage &dest, size_t size)

implement the allocation strategy for static graph-wise storage

Parameters

  • [in] graph: the computing graph that requests the memory

  • [out] dest: output tensor storage; its comp node has been initialized to target comp node

void alloc_dynamic(VarNode *var, DeviceTensorStorage &dest, size_t size)

implement the allocation strategy for dynamic storage of a variable

Note: if allocation fails,

MemAllocError should be raised so VarDevMemDefragmenter can catch the error and do defragmentation.
Parameters

  • [in] var: the variable that needs memory

void defrag_prealloc_contig(ComputingGraph *graph, CompNode comp_node, size_t size)

Ensure a contiguous storage for memory defragmenter.

When doing memory-defragmentation, it is useful to ensure that following allocation requests can be placed in a contiguous storage. This function would be called before calling alloc_dynamic() on the individual vars.

size_t static_alloc_version(ComputingGraph *graph) const

version of static allocation strategy

If version changes before graph exec, static memory would be reallocated. This function would be only called once in each graph execution.

Public Static Attributes

constexpr size_t VERSION_INVALID = ~static_cast<size_t>(0)

a version sentinel value that should never be returned by static_alloc_version()