Class MemAllocPlan

Inheritance Relationships

Base Types

  • public Serializable

  • public NonCopyableObj

Class Documentation

class mgb::cg::MemAllocPlan : public Serializable, public NonCopyableObj

memory allocation plan held by a variable

A MemAllocPlan is a view (i.e. with offset and layout) for some Chunk; Memory sharing between vars is implemented by sharing a Chunk of their mem plans.

Public Functions

MemAllocPlan(VarNode *owner_var)
bool valid() const
DType dtype() const

dtype of owner var

TensorFormat format() const

tensor format of owner var

Chunk &chunk()

get associated chunk

bool is_invalid_cond_exec() const
size_t offset_in_chunk_byte() const

get offset in bytes of this MemAllocPlan in associated chunk

const TensorLayout &layout() const
MemAllocPlan &layout(const TensorLayout &dest, bool allow_shape_change = false)
MemAllocPlan &release_chunk()

release current chunk and decr its refcnt

Release tensor storage if refcnt drops to zero

MemAllocPlan &reset_from_owner_var()

reset chunk to a privately owned chunk, and setup offset and layout from owner var, and clear tensor storage

MemAllocPlan &reset_as_invalid_cond_exec()

reset to a special marker that indicates this var is not computed in conditional execution

This is used in VarNodeMemManager to detect if the var is invalid without adding a new field.

MemAllocPlan &reset_to_uninitialized()

reset to uninitialized status

This is called before calling OperatorNodeBase::init_output_mem_plan and before memplan optimization.

MemAllocPlan &assign(const MemAllocPlan &src)

assign layout, offset and chunk from another mem alloc plan

MemAllocPlan &assign_for_forward(const MemAllocPlan &src, const SubTensorSpec &sub)

assign for readonly forward

MemAllocPlan *next_readonly_fwd_reader() const

next readonly-forward reader of this MemAllocPlan

All the readers of a MemAllocPlan form a singly-linked list which is maintained by assign_for_forward().

VarNode *owner_var() const

the var that owns this mem plan

class Chunk : public NonCopyableObj

identifier for allocated memory

Each Chunk object corresponds to an allocated memory chunk. Memory forwarding and force updating are implemented by sharing Chunk objects between vars.

If mem_alloc_status is not invalid, the memory region for this chunk is owner_var->dev_tensor().storage().

Public Functions

size_t size() const

size of this chunk in bytes

void update_size_for_dynamic_alloc(size_t size)

update value of m_size, only used in dynamic var allocation

std::string id_str() const
Chunk(VarNode *ov)

Public Members

VarNode *const owner_var

var that first creates this chunk

MemAllocStatus mem_alloc_status
class MemAllocStatus

memory allocation status for this chunk

Allocation status can either be INVALID, FROM_OWNER_VAR, or an offset in a static allocation buffer. This status is compactly represented by an integer value. No error check is performed in the accessors.

Note that for static_offset, it is set in SeqMemOptimizer::plan_chunk_allocation() and accessed in VarNodeMemManager::make_static_var_tensor_from_alloc_plan()

Public Functions

bool is_invalid() const

whether memory is not allocated yet

bool is_from_owner_var() const

whether memory comes from owner_var->dev_tensor()

bool is_static_offset() const

whether memory is statically allocated

size_t static_offset() const
void set_invalid()
void set_from_owner_var()
void set_static_offset(size_t offset)