Class GraphExecutable::ExecEnv

Nested Relationships

This class is a nested type of Class GraphExecutable.

Class Documentation

class mgb::cg::GraphExecutable::ExecEnv

operator execution environment

When GraphExecutable::execute() is called, it should add tasks into the ExecEnv. The tasks added to ExecEnv would be invoked by a scheduler to perform actual computing.

Operator code usually only calls dispatch_on_comp_node()

Note: The ExecEnv class exists as an abstraction layer for controlling asynchronous kernel dispatching behavior. CUDA has a limited task queue so we have to use a dedicated CPU thread for each CUDA stream (it can be treated as a blocking queue). But for CPU we have our own unlimited dispatch queue so the ExecEnv can be synchonous.

Public Types

using Task = thin_function<void()>

Public Functions

void dispatch_on_comp_node(CompNode cn, Task &&task) = 0

add a task to the queue corresponding to given comp node

void dispatch_on_comp_node_with_mask(CompNode cn, Task &&task, ExecutionMask *mask) = 0

like dispatch_on_comp_node, but with specific mask other than current opr mask

void pause_exec() = 0

pause execution on all threads if there are async dispatch threads

This is currently only used by memory defragmenter.

void resume_exec() = 0

resume execution (cancel previous pause_exec())

Protected Functions

~ExecEnv() noexcept = default