Struct GraphCommonOptimizeOptions

Inheritance Relationships

Derived Types

Struct Documentation

struct mgb::cg::GraphCommonOptimizeOptions

common optimize options, it both can be used for optimize for inference in graph dump but also used in graph optimization in runtime.

Subclassed by mgb::cg::ComputingGraph::Options::GraphOpt, mgb::gopt::OptimizeForInferenceOptions

Public Types

enum LayoutTransform

Values:

enumerator DEFAULT
enumerator NCHW4

compute using NCHW4 tensor format

enumerator NHWCD4

compute using NHWCD4 tensor format

enumerator NCHW88

compute using NCHW88 tensor format

enumerator NCHW44

compute using NCHW44 tensor format

enumerator NCHW44_DOT

compute using NCHW44_DOT tensor format

enumerator NCHW32

compute using NCHW32 tensor format, used for tensorcore

enumerator CHWN4

compute using CHWN4 tensor format, transformed mainly used for cuda

Public Functions

GraphCommonOptimizeOptions &enable_f16_io_f32_comp()
GraphCommonOptimizeOptions &disable_f16_io_f32_comp()
bool has_set_f16_io_f32_comp()
GraphCommonOptimizeOptions &enable_f16_io_comp()
GraphCommonOptimizeOptions &disable_f16_io_comp()
bool has_set_f16_io_comp()
GraphCommonOptimizeOptions &enable_fuse_conv_bias_nonlinearity()
GraphCommonOptimizeOptions &disable_fuse_conv_bias_nonlinearity()
bool has_set_fuse_conv_bias_nonlinearity()
GraphCommonOptimizeOptions &enable_fuse_conv_bias_with_z()
GraphCommonOptimizeOptions &disable_fuse_conv_bias_with_z()
bool has_set_fuse_conv_bias_with_z()
GraphCommonOptimizeOptions &enable_fuse_preprocess()
GraphCommonOptimizeOptions &disable_fuse_preprocess()
bool has_set_fuse_preprocess()
GraphCommonOptimizeOptions &enable_weight_preprocess()
GraphCommonOptimizeOptions &disable_weight_preprocess()
bool has_set_weight_preprocess()
GraphCommonOptimizeOptions &enable_nchw4()
GraphCommonOptimizeOptions &disable_nchw4()
bool has_set_nchw4() const
GraphCommonOptimizeOptions &enable_nhwcd4()
GraphCommonOptimizeOptions &disable_nhwcd4()
bool has_set_nhwcd4() const
GraphCommonOptimizeOptions &enable_nchw88()
GraphCommonOptimizeOptions &disable_nchw88()
bool has_set_nchw88() const
GraphCommonOptimizeOptions &enable_nchw44()
GraphCommonOptimizeOptions &disable_nchw44()
bool has_set_nchw44() const
GraphCommonOptimizeOptions &enable_nchw44_dot()
GraphCommonOptimizeOptions &disable_nchw44_dot()
bool has_set_nchw44_dot() const
GraphCommonOptimizeOptions &enable_nchw32()
GraphCommonOptimizeOptions &disable_nchw32()
bool has_set_nchw32() const
GraphCommonOptimizeOptions &enable_chwn4()
GraphCommonOptimizeOptions &disable_chwn4()
bool has_set_chwn4() const

Public Members

bool f16_io_f32_comp = false

whether to enable IO in float16 compute in float32

bool f16_io_comp = false

whether to enable tranform to pure float16 model

bool fuse_conv_bias_nonlinearity = false

whether to enable conv bias nonlinearity fusion

bool fuse_conv_bias_with_z = false

fuse pattern like ReLU(conv_bias(x, w, b) + z) or conv_bias(x, w, b)

  • z -> conv_bias(x, w, b, z)

bool weight_preprocess = false

whether to enable weight preprocess, if enabled it may use more memory, default disable now, when weight preprocess is enabled, the input shape should no change

bool fuse_preprocess = false

fuse preprocess patten, like astype + pad_channel + dimshuffle

LayoutTransform layout_transform = LayoutTransform::DEFAULT