DTRConfig¶
- class DTRConfig(eviction_threshold=0, evictee_minimum_size=1 << 20, recomp_memory_factor=1, recomp_time_factor=1)[source]¶
Configuration for DTR memory optimization.
- Parameters
eviction_threshold (
int) – eviction threshold in bytes. When GPU memory usage exceeds this value, DTR will heuristically select and evict resident tensors until the amount of used memory falls below this threshold.evictee_minimum_size (
int) – memory threshold of tensors in bytes. Only tensors whose size exceeds this threshold will be added to the candidate set. A tensor that is not added to the candidate set will never be evicted during its lifetime. Default: 1048576.recomp_memory_factor (
float) – hyperparameter of the estimated memory of recomputing the tensor. The larger this value is, the less memory-consuming tensor will be evicted in heuristic strategies. This value is greater than or equal to 0. Default: 1.recomp_time_factor (
float) – hyperparameter of the estimated time of recomputing the tensor. The larger this value is, the less time-consuming tensor will be evicted in heuristic strategies. This value is greater than or equal to 0. Default: 1.