Frequently Asked Questions about Video Memory Usage#

Note

This guide only applies to the MegEngine Python interface. When using the C++ interface for reasoning, the use and control of video memory will be more refined and complex.

How to correctly observe the memory usage#

MegEngine uses the video memory pool mechanism to accelerate the application and release of video memory. The video memory released in the program will be stored in the video memory pool and will not be actively released to the CUDA driver. Therefore, use the ``nvidia-smi’’ command to observe The video memory usage may be greater than the actual usage value.

You can use get_mem_status_bytes to get the total video memory and free video memory of a computing device (the free video memory at this time includes the unallocated video memory in the video memory pool), and the current accurate value can be obtained according to the difference between the two Video memory usage.

How to release the currently occupied video memory#

When the life cycle of a Python object ends, the video memory will be released, such as：

  • For Tensor objects, when they are not referenced by any variables,:class:~.Module and GradManager, they will be destroyed and the video memory will be released

  • You can manually release the reference through the Tensor or Module object corresponding to :code:

Note

Since Python GC does not guarantee that all objects will be released immediately when the reference count reaches 0, there may be cases where the memory of the object is deleted but not released immediately (especially when multiple objects are referenced circularly). gc.collect to try to force collection immediately.

Warning

The variable life cycle of Python is different from that of C++. At the end of the for loop, you may encounter additional video memory usage caused by the unreleased Tensor object. We can release this part of the video memory immediately by explicitly deleting the variable, an example is as follows：

for i in range(4):
    d = tensor(i * 2)

print(d)    # Still exist!
del d       # Release tensor