Class InputFile

Class Documentation

class mgb::serialization::InputFile

abstract input file interface

Public Functions

~InputFile() = default
void rewind() = 0

reset to beginning of input stream

void skip(size_t bytes) = 0

skip given number of bytes

void read(void *dst, size_t size) = 0

read data into buffer

size_t tell() = 0

return current read offset

void read_into_tensor(HostTensorND &dest, const TensorLayout &layout)

read into a host tensor

The default implementation uses read(); an alternative implementation might directly reset the storage of dest to utilize zero-copy.

SharedBuffer read_shared(size_t size)

read with sharing memory (i.e. use zero-copy if possible)

The default implementation allocates a new buffer and call read().

Note that there is no alignment guarantee.

Public Static Functions

std::unique_ptr<InputFile> make_fs(const char *path)

create an InputFile correspoding to a file on local file system

std::unique_ptr<InputFile> make_mem_proxy(const void *ptr, size_t size)

create an InputFile correspoding to a memory region; the memory region must be alive throughout lifespan of this InputFile

std::unique_ptr<InputFile> make_mem_proxy(std::shared_ptr<void> ptr, size_t size, bool writable = true)

create an InputFile that would directly reuse the memory buffer to load tensor values

  • writable: whether the input memory region can be modified. If this is set to true, tensor storage can be aggressively shared by reusing the buffer for alignment.