Storage Manager

Disk usage manager for dataset caching and temporary file tracking.

Tracks cumulative file sizes against a configurable limit and provides best-effort cleanup of registered temporary files.

class src.utils.storage_manager.StorageManager(limit_gb: float = 500.0)

Bases: object

Manages disk usage to stay under a configurable storage limit.

Tracks cumulative bytes consumed by registered files and provides temporary file creation with automatic registration for later cleanup.

limit_bytes

Maximum allowed cumulative storage in bytes.

used_bytes

Current cumulative bytes consumed by registered files.

temp_files

List of paths to temporary files created via create_temp_file().

__init__(limit_gb: float = 500.0)

Initialize the storage manager.

Parameters:

limit_gb – Storage limit in gigabytes. Defaults to 500 GB.

cleanup()

Remove all registered temporary files and clear the tracking list.

create_temp_file(suffix: str = '.tmp') → str

Create a temporary file under ./temp_data and register it.

Parameters:

suffix – File suffix for the temporary file (default ".tmp").

Returns:

Absolute path to the created temporary file.

register_file(path: str) → bool

Register a file’s size and check whether the limit has been exceeded.

Parameters:

path – Filesystem path to the file to register.

Returns:

True if the cumulative usage is still within the limit, False if the limit has been exceeded.