Pytorch

Caveats

Memory management in Pytorch is not very obvious. The first step is to inspect the logfile for meaningfull messages.

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 50.00 MiB (GPU 0; 79.25 GiB total capacity; 26.95 GiB already allocated; 51.24 GiB free; 27.43 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

In this case the reserved memory is too close for comfort with the allocated memory.
This seems a general problem with pytorch.
The blogs below provide more pointers how to track and save memory usage. For instance:

• Using mixed precision

• Profile Memory Usage with torch.cuda.memory_summary()

• Use torch.no_grad() for Inference

• Pre-allocate memory at the start of your program.

https://www.codegenes.net/blog/memory-resernve-in-totoal-by-pytorch/

https://www.geeksforgeeks.org/deep-learning/how-to-avoid-cuda-out-of-memory-in-pytorch/