'How to release the occupied GPU memory when calling keras model by Apache mod_wsgi django?

My server configuration is as follows:

  1. apache 2.4.23.
  2. Mod_wsgi 4.5.9

By using the Django framework and apache server, we call the Keras deep learning model. And after the successful calling of the model, the model has been always running in the GPU memory, which causes the GPU memory can not be released except by shutting down the apache server.

So, is there any way to control the release of GPU memory when calling a Keras model by Apache+Mod_wsgi+Django?

Thanks!

Runtime memory footprint screenshots

djangoapachememorykeras


Solution 1:[1]

For people who fail to make K.clear_session() work, there is an alternative solution:

from numba import cuda
cuda.select_device(0)
cuda.close()

Tensorflow is just allocating memory to the GPU, while CUDA is responsible for managing the GPU memory.

If CUDA somehow refuses to release the GPU memory after you have cleared all the graph with K.clear_session(), then you can use the cuda library to have a direct control on CUDA to clear GPU memory.

Solution 2:[2]

yes but the latest approach with crashes my spyder console kerner for some reason allthough the GPU memory is free again

from numba import cuda
cuda.select_device(0)
cuda.close()

Solution 3:[3]

from numba import cuda
device = cuda.get_current_device()
device.reset()

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Raven Cheuk
Solution 2 GGEv
Solution 3 MCPMH

Related Questions