'Out of memory issue while slicing scipy.sparse.csr_matrix
We have sparse csr matrix and while slicing the matrix we are getting out of memory (OOM) issue.
X[:, feature_sample]
Here X is of the dimension : [8000000, 138106]
feature_sample is a randomly generated boolean array of the shape (138106,) Ex of it : [True True False False ......True]
Here since column slicing is being done csc matrix is more efficient for the operation but modifying X to a csc matrix didn't help address the OOM issue.
Tried converting this to a dask array but before proceeding need to convert it back to a csr matrix for the next set of operations on X so that resulted in failure.
Looking for suggestions that can help avoid the OOM while slicing the matrix.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
