How to create matrix with specific value in lower triangle and rest values are zero in python?

Question

I am trying to create a matrix of size timesteps x timesteps, which for four timesteps in dense form looks like this:

   M = (1 0 0 0
        1 1 0 0
        1 1 1 0
        1 1 1 1)

For a certain number of timesteps I can create dense matrices as follows:

    import scipy.sparse as sp
    import numpy as np

    ld = np.ones((timesteps, timesteps))
    ld = np.tril(ld)
    ld_sparse=sp.coo_matrix(ld)

However, I need this matrix for timesteps = 35136, which causes a Numpy Memory Error. For this reason I tried to create the matrix using scipy sparse, but I couldn't find a function with which to do this. Thats why I tried to build an array for row, col and data with for loop what is very inefficient.

    row=np.array([])
    column=row=np.array([])
    data=row=np.array([])

    for t in list(range(timesteps)):
      row=np.append(row,np.full((1,(t+1)),t))
      column = np.append(column, np.arange(t + 1))
      data = np.append(data, np.full((1, (t + 1)), 1))

    ld=sp.coo_matrix((data,(row,column)), shape=(timesteps,timesteps))

The final goal is to create a constraint matrix for a linear optimization problem. The matrix "ld" should be used as a submatrix to describe the energy flow of a battery storage. The different submatrices are then combined to the whole constraint matrix. As explained above I could not find an efficient way to create the submatrix "ld".

     lm=sp.identity(timesteps)
     om=sp.coo_matrix((timesteps,timesteps))
     ov=sp.coo_matrix((timesteps,1))
     lv = sp.coo_matrix(np.ones(timesteps).reshape(timesteps, 1))
     ov = sp.coo_matrix((timesteps, 1))
     value = step_duration ** (-1)
     tv = sp.coo_matrix(np.full((timesteps, 1), value))

    A=sp.bmat([
[lm,lm,lm,lm,om,ov,ov],
[lm,om,om,om,om,ov,ov],
[om,om,om,lm,lm,ov,ov],
[om,-lm,om,om,om,ov,ov],
[om,om,lm,om,om,-lv,ov],
[om,om,-lm,om,om,-lv,ov],
[om,om,ld,om,om,ov,-tv],
[om,om,-ld,om,om,ov,ov],
[om,om,om,om,om,ov,-lv]])

Answer 1

MemoryError comes from the default element type in np.ones, which is numpy.float64. Just use another dtype, like bool:

ld = np.ones((35136, 35136), dtype=bool)

Every boolean value is 1 byte instead of 8 in the default case, and this should easily fit in memory.