Save product of a nested For Loop (Storage Grid)

Question

I created a problem space through a nested loop. I am trying to store the result (v_hat), so I can "call" a specific problem P(d,u) and get the corresponding 30 arrays.

outfile = TemporaryFile()
n = 10
w = np.random.normal(60, 3, n)
for d in range(10):
    r = np.random.uniform(0.1 * d, 0.2 * d)
    v = w * r
    for u in range(10):
        c = np.random.uniform(0.1 * u, 0.2 * u)
        sigma = v * c
        for j in range(30):
            v_hat = np.random.normal(v, sigma)  
            np.save(outfile, v_hat)
np.load(outfile)

I tried with np.save but it doesn't work, and even if it did, I wouldn't know how to call the specific P(d,u)

I would really appreciate it if someone could help! Thank you!

Answer 1

Given that you know the number of iterations in each nested loop (=10, 10, 30) and the shape of v_hat (= whatever n is), you could make outfile a NumPy array with shape (10, 10, 30, n). Then during each iteration, you can replaced the appropriate row in outfile (where the appropriate row is outfile[d][u][j]) with v_hat.

Then at the end, you can save outfile. Here's the code:

# You can set n to whatever you want; I tested for various n and the code always works
n = 8

# Make outfile the necessary size
outfile = np.zeros((10, 10, 30, n), dtype=float)

w = np.random.normal(60, 3, n)

for d in range(10):
    r = np.random.uniform(0.1 * d, 0.2 * d)
    v = w * r

    for u in range(10):
        c = np.random.uniform(0.1 * u, 0.2 * u)
        sigma = v * c

        for j in range(30):
            v_hat = np.random.normal(v, sigma)  

            # Index the appropriate row in outfile and replace with this iteration's v_hat
            outfile[d][u][j] = v_hat

# Save outfile to a txt file. Change the path as desired.
np.savetxt("file.txt", outfile)

# How to read in the txt. Make sure the path here and in the line above match
outfile_loaded = np.loadtxt("file.txt")

Let me know if you have any questions.

Answer 2

numpy lets you make separate dimensions for d, u and your n, 30 arrays, leaving you with one (10, 10, n, 30) array that can be indexed into (out[d, u]) to recover your needed array. This can be saved as one organized array rather than a pile of un-indexed sub-arrays:

import numpy as np

def v_hat_gen(n, dmax, umax, i, fn = None):
    w = np.random.normal(60, 3, n)
    r = np.random.uniform(0.1 * np.arange(dmax), 0.2 * np.arange(dmax))
    v = np.einsum('i, jk -> ijk', r, w[None, :])
    c = np.random.uniform(0.1 * np.arange(umax), 0.2 * np.arange(umax), size = (umax, dmax)).T
    sigma = v*c[..., None]
    v_hat = np.random.normal(v[...,None], sigma[...,None], size = sigma.shape + (i,))
    if fn:
        with np.open(fn, 'wb') as f:
            np.save(f, v_hat)
    return v_hat

out = v_hat_gen(5, 10, 10, 30)
print(out.shape)

Out[]: (10, 10, 5, 30)