Append an array to an image in python

Question

I'm trying to create a piece of data for a CNN in tensorflow. The image corresponds to a state in my environment. I'd like to take the state (the array) and append it to the image for model input.

Some conditions are, that the original structure of the image needs to be intact. So when I convert the array to an image, I can see the original image plus the appended array. The final array can have any number of rows, but the number of columns and channels cannot change.

So in other words, I'd like to reshape an array of max_length (say, 2,000,000) a long a matrix of n rows and 1920 columns and 3 channels. Padding with 0s if necessary, and then append that to an image of shape (1080, 1920, 3). I have a feeling my approach is more complicated than it needs to be.

My attempt at doing this was to convert the state and image to a list, for faster array transformations. Add them together sequentially and then reshape them into an numpy array. I use padding to constrain the array to 'n rows x 1920 x 3'.

The variable max_length is the max length of the padding.

input_space is the state or array I want to append to the image. It's length can be anywhere between 1 and max_length which is why I use padding. So the state size isn't changing per loop.

The code described under aspect ratio, is used for more padding to ensure we constrain the array to required dimensions.

def get_state(input_space, img):
    img = list(np.concatenate(img).flat)
    pad_length = (max_length - len(input_space))
    state = img + input_space + [0] * pad_length
    # get pad length for 1920 aspect ratio
    aspect_ratio = (len(state) / 3) / 1920
    remainder = aspect_ratio - np.fix(aspect_ratio)
    aspect_ratio = round(((1 - remainder) * 1920) * 3)
    # pad state to shape into 1920 x n rows and 3 channels
    state = state + [0] * aspect_ratio
    rows = round((len(state)/3)/1920)
    return np.reshape(state, (rows, 1920, 3)).T

The function successfully produces an array constrained too n rows, 1920 columns and 3 channels, except when I go to preview the array as an image the original image structure is lost.

Is there a better way to approach this? I should add that performance is important because the function runs in a loop.

Answer 1

Ok so I put together something that works. Instead of appending together the image and state to start, and then reshaping them in the desired shape, I reshape the state and then just append it to the image. Seems to work out.

def get_state(input_space, img):
    pad_length = (max_length - len(input_space))
    state = input_space + [0] * pad_length
    # get pad length for 1920 aspect ratio
    aspect_ratio = (len(state) / 3) / 1920
    remainder = aspect_ratio - np.fix(aspect_ratio)
    aspect_ratio = round(((1 - remainder) * 1920) * 3)
    # pad state to shape into 1920 x n rows and 3 channels
    state = state + [0] * aspect_ratio
    rows = round((len(state)/3)/1920)
    x = list(np.reshape(state, (3, 1920, rows)).T)
    state = list(img) + x
    return np.array(state)