ValueError: Input 0 of layer "max_pooling2d" is incompatible with the layer: expected ndim=4, found ndim=5. Full shape received: (None, 3, 51, 39, 32)

Question

I have two different problems occurs at the same time.

I am having dimensionality problems with MaxPooling2d and having same dimensionality problem with DQNAgent.

The thing is, I can fix them seperately but cannot at the same time.

First Problem

I am trying to build a CNN network with several layers. After I build my model, when I try to run it, it gives me an error.

!pip install PyOpenGL==3.1.* PyOpenGL-accelerate==3.1.*
!pip install tensorflow gym keras-rl2 gym[atari] keras pyvirtualdisplay 

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Convolution2D, MaxPooling2D, Activation
from keras_visualizer import visualizer 
from tensorflow.keras.optimizers import Adam

env = gym.make('Boxing-v0')
height, width, channels = env.observation_space.shape
actions = env.action_space.n

input_shape = (3, 210, 160, 3)   ## input_shape = (batch_size, height, width, channels)

def build_model(height, width, channels, actions):
  model = Sequential()
  model.add(Convolution2D(32, (8,8), strides=(4,4), activation="relu", input_shape=input_shape, data_format="channels_last"))
  model.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_last"))
  model.add(Convolution2D(64, (4,4), strides=(1,1), activation="relu"))
  model.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_last"))
  model.add(Convolution2D(64, (3,3), activation="relu"))
  model.add(Flatten())
  model.add(Dense(512, activation="relu"))
  model.add(Dense(256, activation="relu"))
  model.add(Dense(actions, activation="linear"))
  return model

model = build_model(height, width, channels, actions)

It gives below error:

ValueError: Input 0 of layer "max_pooling2d_12" is incompatible with the layer: expected ndim=4, found ndim=5. Full shape received: (None, 3, 51, 39, 32)

Second Problem

My input_shape is (3, 210, 160, 3). I am using the first 3 on purpose due to I have to specify the batch_size before. If I do not specify it before and pass it as (210, 160, 3) to the build_model function, below build_agent function gives me an another error:

def build_agent(model, actions):
  policy = LinearAnnealedPolicy(EpsGreedyQPolicy(), attr="eps", value_max=1., value_min=.1, value_test=.2, nb_steps=10000)
  memory = SequentialMemory(limit=1000, window_length=3)
  dqn = DQNAgent(model=model, memory=memory, policy=policy,
                 enable_dueling_network=True, dueling_type="avg",
                 nb_actions=actions, nb_steps_warmup=1000)
  return dqn

dqn = build_agent(model, actions)
dqn.compile(Adam(learning_rate=1e-4))

dqn.fit(env, nb_steps=10000, visualize=False, verbose=1)

ValueError: Error when checking input: expected conv2d_11_input to have 4 dimensions, but got array with shape (1, 3, 210, 160, 3)

Deleting batch size number in the model construction phase, removes the MaxPooling2D incompatibility error but throws DQNAgent dimensionality error. Adding the batch size to the model construction phase removes DQNAgent dimensionality error but throws the MaxPooling2D incompatibility error.

I am really stucked.