'Reinforcement Learning using PPO2- Why my ep_rewmean (reward) during training is always showing nan?
I am currently implementing a reinforcement learning problem similar to buying/selling shares in the stock exchange. It's an online implementation and I am creating steps based on a dataset and its a customized environment. Now, the problem is that during my training timesteps, I observe that the value of one of the performance metrics ep_rewmean and eplenmean is always nan as follows,
My environment is
env = DummyVecEnv([lambda: ETradingEnv(df)])
model = PPO2('MlpPolicy', env, verbose=1)
model.learn(total_timesteps=200)
My Step Function is,
def step(self, action):
# Execute one time step within the environment
amount=self._take_action(action)
self.amount += amount
self.current_step += 1
if self.current_step > len(self.df.loc[:, 'current_l'].values) - 6:
self.current_step = 0
delay_modifier = (self.current_step / MAX_STEPS)
# Reward 1
r1 = -1 * self.delta * amount* self.current_price
# Reward 2
d=self.df.loc[self.current_step, 'd']
if del == 1:
r2= -1 * (self.s-MAX_S)**2
else:
r2=0
reward = r1 + r2
# reward=100
done = d == 1
obs = self._next_observation()
print(f'reward: {reward}')
return obs, reward, done, {}
How can I resolve this issue? Any help would be highly appreciated. Thanks in advance.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|

