PyGame Learning Environment and NEAT-Python

Question

I have been trying to make a genetic algorithm to play FlappyBird using the library NEAT-python. I had problems where the for loop over genomes. After the iteration over genomes starts to for loop only completes the action for one bird instead of continuing and restarting the game for a new genome. here is the function where I process this and how I initialize the game environment.

game = FlappyBird()
p = PLE(game, fps=30, display_screen=True, force_fps=False)
p.init()

def eval_genomes(genomes, config):

    for genome_id, genome in genomes:
        p.reset_game()
        net = FeedForwardNetwork.create(genome, config)
        genome.set_actions(p.getActionSet())
        genome.fitness = 0

        while not p.game_over():
            game_state = game.getGameState()
            decision = genome.play(net, game_state)
            if decision == 1:
                p.act(0)

        else:
            genome.fitness = p.score()
            genome.is_dead = True

Answer 1

Just Today I did the same! :D

With the difference that it worked for me....

You may wanna have a look at the following repository on github. https://github.com/rsk2327/NEAT_FlappyBird

I don't think it's updated anymore. but there's no need to. The only thing I had to change at his code was to remove all: "/home/roshan/Documents/FlappyBird/". You can just leave them away and put all ressources within the same folder.

Furthermore I changed the pickle-dump of the winner to this:

#outputDir = 'bestGenomes/'
#os.chdir(outputDir)
#serialNo = len(os.listdir(outputDir))+1
#outputFile = open(str(serialNo)+'_'+str(int(MAX_FITNESS))+'.p','wb' )
outputFile = 'bestGenome/winner.p'
pickle.dump(winner, outputFile)

This because on the first few runs I saved every genome, and changing the directory messes everything up.

Other than that it works fine. as soon as it works you can just replace the game, or the whole eval.

Else, have a look at the examples of Neat-python. In the evolve-minimal example all they need is following code, and it runs for every genome.

def eval_genomes(genomes, config):
    for genome_id, genome in genomes:
        genome.fitness = 4.0
        net = neat.nn.FeedForwardNetwork.create(genome, config)
        for xi, xo in zip(xor_inputs, xor_outputs):
            output = net.activate(xi)
            genome.fitness -= (output[0] - xo[0]) ** 2

Although I have to say, for me it looks functionally the same. So there's probably something wrong within the while loop, isn't it?

Answer 2

1. When using the PyGame Learning Environment, the p.reset_game method does not work as it should. For me, the p.init() instead of reset_game did the trick.
2. Another thing I noticed with PLE.FlappyBird() is that the game crashes if you haven't added the NOOP action(clarification: when the NN output is negative and you don't enter an act() input , the flappy bird window freezed during my testing).

Example:

In the fitness function method

Start of generation

environment.init()
for genome_id, genome in genomes:
    net = neat.nn.FeedForwardNetwork.create(genome, configuration)    
    pipe_count = 0                                    # this is the score
    
    # Game loop for a single genome
    while True:  
        last_score = environment.game.getScore()      # score before action
        
        output = networkOutput(net, environment)      # Get output from network
        if output[0] > 0.5:
            environment.act(environment.getActionSet()[0])  # Jump   
        else:
            environment.act(environment.getActionSet()[1])  # Don't jump (NOOP)
        
        # keep track of score
        score = environment.game.getScore() - last_score
        if(score >= 1): 
            # print("and anotha one: " + str(score))
            pipe_count += 1
        
        # when genome dies
        if(environment.lives() <= 0):
            genome.fitness = environment.game.score
            environment.init()            # Take note (init(), not reset_game)
            print("Dead: " + str(genome_id) + " score: " + str(pipe_count))
            break