Wrong value coloration in 2D distribution (Matplotlib, imshow())

Question

I have a problem with the imshow() function in matplotlib, namely it sometimes colors zero values in a wrong manner or rather it doesnt set it to white but to the next smaller value besides zero. Sometimes it also neglects non zero values which are too small in comparison with other values. I am not too familiar with matplotlib so it might be that I am just missing something but I will give you an example.

I have the following testfile which looks like this:

0 0 0
1 0 0
2 0 0
3 0 0
4 0 0
0 1 0
1 1 1000
2 1 500
3 1 1
4 1 0
0 2 0
1 2 0
2 2 0
3 2 0
4 2 0
0 3 0
1 3 0
2 3 0
3 3 0
4 3 0
0 4 0
1 4 0
2 4 0
3 4 0
4 4 0

Now I want to plot a 2d histogram of this dataset. First column should represent my x-axis, second my y-axis and the third is the relative probability/entries.

With the following code:

import numpy as np
import math
import matplotlib.pyplot as plt
import sys

###########
#READ DATA#
###########
f=open(sys.argv[1],'r')
lines=f.readlines()
f.close()

z=[]

for i,line in enumerate(lines):
    l=line.split()
    z.append(float(l[2]))
z=np.array(z)
bins=int(float(math.sqrt(len(z))))
z=z.reshape((bins,bins),order='F')

###########
#PLOT DATA#
###########
fig = plt.figure(figsize=(18,10))
ax = fig.add_subplot(111)

im = ax.imshow(z, cmap='Greens')
ax.set_xlabel('A',fontsize=30)
ax.set_ylabel('B',fontsize=30)

plt.colorbar(im, fraction=0.045, pad=0.04)

#set x ticks
ax.set_xlim(-0.5,4.5)

#set y ticks
ax.invert_yaxis()
ax.set_ylim(-0.5,4.5)

plt.savefig('test.pdf',bbox_inches='tight', dpi=600)

I obtain following image: Wrong Coloration

As can be seen, all zero values are somehow set to the same color as the entry "2 1 500". Also, there is no difference between "3 1 1" and all the zero values. If I change the x- and ylim according to:

ax.set_xlim(-0.5,5.5)
ax.set_ylim(-0.5,5.5)

I get this image: Still wrong but better

Now everything has the color of "3 1 1" which is still wrong because obviously I want to differentiate between this entry and all the other zero values.

Can someone clarify what exactly is happening and why? Best