unable to reproduce coordconv results

Question

Hey I've been trying to implement this paper's Supervised Coordinate Regression part https://paperswithcode.com/method/coordconv

https://arxiv.org/pdf/1807.03247.pdf (page 6 section 4.2)

here's the model I've been trying to use

class CoordConv(nn.Module):
    def __init__(self, with_r=False, **kwargs):
        super().__init__()
        self.addcoords = AddCoords(with_r=with_r)
        self.conv = nn.Sequential(
            nn.Conv2d(5, 8, kernel_size=(1, 1), padding='same', bias=False),
            nn.Conv2d(8, 8, kernel_size=(1, 1), padding='same', bias=False),
            nn.Conv2d(8, 8, kernel_size=(1, 1), padding='same', bias=False),
            nn.Conv2d(8, 8, kernel_size=(3, 3), padding='same', bias=False),
            nn.Conv2d(8, 2, kernel_size=(3, 3), padding='same', bias=False),
            nn.MaxPool2d(kernel_size=64,stride=64,padding=0)
        )
    def forward(self, x):
        ret = self.addcoords(x)
        ret = self.conv(ret)
        return ret

as mentioned in paper (page 15 S3)

with optimizer

optimizer_ft = torch.optim.Adam(model_ft.parameters(),lr=1e-3)

using MSE loss

But the best loss I've got is 33, which is very far from the 100% accuracy the paper claims.

what am I doing wrong here?