Can't zoom in after implementing fitInView() in Qt

Question

I'm working on a project in Qt and the thing I've to do is to have an image in the background(can be png or jpg).

I've created a view with a scene with QGraphicsView and QGraphicsScene.

QGraphicsPixmapItem * image = new QGraphicsPixmapItem(QPixmap("...../world_map.png"));
int imageWidth = image->pixmap().width();
int imageHeight = image->pixmap().height();
image->setOffset(- imageWidth / 2, -imageHeight / 2);
image->setPos(0, 0);

QGraphicsScene *scene = new QGraphicsScene();
scene->setSceneRect(-imageWidth / 2, -imageHeight / 2, imageWidth, imageHeight);
QGraphicsView * gv = new QGraphicsView();
gv->setScene(scene);
gv->scene()->addItem(image);

I get the following output:

But I wanted the whole image to fit the view while maintaining the aspect ratio. So, I made a custom class inherited from QGraphicsView and wrote the following:

void MyView::resizeEvent(QResizeEvent *event)
{
    QGraphicsView::resizeEvent(event);
    fitInView(sceneRect(), Qt::KeepAspectRatio);
}

Now, I got the following output:

This is desirable but I can't zoom in on the view now. I can only zoom out. P.S. - I wrote a mouseWheelEvent function to zoom in and out.

What can be done to implement zooming in facility??

Edit : This is how I implemented zoom in/out:

void MyView::wheelEvent(QWheelEvent *e)
{
    static const double factor = 1.1;
    static double currentScale = 1.0;
    static const double scaleMin = 1.0;
    ViewportAnchor oldAnchor = transformationAnchor();

    setTransformationAnchor(QGraphicsView::AnchorUnderMouse); // set focus to mouse coords

    //if (e->delta() > 0)
    if (e->angleDelta().y() > 0){
        scale(factor, factor);
        currentScale *= factor;
    }
    else if (currentScale > scaleMin){
        scale(1 / factor, 1 / factor);
        currentScale /= factor;
    }
    setTransformationAnchor(oldAnchor); // reset anchor
}