How do I patch a class attribute in pytest?

Question

I have a service class that connections to AWS S3. The connection uses boto3 within the __init__() method. I would like to mock this to use a moto s3 instance I've defined in a fixture, but I just can't get the mock to do anything.

Let's say I have a service class that looks like this:

import boto3

class S3Storage:
    def __init__(self):
        self._s3 = boto3.resource('s3')

    def do_download(self):
        self._s3 .download_file(
            Key='file.txt',
            Bucket='mybucket',
            Filename='path/to/destination/file.txt',
        )

and then I create a conftest file that has these moto fixtures:

# Fixtures
@pytest.fixture(scope='function')
def mocked_s3r():
    with mock_s3():
        yield boto3.resource('s3')

@pytest.fixture(scope='function')
def mocked_s3client():
    with mock_s3():
        yield boto3.client('s3')

@pytest.fixture(scope='function', autouse=True)
def upload_s3_resources(mocked_s3client, s3files):
    mocked_s3client.create_bucket(Bucket='mybucket')
    mocked_s3client.upload_file(
        Filename='path/to/destination/file.txt',
        Bucket='mybucket',
        Key='file.txt',
    )

The bottom fixture will grab a local file and place it in the moto s3 instance, which can be accessed from the mocked_s3r client mock.

My problem is that I cannot make a successful patch for the S3Storage._s3 attribute that holds the boto resource (I know I'm mixing boto clients and resources here, but I don't think that's causing the issue).

So I tried writing some fixtures to patch (using pytest-mock) or monkeypatch the boto resource and/or client.

# This is what I can't make work...
@pytest.fixture(autouse=True)
def mocked_s3(mocked_s3client, mocker):
    mocker.patch('app.utils.s3_storage.boto3.resource', return_value=mocked_s3r)
    return mocked_s3client

# This other approach also doesn't work...
@pytest.fixture(autouse=True)
def mocked_s3(mocked_s3client, mocker):
    mocker_s3storage = mocker.patch('app.utils.s3_storage.boto3.resource')
    mocker_s3storage()._s3 = mocked_s3client
    return mocked_s3client

# Nor this...
@pytest.fixture(autouse=True)
def mocked_s3(mocked_s3client, monkeypatch):
    monkeypatch.setattr('app.utils.s3_storage.S3Storage._s3', mocked_s3client)
    return mocked_s3client

But nothing works. I think I might be fundamentally misunderstanding how to patch an attribute that belongs to an instance of a class.

I'd rather do all this in a fixture, not in each individual test, such that I can write a test like:

def test_download_file(mocked_s3client):
    s3storage = S3Storage()
    s3storage._s3 # This should be a mock object, but it just connects to the real AWS
    s3storage.do_download()

and I don't have to specify the mock each time.

Answer 1

It is not necessary to patch the client, in order for Moto to work. As long as clients/resources are created while the mock is active, they are automatically patched.

Using your example fixtures, the following test works:

def test_file_exists(upload_s3_resources):
    S3Storage().do_download()
    test_file_exists()  # todo: actually verify somethign happened

---

Note that the download-file call in your logic should be slightly modified, but I'm assuming that was just a example to keep things simple. I had to change to the following to get the test to succeed: self._s3.Bucket('mybucket').download_file('file.txt', 'test.txt')