How to mark one function not a test for pytest?

Question

I'm using pytest to test some code based on TensorFlow.

A TestCase is defined for simplicity like:

class TestCase(tf.test.TestCase):
    # ...

The problem is tf.test.TestCase provide a useful function self.test_session(), which was treated as a test method in pytest since its name starts with test_.

The result pytest report more succeed tests than test methods I defined due to test_session() methods.

I use the following code to skip test_session:

class TestCase(tf.test.TestCase):
    @pytest.mark.skip
    @contextmanager
    def test_session(self):
        with super().test_session() as sess:
            yield sess

However there would be some "s" in test report indicating there are some skip tests.

Is there anyway I can mark one exact method not a test method without changing pytest test discovery rules globally?

Answer 1

You can set __test__ = False, either directly, or by writing a simple decorator. The latter should behave similarly to Nose's nottest decorator.

def nottest(obj):
    obj.__test__ = False
    return obj

class TestMyTest:

    def test_should_not_collect_1(self):
        assert False
    test_should_not_collect_1.__test__ = False

    @nottest
    def test_should_not_collect_2(self):
        assert False

    def test_should_collect(self):
        assert True


def test_should_not_collect_1():
    assert False
test_should_not_collect_1.__test__ = False

@nottest
def test_should_not_collect_2():
    assert False


def test_should_collect():
    assert True

When running pytest, this only runs the methods which are not marked:

$ pytest test.py -v
====================================== test session starts ======================================
platform darwin -- Python 3.9.1, pytest-7.0.1, pluggy-1.0.0 -- /Users/lucaswiman/.pyenv/versions/3.9.1/bin/python3.9
cachedir: .pytest_cache
rootdir: /private/tmp
plugins: anyio-2.2.0
collected 2 items                                                                               

test.py::TestMyTest::test_should_collect PASSED                                           [ 50%]
test.py::test_should_collect PASSED                                                       [100%]

======================================= 2 passed in 0.04s =======================================

This behavior is documented here:

Since Pytest 2.6, users can prevent pytest from discovering classes that start with Test by setting a boolean __test__ attribute to False.

Answer 2

There is way to do it in unittest

@unittest.skip("skipping reason")

tf.test has skipTest(reason) read more at https://www.tensorflow.org/api_docs/python/tf/test/TestCase#skipTest