pytest分布式执行插件 pytest-xdist 的高级用法

2023-12-26 18:49:18

想要使用多个CPU核心来进行测试，可以使用 -n 参数( 或者 --numprocesses)
(使用8个核心来跑测试用例)

1	`pytest -n 8`

使用?-n auto?参数可以利用电脑的所有核心来跑测试用例
测试时使用的算法可以根据--dist命令参数定制：

--dist load(默认选项)：给每个CPU核心随机分配用例，不保证执行顺序。
--dist loadscope：对于测试函数，测试按模块分组，对于测试方法，测试按类分组。每组作为一个整体分配给可用的worker。这保证了组中的所有测试都在同一进程中运行。如果的模块级或类级fixtures，这将非常有用。按类分组优先于按模块分组。
--dist loadfile: 测试用例按其所在文件分组。每组作为一个整体分配给可用的worker。这保证了文件中的所有测试都在同一个辅助进程中运行。
--dist loadgroup: 测试按xdist_group标记分组。每组作为一个整体分配给可用的执行器。这保证了具有相同xdist_ group名称的所有测试都在同一个worker中运行。

@pytest.mark.xdist_group(name="group1")

def test1():

????pass

class TestA:

????@pytest.mark.xdist_group("group1")

????def test2():

????????pass-

这将确保test1和TestA:：test2将在同一个worker中运行。没有xdist_ group标记的测试在--dist=load模式下正常运行。

--dist no：正常的pytest执行模式，一次运行一个测试（完全没有分发）。

例子：
项目目录结构

xdist_test.py

import logging

import pytest

class TestXdist(object):

????@pytest.mark.xdist_group("group1")

????def test_one(self):

????????logging.info("1")

????????assert True

????@pytest.mark.xdist_group("group1")

????def test_two(self):

????????logging.info("2")

????????assert True

????@pytest.mark.xdist_group("group2")

????def test_three(self):

????????logging.info("3")

????????assert True

????@pytest.mark.xdist_group("group2")

????def test_four(self):

????????logging.info("4")

????????assert True

xdist_dummy_test.py

import logging

import pytest

class TestXdist(object):

????@pytest.mark.run(order=1)

????@pytest.mark.xdist_group("group1")

????def test_dummy_one(self):

????????logging.info("d1")

????????assert True

????@pytest.mark.xdist_group("group1")

????@pytest.mark.run(order=2)

????def test_dummy_two(self):

????????logging.info("d2")

????????assert True

????@pytest.mark.run(order=3)

????@pytest.mark.xdist_group("group2")

????def test_dummy_three(self):

????????logging.info("d3")

????????assert True

????@pytest.mark.xdist_group("group2")

????@pytest.mark.run(order=4)

????def test_dummy_four(self):

????????logging.info("d4")

????????assert True

(venv) ?? pytest pytest -n auto --dist loadscope

=========================================================================== test session starts ===========================================================================

platform darwin -- Python 3.8.9, pytest-7.1.2, pluggy-1.0.0

rootdir: /Users/spock/PycharmProjects/pytest, configfile: pytest.ini

plugins: xdist-2.5.0, forked-1.4.0, ordering-0.6

[gw0] Python 3.8.9 (default, May 17 2022, 12:55:41)? -- [Clang 13.1.6 (clang-1316.0.21.2.5)]

[gw1] Python 3.8.9 (default, May 17 2022, 12:55:41)? -- [Clang 13.1.6 (clang-1316.0.21.2.5)]

[gw2] Python 3.8.9 (default, May 17 2022, 12:55:41)? -- [Clang 13.1.6 (clang-1316.0.21.2.5)]

[gw3] Python 3.8.9 (default, May 17 2022, 12:55:41)? -- [Clang 13.1.6 (clang-1316.0.21.2.5)]

[gw4] Python 3.8.9 (default, May 17 2022, 12:55:41)? -- [Clang 13.1.6 (clang-1316.0.21.2.5)]

[gw5] Python 3.8.9 (default, May 17 2022, 12:55:41)? -- [Clang 13.1.6 (clang-1316.0.21.2.5)]

[gw6] Python 3.8.9 (default, May 17 2022, 12:55:41)? -- [Clang 13.1.6 (clang-1316.0.21.2.5)]

[gw7] Python 3.8.9 (default, May 17 2022, 12:55:41)? -- [Clang 13.1.6 (clang-1316.0.21.2.5)]

gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]

scheduling tests via LoadScopeScheduling

test2/xdist_dummy_test.py::TestXdist::test_dummy_one

test/xdist_test.py::TestXdist::test_one

[gw0] [ 12%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_one

[gw1] [ 25%] PASSED test/xdist_test.py::TestXdist::test_one

test2/xdist_dummy_test.py::TestXdist::test_dummy_two

test/xdist_test.py::TestXdist::test_two

[gw1] [ 37%] PASSED test/xdist_test.py::TestXdist::test_two

test/xdist_test.py::TestXdist::test_three

[gw0] [ 50%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_two

test2/xdist_dummy_test.py::TestXdist::test_dummy_three

[gw1] [ 62%] PASSED test/xdist_test.py::TestXdist::test_three

test/xdist_test.py::TestXdist::test_four

[gw0] [ 75%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_three

test2/xdist_dummy_test.py::TestXdist::test_dummy_four

[gw1] [ 87%] PASSED test/xdist_test.py::TestXdist::test_four

[gw0] [100%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_four

============================================================================ 8 passed in 0.40s ============================================================================

(venv) ?? pytest pytest -n auto --dist loadfile