37.3: pytest fails in in 6 units

[wuch-g2v]

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using 'installer` module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-readme_renderer-37.3-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-readme_renderer-37.3-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.16, pytest-7.3.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/readme_renderer-37.3
collected 48 items

tests/test_clean.py .                                                                                                                                                                 [  2%]
tests/test_cli.py FF..FF...F.F...                                                                                                                                                     [ 33%]
tests/test_markdown.py s.                                                                                                                                                             [ 37%]
tests/test_noextra.py ..                                                                                                                                                              [ 41%]
tests/test_rst.py ...........................                                                                                                                                         [ 97%]
tests/test_txt.py .                                                                                                                                                                   [100%]

========================================================================================= FAILURES ==========================================================================================
_____________________________________________________________________ test_cli_input_file[test_CommonMark_001.md-False] _____________________________________________________________________

input_file = PosixPath('tests/fixtures/test_CommonMark_001.md'), output_file = False

    @pytest.mark.parametrize("output_file", [False, True])
    def test_cli_input_file(input_file, output_file):
        with mock.patch("builtins.print") as print_:
            if output_file:
                with tempfile.TemporaryDirectory() as tmpdir:
                    output = pathlib.Path(tmpdir) / "output.html"
                    main(["-o", str(output), str(input_file)])
                    with output.open() as fp:
                        result = fp.read()
            else:
>               main([str(input_file)])

tests/test_cli.py:24:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cli_args = ['tests/fixtures/test_CommonMark_001.md']

    def main(cli_args: Optional[List[str]] = None) -> None:
        parser = argparse.ArgumentParser(
            description="Renders a .md, .rst, or .txt README to HTML",
        )
        parser.add_argument("-p", "--package", help="Get README from package metadata",
                            action="store_true")
        parser.add_argument("-f", "--format", choices=["md", "rst", "txt"],
                            help="README format (inferred from input file name or package)")
        parser.add_argument('input', help="Input README file or package name")
        parser.add_argument('-o', '--output', help="Output file (default: stdout)",
                            type=argparse.FileType('w'), default='-')
        args = parser.parse_args(cli_args)

        content_format = args.format
        if args.package:
            distribution = get_distribution(args.input)
            pkg_info = distribution.get_metadata(distribution.PKG_INFO)
            message = email.message_from_string(pkg_info)
            source = message.get_payload()

            # Infer the format of the description from package metadata.
            if not content_format:
                content_type = message.get("Description-Content-Type", "text/x-rst")
                if content_type == "text/x-rst":
                    content_format = "rst"
                elif content_type == "text/markdown":
                    content_format = "md"
                elif content_type == "text/plain":
                    content_format = "txt"
                else:
                    raise ValueError(f"invalid content type {content_type} for package "
                                     "`long_description`")
        else:
            filename = pathlib.Path(args.input)
            content_format = content_format or filename.suffix.lstrip(".")
            with filename.open() as fp:
                source = fp.read()

        if content_format == "md":
            rendered = render_md(source, stream=sys.stderr)
        elif content_format == "rst":
            rendered = render_rst(source, stream=sys.stderr)
        elif content_format == "txt":
            rendered = render_txt(source, stream=sys.stderr)
        else:
            raise ValueError(f"invalid README format: {content_format} (expected `md`, "
                             "`rst`, or `txt`)")
        if rendered is None:
>           sys.exit(1)
E           SystemExit: 1

readme_renderer/__main__.py:60: SystemExit
_____________________________________________________________________ test_cli_input_file[test_CommonMark_001.md-True] ______________________________________________________________________

input_file = PosixPath('tests/fixtures/test_CommonMark_001.md'), output_file = True

    @pytest.mark.parametrize("output_file", [False, True])
    def test_cli_input_file(input_file, output_file):
        with mock.patch("builtins.print") as print_:
            if output_file:
                with tempfile.TemporaryDirectory() as tmpdir:
                    output = pathlib.Path(tmpdir) / "output.html"
>                   main(["-o", str(output), str(input_file)])

tests/test_cli.py:20:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cli_args = ['-o', '/tmp/tmp33u9of8i/output.html', 'tests/fixtures/test_CommonMark_001.md']

    def main(cli_args: Optional[List[str]] = None) -> None:
        parser = argparse.ArgumentParser(
            description="Renders a .md, .rst, or .txt README to HTML",
        )
        parser.add_argument("-p", "--package", help="Get README from package metadata",
                            action="store_true")
        parser.add_argument("-f", "--format", choices=["md", "rst", "txt"],
                            help="README format (inferred from input file name or package)")
        parser.add_argument('input', help="Input README file or package name")
        parser.add_argument('-o', '--output', help="Output file (default: stdout)",
                            type=argparse.FileType('w'), default='-')
        args = parser.parse_args(cli_args)

        content_format = args.format
        if args.package:
            distribution = get_distribution(args.input)
            pkg_info = distribution.get_metadata(distribution.PKG_INFO)
            message = email.message_from_string(pkg_info)
            source = message.get_payload()

            # Infer the format of the description from package metadata.
            if not content_format:
                content_type = message.get("Description-Content-Type", "text/x-rst")
                if content_type == "text/x-rst":
                    content_format = "rst"
                elif content_type == "text/markdown":
                    content_format = "md"
                elif content_type == "text/plain":
                    content_format = "txt"
                else:
                    raise ValueError(f"invalid content type {content_type} for package "
                                     "`long_description`")
        else:
            filename = pathlib.Path(args.input)
            content_format = content_format or filename.suffix.lstrip(".")
            with filename.open() as fp:
                source = fp.read()

        if content_format == "md":
            rendered = render_md(source, stream=sys.stderr)
        elif content_format == "rst":
            rendered = render_rst(source, stream=sys.stderr)
        elif content_format == "txt":
            rendered = render_txt(source, stream=sys.stderr)
        else:
            raise ValueError(f"invalid README format: {content_format} (expected `md`, "
                             "`rst`, or `txt`)")
        if rendered is None:
>           sys.exit(1)
E           SystemExit: 1

readme_renderer/__main__.py:60: SystemExit
________________________________________________________________________ test_cli_input_file[test_GFM_001.md-False] _________________________________________________________________________

input_file = PosixPath('tests/fixtures/test_GFM_001.md'), output_file = False

    @pytest.mark.parametrize("output_file", [False, True])
    def test_cli_input_file(input_file, output_file):
        with mock.patch("builtins.print") as print_:
            if output_file:
                with tempfile.TemporaryDirectory() as tmpdir:
                    output = pathlib.Path(tmpdir) / "output.html"
                    main(["-o", str(output), str(input_file)])
                    with output.open() as fp:
                        result = fp.read()
            else:
>               main([str(input_file)])

tests/test_cli.py:24:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cli_args = ['tests/fixtures/test_GFM_001.md']

    def main(cli_args: Optional[List[str]] = None) -> None:
        parser = argparse.ArgumentParser(
            description="Renders a .md, .rst, or .txt README to HTML",
        )
        parser.add_argument("-p", "--package", help="Get README from package metadata",
                            action="store_true")
        parser.add_argument("-f", "--format", choices=["md", "rst", "txt"],
                            help="README format (inferred from input file name or package)")
        parser.add_argument('input', help="Input README file or package name")
        parser.add_argument('-o', '--output', help="Output file (default: stdout)",
                            type=argparse.FileType('w'), default='-')
        args = parser.parse_args(cli_args)

        content_format = args.format
        if args.package:
            distribution = get_distribution(args.input)
            pkg_info = distribution.get_metadata(distribution.PKG_INFO)
            message = email.message_from_string(pkg_info)
            source = message.get_payload()

            # Infer the format of the description from package metadata.
            if not content_format:
                content_type = message.get("Description-Content-Type", "text/x-rst")
                if content_type == "text/x-rst":
                    content_format = "rst"
                elif content_type == "text/markdown":
                    content_format = "md"
                elif content_type == "text/plain":
                    content_format = "txt"
                else:
                    raise ValueError(f"invalid content type {content_type} for package "
                                     "`long_description`")
        else:
            filename = pathlib.Path(args.input)
            content_format = content_format or filename.suffix.lstrip(".")
            with filename.open() as fp:
                source = fp.read()

        if content_format == "md":
            rendered = render_md(source, stream=sys.stderr)
        elif content_format == "rst":
            rendered = render_rst(source, stream=sys.stderr)
        elif content_format == "txt":
            rendered = render_txt(source, stream=sys.stderr)
        else:
            raise ValueError(f"invalid README format: {content_format} (expected `md`, "
                             "`rst`, or `txt`)")
        if rendered is None:
>           sys.exit(1)
E           SystemExit: 1

readme_renderer/__main__.py:60: SystemExit
_________________________________________________________________________ test_cli_input_file[test_GFM_001.md-True] _________________________________________________________________________

input_file = PosixPath('tests/fixtures/test_GFM_001.md'), output_file = True

    @pytest.mark.parametrize("output_file", [False, True])
    def test_cli_input_file(input_file, output_file):
        with mock.patch("builtins.print") as print_:
            if output_file:
                with tempfile.TemporaryDirectory() as tmpdir:
                    output = pathlib.Path(tmpdir) / "output.html"
>                   main(["-o", str(output), str(input_file)])

tests/test_cli.py:20:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cli_args = ['-o', '/tmp/tmp6if1era7/output.html', 'tests/fixtures/test_GFM_001.md']

    def main(cli_args: Optional[List[str]] = None) -> None:
        parser = argparse.ArgumentParser(
            description="Renders a .md, .rst, or .txt README to HTML",
        )
        parser.add_argument("-p", "--package", help="Get README from package metadata",
                            action="store_true")
        parser.add_argument("-f", "--format", choices=["md", "rst", "txt"],
                            help="README format (inferred from input file name or package)")
        parser.add_argument('input', help="Input README file or package name")
        parser.add_argument('-o', '--output', help="Output file (default: stdout)",
                            type=argparse.FileType('w'), default='-')
        args = parser.parse_args(cli_args)

        content_format = args.format
        if args.package:
            distribution = get_distribution(args.input)
            pkg_info = distribution.get_metadata(distribution.PKG_INFO)
            message = email.message_from_string(pkg_info)
            source = message.get_payload()

            # Infer the format of the description from package metadata.
            if not content_format:
                content_type = message.get("Description-Content-Type", "text/x-rst")
                if content_type == "text/x-rst":
                    content_format = "rst"
                elif content_type == "text/markdown":
                    content_format = "md"
                elif content_type == "text/plain":
                    content_format = "txt"
                else:
                    raise ValueError(f"invalid content type {content_type} for package "
                                     "`long_description`")
        else:
            filename = pathlib.Path(args.input)
            content_format = content_format or filename.suffix.lstrip(".")
            with filename.open() as fp:
                source = fp.read()

        if content_format == "md":
            rendered = render_md(source, stream=sys.stderr)
        elif content_format == "rst":
            rendered = render_rst(source, stream=sys.stderr)
        elif content_format == "txt":
            rendered = render_txt(source, stream=sys.stderr)
        else:
            raise ValueError(f"invalid README format: {content_format} (expected `md`, "
                             "`rst`, or `txt`)")
        if rendered is None:
>           sys.exit(1)
E           SystemExit: 1

readme_renderer/__main__.py:60: SystemExit
_____________________________________________________________________ test_cli_explicit_format[test_CommonMark_001.md] ______________________________________________________________________

input_file = PosixPath('tests/fixtures/test_CommonMark_001.md')

    def test_cli_explicit_format(input_file):
        fmt = input_file.suffix.lstrip(".")
        with input_file.open() as fp, \
                mock.patch("pathlib.Path.open", return_value=fp), \
                mock.patch("builtins.print") as print_:
>           main(["-f", fmt, "no-file.invalid"])

tests/test_cli.py:48:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cli_args = ['-f', 'md', 'no-file.invalid']

    def main(cli_args: Optional[List[str]] = None) -> None:
        parser = argparse.ArgumentParser(
            description="Renders a .md, .rst, or .txt README to HTML",
        )
        parser.add_argument("-p", "--package", help="Get README from package metadata",
                            action="store_true")
        parser.add_argument("-f", "--format", choices=["md", "rst", "txt"],
                            help="README format (inferred from input file name or package)")
        parser.add_argument('input', help="Input README file or package name")
        parser.add_argument('-o', '--output', help="Output file (default: stdout)",
                            type=argparse.FileType('w'), default='-')
        args = parser.parse_args(cli_args)

        content_format = args.format
        if args.package:
            distribution = get_distribution(args.input)
            pkg_info = distribution.get_metadata(distribution.PKG_INFO)
            message = email.message_from_string(pkg_info)
            source = message.get_payload()

            # Infer the format of the description from package metadata.
            if not content_format:
                content_type = message.get("Description-Content-Type", "text/x-rst")
                if content_type == "text/x-rst":
                    content_format = "rst"
                elif content_type == "text/markdown":
                    content_format = "md"
                elif content_type == "text/plain":
                    content_format = "txt"
                else:
                    raise ValueError(f"invalid content type {content_type} for package "
                                     "`long_description`")
        else:
            filename = pathlib.Path(args.input)
            content_format = content_format or filename.suffix.lstrip(".")
            with filename.open() as fp:
                source = fp.read()

        if content_format == "md":
            rendered = render_md(source, stream=sys.stderr)
        elif content_format == "rst":
            rendered = render_rst(source, stream=sys.stderr)
        elif content_format == "txt":
            rendered = render_txt(source, stream=sys.stderr)
        else:
            raise ValueError(f"invalid README format: {content_format} (expected `md`, "
                             "`rst`, or `txt`)")
        if rendered is None:
>           sys.exit(1)
E           SystemExit: 1

readme_renderer/__main__.py:60: SystemExit
_________________________________________________________________________ test_cli_explicit_format[test_GFM_001.md] _________________________________________________________________________

input_file = PosixPath('tests/fixtures/test_GFM_001.md')

    def test_cli_explicit_format(input_file):
        fmt = input_file.suffix.lstrip(".")
        with input_file.open() as fp, \
                mock.patch("pathlib.Path.open", return_value=fp), \
                mock.patch("builtins.print") as print_:
>           main(["-f", fmt, "no-file.invalid"])

tests/test_cli.py:48:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cli_args = ['-f', 'md', 'no-file.invalid']

    def main(cli_args: Optional[List[str]] = None) -> None:
        parser = argparse.ArgumentParser(
            description="Renders a .md, .rst, or .txt README to HTML",
        )
        parser.add_argument("-p", "--package", help="Get README from package metadata",
                            action="store_true")
        parser.add_argument("-f", "--format", choices=["md", "rst", "txt"],
                            help="README format (inferred from input file name or package)")
        parser.add_argument('input', help="Input README file or package name")
        parser.add_argument('-o', '--output', help="Output file (default: stdout)",
                            type=argparse.FileType('w'), default='-')
        args = parser.parse_args(cli_args)

        content_format = args.format
        if args.package:
            distribution = get_distribution(args.input)
            pkg_info = distribution.get_metadata(distribution.PKG_INFO)
            message = email.message_from_string(pkg_info)
            source = message.get_payload()

            # Infer the format of the description from package metadata.
            if not content_format:
                content_type = message.get("Description-Content-Type", "text/x-rst")
                if content_type == "text/x-rst":
                    content_format = "rst"
                elif content_type == "text/markdown":
                    content_format = "md"
                elif content_type == "text/plain":
                    content_format = "txt"
                else:
                    raise ValueError(f"invalid content type {content_type} for package "
                                     "`long_description`")
        else:
            filename = pathlib.Path(args.input)
            content_format = content_format or filename.suffix.lstrip(".")
            with filename.open() as fp:
                source = fp.read()

        if content_format == "md":
            rendered = render_md(source, stream=sys.stderr)
        elif content_format == "rst":
            rendered = render_rst(source, stream=sys.stderr)
        elif content_format == "txt":
            rendered = render_txt(source, stream=sys.stderr)
        else:
            raise ValueError(f"invalid README format: {content_format} (expected `md`, "
                             "`rst`, or `txt`)")
        if rendered is None:
>           sys.exit(1)
E           SystemExit: 1

readme_renderer/__main__.py:60: SystemExit
===================================================================================== warnings summary ======================================================================================
readme_renderer/markdown.py:44
  /home/tkloczko/rpmbuild/BUILD/readme_renderer-37.3/readme_renderer/markdown.py:44: UserWarning: Markdown renderers are not available. Install 'readme_renderer[md]' to enable Markdown rendering.
    warnings.warn(_EXTRA_WARNING)

../../../../../usr/lib/python3.8/site-packages/pkg_resources/__init__.py:121
  /usr/lib/python3.8/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
    warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)

tests/test_cli.py::test_cli_input_file[test_CommonMark_001.md-False]
tests/test_cli.py::test_cli_input_file[test_CommonMark_001.md-True]
tests/test_cli.py::test_cli_input_file[test_GFM_001.md-False]
tests/test_cli.py::test_cli_input_file[test_GFM_001.md-True]
tests/test_cli.py::test_cli_explicit_format[test_CommonMark_001.md]
tests/test_cli.py::test_cli_explicit_format[test_GFM_001.md]
tests/test_markdown.py::test_missing_variant
  /home/tkloczko/rpmbuild/BUILD/readme_renderer-37.3/readme_renderer/markdown.py:60: UserWarning: Markdown renderers are not available. Install 'readme_renderer[md]' to enable Markdown rendering.
    warnings.warn(_EXTRA_WARNING)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== short test summary info ==================================================================================
SKIPPED [1] tests/test_markdown.py:8: got empty parameter set ('md_filename', 'html_filename', 'variant'), function test_md_fixtures at /home/tkloczko/rpmbuild/BUILD/readme_renderer-37.3/tests/test_markdown.py:7
FAILED tests/test_cli.py::test_cli_input_file[test_CommonMark_001.md-False] - SystemExit: 1
FAILED tests/test_cli.py::test_cli_input_file[test_CommonMark_001.md-True] - SystemExit: 1
FAILED tests/test_cli.py::test_cli_input_file[test_GFM_001.md-False] - SystemExit: 1
FAILED tests/test_cli.py::test_cli_input_file[test_GFM_001.md-True] - SystemExit: 1
FAILED tests/test_cli.py::test_cli_explicit_format[test_CommonMark_001.md] - SystemExit: 1
FAILED tests/test_cli.py::test_cli_explicit_format[test_GFM_001.md] - SystemExit: 1
==================================================================== 6 failed, 41 passed, 1 skipped, 9 warnings in 0.83s ====================================================================

Here is list of installed modules in build env

Package         Version
--------------- -------
bleach          6.0.0
build           0.10.0
distro          1.8.0
docutils        0.19
exceptiongroup  1.0.0
gpg             1.19.0
html5lib        1.1
iniconfig       2.0.0
installer       0.7.0
libcomps        0.1.19
packaging       23.0
pluggy          1.0.0
Pygments        2.15.0
pyproject_hooks 1.0.0
pytest          7.3.0
python-dateutil 2.8.2
setuptools      67.6.1
six             1.16.0
tomli           2.0.1
webencodings    0.5.1
wheel           0.40.0

[miketheman]

Thanks for opening this.

@wuch-g2v looks like you're pulling unreleased code from github, is that correct?

Looks like the recent changes in #271 did not account for the noextra use case where the markdown extensions are not installed, and only for running the test case

Can also be reproduced via tox -e noextra.

@tillahoffmann would you want to take a pass at fixing this?