pyproject.toml with a default configuration causes "Can only merge Directories with no duplicates" bug.

[bastianwegge]

Describe the bug
When I add a pyproject toml with the contents like this, pants gives me a weird comparison error:

❯ ./pants test ::
21:31:56.75 [INFO] Initializing scheduler...
21:31:56.93 [INFO] Scheduler initialized.
21:31:59.36 [ERROR] 1 Exception encountered:

  Exception: String("Can only merge Directories with no duplicates, but found 2 duplicate entries in :\n\n`pyproject.toml`: 1.) file digest=9d7d5d9df3e4ae5a4842a380891776c10adfc04ffdf0d48728db8ff4e7a05943 size=1006:\n\n[tool.black]\nline-length = 100\ntarget-version = [ \"py38\",]\ninclude = \"\\\\.pyi?$\"\nforce-exclude = \"/(\\n    .git\\n  | .hg\\n  | .eggs\\n  | .mypy_cache\\n  | .tox\\n  | .venv\\n  | .venv\\n  | _build\\n  | buck-out\\n  | build\\n  | dist\\n  | migrations\\n)/\\n\"\n\n[tool.isort]\nprofile = \"black\"\nline_length = 100\nsections = [ \"FUTURE\", \"STDLIB\", \"THIRDPARTY\", \"FIRSTPARTY\", \"LOCALFOLDER\",]\nno_lines_before = \"LOCALFOLDER\"\nskip_glob = [ \"*/migrations/*.py\",]\nsrc_paths = [ \"src\", \".\",]\nknown_thirdparty = [ \"django\",]\n\n[tool.coverage.run]\nomit = [ \"*/tests/*\", \"*/migrations/*\", \"*/prototypes/*\", \"*/site-packages/*\", \"*/test_*\", \"scripts/*\", \"docker/*\", \"config/*\", \"pytest.pex/*\",]\nrelative_files = true\n\n[tool.coverage.report]\nprecision = 2\n\n[tool.pytest.ini_options]\npython_files = \"tests.py test_*.py *_tests.py\"\nDJANGO_SETTINGS_MODULE = \"config.settings_testing\"\nmarkers = [ \"slow: marks tests as slow (deselect with '-m \\\"not slow\\\"')\", \"jwt_scope: sets required scope for api test\",]\nenv = [ \"ENVIRONMENT=test\",]\n\n\n`pyproject.toml`: 2.) file digest=ca030f02c34b88ba4f34738eed6a6b0f6fadc4184f1ee0bd6d8d38186066118c size=995:\n\n[tool.black]\nline-length = 100\ntarget-version = ['py38']\ninclude = '\\.pyi?$'\nforce-exclude = \"\"\"\n/(\n    .git\n  | .hg\n  | .eggs\n  | .mypy_cache\n  | .tox\n  | .venv\n  | .venv\n  | _build\n  | buck-out\n  | build\n  | dist\n  | migrations\n)/\n\"\"\"\n\n[tool.isort]\nprofile = \"black\"\nline_length = 100\nsections = [\"FUTURE\", \"STDLIB\", \"THIRDPARTY\", \"FIRSTPARTY\", \"LOCALFOLDER\"]\nno_lines_before = \"LOCALFOLDER\"\nskip_glob = [\"*/migrations/*.py\"]\nsrc_paths = [\"src\", \".\"]\nknown_thirdparty = [\"django\"]\n\n[tool.coverage.run]\nomit = [\n    \"*/tests/*\",\n    \"*/migrations/*\",\n    \"*/prototypes/*\",\n    \"*/site-packages/*\",\n    \"*/test_*\",\n    \"scripts/*\",\n    \"docker/*\",\n    \"config/*\",\n]\n\n[tool.coverage.report]\nprecision = 2\n\n[tool.pytest.ini_options]\npython_files = \"tests.py test_*.py *_tests.py\"\nDJANGO_SETTINGS_MODULE = \"config.settings_testing\"\nmarkers = [\n    \"slow: marks tests as slow (deselect with '-m \\\"not slow\\\"')\",\n    \"jwt_scope: sets required scope for api test\",\n]\nenv = [\n    \"ENVIRONMENT=test\",\n]\n")

Pants version
2.9

OS
MacOS

Additional info
I created a reproduction using the example-django project here: https://github.com/bastianwegge/example-django/tree/pyproject-toml-diff-bug

[benjyw]

Confirming that I reproduce this. This seems like a very bad bug.

[Eric-Arellano]

I encountered this with pantsbuild/pants #12679 (comment), but I assumed it was only due to Pants synthetically creating pyproject.toml in our integration tests, which I reasoned users aren't doing so lower priority.

[benjyw]

This is due to coverage config and pytest config colliding - they both pick up pyproject.toml.

Although why they aren't identical is unclear to me so far.

[benjyw]

Ah, it's because we update the coverage config and write it back out. Ugh.

[benjyw]

Will figure out a hack around this.

[bastianwegge]

That was fast! I'm super stoked that you both took the time! 🎉 👏
As I am on 2.9 currently, I guess it will take some time for me to catch up.
Feel free to close this if you cannot reproduce the bug within the example-django project with your fixes

[Eric-Arellano]

It was cherry-picked to Pants 2.10, which we hope to have a stable release of this week :)

[Eric-Arellano]

Let's reopen this if the bug fix doesn't do the trick.