tox should crash when an unknown key is used

[obestwalter]

If the user mistypes a key name, they should get appropriate feedback that what they might want to happen will not happen.

One obvious example:

[testenv]
deps = pytest
command = pytest

oops - forget the s and the tests are not run.

Error could look something like:

ERROR: unknown key 'command' in testenv 'whatever'

On first look this seems an easy fix. One thing that needs to be kept in mind is that plugins can also add configuration options, so the check needs to happen after the plugins had the chance to do that.

Also: I sincerely hope that nobody uses the existing behaviour to add their own keys to tox testenvs and do something with them in a different tool, but even if it is I would regard this a breaking change triggering major version jump.

[gaborbernat]

I'm opposed to this myself, for multiple reasons, I think this would cause lots of issues:

• some of the keys might be there for some optional plugins (aka virtual env no download - without the plugin the process should finish just as fine, why would we force users to install the plugin if it's optional - definitely not until we have plugin auto provision implemented),
• this would be breaking change and not bound well with the current substitution logic possibilities ( I use this technique myself multiple times):

[tox]
magic_dep = a

[tox:A]
deps = {[tox]magic_dep}
       d
[tox:B]
deps = {[tox]magic_dep}
       c 

[gaborbernat]

To be fair I don't even think we have a reliable way to find out what option is needed and what's not. We load all in the file (the defined one are post-processed), but some of them might only be accessed by downstream plugins and I don't believe they can at the moment even register new file configs. At minimum we would be looking at a long migration path until the plugins would adopt. I don't think it's worth the effort, so I'll close this unless someone can convince me otherwise by tomorrow.

[obestwalter]

I think you misunderstand the issue: It's not about not needed keys, it's about unknown keys. If plugins do not make their keys known in a well defined way yet than this would be something that could be improved.

[obestwalter]

But you're right: self defined substitutions would need to be handled also. I don't know if it's worth the effort, but we should at least discuss it.

If there is no general solutions, we could maybe catch the most common mistakes, but that would be ugly.

If there is no programmatic solution for this then it should be documented clearly that the user can define arbitrary keys and reuse them later, because AFAIK this is not explicitly documented.

[gaborbernat]

there's no way to 100% classify what's unneeded and what's unknown AFAIK, so I don't think it's doable. yes we should document the substitution part.

[obestwalter]

there's no way to 100% classify what's unneeded and what's unknown AFAIK, so I don't think it's doable.

Looking at it again, I am afraid you're right. There is no clean way to do this. I guess it is better to come around to the idea that the way how configuration with tox.ini works is what it is and it is good enough although it has quite a few qualities which make it less newbie friendly and harder to debug problems.

Rather than trying to fix it here, we can learn from this and let this flow into the design of #999.

If we can agree on this, maybe we should even announce (and add to the (contributor) docs) that new features or major changes in tox.ini are not wanted and it is considered frozen (except for bug fixes). Then we can concentrate on making it better with the new config format.

Closing this then and opening issues for what came up here, as it is off-topic here.