Suppress tracebacks and provide a meaningful solution

[tomschr]

Situation and problem

Let's assume we have the following pyproject.toml file:

Example pyproject.toml (click me)

[build-system]
requires = [
    "hatchling>=1.8.0",
]
build-backend = "hatchling.build"

[project]
name = "semver"
description = "Python helper for Semantic Versioning (https://semver.org)"
# [...]

[tool.hatch.envs.docs]
dependencies = [
    #"sphinx",
    "sphinx-argparse",
    "sphinx-autodoc-typehints",
]
platforms = ["linux", "windows",]

[tool.hatch.envs.docs.scripts]
build = "make -C docs html"
linkcheck = "make -C docs linkcheck"
serve = "python3 -m webbrowser -t docs/_build/html/index.html"

When running hatch env show it gives me the following output:

$ hatch env show
                         Standalone                         
┏━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Name    ┃ Type    ┃ Dependencies             ┃ Scripts   ┃
┡━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ default │ virtual │                          │           │
├─────────┼─────────┼──────────────────────────┼───────────┤
│ docs    │ virtual │ sphinx-argparse          │ build     │
│         │         │ sphinx-autodoc-typehints │ linkcheck │
│         │         │                          │ serve     │
└─────────┴─────────┴──────────────────────────┴───────────┘

That's good! A nicely formatted table.

Let's assume I would like to amend the pyproject.toml file above with platform dependent commands. However, I don't know how exactly what to do so I end up with the following (wrong) line:

[tool.hatch.envs.docs.scripts.overrides]
wrong = "wrong line"

When I run hatch env show again, I get this traceback:

$ hatch env show
[ ... pruned long detailed traceback ...]
TypeError: Field `tool.hatch.envs.docs.scripts.overrides` must be a string or an array of strings

Possible solutions

In my humble opinion, showing these detailed traceback is wrong and only useful for Hatch developers. A traceback should never be presented to an end user. Of course, the error itself must be visible, but not the traceback itself.

A user doesn't care where the problem was raised nor which internal file was affected. All the user cares is:

• where does it happen in the config file, and
• how to fix it.

The first part (where does it happen) is kind of presented in the error message. It would be nice to have a line number, although it's not completely necessary. A line number may be helpful if you have a section with different options, but only one is broken.

What's more helpful (and missing from the error message) is how to solve it. The error message tells us that it expects a string or array of strings. Well, that's nice, but "developer lingo". 😉 As a newbie to Hatch I would like to know how does the string has to look like. Especially when people try Hatch for the first time, it makes a real difference if Hatch tries to be as user friendly as possible.

So here is my suggestion:

• Tell the user what exactly Hatch expects, not only data types.
• Provide a link to the reference where the user can find more helpful examples and details.

In an ideal case, the user does NOT have to open the reference documentation. It should be clear from the message what Hatch expects and how to solve it. Nevertheless, Hatch should show the URL to the respective section of the reference if a user wants to know details or examples.

This could look something like this:

$ hatch env show
ERROR: Missing `project` metadata table in configuration.
Solution: Add a [project] section in your config file with at least the following minimal options:

  [project]
  name = "foobar"
  description = "Makes foo to bars"

For details, see also: <URL_to_the_reference_doc>

The real output and format can vary and could be subject to a user configuration. A beginner would probably be happy to see more helpful messages whereas an expert is satisfied if it gets the problem shown. I think, git has a similar system.

This would focus on the essential parts and avoid distraction for the user.

TODOs

If you consider this issue helpful, the following tasks need to be taken into account:

• Write meaningful solutions.
• Suppress tracebacks for end users.
• If really needed, make tracebacks visible for Hatch developers.
• Test URLs to reference docs. If you restructure the doc, change anchors etc. the tests should fail and tell which URL is broken. That prevents that URLs for the solution gets out of sync and users see a 404 error when accessing the URL.
• Investigate if it's useful to have a user configuration to switch the verbosity of the messages (more for beginners, less for experts). This could also be implemented in a different PR.

[Bobronium]

Love the proposal and excited for these changes!

This PR might be helpful as a reference: python-poetry/cleo#132
As well as the following issue: python-poetry/poetry#5846 and following discussion and resolution.

TL;DR: cleo framework which poetry built on has a "solutions" concept bound to errors. PR mentioned above adressed the default verbosity. Issue linked above addressed an edge case when hiding tracebacks wasn't a good strategy.

This experiene may have less value in the context of hatch codebase, but nontheless, IMO, should be considered.

[warsaw]

Big +1 for this issue.

I am writing a hatch.toml file for $work and messed up my matrix context field substitutions, yielding a very long traceback and this message:

ValueError: Unknown context field `lib`

It does not give me the line number in my config file where the error occurs, at least AFAICT.

1. The error really should print the file and line number where the misconfiguration was found
2. The error should not print the very detailed traceback, because as you say, that's for developers and isn't helpful to end users. It's fine if there's a switch (like increased verbosity) to produce the traceback for issue reporting.

[ofek]

I plan to improve messages and also prevent the user from ever seeing tracebacks from the CLI. I would appreciate it if someone could look at what other CLIs (in any language) do in this regard since this improvement is not extremely high priority and therefore I don't have too much time.