about/long_about and doc-comment precedence is unpredictable

[ssokolow]

In my experiments, I discovered that, if a StructOpt-deriving structure has a doc-comment that is not intended to be included in the --help output:

1. Normally, you must specify long_about or the doc-comment will overrule about for both -h and --help.
2. When you specify a template value, the relationship is flipped and you must specify about because the doc-comment will overrule long_about.

This is a footgun cocked and ready to go off. (Because I believe in having polished, meaningful rustdoc output, I almost pushed a change which had Command-line argument schema as my command's --help description before I noticed the unexpected change.)

As I couldn't figure out how I might even attempt to replicate a "doc comment on a derived struct" bug like this with Clap's API, I'm going to assume it's a bug in StructOpt.

Here's a minimal reproducer. Just try adding and removing the about, long_about, and template lines and you'll see what I mean in the --help output:

use structopt::StructOpt;

const HELP_TEMPLATE: &str = "{bin} {version}

{about}

USAGE:
    {usage}

{all-args}
";

/// DOC COMMENT
#[derive(StructOpt, Debug)]
#[structopt(
            about="ABOUT STRING",
            long_about="LONG ABOUT STRING",
            raw(template="HELP_TEMPLATE")
            )]
struct Opt {}

fn main() { Opt::from_args(); }

[TeXitoi]

Maybe that's not really a bug but #146 that breaks your usecase?

[ssokolow]

I'm not sure how it would be.

The problem happens with both the single-line test case above and this multi-line doc-comment block from my project boilerplate:

/// Command-line argument schema
///
/// ## Relevant Conventions:
///
///  * Make sure that there is a blank space between the `<name>` `<version>` line and the
///    description text or the `--help` output won't comply with the platform conventions that
///    `help2man` depends on to generate your manpage.
///    (Specifically, it will mistake the `<name> <version>` line for part of the description.)
///  * StructOpt's default behaviour of including the author name in the `--help` output is an
///    oddity among Linux commands and, if you don't disable it, you run the risk of people
///    unfamiliar with `StructOpt` assuming that you are an egotistical person who made a conscious
///    choice to add it.
///
///    The proper standardized location for author information is the `AUTHOR` section which you
///    can read about by typing `man help2man`.
///
/// ## Cautions:
///  * Subcommands do not inherit `template` and it must be re-specified for each one.
///    ([clap-rs/clap#1184](https://github.com/clap-rs/clap/issues/1184))
///  * Double-check that your choice of `about` or `long_about` is actually overriding this
///    doc comment. The precedence is affected by things you wouldn't expect, such as the presence
///    or absence of `template` and it's easy to wind up with this doc-comment as your `--help`
///    ([TeXitoi/structopt#173](https://github.com/TeXitoi/structopt/issues/173))
///  * Do not begin the description text for subcommands with `\n`. It will break the formatting
///    in the top-level help output's list of subcommands.

Even if it is somehow caused by that, having predecence change like that in response to something that, intuitively, should have no effect on it is still a massive footgun of the sort I came to Rust to avoid wherever possible.

[0ndorio]

@ssokolow @TeXitoi

I just discovered this issue and took a short moment to look into it as I wrote #146

The underlying issue seems to be a combination of the following points:

1. The clap default template for a simple struct like yours only displays long_about even if -h is used (Maybe also in other cases but I didn't checked this).
2. Your custom template only displays about even if --help is used.
3. Writing a doc-comment automatically applies the content of the comment to about and long_about as long as you don't define a custom value.

Based on this I would guess that this issue is either ...

• a problem with claps default behavior as it should display about if -h is used as an argument.
• or a problem with claps template engine as it should include an auto toggle {about/long_about} placeholder.

Please correct me if I am wrong but the only chance I can see to fix this from the perspective of structopt would be to apply a new hidden default template to fix the default behavior of clap.

[ssokolow]

A new hidden default template wouldn't affect me at all unless it also reworks things to match GNU conventions so I don't need to override it. Specifically:

1. Remove {author} because it's not normally part of --help output and anyone familiar with other command-line tools but not familiar with structopt will mistake it for a conscious choice made by an egotist.

   (If nothing else, GNU convention plans for a project to accumulate a large number of contributors over its lifetime, which makes putting them in --help unfeasible... plus, it's irrelevant clutter, given what question users run --help looking to answer. You might as well automatically query crates.io and GitHub/Bitbucket/GitLab at build time and complement what's supposed to go in the AUTHORS manpage section with equivalents to the manpage sections for where to report bugs and what known bugs exist in this release.)

2. Add a newline between the {bin} {version} line and {about} so help2man doesn't mistake the {bin} {version} line for a word-wrapped first line of the {about} block.

[0ndorio]

Seems like your are not the only one with this issue see: clap-rs/clap#1472.

**Issue number updated as I accidentally linked an old issue. **

[TeXitoi]

Structopt will not use a default hidden template. This sort of things must be done in clap.

[ssokolow]

structopt already sort of does have a de facto default hidden template in that Clap won't populate {author} unless asked while, with structopt, I have to either set a custom template or manually override it to nothing and deal with bug #172

[TeXitoi]

Then, maybe the solution can be that no author is setted by default, but structopt(author) do the magic? Same for version and about?

[ssokolow]

Yeah. That and getting a fix to clap-rs/clap#1432 would remove my need for a custom template.

[0ndorio]

Just realized I linked the wrong issue above as I missed the leading number: clap-rs/clap#1472

[CreepySkeleton]

I confirm the bug is still present.

I propose to institute a strict priority order among help/about//// ......
Doc comments are the lowest priority.

// help/about overrides one-line doc comment

/// This is not displayed
#[structopt(help = "This is displayed")]


// long_help/long_about overrides multi-line doc comment 
// with second line empty 

/// This is not displayed
///
/// More docs...
#[structopt(long_help = "This is displayed")]


// `help` does NOT override "long" doc comment just as
// `long_help` does NOT override one-line doc comment

/// This is displayed on -h
#[structopt(long_help = "This is displayed on --help")]

/// This is displayed on --help
///
/// More docs
#[structopt(help = "This is displayed on -h")]

cc @TeXitoi @ssokolow @0ndorio

[TeXitoi]

OK for me.

[ssokolow]

As long as the docs on that behaviour are as clear as humanly possible, I'd be OK with that.

[CreepySkeleton]

docs on that behaviour are as clear as humanly possible

Could you please elaborate? What is "as clear as possible" in your understanding is? I think the design presented is quite clear and sensible.

[TeXitoi]

I think it means "please document explicitly these rules".

[CreepySkeleton]

I think is mean "please document explicitly these rules".

That's integral :) I would still like to hear @ssokolow answer though.

[ssokolow]

I think it means "please document explicitly these rules".

Not just explicitly but with care paid to the way in which it's made explicit. I've seen lots of things where what was written was perfectly explicit... but it took an outsized amount of time to convert what was being read into a mental model of what was being described.

(Typically stuff written by academics who have spent so much time writing papers for their peers that they've forgotten how to write anything else. The manual for splint has this problem.)

Something combining both a point-form list of the rules in simple English (ie. high-school-level vocabulary and no overly complex sentence structures) and some examples... ideally with the examples including <pre> copy-pastes of shell sessions to minimize the amount of mental energy needed to connect the dots.

The point-form list should be a unified place to understand help behaviour, so it should also serve to answer questions like these:

1. If a user specifies -h and there's no source for short help, what happens?
2. If a user specifies --help and there's no source for long help, what happens?

(Same design philosophy as a well-written manpage, actually... just applied to a slightly different kind of interface.)

[CreepySkeleton]

Something combining both a point-form list of the rules in simple English (ie. high-school-level vocabulary and no overly complex sentence structures) and some examples... ideally with the examples including <pre> copy-pastes of shell sessions to minimize the amount of mental energy needed to connect the dots.

We could really use some help on improving the documentation. It looks like your English-fu is better than mine and @TeXitoi 's put together (no offense). I'll ping you once we have the implementing PR merged if you don't mind.

• If a user specifies -h and there's no source for short help, what happens?

• If a user specifies --help and there's no source for long help, what happens?

Those are up to clap to answer.

[ssokolow]

I'll ping you once we have the implementing PR merged if you don't mind.

I can't guarantee how much time I'll have, but I'm certainly open to it.

Those are up to clap to answer.

Yes, but you don't want users to feel that they're being passed around like a hot potato, so it'd be a good idea for structopt's docs to either provide a unified overview of the behaviour or provide a link directly to the relevant part of the clap docs.

(Otherwise, you wind up in a situation analogous to how man tesseract says that it supports "anything readable by Leptonica" as input formats, and the user (me) is stuck tracking down the right project named leptonica and reading through the developer docs (officially unofficial docs at that), just to figure out which formats can be fed to an OCR tool. The proper solution would have been to say something like "JPEG, PNG, TIFF, BMP, GIF, WebP, and anything else readable by Leptonica", thus covering the common cases nicely, while also making the actual constraint clear.)

[CreepySkeleton]

feel that they're being passed around like a hot potato

I'm sure I've seen a porn starting just like you describe...

provide a link directly to the relevant part of the clap docs.

Yeah, this is what we typically do in a situation like this.

[ssokolow]

I'm sure I've seen a porn starting just like you describe...

I was thinking more about how, when company support desks insist on doing that, I make sure to waste paid employee hours with value equivalent to whatever they cheated me out of, but whatever floats your boat.