instead of writing a .gitignore with /target -- output a .gitignore with * *inside* target/

[asottile]

Problem

currently cargo init helpfully writes out a .gitignore with the following contents:

/target

this ~generally means that any rust project now needs to have this /target entry in .gitignore

instead of needing this in .gitignore, one can create a self-ignoring directory by writing a .gitignore with the contents * inside of it. this is (for example) what virtualenv does:

$ virtualenv venv >& /dev/null
$ cat venv/.gitignore
# created by virtualenv automatically
*

this would remove the need for everyone having /target in their gitignore and instead have it be automatic!

Proposed Solution

the proposal is to create a .gitignore with * in the target directory upon creation and phase out the top-level .gitignore's /target entry

Notes

No response

[weihanglo]

This looks pretty interesting! Thanks for your proposal.

There are several things needs to consider. One is backward compatibility. I believe it is rare that a user commits stuff under target/ directory. It may be fine to completely ignore target dir, although I am not 120% sure there is no one what to commit target/cargo-timings for further build pipeline investigation. (Cargo should provide a better way to store such information 🤪)

Cargo supports various kinds of VCS, e.g. Git, Mecurial, Pijul, and Fossil. We don't need to do it all at once, as 99% of Rust projects choose Git. Still, we can perform more researches on how other VCS works.

Going further, “which VCS ignore file should be generated” becomes a question. Is it a good practice for Cargo to check VCS information for each run of cargo build? Could we store VCS info somewhere to avoid that cost? Or just generate 4 kinds of ignore files when every time Cargo recreates target?

The idea is cool anyway. Thank you again.

[LunarLambda]

Writing 4 tiny (single line) files should not be a significant cost compared to any kind of average cargo compile, even hello world.

Meson for example generates ones for git and mercurial only, but others wouldn't be difficult, assuming they provide glob support. However, it only generates them once. Cargo could do the same if cargo clean didn't delete target/ entirely, but left the VCS ignores

[mathstuf]

Please keep /target in the ignore as putting in the directory does not support target-as-symlink.

[LunarLambda]

In what case would you need target to be a symlink?

[mathstuf]

I keep my build artifacts on a separate partition to keep my main one from croaking when it gets too full. It also makes it easy to keep them out of backups by just ignoring where all build artifacts live.

[LunarLambda]

Can't the target directory be controlled via a config option or environment variable?

Alternatively, a bind mount is also possible.

I don't know how the cargo team sees it, but this seems like a fairly niche use case, it feels like the usability improvement would be greater in the general case. Besides, you can still have an additional .gitignore file in the project directory itself, it just wouldn't be there by default (just as target as a symlink isn't there by default, so I think this could be easily added to whatever setup script you may have)

[mathstuf]

Can't the target directory be controlled via a config option or environment variable?

Can it include variables? Say, $buildroot/$(basename ../..)/target? If not..I'm not sure how one is supposed to make sure all tools agree on this path in such a way. A symlink Just Works and everything agrees about where it goes.

Alternatively, a bind mount is also possible.

I don't think involving root into any cargo-using project I want to build is scalable.

Besides, you can still have an additional .gitignore file in the project directory itself, it just wouldn't be there by default

That doesn't help with projects I contribute to (I've made quite a few PRs removing the trailing slash in projects' .gitignore after fixing the template here).

[senekor]

@mathstuf you can add target to your global gitignore file.

For the other VCS, we could either take the same approach (if possible) or simply keep the current approach. Because the target dir will be ignored completely, the technically redundant .gitignore won't hurt anyone using other VCS.

It could be interesting to check if other VCS support this approach. At least judging from mercurial's documentation, it doesn't support it. So we'd have to keep the code paths of the "old" approach around either way.

I am not 120% sure there is no one what to commit target/cargo-timings

This shouldn't be a problem. Git keeps tracking already tracked files, even if they are matched by a gitignore file. It is also possible to track new files that are ignored with git add -f. Other VCS are not affected either, at least if we keep the current approach for them. Otherwise we'd have to check how each VCS behaves in this case.

I think this is a nice cleanup for simple Rust projects with basically no downsides.

[mathstuf]

@mathstuf you can add target to your global gitignore file.

IMO, it doesn't belong there. Global ignores are for things I do (e.g., vim backup files or other editor detritus, macOS users and the .DS_Store noise). Projects should have ignore entries for things they are responsible for (e.g., Rust projects' target and JavaScript's node_modules). If there was a way to make a global ignore conditional on the type of project…sure. But I've had to deal with overzealous global ignores masking files that should have been usable from projects not using the tools those ignores were for.

[senekor]

IMO, it doesn't belong there. Global ignores are for things I do

Well, you are the one creating the symlink, right?

You can also put it in .git/info/exclude. Now it's project-specifc and individual. You can even automate that with a script and shell hook. Fundamentally, the symlink workflow is very specific to you and therefore it makes sense for you to configure it idividually.

It's not cargo's responsibility to advertise the preferred setup of some people with a very niche use case, especially if that use case can easily be achieved in multiple different ways.

[mathstuf]

Hmm. Perhaps if there were a way to exclude it if it is a symlink (like trailing-/ does for directories)…

[senekor]

if [ -d .git ] && [ -L target ] && ! grep -q "/target" .git/info/exclude
then
    echo "/target" >> .git/info/exclude
fi

Run that as a shell hook whenever you cd into a new directory.

[weihanglo]

Writing a .gitignore inside target directory seems good on its own. However, changing the cargo new/init template needs more considerations on backward-compatibility, depending on how much we value it.

For example, one could cargo new a package with the new self-ignore behavior, so there is no .gitignore in that new package. Other folks then cloned the repo and found out the target directory was not ignored because they were using an older toolchain, and that requires extra efforts to figure out and fix than having .gitignore in the first place.

There might be some other tools in the wild only understanding the root .gitignore, but do not recognize the self-ignored directory, and that might lead to unexpected results. I cannot think of a tool may fail in this case atm though.

[senekor]

Other folks then cloned the repo and found out the target directory was not ignored because they were using an older toolchain

That's a good point. I agree that we should keep the template for a (long) while because of that.

People can still "opt-in" early by removing the .gitignore file or configuring cargo with cargo-new.vcs = "none".

There might be some other tools in the wild only understanding the root .gitignore, but do not recognize the self-ignored directory

I'd say a tool like that is just fundamentally broken, because it doesn't match git's behavior. Users of these tools would also have problems with other existing tools that use this technique (e.g. virtualenv).

[mathstuf]

Run that as a shell hook whenever you cd into a new directory.

Just a note that this hook is not sufficient, namely in that it fails to consider worktrees, and that it waits until coming back to a directory to update the set. In any case a hook on cd is way too broad; I'd probably write it up as a git-exclude-cargo-target script instead (or, more likely, part of my mkbuilddir script that sets up the symlinks in the first place), but it's something I'll look at when .gitignore becomes rare rather than ASAP.

[senekor]

I created a new issue (#15061) that only deals with adding the new file to the target directory.