virtualfilesystem: don't run the virtual file system hook if the index has been redirected #15

benpeart · 2018-08-02T16:12:05Z

Closes #13

Some git commands spawn helpers and redirect the index to a different
location. These include "difftool -d" and the sequencer
(i.e. git rebase -i, git cherry-pick and git revert) and others.
In those instances we don't want to update their temporary index with
our virtualization data.

Helped-by: Johannes Schindelin johannes.schindelin@gmx.de
Signed-off-by: Ben Peart Ben.Peart@microsoft.com

kewillford · 2018-08-02T16:53:40Z

config.c

+			/* virtual file system relies on the sparse checkout logic so force it on */
+			core_apply_sparse_checkout = 1;
+			return 1;
+		}


Do we need to turn off the core_virtualfilesystem flag in an else here or are all the checks of that flag inside a check of core_apply_sparse_checkout? Because not it looks like we are in a state where the one is set but the other isn't.

Good catch. Yes, I'll update it to ensure they stay consistent.

…x has been redirected Fixes #13 Some git commands spawn helpers and redirect the index to a different location. These include "difftool -d" and the sequencer (i.e. `git rebase -i`, `git cherry-pick` and `git revert`) and others. In those instances we don't want to update their temporary index with our virtualization data. Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>

virtualfilesystem: don't run the virtual file system hook if the index has been redirected

…racked files Update the package for GitForWindows to include tracing information. Includes the following Git PRs: * [36 Avoid `sane_execvp` in `git rebase` and `git stash`](microsoft/git#36) * [34 Add Trace2 regions to 'pack-objects'](microsoft/git#34) * [28 Trace2 base plus GVFS extensions](microsoft/git#28) * [33 virtualfilesystem: check if directory is included](microsoft/git#33) * [31 Revert "gvfs: add a perf test for reading the index"](microsoft/git#31) * [32 compat/poll: prepare for targeting Windows Vista](microsoft/git#32) * [22 Enable the filesystem cache (fscache) in refresh_index()](microsoft/git#22) * [27 virtualfilesystem: fix bug with symlinks being ignored](microsoft/git#27) * [23 Unpack trees with cache tree gvfs](microsoft/git#23) * [15 virtualfilesystem: don't run the virtual file system hook if the index has been redirected](microsoft/git#15) * Includes Git 2.19.0. This also includes VFS for Git updates from Kevin Willford's work: **Fix missing untracked files when created in subfolder** Add to the test to create multiple level of folders and files in those folder to make sure they show up as untracked files. Update the git for windows version that has the fix. Fixes #358

virtualfilesystem: don't run the virtual file system hook if the index has been redirected

Add virtual file system settings and hook proc. On index load, clear/set the skip worktree bits based on the virtual file system data. Use virtual file system data to update skip-worktree bit in unpack-trees. Use virtual file system data to exclude files and folders not explicitly requested. The hook was first contributed in private, but was extended via the following pull requests: #15 #27 #33 #70 Signed-off-by: Ben Peart <Ben.Peart@microsoft.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>

An internal customer reported a segfault when running `git sparse-checkout set` with the `index.sparse` config enabled. I was unable to reproduce it locally, but with their help we debugged into the failing process and discovered the following stacktrace: ``` #0 0x00007ff6318fb7b0 in rehash (map=0x3dfb00d0440, newsize=1048576) at hashmap.c:125 #1 0x00007ff6318fbc66 in hashmap_add (map=0x3dfb00d0440, entry=0x3dfb5c58bc8) at hashmap.c:247 #2 0x00007ff631937a70 in hash_index_entry (istate=0x3dfb00d0400, ce=0x3dfb5c58bc8) at name-hash.c:122 #3 0x00007ff631938a2f in add_name_hash (istate=0x3dfb00d0400, ce=0x3dfb5c58bc8) at name-hash.c:638 #4 0x00007ff631a064de in set_index_entry (istate=0x3dfb00d0400, nr=8291, ce=0x3dfb5c58bc8) at sparse-index.c:255 #5 0x00007ff631a06692 in add_path_to_index (oid=0x5ff130, base=0x5ff580, path=0x3dfb4b725da "<redacted>", mode=33188, context=0x5ff570) at sparse-index.c:307 #6 0x00007ff631a3b48c in read_tree_at (r=0x7ff631c026a0 <the_repo>, tree=0x3dfb5b41f60, base=0x5ff580, depth=2, pathspec=0x5ff5a0, fn=0x7ff631a064e5 <add_path_to_index>, context=0x5ff570) at tree.c:46 #7 0x00007ff631a3b60b in read_tree_at (r=0x7ff631c026a0 <the_repo>, tree=0x3dfb5b41e80, base=0x5ff580, depth=1, pathspec=0x5ff5a0, fn=0x7ff631a064e5 <add_path_to_index>, context=0x5ff570) at tree.c:80 #8 0x00007ff631a3b60b in read_tree_at (r=0x7ff631c026a0 <the_repo>, tree=0x3dfb5b41ac8, base=0x5ff580, depth=0, pathspec=0x5ff5a0, fn=0x7ff631a064e5 <add_path_to_index>, context=0x5ff570) at tree.c:80 #9 0x00007ff631a06a95 in expand_index (istate=0x3dfb00d0100, pl=0x0) at sparse-index.c:422 #10 0x00007ff631a06cbd in ensure_full_index (istate=0x3dfb00d0100) at sparse-index.c:456 #11 0x00007ff631990d08 in index_name_stage_pos (istate=0x3dfb00d0100, name=0x3dfb0020080 "algorithm/levenshtein", namelen=21, stage=0, search_mode=EXPAND_SPARSE) at read-cache.c:556 #12 0x00007ff631990d6c in index_name_pos (istate=0x3dfb00d0100, name=0x3dfb0020080 "algorithm/levenshtein", namelen=21) at read-cache.c:566 #13 0x00007ff63180dbb5 in sanitize_paths (argc=185, argv=0x3dfb0030018, prefix=0x0, skip_checks=0) at builtin/sparse-checkout.c:756 #14 0x00007ff63180de50 in sparse_checkout_set (argc=185, argv=0x3dfb0030018, prefix=0x0) at builtin/sparse-checkout.c:860 #15 0x00007ff63180e6c5 in cmd_sparse_checkout (argc=186, argv=0x3dfb0030018, prefix=0x0) at builtin/sparse-checkout.c:1063 #16 0x00007ff6317234cb in run_builtin (p=0x7ff631ad9b38 <commands+2808>, argc=187, argv=0x3dfb0030018) at git.c:548 #17 0x00007ff6317239c0 in handle_builtin (argc=187, argv=0x3dfb0030018) at git.c:808 #18 0x00007ff631723c7d in run_argv (argcp=0x5ffdd0, argv=0x5ffd78) at git.c:877 #19 0x00007ff6317241d1 in cmd_main (argc=187, argv=0x3dfb0030018) at git.c:1017 #20 0x00007ff631838b60 in main (argc=190, argv=0x3dfb0030000) at common-main.c:64 ``` The very bottom of the stack being the `rehash()` method from `hashmap.c` as called within the `name-hash` API made me look at where these hashmaps were being used in the sparse index logic. These were being copied across indexes, which seems dangerous. Indeed, clearing these hashmaps and setting them as not initialized fixes the segfault. The second commit is a response to a test failure that happens in `t1092-sparse-checkout-compatibility.sh` where `git stash pop` starts to fail because the underlying `git checkout-index` process fails due to colliding files. Passing the `-f` flag appears to work, but it's unclear why this name-hash change causes that change in behavior.