Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ScalarVerb: Remove EnsureLocalCacheIsHealthy #146

Merged
merged 2 commits into from
Oct 4, 2019

Conversation

wilbaker
Copy link
Member

@wilbaker wilbaker commented Oct 3, 2019

Part of the work required for #135

With the deprecating of the mount process (and verb)
EnsureLocalCacheIsHealthy is no longer needed.

In VFS4G, EnsureLocalCacheIsHealthy is needed because:

  • A mapping file is used in the .gvfsCache directory that
    may need to be regenerated if the user deletes it.
  • The decision was made that the mount verb should ensure
    that the alternates file is properly configured to use
    the cache directory.

However, in Scalar:

  • A mapping file will not be used (Scalar will rely on
    repository IDs and stable hashes.
  • There will be no 'mount' verb run by users
  • Scalar will not be responsible for automatically fixing
    up modifications made by users that would remove the
    .scalarCache directory from the alternates files

And so EnsureLocalCacheIsHealthy is being removed.

@wilbaker wilbaker force-pushed the remove_alternates_fixup_code branch 2 times, most recently from 343266e to 3dd7149 Compare October 3, 2019 22:11
@wilbaker wilbaker added the disk-layout Issues that impact the disk layout used by Scalar label Oct 3, 2019
@wilbaker wilbaker changed the title [WIP] ScalarVerb: Remove EnsureLocalCacheIsHealthy ScalarVerb: Remove EnsureLocalCacheIsHealthy Oct 3, 2019
@wilbaker wilbaker marked this pull request as ready for review October 3, 2019 22:31
@wilbaker wilbaker requested a review from derrickstolee October 3, 2019 22:31
Copy link
Contributor

@derrickstolee derrickstolee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's wait for #122 to merge, and see if we can remove the [Ignore] annotation.

@@ -257,7 +220,7 @@ private void LoadBlobsViaGit(ScalarFunctionalTestEnlistment enlistment)
// 'git rev-list --objects' will check for all objects' existence, which
// triggers an object download on every missing blob.
ProcessResult result = GitHelpers.InvokeGitAgainstScalarRepo(enlistment.RepoRoot, "rev-list --all --objects");
result.ExitCode.ShouldEqual(0);
result.ExitCode.ShouldEqual(0, result.Errors);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have this change in #122.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops I saw you change there and added it locally, did not mean to push it to this PR. I will remove it thanks!

With the deprecating of the mount process (and verb)
EnsureLocalCacheIsHealthy is no longer needed.

In VFS4G, EnsureLocalCacheIsHealthy is needed because:

- A mapping file is used in the .gvfsCache directory that
  may need to be regenerated if the user deletes it.
- The decision was made that the mount verb should ensure
  that the alternates file is properly configured to use
  the cache directory.

However, in Scalar:

- A mapping file will not be used (Scalar will rely on
  repository IDs and stable hashes.
- There will be no 'mount' verb run by users
- Scalar will not be responsible for automatically fixing
  up modifications made by users that would remove the
  .scalarCache directory from the alternates files

And so EnsureLocalCacheIsHealthy is being removed.
@wilbaker wilbaker force-pushed the remove_alternates_fixup_code branch from 5fa3ba8 to ad64ba6 Compare October 4, 2019 16:06
@wilbaker
Copy link
Member Author

wilbaker commented Oct 4, 2019

@derrickstolee I need to add back the [Ignore]. I tested with the changes in #122 and found that when git can't find the sharedCache directory it falls back on using the local .git/objects directory:

~/Scalar.FT/test/3c38712ba97b430b8d12/src>git rev-list --all --objects
error: object directory /Users/wilbaker/Scalar.FT/b09f63271332491b847d8218268260c2/.customScalarCache/b2747b29d0cc492fb757b463b9f5d5fb does not exist; check .git/objects/info/alternates
~/Scalar.FT/test/3c38712ba97b430b8d12/src>ls .git/objects/
00	0a	14	1e	28	32	3c	46	50	5a	64	6e	78	82	8c	96	a0	aa	b4	be	c8	d2	dc	e6	f0	fa
01	0b	15	1f	29	33	3d	47	51	5b	65	6f	79	83	8d	97	a1	ab	b5	bf	c9	d3	dd	e7	f1	fb
02	0c	16	20	2a	34	3e	48	52	5c	66	70	7a	84	8e	98	a2	ac	b6	c0	ca	d4	de	e8	f2	fc
03	0d	17	21	2b	35	3f	49	53	5d	67	71	7b	85	8f	99	a3	ad	b7	c1	cb	d5	df	e9	f3	fd
04	0e	18	22	2c	36	40	4a	54	5e	68	72	7c	86	90	9a	a4	ae	b8	c2	cc	d6	e0	ea	f4	fe
05	0f	19	23	2d	37	41	4b	55	5f	69	73	7d	87	91	9b	a5	af	b9	c3	cd	d7	e1	eb	f5	ff
06	10	1a	24	2e	38	42	4c	56	60	6a	74	7e	88	92	9c	a6	b0	ba	c4	ce	d8	e2	ec	f6	info
07	11	1b	25	2f	39	43	4d	57	61	6b	75	7f	89	93	9d	a7	b1	bb	c5	cf	d9	e3	ed	f7	pack
08	12	1c	26	30	3a	44	4e	58	62	6c	76	80	8a	94	9e	a8	b2	bc	c6	d0	da	e4	ee	f8
09	13	1d	27	31	3b	45	4f	59	63	6d	77	81	8b	95	9f	a9	b3	bd	c7	d1	db	e5	ef	f9

Interestingly it must be returning 0 as the test fails not in the call to git, but when validating that the shared cache folder was recreated.

@derrickstolee
Copy link
Contributor

@derrickstolee I need to add back the [Ignore]. I tested with the changes in #122 and found that when git can't find the sharedCache directory it falls back on using the local .git/objects directory

Adding the [Ignore] is fine for now. Adding @jeffhostetler to think about the right thing to do here.

@wilbaker wilbaker merged commit 73421a7 into microsoft:master Oct 4, 2019
@jeffhostetler
Copy link
Contributor

@wilbaker @derrickstolee
I added code to try to get the sharedCache pathname and fall back to .git/objects/ if the config var
isn't set. That was for safety -- the top-level git command (not just gvfs-helper) should not fail if the
shared cache isn't present, right?

With the last round of changes, git.exe passes the sharedCache pathname on the
command line to gvfs-helper (which means that git.exe looks up config variable
usually) so that there are no surprises.

I haven't yet looked at the unit test in question, so I can't say which is the right answer.
That is, is my code wrong or does the test need fixing.

@wilbaker
Copy link
Member Author

wilbaker commented Oct 4, 2019

@jeffhostetler @derrickstolee

That was for safety -- the top-level git command (not just gvfs-helper) should not fail if the
shared cache isn't present, right?

Prior to gvfs-helper the mount process would create the shared cache folder if it was missing before downloading the object(s) requested by git (see https://github.com/microsoft/VFSForGit/blob/4736473e8fc5c1703bb011ad314faf85a14a624b/GVFS/GVFS.Common/Git/GitObjects.cs#L606).

That code was put in to VFS4G because:

  • Users expected that they could delete the shared cache and things would still work properly without manual intervention.
  • Auto-healing in this scenario reduces customer frustration and support load (if the fix is to tell the user to recreate the folder, we might as well recreate it ourselves)

I think if the gvfs.sharedCache setting is interpreted to mean "this is where git should download missing objects (when using the gvfs protocol)" then I think it's okay for git to create the folder, because that is where the config file indicates that downloaded files should be placed.

As an added benefit, it might allow us to further simplify scalar clone by removing the code for creating the sharedCache directory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
disk-layout Issues that impact the disk layout used by Scalar
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants