cephfs: add SelectFilesystem, a 2nd fs to micro-osd.sh etc, and tests #827

phlogistonjohn · 2023-02-06T21:56:27Z

This PR updates the micro-osd.sh script to host an alternate cephfs filesystem, on pacific and later.
The entrypoint.sh and makefile are updated to plumb information about the alternate fs to the test suite.
The SelecFilesystem function is added implementing ceph_select_filesystem.
A group of tests, gated by the name of the alternate filesystem being provided, is added, along with API status updates.

Fixes: #262
Fixes: #818
Original-Version-By: "Arvid E. Picciani" arvid@kraud.cloud
Original-PR-URL: #819

Checklist

Added tests for features and functional changes
Public functions and types are documented
Standard formatting is applied to Go code
Is this a new API? Added a new file that begins with //go:build ceph_preview
Ran make api-update to record new APIs

New or infrequent contributors may want to review the go-ceph Developer'sGuide including the section on how we track API Status and the API Stability Plan.

The go-ceph project uses mergify. View the mergify command guide for information on how to interact with mergify. Add a comment with @Mergifyio rebase to rebase your PR when github indicates that the PR is out of date with the base branch.

phlogistonjohn · 2023-02-06T21:57:19Z

Keeping it in draft to run the CI on the various branches which I haven't done locally.

phlogistonjohn · 2023-02-06T22:08:48Z

Mystery failures on nautilus and octopus. Failures in cephfs admin tests that assume there's exactly one fs. The latter should be easy to fix. I'll investigate the former soon. Watch this space! ;-)

Add a function to bring up a 2nd cephfs file system. Execute it on ceph versions pacific and later. While the API exists on nautilus and octopus those versions are already deprecated in go-ceph. Additionally, the ceph docs for "multifs" state, "Beginning with the Pacific release, multiple file system support is stable and ready to use." [1] Therefore we won't bother running our test automation on anything earlier. 1 - https://docs.ceph.com/en/latest/cephfs/multifs/#multiple-ceph-file-systems Signed-off-by: John Mulligan <jmulligan@redhat.com>

With micro-osd.sh supporting a 2nd alternate fs we add basic support to entrypoint.sh to help the tests find and use that alternate fs. The command line option is flexible, you can supply it with a literal fs name or, when prefixed with an `@`, it will read the name from a file. Signed-off-by: John Mulligan <jmulligan@redhat.com>

Signed-off-by: John Mulligan <jmulligan@redhat.com>

ansiwen

LGTM, set DNM label to give you a chance to read my nit comments.

cephfs/admin/volume_test.go

cephfs/select_fs.go

anoopcs9

All good, just a minor comment(see below).

docs/api-status.json

Add SelectFileystem implementing ceph_select_filesystem which can be used to select a cephfs by name, prior to calling mount, when the ceph cluster hosts more than one cephfs file system. Fixes: ceph#262 Fixes: ceph#818 Original-Version-By: "Arvid E. Picciani" <arvid@kraud.cloud> Original-PR-URL: ceph#819 Signed-off-by: John Mulligan <jmulligan@redhat.com>

These tests are all scoped within a test function that will only run if the alternate fs name is supplied through the environment. This way the tests can be run by hand on older versions if need be. The tests verify the use of the SelectFilesystem function and that the file systems are distinct namespaces. Signed-off-by: John Mulligan <jmulligan@redhat.com>

If the ceph cluster we're testing against supports more then one fs our tests shouldn't fail. Update the tests to ensure a cephfs exists as expected but don't assume it's the only one (or that it's the first one returned). Signed-off-by: John Mulligan <jmulligan@redhat.com>

Signed-off-by: John Mulligan <jmulligan@redhat.com>

Pull request has been modified.

phlogistonjohn · 2023-02-11T14:47:20Z

I updated the doc comment. To save a little time I already set the version numbers in the API status by running make api-fix-versions myself. IF we don't merge before the release I'll manually adjust the numbers myself. I just hope this isn't needed (nudge, nudge :-))

This has dismissed the previous reviews, so @ansiwen @anoopcs9 please take another look. Thanks.

anoopcs9

lgtm, thanks.

ansiwen · 2023-02-13T11:41:13Z

@Mergifyio rebase

ansiwen · 2023-02-13T11:45:32Z

@Mergifyio refresh

ansiwen · 2023-02-13T11:47:37Z

Mergify is on vacation?

phlogistonjohn · 2023-02-13T15:10:55Z

Looks like we need to make a change to mergify config. It shows the following when I look at the status:



    The configuration uses the deprecated rebase_fallback mode of the queue and/or merge action.
    A brownout is planned on February 13th, 2023.
    This option will be removed on March 13th, 2023.
    For more information: https://docs.mergify.com/actions/queue/ or https://docs.mergify.com/actions/merge/

So we're hitting this brownout I guess. As I would like to expidite this PR for the review. I am going to hit the merge button as I have two approvals for this PR. I will follow up with a fix for mergify config later today, but that won't be desired for the release this week.

mergify · 2023-02-16T21:58:10Z

rebase

☑️ Nothing to do

-closed [:pushpin: rebase requirement]

mergify · 2023-02-16T21:58:11Z

refresh

✅ Pull request refreshed

phlogistonjohn added the API This PR includes a change to the public API of a go-ceph package label Feb 6, 2023

phlogistonjohn force-pushed the jjm-alt-fs branch from 3033965 to 6a9f009 Compare February 7, 2023 21:00

phlogistonjohn added 3 commits February 7, 2023 16:25

makefile: add ability to use alternate fs on multi container tests

480c3f9

Signed-off-by: John Mulligan <jmulligan@redhat.com>

phlogistonjohn force-pushed the jjm-alt-fs branch from 8957195 to 1bb065f Compare February 7, 2023 21:25

phlogistonjohn marked this pull request as ready for review February 7, 2023 23:34

phlogistonjohn requested review from anoopcs9 and ansiwen February 7, 2023 23:34

ansiwen added the do-not-merge label Feb 8, 2023

ansiwen previously approved these changes Feb 8, 2023

View reviewed changes

cephfs/admin/volume_test.go Show resolved Hide resolved

cephfs/admin/volume_test.go Show resolved Hide resolved

cephfs/select_fs.go Show resolved Hide resolved

anoopcs9 reviewed Feb 9, 2023

View reviewed changes

docs/api-status.json Outdated Show resolved Hide resolved

phlogistonjohn added 4 commits February 11, 2023 09:40

docs: add SelectFilesystem to api status

b4e6675

Signed-off-by: John Mulligan <jmulligan@redhat.com>

phlogistonjohn force-pushed the jjm-alt-fs branch from 1bb065f to b4e6675 Compare February 11, 2023 14:44

phlogistonjohn removed the do-not-merge label Feb 11, 2023

phlogistonjohn requested review from anoopcs9 and ansiwen February 11, 2023 14:47

anoopcs9 reviewed Feb 13, 2023

View reviewed changes

anoopcs9 approved these changes Feb 13, 2023

View reviewed changes

ansiwen approved these changes Feb 13, 2023

View reviewed changes

phlogistonjohn merged commit 45e9a80 into ceph:master Feb 13, 2023

phlogistonjohn deleted the jjm-alt-fs branch March 6, 2023 15:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cephfs: add SelectFilesystem, a 2nd fs to micro-osd.sh etc, and tests #827

cephfs: add SelectFilesystem, a 2nd fs to micro-osd.sh etc, and tests #827

phlogistonjohn commented Feb 6, 2023 •

edited

Loading

phlogistonjohn commented Feb 6, 2023

phlogistonjohn commented Feb 6, 2023

ansiwen left a comment •

edited

Loading

anoopcs9 left a comment

phlogistonjohn commented Feb 11, 2023

anoopcs9 left a comment

ansiwen commented Feb 13, 2023

ansiwen commented Feb 13, 2023

ansiwen commented Feb 13, 2023

phlogistonjohn commented Feb 13, 2023

mergify bot commented Feb 16, 2023

mergify bot commented Feb 16, 2023

cephfs: add SelectFilesystem, a 2nd fs to micro-osd.sh etc, and tests #827

cephfs: add SelectFilesystem, a 2nd fs to micro-osd.sh etc, and tests #827

Conversation

phlogistonjohn commented Feb 6, 2023 • edited Loading

Checklist

phlogistonjohn commented Feb 6, 2023

phlogistonjohn commented Feb 6, 2023

ansiwen left a comment • edited Loading

Choose a reason for hiding this comment

anoopcs9 left a comment

Choose a reason for hiding this comment

phlogistonjohn commented Feb 11, 2023

anoopcs9 left a comment

Choose a reason for hiding this comment

ansiwen commented Feb 13, 2023

ansiwen commented Feb 13, 2023

ansiwen commented Feb 13, 2023

phlogistonjohn commented Feb 13, 2023

mergify bot commented Feb 16, 2023

☑️ Nothing to do

mergify bot commented Feb 16, 2023

✅ Pull request refreshed

phlogistonjohn commented Feb 6, 2023 •

edited

Loading

ansiwen left a comment •

edited

Loading