Synchronize compat_features with mdn/browser-compat-data tags

[ddbeck]

Goal

When we override compat_features in a YAML file here, those changes ought to propagate to mdn/browser-compat-data. Likewise, new features originating in mdn/browser-compat-data ought to bubble up to web-features.

Background

Open Web Docs is working on several tasks including:

Provide synchronization of browser-compat-data keys between the mdn/browser-compat-data and web-platform-dx/web-features repositories. Contribute source code required to synchronize keys to their respective repositories on GitHub.

(see openwebdocs/project#208)

But we're not capturing this work here on the repo anywhere. This issue is attempting cover it.

[ddbeck]

There's two sides of this: upstreaming and downstreaming.

Downstreaming

Downstreaming is sending changes from the upstream side (mdn/browser-compat-data) to the downstream (web-platform-dx/web-features).

I'd expect a web-features workflow along these lines:

1. Start by upgrading @mdn/browser-compat-data in web-features (e.g., via Dependabot, repository/workflow dispatch, or cron workflow).
2. Run npm run bcd-sync (or similar). Do these steps:
   1. If there exists a tag in BCD that matches a compat_features list, then delete the compat_features list from the file and commit.
   2. If there exists a tag in BCD that differs from the compat_features list, then annotate the .yml file’s compat_features line with the discrepancy. Note: some discrepancies will be expected/desired (e.g., BCD is “stale”); this should not fail tests or block merging.
   3. Run npm run dist and commit.
   4. Push the changes.
3. Review. If applicable, push corrective changes to the PR. Finally, approve the PR.
4. Merge the PR.
5. After merging, if there any web-features: keys that exist in BCD that do not have a corresponding feature ID, then generate an issue (or perhaps even a PR from a template) in web-features reporting a "missing" feature.

Upstreaming

Upstreaming is sending changes from the downstream side (web-platform-dx/web-features) to the upstream side (mdn/browser-compat-data).

I'd expect an mdn/browser-compat-data workflow:

1. Start by upgrading web-features in a PR (e.g., via Dependabot, repository/workflow dispatch, or cron workflow).
2. Run some script that does these steps:
   1. Compare the list of feature IDs in web-features with the list of tags in BCD.
   2. Produce a Checks report (e.g., some Markdown) that shows what tags/features will change.
   3. Run migrate-to-bcd.
   4. Push the results to the PR.
3. Review and approve the PR.
4. Merge the PR.

[ddbeck]

Today, I met with @atopal and @Elchi3 to figure out next steps for this work.

Highlights

• We're going to continue to work on the scripts and workflows needed for this, but…
• We're not going to turn on synchronization workflows until the feature backfilling is complete. This is to minimize churn and challenging changes to YAML files.
• That means we need to…
  • Complete these PRs:
    • Add script to cleanup redundant BCD overrides #2333 — @queengooborg to follow up on naming
    • Remove redundant compat_features keys from features with comments #2345 (partially, only moving or removing the comments) — @queengooborg to rebase and refresh
    • Add undist script #2474
  • Create new mdn/browser-compat-data PRs to:
    • Prevent authors from creating new tags (i.e., BCD uses web-features IDs as an allowlist for tags). See Allowlist web-feature tags mdn/browser-compat-data#25614
    • Apply tags to BCD keys based on compat_features lists from web-features. See Add action to update web-features tags mdn/browser-compat-data#25704
  • Create a new PR in web-features to document the overall new BCD key workflow (see next section for an overview). See Add docs for how web-features syncs with BCD #2560
  • Close these PRs:
    • Cleanup compat_features keys #2334
    • Create action to automatically migrate web-features tags to BCD #2471

Workflow summary

After some discussion, we're expecting two typical workflows for synchronizing compat_features with BCD tags:

1. New keys are tagged in BCD directly. In this workflow, new keys in BCD are tagged from inception. This is more likely for small, incremental changes to existing features.

   For example, a new key appears in BCD (e.g., via a Collector PR). The author of that PR knew which feature the key belongs to (e.g., due to a key being renamed) and tags the key with the correct web-features:* tag. A few days later, that data is released by BCD. That triggers the next @mdn/browser-compat-data Dependabot upgrade PR, where we regenerate web-features data based on the updated tag and remove the compat_features list from the YAML file (if the list and tagged keys match). Done.

   If necessary (e.g., a tag was assigned in error), a web-features maintainer can update a compat_features list in web-features and the work syncs to BCD as in workflow 2.

2. New keys are tagged in web-features. In this workflow, web-features receives unaccounted-for keys in the features/draft folder and BCD periodically incorporates compat_features as tags. This is more likely for new features.

   For example, a new key appears in BCD (e.g., via a Collector PR). The author of that PR doesn't know or doesn't care which feature that key belongs to and does not tag it. A few days later, that data is released by BCD. That triggers the next @mdn/browser-compat-data Dependabot upgrade PR. After that PR merges, we run the "Update draft features" workflow, which makes the new keys to appear in the features/draft folder. At this point, a web-features maintainer can assign the key to an existing feature or author a new feature. A few days later, that data is released by web-features. When BCD upgrades web-features, it'll automatically apply tags to each feature according to the compat_features list.

To prevent circular changes:

• BCD contributors must never create a new tag before the corresponding ID appears in web-features.
• web-features contributors must never manually remove a compat_features list from a feature.

To ensure ongoing completeness, web-features maintainers ought to strive to keep generated drafts to an absolute minimum.

[Elchi3]

I just merged the initial BCD sync from the GitHub action. mdn/browser-compat-data#25762

[Elchi3]

Do we need to land anything else other than the documentation in #2560?

[ddbeck]

@Elchi3 There's one last thing, which I forgot to include in the to-do list above:

We must periodically run scripts/remove-tagged-compat-features.ts. I think that should be a follow-up to browser-compat-data Dependabot PRs, but not part of that PR. So I propose the following:

• Make a "Remove tagged compat features" workflow. Add a workflow to remove tagged compat features #2587

  This would be a copy of the "Update Draft Features" workflow, with these exceptions:

  • Run scripts/remove-tagged-compat-features.ts instead of the drafts updater.
  • Comment out the cron schedule (until we're ready to run that routinely).

• Add running the above workflow to the Dependabot upgrade checklist. Add a workflow to remove tagged compat features #2587

Since we're not likely to turn this on for a couple of weeks, I consider OWD's part in this done when #2560 lands. I will absolutely not turn down the help, of course. 😄

[Elchi3]

I consider OWD's part in this done when #2560 lands

Thanks Daniel. I just landed this. cc @atopal

I will absolutely not turn down the help, of course. 😄

Absolutely, I will continue to help out :)

[atopal]

Perfect! Thank you all for your work on this!

[ddbeck]

With #2587, the work here is complete except for turning on the automatic workflow runs.

Once we've completed mapping the back catalog of compat keys, we need to uncomment these lines from the workflow:

web-features/.github/workflows/remove_tagged_compat_features.yml

Lines 5 to 6 in c69f545

	# schedule:
	# - cron: "30 5 * * 1,4"

So this is blocked for now, but very nearly complete. I'll leave this issue open until that uncommenting has happened.

Thank you @Elchi3 and @queengooborg for your work on getting this into good shape.