Normative: specify time zone ID requirements to reduce divergence between engines #877

justingrant · 2024-03-31T08:28:27Z

This proposed change resolves #825 by adding normative spec text to clarify how ECMA-402 implementations should decide which IANA time zone IDs should be primary vs. non-primary. This will enable more consistency between ECMAScript implementations and prevent future divergence.

This PR also accommodates to web reality by aligning ECMA-402 with CLDR and ICU. This should make it easier for all ECMAScript engines to comply with the spec while still being able to use ICU.

This PR is stacked on #876, so please ignore the first commit when reviewing this PR.

Note that the problem of out-of-date primary IDs for renamed Zones (like Asia/Calcutta) is out of scope to this PR, because the spec already requires current IDs, and there's already a plan to fix it that requires no spec changes.

Summary of proposed changes

This PR implements "Option C" in #825 by deterministically defining ECMAScript's exceptions from the IANA Time Zone Database's defaults, and then pointing implementers at ICU as a convenient implementation of those exceptions.

We'll start with a baseline of IANA's Zone and Link names and specify a few exceptions:

Existing special-cases for Etc/UTC, Etc/GMT, and GMT will be retained.
Primary and non-primary IDs must share the same ISO-3166-2 country code, so IANA's time zone merges between countries will be reverted. CLDR and ICU already disallow merges across country boundaries. Note that IANA's merges of time zones *inside* of a country (e.g. Asia/Chongquing=>Asia/Shanghai) will be accepted, also matching current ICU/CLDR behavior.

This PR also makes two other smaller text changes that we expect to have zero impact on current engines:

Changes Editorial: align time zone identifier text and AOs with ECMA-262 #876's recommendations into requirements to limit observable dynamic updating of TZDB info. AFAIK, no existing ECMAScript engine updates TZDB (observably or otherwise!) during the lifetime of the surrounding agent, so this is really a future-compatibility change.
Adds text recommending a two-year waiting period before a newly-renamed ID becomes primary. These are rare (the last was 2022's Europe/Kiev=>Europe/Kyiv) and when the next rename happens, we'll try to convince CLDR to implement this requirement for us. There are no pending renames, so this is also a future-compatibility change.

Per-engine changes required

Implementing the changes in this PR will impact JS engines differently, given the current divergence between engines:

For V8 (cc @FrankYFTang) and JSC (cc @Constellation), there should be no change because they rely on ICU and CLDR which already behave like 1-3 above. Note that this PR doesn't affect the plan to fix out-of-date canonicalizations like Asia/Calcutta and Europe/Kiev. This plan is unchanged: as part of landing Temporal Stage 4, switch to use ICU's new icu::TimeZone::getIanaID(), which returns the latest IANA IDs instead of out-of-date canonical IDs like Asia/Calcutta.
For SpiderMonkey (cc @anba), more changes are needed because currently SpiderMonkey conforms to the spec which requires using backward in TZDB to determine canonicalization. SM could use icu::TimeZone::getIanaID() to implement (2) and (3) above, or could implement the same behavior by reading CLDR data or IANA data directly. Also, this PR will reduce SM's differences in Intl.supportedValuesOf('timeZone') vs. V8/JSC.

Testing

Test262 changes will be needed to validate these normative changes, but I'm not sure how we can run those tests except using the Temporal polyfill. @ptomato I'll be looking for your advice (and perhaps help writing tests!) on this point.

Feedback requested

Feedback is welcome on any part of this proposal, but I'm most interested in making sure that the spec text actually accomplishes what the summary above claims that it does.

spec/locales-currencies-tz.html

sffc · 2024-04-02T20:46:11Z

Thanks for putting this together @justingrant! We can discuss this at the next TG2 meeting. In the mean time, I encourage the listed reviewers to take a look.

ptomato · 2024-04-03T00:37:00Z

Test262 changes will be needed to validate these normative changes, but I'm not sure how we can run those tests except using the Temporal polyfill.

It's fine to submit tests to test262 that no engine can pass yet, as long as they are correct according to the current snapshot of ECMA-262 or 402 or a Stage 3 proposal.

ptomato

I don't feel qualified to review the details — I mostly haven't followed those discussions. At a general level, this all looks very reasonable.

spec/locales-currencies-tz.html

gibson042

Mostly editorial comments, but I am not comfortable with making the rename waiting period mandatory.

spec/locales-currencies-tz.html

FrankYFTang · 2024-04-25T18:44:56Z

4. e.g. PST7PDT=>America/Los_Angeles

I assuem you mean to say "e.g. PST8PDT=>America/Los_Angeles" ?
not "PST7PDT=>America/Los_Angeles" right?

FrankYFTang · 2024-04-25T19:05:44Z

I am not sure what these CLDR tool files for but it mention
https://github.com/unicode-org/cldr/blob/a6fbd0ffa1db0c9f60dc0f678ace589366ae7f3a/tools/cldr-code/src/main/resources/org/unicode/cldr/icu/timezone_aliases.txt#L100
https://github.com/unicode-org/cldr/blob/a6fbd0ffa1db0c9f60dc0f678ace589366ae7f3a/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/TimeZoneAliases.txt#L86

Here is good one
https://github.com/unicode-org/cldr/blob/a6fbd0ffa1db0c9f60dc0f678ace589366ae7f3a/common/supplemental/supplementalMetadata.xml#L1800

Look at

<zoneAlias type="EST" ,

etc

Add agenda item for: Normative: specify time zone ID requirements to reduce divergence between engines tc39/ecma402#877

sffc · 2024-07-18T19:21:25Z

I believe this got TG1 consensus in June 2024.

@justingrant Is this ready to merge, or should we wait for CLDR-17111?

justingrant · 2024-07-18T19:31:19Z

AFAIK this is ready to merge. It lacks Test262 tests, but I assume these can be added later?

sffc · 2024-07-30T18:41:37Z

@ben-allen If this is ready to merge, please merge it :)

"PST8PDT" is a link in tzdata, but since we still use `backzone` entries, it's rewritten to a zone in "TimeZoneDataGenerated.h". This will change when we implement <tc39/ecma402#877>. Differential Revision: https://phabricator.services.mozilla.com/D224469

"PST8PDT" is a link in tzdata, but since we still use `backzone` entries, it's rewritten to a zone in "TimeZoneDataGenerated.h". This will change when we implement <tc39/ecma402#877>. Differential Revision: https://phabricator.services.mozilla.com/D224469 UltraBlame original commit: 8f0baf6a61b721bcc08162a241467e4c037b7faf

"PST8PDT" is a link in tzdata, but since we still use `backzone` entries, it's rewritten to a zone in "TimeZoneDataGenerated.h". This will change when we implement <tc39/ecma402#877>. Differential Revision: https://phabricator.services.mozilla.com/D224469

…n. r=dminor Implement the changes from <tc39/ecma402#877> to reduce differences in time zone canonicalisation when compared to V8 and JSC (which both use CLDR time zone data instead of IANA time zone data). Implementing the `AvailableNamedTimeZoneIdentifiers` spec operation requires to link time zone identifiers to region codes. When the time zone is listed in the "zone.tab" file, we can get the region code from "zone.tab". In all other cases we need to manually map the time zone to a matching region, because CLDR doesn't have "public" data for this [1]. This is implemented using the new file "intl/TimeZoneMapping.yaml". ICU 74 added `ucal_getIanaTimeZoneID` to get the canonical IANA time zone id. Internally `ucal_getIanaTimeZoneID` first calls `ucal_getCanonicalTimeZoneID` and then loads a resource bundle to check if there are any time zone ids which need to be replaced with other ids for compatibility with IANA data. Unfortunately the resource bundle is not cached, so calling `ucal_getIanaTimeZoneID` instead of `ucal_getIanaTimeZoneID` adds a considerable performance overhead. To avoid any performance regressions, we keep our own time zone rewriting code for the time being. Using our own code also means we don't have to add a workaround for this CLDR bug: <https://unicode-org.atlassian.net/browse/CLDR-16439>. Also remove "Factory" from the list of supported time zone identifiers, because supporting it was always a bit questionable and latest V8 also doesn't support it anymore, so we shouldn't run into web-compat issues. Remove the old generated tests and add "timeZone_links.js" to ensure time zone links are correctly resolved. [1] Neither of these two files look like "public" data to me: - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/TZID.txt - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/icu/idList.txt Differential Revision: https://phabricator.services.mozilla.com/D228584

…n. r=dminor Implement the changes from <tc39/ecma402#877> to reduce differences in time zone canonicalisation when compared to V8 and JSC (which both use CLDR time zone data instead of IANA time zone data). Implementing the `AvailableNamedTimeZoneIdentifiers` spec operation requires to link time zone identifiers to region codes. When the time zone is listed in the "zone.tab" file, we can get the region code from "zone.tab". In all other cases we need to manually map the time zone to a matching region, because CLDR doesn't have "public" data for this [1]. This is implemented using the new file "intl/TimeZoneMapping.yaml". ICU 74 added `ucal_getIanaTimeZoneID` to get the canonical IANA time zone id. Internally `ucal_getIanaTimeZoneID` first calls `ucal_getCanonicalTimeZoneID` and then loads a resource bundle to check if there are any time zone ids which need to be replaced with other ids for compatibility with IANA data. Unfortunately the resource bundle is not cached, so calling `ucal_getIanaTimeZoneID` instead of `ucal_getIanaTimeZoneID` adds a considerable performance overhead. To avoid any performance regressions, we keep our own time zone rewriting code for the time being. Using our own code also means we don't have to add a workaround for this CLDR bug: <https://unicode-org.atlassian.net/browse/CLDR-16439>. Also remove "Factory" from the list of supported time zone identifiers, because supporting it was always a bit questionable and latest V8 also doesn't support it anymore, so we shouldn't run into web-compat issues. Remove the old generated tests and add "timeZone_links.js" to ensure time zone links are correctly resolved. [1] Neither of these two files look like "public" data to me: - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/TZID.txt - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/icu/idList.txt Differential Revision: https://phabricator.services.mozilla.com/D228584 UltraBlame original commit: 880f2288640c43653f56bc919e9d16434426bf84

…n. r=dminor Implement the changes from <tc39/ecma402#877> to reduce differences in time zone canonicalisation when compared to V8 and JSC (which both use CLDR time zone data instead of IANA time zone data). Implementing the `AvailableNamedTimeZoneIdentifiers` spec operation requires to link time zone identifiers to region codes. When the time zone is listed in the "zone.tab" file, we can get the region code from "zone.tab". In all other cases we need to manually map the time zone to a matching region, because CLDR doesn't have "public" data for this [1]. This is implemented using the new file "intl/TimeZoneMapping.yaml". ICU 74 added `ucal_getIanaTimeZoneID` to get the canonical IANA time zone id. Internally `ucal_getIanaTimeZoneID` first calls `ucal_getCanonicalTimeZoneID` and then loads a resource bundle to check if there are any time zone ids which need to be replaced with other ids for compatibility with IANA data. Unfortunately the resource bundle is not cached, so calling `ucal_getIanaTimeZoneID` instead of `ucal_getIanaTimeZoneID` adds a considerable performance overhead. To avoid any performance regressions, we keep our own time zone rewriting code for the time being. Using our own code also means we don't have to add a workaround for this CLDR bug: <https://unicode-org.atlassian.net/browse/CLDR-16439>. Also remove "Factory" from the list of supported time zone identifiers, because supporting it was always a bit questionable and latest V8 also doesn't support it anymore, so we shouldn't run into web-compat issues. Remove the old generated tests and add "timeZone_links.js" to ensure time zone links are correctly resolved. [1] Neither of these two files look like "public" data to me: - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/TZID.txt - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/icu/idList.txt Differential Revision: https://phabricator.services.mozilla.com/D228584 UltraBlame original commit: a33fb69abc60581a92b8ebba42fc9cbf193df2e9

…n. r=dminor Implement the changes from <tc39/ecma402#877> to reduce differences in time zone canonicalisation when compared to V8 and JSC (which both use CLDR time zone data instead of IANA time zone data). Implementing the `AvailableNamedTimeZoneIdentifiers` spec operation requires to link time zone identifiers to region codes. When the time zone is listed in the "zone.tab" file, we can get the region code from "zone.tab". In all other cases we need to manually map the time zone to a matching region, because CLDR doesn't have "public" data for this [1]. This is implemented using the new file "intl/TimeZoneMapping.yaml". ICU 74 added `ucal_getIanaTimeZoneID` to get the canonical IANA time zone id. Internally `ucal_getIanaTimeZoneID` first calls `ucal_getCanonicalTimeZoneID` and then loads a resource bundle to check if there are any time zone ids which need to be replaced with other ids for compatibility with IANA data. Unfortunately the resource bundle is not cached, so calling `ucal_getIanaTimeZoneID` instead of `ucal_getIanaTimeZoneID` adds a considerable performance overhead. To avoid any performance regressions, we keep our own time zone rewriting code for the time being. Using our own code also means we don't have to add a workaround for this CLDR bug: <https://unicode-org.atlassian.net/browse/CLDR-16439>. Also remove "Factory" from the list of supported time zone identifiers, because supporting it was always a bit questionable and latest V8 also doesn't support it anymore, so we shouldn't run into web-compat issues. Remove the old generated tests and add "timeZone_links.js" to ensure time zone links are correctly resolved. [1] Neither of these two files look like "public" data to me: - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/TZID.txt - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/icu/idList.txt Differential Revision: https://phabricator.services.mozilla.com/D228584 UltraBlame original commit: 880f2288640c43653f56bc919e9d16434426bf84

…n. r=dminor Implement the changes from <tc39/ecma402#877> to reduce differences in time zone canonicalisation when compared to V8 and JSC (which both use CLDR time zone data instead of IANA time zone data). Implementing the `AvailableNamedTimeZoneIdentifiers` spec operation requires to link time zone identifiers to region codes. When the time zone is listed in the "zone.tab" file, we can get the region code from "zone.tab". In all other cases we need to manually map the time zone to a matching region, because CLDR doesn't have "public" data for this [1]. This is implemented using the new file "intl/TimeZoneMapping.yaml". ICU 74 added `ucal_getIanaTimeZoneID` to get the canonical IANA time zone id. Internally `ucal_getIanaTimeZoneID` first calls `ucal_getCanonicalTimeZoneID` and then loads a resource bundle to check if there are any time zone ids which need to be replaced with other ids for compatibility with IANA data. Unfortunately the resource bundle is not cached, so calling `ucal_getIanaTimeZoneID` instead of `ucal_getIanaTimeZoneID` adds a considerable performance overhead. To avoid any performance regressions, we keep our own time zone rewriting code for the time being. Using our own code also means we don't have to add a workaround for this CLDR bug: <https://unicode-org.atlassian.net/browse/CLDR-16439>. Also remove "Factory" from the list of supported time zone identifiers, because supporting it was always a bit questionable and latest V8 also doesn't support it anymore, so we shouldn't run into web-compat issues. Remove the old generated tests and add "timeZone_links.js" to ensure time zone links are correctly resolved. [1] Neither of these two files look like "public" data to me: - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/TZID.txt - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/icu/idList.txt Differential Revision: https://phabricator.services.mozilla.com/D228584 UltraBlame original commit: a33fb69abc60581a92b8ebba42fc9cbf193df2e9

…n. r=dminor Implement the changes from <tc39/ecma402#877> to reduce differences in time zone canonicalisation when compared to V8 and JSC (which both use CLDR time zone data instead of IANA time zone data). Implementing the `AvailableNamedTimeZoneIdentifiers` spec operation requires to link time zone identifiers to region codes. When the time zone is listed in the "zone.tab" file, we can get the region code from "zone.tab". In all other cases we need to manually map the time zone to a matching region, because CLDR doesn't have "public" data for this [1]. This is implemented using the new file "intl/TimeZoneMapping.yaml". ICU 74 added `ucal_getIanaTimeZoneID` to get the canonical IANA time zone id. Internally `ucal_getIanaTimeZoneID` first calls `ucal_getCanonicalTimeZoneID` and then loads a resource bundle to check if there are any time zone ids which need to be replaced with other ids for compatibility with IANA data. Unfortunately the resource bundle is not cached, so calling `ucal_getIanaTimeZoneID` instead of `ucal_getIanaTimeZoneID` adds a considerable performance overhead. To avoid any performance regressions, we keep our own time zone rewriting code for the time being. Using our own code also means we don't have to add a workaround for this CLDR bug: <https://unicode-org.atlassian.net/browse/CLDR-16439>. Also remove "Factory" from the list of supported time zone identifiers, because supporting it was always a bit questionable and latest V8 also doesn't support it anymore, so we shouldn't run into web-compat issues. Remove the old generated tests and add "timeZone_links.js" to ensure time zone links are correctly resolved. [1] Neither of these two files look like "public" data to me: - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/TZID.txt - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/icu/idList.txt Differential Revision: https://phabricator.services.mozilla.com/D228584 UltraBlame original commit: 880f2288640c43653f56bc919e9d16434426bf84

…n. r=dminor Implement the changes from <tc39/ecma402#877> to reduce differences in time zone canonicalisation when compared to V8 and JSC (which both use CLDR time zone data instead of IANA time zone data). Implementing the `AvailableNamedTimeZoneIdentifiers` spec operation requires to link time zone identifiers to region codes. When the time zone is listed in the "zone.tab" file, we can get the region code from "zone.tab". In all other cases we need to manually map the time zone to a matching region, because CLDR doesn't have "public" data for this [1]. This is implemented using the new file "intl/TimeZoneMapping.yaml". ICU 74 added `ucal_getIanaTimeZoneID` to get the canonical IANA time zone id. Internally `ucal_getIanaTimeZoneID` first calls `ucal_getCanonicalTimeZoneID` and then loads a resource bundle to check if there are any time zone ids which need to be replaced with other ids for compatibility with IANA data. Unfortunately the resource bundle is not cached, so calling `ucal_getIanaTimeZoneID` instead of `ucal_getIanaTimeZoneID` adds a considerable performance overhead. To avoid any performance regressions, we keep our own time zone rewriting code for the time being. Using our own code also means we don't have to add a workaround for this CLDR bug: <https://unicode-org.atlassian.net/browse/CLDR-16439>. Also remove "Factory" from the list of supported time zone identifiers, because supporting it was always a bit questionable and latest V8 also doesn't support it anymore, so we shouldn't run into web-compat issues. Remove the old generated tests and add "timeZone_links.js" to ensure time zone links are correctly resolved. [1] Neither of these two files look like "public" data to me: - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/TZID.txt - https://github.com/unicode-org/cldr/blob/main/tools/cldr-code/src/main/resources/org/unicode/cldr/icu/idList.txt Differential Revision: https://phabricator.services.mozilla.com/D228584 UltraBlame original commit: a33fb69abc60581a92b8ebba42fc9cbf193df2e9

justingrant requested review from Constellation, ptomato, sffc, gibson042, anba and FrankYFTang March 31, 2024 08:28

justingrant force-pushed the time-zone-id-requirements branch 3 times, most recently from aa65347 to 3ea9cee Compare March 31, 2024 08:35

justingrant mentioned this pull request Mar 31, 2024

Decouple 262 IsValidTimeZone from 402 IsValidTimeZone tc39/proposal-temporal#1996

Closed

justingrant commented Mar 31, 2024

View reviewed changes

spec/locales-currencies-tz.html Outdated Show resolved Hide resolved

justingrant force-pushed the time-zone-id-requirements branch from 3ea9cee to 6a544eb Compare March 31, 2024 19:51

justingrant mentioned this pull request Mar 31, 2024

Should ECMA-402 spec text for time zone canonicalization refer to CLDR or to IANA as authoritative? #825

Closed

justingrant added normative needs tests needs review needs consensus labels Mar 31, 2024

ptomato reviewed Apr 3, 2024

View reviewed changes

spec/locales-currencies-tz.html Outdated Show resolved Hide resolved

spec/locales-currencies-tz.html Outdated Show resolved Hide resolved

justingrant force-pushed the time-zone-id-requirements branch 4 times, most recently from c78de1d to 3bb6dee Compare April 3, 2024 01:48

srl295 reviewed Apr 4, 2024

View reviewed changes

spec/locales-currencies-tz.html Outdated Show resolved Hide resolved

spec/locales-currencies-tz.html Outdated Show resolved Hide resolved

gibson042 requested changes Apr 25, 2024

View reviewed changes

michaelficarra pushed a commit to tc39/agendas that referenced this pull request Jun 3, 2024

Add needs-consensus PR item for ECMA-402 #877

d4ce62e

Add agenda item for: Normative: specify time zone ID requirements to reduce divergence between engines tc39/ecma402#877

JLHwung mentioned this pull request Jun 12, 2024

June 2024 babel/proposals#89

Open

sffc added has consensus (TG1) Has consensus from TC39-TG1 and removed needs consensus labels Jul 18, 2024

justingrant mentioned this pull request Jul 30, 2024

Normative: Handle "start of day" separately from "midnight" tc39/proposal-temporal#2918

Merged

sffc requested a review from ben-allen July 30, 2024 18:41

ben-allen merged commit 602aaf0 into tc39:main Aug 1, 2024
2 checks passed

FrankYFTang mentioned this pull request Dec 19, 2024

Need to add test to test ECMA402 PR 877 tc39/test262#4363

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Normative: specify time zone ID requirements to reduce divergence between engines #877

Normative: specify time zone ID requirements to reduce divergence between engines #877

justingrant commented Mar 31, 2024 •

edited

Loading

sffc commented Apr 2, 2024

ptomato commented Apr 3, 2024

ptomato left a comment

gibson042 left a comment

FrankYFTang commented Apr 25, 2024

FrankYFTang commented Apr 25, 2024 •

edited

Loading

sffc commented Jul 18, 2024 •

edited

Loading

justingrant commented Jul 18, 2024

sffc commented Jul 30, 2024

Normative: specify time zone ID requirements to reduce divergence between engines #877

Normative: specify time zone ID requirements to reduce divergence between engines #877

Conversation

justingrant commented Mar 31, 2024 • edited Loading

Summary of proposed changes

Per-engine changes required

Testing

Feedback requested

sffc commented Apr 2, 2024

ptomato commented Apr 3, 2024

ptomato left a comment

Choose a reason for hiding this comment

gibson042 left a comment

Choose a reason for hiding this comment

FrankYFTang commented Apr 25, 2024

FrankYFTang commented Apr 25, 2024 • edited Loading

sffc commented Jul 18, 2024 • edited Loading

justingrant commented Jul 18, 2024

sffc commented Jul 30, 2024

justingrant commented Mar 31, 2024 •

edited

Loading

FrankYFTang commented Apr 25, 2024 •

edited

Loading

sffc commented Jul 18, 2024 •

edited

Loading