Accurate JSON representation for duration

[dsnet]

Hello, thank you for all your hard work improving time support in ECMAScript.

I'm one of the maintainers of the "encoding/json" package in Go, where the community is currently pursuing a prospective "encoding/json/v2" package (golang/go#71497). There's an unsettled debate (golang/go#71631) regarding what the right JSON representation for a Go time.Duration should be, where the community is split between a custom Go representation (e.g., 123h4m56.789s) versus a subset of ISO 8601 (e.g., PT123H4M56.789S). Since JSON is often used as a format for interacting with systems written in other languages, there's value in being consistent with what other implementations support. Furthermore, JSON finds its heritage in JavaScript, so the precedence set by JavaScript does impact the rest of the industry.

While ISO 8601 specifies a set of grammars for a duration, it is unfortunately ill-suited for Go since it supports both nominal units (e.g., year, month, week, day) and accurate units (hour, minute, second). For implementations with duration data structures that represent both nominal and accurate units individually (e.g., Temporal.Duration), this is perfectly fine. However, for languages that can only represent accurate durations, this is impossible to fully support (e.g., Go's time.Duration, java.time.Duration, google.protobuf.Duration, etc.).

Given the existence of many implementations that can only represent accurate durations, I suspect this has significantly impeded the adoption of the ISO 8601 duration format since interoperability is poor if nominal units are ever used. In an attempt remedy this problem, we authored an RFC Internet-Draft that proposes a strict subset of ISO 8601 that prioritizes interoperability. This RFC I-D is merely a proposal intended to kickstart discussion and the grammar is subject to change based on feedback.

The goal of this issue is to raise the concern of interopability and see if we can cooperatively move towards greater interopability. The Internet is comprised of many systems written by many different languages and would benefit from this endeavor.

Overall, it seems the TC39's design is almost entirely compatible with the proposed RFC I-D, but a minor thought we can consider:

• Have the Temporal.Duration.toJSON method call round({largestUnit: "hour"}) before formatting as ISO 8601 if all the nominal units are zero. This does mean that formatting and parsing from JSON does not round-trip identically, but the Temporal.Duration.compare method does correctly treat PT90S and PT1M30S as equal. One advantage of always rounding is that it avoids leaking details about the fact that Temporal.Duration is able to handle each unit independently (which other languages cannot do). I believe always rounding by the largest hour would make the output compliant with the proposed RFC I-D.

• The above suggestion is however impossible if nominal units are non-zero. In such a case, the toJSON method would have to output an ISO 8601 duration that may not be parsable by other implementations, but this is the best that can be done when lacking any "reference point". Fortunately, the arguably most common way to obtain a duration is by measuring the passage of time through the use of zonedDateTime.until (or since) methods. The until method defaults to measuring time in terms of accurate units (e.g., up to hours if no options are specified). Thus, I suspect that most common usages of the Temporal types will only have accurate units unless the programmer went out of their way to use nominal units.

• The above suggestions are targeted toward toJSON, while toString remains unchanged. The assumption is that toJSON is targeted towards machine consumption where accuracy and interopability is a priority, while toString is targeted towards human consumption where readability is a priority.

• It is far more important that the formatted durations be interoperable (especially in JSON). If Temporal.Duration is able to parse the durations according to the full grammar of ISO 8601, then more power to ECMAScript!

Thank you for considering our thoughts.

\cc @timbray @mvdan @johanbrandhorst

[dsnet]

In case the Temporal proposal is too far along to change, it's probably fine if no behavior is made to the Temporal proposal.

Fortunately, I believe the following statements are true:

• The output of Temporal.Duration.toJSON is exactly identical to the I-D if the duration has already been balanced to the largest unit of hours.
• By default, the typical production of a Temporal.Duration value is already balanced to the largest unit of hours (e.g., zonedDateTime.until or zonedDateTime.since).
• The I-D grammar is always a valid input to Temporal.Duration.from except for overly large or overly precise durations, where I-D says that implementations are free to choose practical limits. Thus, TC39 is free to choose to only support nanosecond precision.

[gibson042]

In case the Temporal proposal is too far along to change, it's probably fine if no behavior is made to the Temporal proposal.

That is probably the case, although I will note that the proposal is still at Stage 3, whose documented purpose is "Gaining implementation experience and discovering any web compatibility or integration issues". So changes like this might still be possible.

EDIT: Removed text representing a misunderstanding that this suggestion would include mapping P1D to P24H.

[dsnet]

Playing around with the Temporal package, I found the following behavior surprising:

t0 = Temporal.Now.zonedDateTimeISO()
// sleep 45 seconds
t1 = Temporal.Now.zonedDateTimeISO()
// sleep 45 seconds
t2 = Temporal.Now.zonedDateTimeISO()

d1 = t2.since(t0)                   // produces P1M30S; makes sense
d2 = t2.since(t1).add(t1.since(t0)) // produces PT90S; huh?

I was surprised that d1 and d2 produce different results when I would semantically expect them to be the same thing.

I understand why this occurs since the balancing to largestUnits: "hours" occurs within the context of the Temporal.ZonedDateTime.since method call, but the semantic intent to balance according to "hours" does not seem to be encoded in the Temporal.Duration object itself. Thus, when Temporal.Duration.add occurs, it doesn't doesn't balance upwards and maintains values in the current largest units (which is seconds).

I wonder if the intended "largestUnits" should be a stateful property in the Temporal.Duration such that arithmetic with a duration respect that intent (or the largest unit between two durations).

[arshaw]

I had similar concerns about a data-inferred largestUnit that I expressed in an older ticket, the "LargestUnit for time parts? Balancing up?" section

[gibson042]

This was discussed in the Temporal champions meeting, and there was no appetite for it at this time. While changing the output of toJSON() wouldn't be harmful, and it would certainly be good to align with the external ecosystem, a lot could still change with https://datatracker.ietf.org/doc/draft-tsai-duration/ and it would be bad to guess wrong. Please post an update here if/when that draft advances, at which point it would be more enthusiastically accepted (if this proposal has not yet been merged into ECMA-262).

The interoperability arguments are completely valid; it's just unfortunate that there's not yet a mature specification to support them.

[timbray]

FWIW my best guess as to what will happen with draft-tsai-duration is: Nothing.

Because nothing happens in the IETF unless there are people who want to get behind a piece of work and put energy into pushing it through the process, which is quite heavyweight. I've done this myself a couple of times but don't have room in my life to do it for this one.

If Joe wants to put a bunch of energy into the task it might advance and it might not. The IETF meets later this month and unless there's a surprise burst of energy there, I think this community could safely consider the spec stable.

[dsnet]

There's a cause-and-effect dilemma going on.

The Go community is hesitant to adopt ISO 8601 because the full set of the standard is impossible for us to support and there exist no specification to constrain ISO 8601 into something interoperable. However, if there were an RFC 3339 equivalent for ISO 8601 durations, then we'd almost certainly adopt that. However, no such RFC exists, so there's pressure to stay the course and do something custom. This makes the industry more fragmented, and thus produces less incentive to pursue agreement for the sake of interoperability, and the cycle goes on...

The status quo is unfortunate, which means that some party (or parties) in the industry need to be the first movers to try to agree on something and break the cycle.

I believe the default temporal-proposal output is sufficiently close to the RFC draft that I'm leaning towards championing the use of ISO 8601 in Go's json/v2 package. There's a relatively minor difference to work out (dsnet/rfc-internet-duration#7), but interoperability is definitely within reach.

If TC39 and Go were compatible, that'd be 2 major implementations with compatible formats that a RFC Standard might come to light eventually after the fact. I'd be willing to invest energy into seeing it advance, but I suspect the existence of two existing implementations will be weight to help progress it along.

A potential candidate for a 3rd implementation would be java.time.Duration, which outputs exactly according to the RFC draft for all non-negative duration values. However, it formats negative durations as PT-1H-2M-3S instead of -PT1H2M3S (even though it can parse the latter as well). This representation is both less readable and has a harder grammar to represent that I don't think TC39 or Go would be willing to adopt it. If TC39 and Go were both compatible with each other, it might serve as incentive for Java to come alongside and be compatible as well in their formatting.

Note that interoperability is primarily focused on the formatting of durations, and less so about parsing. It's fine if implementations parse a larger grammar than the RFC. This could be seen as an application of Postel's Law: "be conservative in what you send, be liberal in what you accept."

[BurntSushi]

@dsnet You can add Rust's jiff to that list too. It's not yet ubiquitous in the Rust ecosystem, but it's steadily making its way.

Jiff behaves the same as Temporal (quirks and all):

use jiff::Zoned;

fn main() -> anyhow::Result<()> {
    let zdt0: Zoned = "2025-03-11T20:13:00-04[America/New_York]".parse()?;
    let zdt1: Zoned = "2025-03-11T20:13:45-04[America/New_York]".parse()?;
    let zdt2: Zoned = "2025-03-11T20:14:30-04[America/New_York]".parse()?;

    let d1 = zdt2.since(&zdt0)?;
    let d2 = zdt2.since(&zdt1)?.checked_add(zdt1.since(&zdt0)?)?;

    assert_eq!(d1.to_string(), "PT1M30S");
    assert_eq!(d2.to_string(), "PT90S");

    Ok(())
}

With this Cargo.toml:

[package]
publish = false
name = "dsnet-duration-rfc"
version = "0.1.0"
edition = "2024"

[dependencies]
anyhow = "1.0.97"
jiff = "0.2.4"

[[bin]]
name = "dsnet-duration-rfc"
path = "main.rs"

[profile.release]
debug = true

The assertions pass when you do cargo run.

Jiff has the same "balance to hours by default" when computing spans between zoned datetimes.

Jiff also has a bespoke duration format that you can opt into. The docs motivate why it exists and they contain a grammar.

[dsnet]

Thanks @BurntSushi, this is really helpful.

Jiff behaves the same as Temporal (quirks and all):

This is further evidence to expand the grammar in the RFC draft to allow overflow of minutes and seconds.

[justingrant]

A potential candidate for a 3rd implementation would be java.time.Duration, which outputs exactly according to the RFC draft for all non-negative duration values. However, it formats negative durations as PT-1H-2M-3S instead of -PT1H2M3S (even though it can parse the latter as well).

Some context: in Temporal, we disallowed durations where units had different signs. Excerpted from #782:

2. All Duration units must share the same sign; intra-duration sign variation is not supported

• 2.1 Intra-duration sign variation (e.g. "2 days and negative 12 hours") seems to have only one major use case: the ability to combine a math operation with construction of a Duration. We explicitly stopped doing this in Remove {disambiguation: 'balance'} in with() and from() of non-Duration types #642, so I don't see any need to support it now. The (easy) workaround is to construct a single-sign duration, and then apply the math operation.

3. The string persistence format for Duration will be extended with an optional leading sign

• 3.1 The string format can optionally include a leading minus or (no-op) plus sign, e.g. -P2D means a negative 2-day duration, while P2D and +P2D both mean a positive 2-day duration.
• 3.2 The leading plus/minus format is used by RFC 5545 and many other libraries and platforms. AFAIK there is no other alternative format used in mainstream platforms.
• 3.3 Duration.prototype.toString will emit the leading negative sign for negative durations, but will NOT emit a leading plus for positive durations, so that users who are using ISO8601-compliant positive duration will get an ISO8601-compliant string persistence format.
• 3.4 Duration.from will accept a leading minus, a leading plus, or no leading sign. Intra-duration (non-leading) plus or minus characters are not supported and must throw when parsed.

When we made these decisions in 2020, I don't think we knew that Java accepted mixed-sign durations, but otherwise I think these 5-year-old decisions still seem like the right call.

Since we made that decision, there are several other reasons found for why mixed-sign durations are a bad idea:

• Mixed-sign units would make it harder (and impossible for durations with date units) to know if a duration is zero-length, because positive and negative units can cancel each other out.
• MIxed-sign units makes it harder (time units) or impossible (date units) to reliably know if a duration overall is positive, negative, or zero. Knowing the sign of a duration is important for many operations, including localization, UI (e.g. which direction to render the arrow), validation (e.g. use cases where negative durations are disallowed, or when a nonzero duration is required), etc.
• Localized formatting of a mixed-sign duration will be very challenging, both to implement and for end users to understand.
• Implementing mixed-sign duration makes pretty much everything harder: parsing, arithmetic, etc. which increases the surface area for bugs.

Given that we weren't supporting mixed-sign duration, forbidding intra-duration-string sign characters seemed to be the best solution. It:

• Prevents bugs where callers forget to apply minus signs to all units
• Makes negative durations shorter when persisted or sent over a network
• Makes parsing simpler, because you only need to look at the first character
• Corresponds more closely to the mental model we want callers to have, which is that the duration as a whole (not its individual components) is either positive, negative, or zero.

Hopefully this context is helpful.

[justingrant]

BTW, here's more context that may be helpful, listing changes in ISO 8601-2:2019:

#819 (comment)

This reminded me that Temporal also does not permit fractional units because of the complexity introduced. Instead users are responsible for dealing with that complexity outside Temporal, so that when creating durations only integers are involved.

[gibson042]

Copying from dsnet/rfc-internet-duration#7 to bring it in to this proposal:

Would the tc39/temporal-proposal be willing to consider making arithmetic operations balance to hours by default to avoid odd behavior like tc39/proposal-temporal#3095 (comment)? In some ways, I would argue that the temporal-proposal would be more consistent across the API since zonedDateTime.until and zonedDateTime.since balance to largestUnit:'hour' by default, but Temporal.Duration.add and Temporal.Duration.subtract do not… This seems like a beneficial change regardless of what the RFC draft ends up being.

I'm a reviewer of that proposal rather than a champion, which at this level of maturity may be irrelevant. But that is an interesting argument, and merits its own issue in my opinion (which if opened before March 20 should be discussed at the next Temporal meeting).

However, there is at least plausible justification for operations balancing to different largest units by default.

If arithmetic balances to hours by default, then I believe the temporal-proposal would therefore be identical to the RFC draft by default. If a user explicitly choses to balance to other units, then that's fine and the explicit choice to do so is a signal that they actually wanted to preserve odd representations (e.g., PT1M90S) for some reason.

That's true, but the unit to which any given arithmetic operation balances may differ from the unit for a different operation, which may be important—especially when dealing with variable-length units. The current state is visible in uses of GetDifferenceSettings and AddDurations:

Type	Default Largest Unit
PlainYearMonth	year
PlainDate	day
PlainDateTime	day
PlainTime	hour
ZonedDateTime	hour
Instant	second
Duration	<largest non-zero unit in any operand>

For example,

Temporal.Duration.from("PT59M59S").add("PT2M2S"); // PT62M1S
const t1 = "2024-01-01T00:00:00+00:00";
const t2 = "2024-01-01T01:02:01+00:00";
const dt1 = Temporal.PlainDateTime.from(t1);
const ts1 = Temporal.Instant.from(t1);
dt1.until(t2); // PT1H2M1S
ts1.until(t2); // PT3721S

If I understand correctly, what you want in ECMAScript Temporal would require changing Instant {since,until} and Duration {add,subtract}, with the latter applying only when neither operand has a non-zero calendar unit and probably also requiring introduction of an options parameter for overriding the default largestUnit, and would provide conformance with the current state of this draft for generated-with-defaults durations that lack a non-zero calendar unit, with the cost of arguably worse ergonomics for e.g. single-unit duration arithmetic like Temporal.Duration.from("PT40M").add("PT30M").

[timbray]

Just want to say that I would hate to see the chosen representation lose the automatically-canonical feature of the current draft. I maintain a very high performance event-filtering library and I ended up writing my own JSON parser which is much faster than Go's, not because I wrote better code but because it is able to operate on the utf-8 bytes of the object names/values rather than bringing in heavyweight apparatus like a general-purpose time/duration library. You can do a lot by just looking at bytes.