Bech32 length limitations

[msparks]

Hey, I ran into this limitation while working on an age plugin. Currently, age rejects plugin identities that are 'too long', i.e., more than 43 bytes of data. That's technically correct per the draft plugin specification, but in practice it creates complexity for plugins, so it seems worth a discussion, especially since it may mean changes to the spec.

Background

The Bech32 encoding, as defined in BIP-0173, produces strings that are at most 90 characters long, composed of 1–83 characters for HRP, 1 character for the separator ("1"), and at least 6 characters for the data and checksum.

The draft age plugin spec requires Bech32 encoding for recipients and identities:

• Plugin-compatible recipients are encoded using Bech32 with the HRP age1name (lowercase).
• Plugin-compatible identities are encoded using Bech32 with the HRP AGE-PLUGIN-NAME- (uppercase).

And although Bech32 implementations vary in terms of HRP and data length enforcement, a strict reading and implementation of the age plugin specification would indeed be extremely limiting: age plugin identities can have only 43 bytes of data in the best case.

To illustrate:

• HRP: AGE-PLUGIN-X-: 13 characters (at least)
• Separator: 1: 1 character
• Checksum: 6 characters
• Data: (90 - 13 - 1 - 6) * (5 bits / 8 bits for encoding overhead) = 43 bytes for raw data.

Problem statement

The Bech32 encoding is too restrictive for plugin recipients and identities. While some plugins can be implemented within the limits, such as age-plugin-yubikey, others using longer keys and/or storing metadata in identities cannot.

My use case is a wrap plugin that encrypts and decrypts a native age X25519 identity transparently, using age itself and potentially other plugins. The goal is to get an X25519 recipient, which can be used without plugins, but with an identity that is protected by a hardware device, for example. Thus, plugins are required for decryption, but not encryption. How the identity is wrapped is arbitrary and can include interesting operations like splitting the identity or adding a timelock. It's possible to do this now with multiple age invocations (decrypt the identity first, then decrypt the message with the decrypted identity), but a plugin would make it transparent and therefore a better user experience. Specifically, the recipient is native X25519 age1... and the identity is AGE-PLUGIN-WRAP-1.... The problem is that a wrapped identity easily can be hundreds of bytes, so implementing this plugin is currently infeasible.

Other plugins will likely run into the same problem. One example is Kʏʙᴇʀ, discussed in #231. @FiloSottile experimented with a Kʏʙᴇʀ768+X25519 plugin and tweeted an example recipient, which is very long: https://twitter.com/FiloSottile/status/1544803635237998592.

Existing implementations

As mentioned above, implementations vary:

• age uses a slightly modified version of the reference Go implementation of the Bech32 specification. Both implementations are strict and enforce the 90-character limit on encode and decode.
• rage uses the bech32 crate. It enforces the length on the HRP but not the data.
  • Note: The bech32 crate is not the reference implementation. Interestingly, the reference implementation in Rust has a length limitation only on decode. The same is true for the reference Python implementation and others (but not Go!).

Potential solutions

The simplest approach here is to specify the maximum acceptable length of Bech32-encoded data, or remove the limitation explicitly, in the age plugin spec. The length limitation in Bech32 exists to make guarantees on the error detection of the checksum (see the end of the Checksum design section in BIP-0173), but age identities and recipients perhaps have weaker requirements for the checksum strength than the motivating use case for BIP-0173, so allowing longer data lengths is reasonable.

Note that this approach would mean that the encoding isn't strict Bech32, so plugin authors would be unable to use strict Bech32 libraries like the reference Go implementation. That'll be confusing, but likely minor in practice.

Some possible alternatives:

• Remove the length limitation for identities only, with the assumption that identities are less prone to errors than recipients. This would be insufficient for the Kʏʙᴇʀ768+X25519 use case.
• Use side channels in plugins (e.g., environment variables, files) for longer identities, and keep only a stub or index in the identity file that age reads. Mentioned only for completeness; this option goes against age's simplicity philosophy.
• Support a different encoding scheme for identities. Mentioned for completeness.

[FiloSottile]

Thank you for the detailed investigation, and for addressing the potential issues @msparks, I really appreciate it!

I think I agree, such long recipients/identities are going to be copy-pasted, so I am less concerned about the error-correction effectiveness. I'll drop the length restriction before v1.1.0 final. @str4d, do you agree? You might want to add a test to lock this in.

[str4d]

Yes, I'm fine with this. Bech32 has been used almost since its inception with a longer data length limit: the Lightning Network raised the limit to 1024 bytes (the next cliff in Bech32) to accommodate their invoice data structures, and this limit (or lack thereof) has propagated elsewhere. I expect 1024 bytes to likely be sufficient here, but I also wouldn't object to just removing the limit.

[michaelforney]

Perhaps it should be mentioned in the specification somewhere that age uses a relaxed version of Bech32 without length limitations? I had the same concerns as @msparks, since there isn't any mention of the length restriction in https://c2sp.org/age or https://c2sp.org/age-plugin.

As is, it states that recipients and identities are encoded with Bech32 with a link to BIP-0173, which defines

A Bech32 string is at most 90 characters long.

It was only after some digging that I found this discussion and learned that larger recipients and identities are acceptable.

My worry is for potential incompatibilities between clients written to the spec and plugins requiring large strings, or that potential plugin authors might deem their scheme incompatible with the spec and be deterred.