Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ulid): port /x/ulid module #3582

Merged
merged 30 commits into from
Sep 12, 2023
Merged

feat(ulid): port /x/ulid module #3582

merged 30 commits into from
Sep 12, 2023

Conversation

lino-levan
Copy link
Contributor

Closes #3522.

I think this is valuable, particularly for Deno KV.

Copy link
Contributor

@iuioiua iuioiua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably only want to export the following from mod.ts:

  • decodeTime()
  • factory()
  • monotonicFactory()
  • PRNG
  • ULID
    I think the others should be moved to a _util.ts or similar file.

@iuioiua
Copy link
Contributor

iuioiua commented Aug 26, 2023

We also probably want to include the examples from the README in the JSDocs.

ulid/test.ts Outdated Show resolved Hide resolved
Copy link
Contributor

@iuioiua iuioiua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

ulid/mod.ts Outdated Show resolved Hide resolved
ulid/mod.ts Outdated Show resolved Hide resolved
Copy link
Contributor

@iuioiua iuioiua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM again

@kt3k
Copy link
Member

kt3k commented Aug 29, 2023

@iuioiua Do you know the usage of monotonicFactory? I'm not sure what it's used for.

I'm also unsure we need to export factory. I guess nobody would replace pseudo random number generator.

I'd suggest we should only expose ulid and decodeTime functions (and related types) at least for first pass. What do you think?

@iuioiua
Copy link
Contributor

iuioiua commented Aug 29, 2023

I think monotonicFactory() is worth keeping. It's used for generating monotonically increasing ULIDs, which would be valuable in guaranteeing no conflicts when high numbers of ULIDs are generated within the same millisecond.

I agree with the rest of your points.

@kt3k
Copy link
Member

kt3k commented Aug 29, 2023

Ok. Monotonicity looks useful. (This doc also mentions about guaranteeing of generation order).

While monotonicity is useful, monotonicFactory looks still redundant as it is for specifying alternative random number generator. I'd suggest we should replace it with non-factory version (such as ulidMonotonic or maybe generateMonotonic)

@iuioiua
Copy link
Contributor

iuioiua commented Aug 30, 2023

Yeah, it seems a little funny. It only increments monotonically if the last timestamp is less than or equal to the current timestamp (now). I think it's fine how it is with the name monotonicFactory. However, that's not a strong opinion.

Half-related: I think deviating from the ULID's JavaScript implementation is fine. Otherwise, we'd be putting ourselves into a box regarding capability. E.g. I found a possible tweak that could improve the performance of ULID generation by 10x while maintaining cryptographically random strings. Though, I'd have to validate it's correctness first 🤣

@iuioiua
Copy link
Contributor

iuioiua commented Sep 1, 2023

WDYT, @kt3k?

@lino-levan
Copy link
Contributor Author

I would almost flip the naming to have ulid be monotonically increasing and some sort of non-monotonic ulid exist on the side? It feels like outside of migration, everyone should really use monotonic ulids.

@kt3k
Copy link
Member

kt3k commented Sep 1, 2023

With monotonic version, the random bits are not random anymore, and it seems having far higher possibility of collision in distributed usage. I think monotonic version is not something we should recommend by default.

@lino-levan
Copy link
Contributor Author

With monotonic version, the random bits are not random anymore

I missed that. This makes sense then. Completely agree that no one will replace the PRNG function. I'll rework the PR tonight if I can.

@iuioiua
Copy link
Contributor

iuioiua commented Sep 1, 2023

Oh, I completely missed that you suggested removing the random number generator parameter. Yes, that parameter looks mostly useless to me.

ulid/_util.ts Outdated Show resolved Hide resolved
@lino-levan
Copy link
Contributor Author

Return of the flaky tests I guess. This is ready for review @iuioiua.

Copy link
Contributor

@iuioiua iuioiua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool to see that change implemented! Few things:

  • I don't think monotonicFactory() should have an encodeRand parameter or any parameter.
  • This implementation deviates from the original JS implementation (e.g. faster encodeRandom() and no prng parameter. I think these differences should be pointed out in the JSDocs somewhere.

ulid/_util.ts Outdated Show resolved Hide resolved
ulid/_util.ts Outdated Show resolved Hide resolved
ulid/_util.ts Outdated Show resolved Hide resolved
ulid/_util.ts Show resolved Hide resolved
ulid/_util.ts Outdated Show resolved Hide resolved
Co-authored-by: Asher Gomez <ashersaupingomez@gmail.com>
@lino-levan
Copy link
Contributor Author

lino-levan commented Sep 3, 2023

I don't think monotonicFactory() should have an encodeRand parameter or any parameter.

Strongly disagree. It is much harder to test without this parameter. It is not exposed to end users so I don't think this matters.

This implementation deviates from the original JS implementation (e.g. faster encodeRandom() and no prng parameter. I think these differences should be pointed out in the JSDocs somewhere.

Again, not exposed to the user so I don't know if that's relevant.

lino-levan and others added 3 commits September 4, 2023 09:52
Co-authored-by: Asher Gomez <ashersaupingomez@gmail.com>
Co-authored-by: Asher Gomez <ashersaupingomez@gmail.com>
ulid/_util.ts Show resolved Hide resolved
Copy link
Contributor

@iuioiua iuioiua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless of my previous unreachable line comment, LGTM! This implementation is actually better than the original JavaScript implementation because:

  1. Random strings are generated 10x faster.
  2. The redundant prng parameter is removed. Fewer options are better.
  3. Monontic ULIDs are possible without having first to create the factory.

Good stuff, @lino-levan and @kt3k!

* monotonicUlid(100000); // 000XAL6S41ACTAV9WEVGEMMVRD
* ```
*/
export const monotonicUlid = monotonicFactory();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I totally misread your initial suggestion, @kt3k. Now, I understand, and I like it 😎

@kt3k kt3k changed the title feat(ulid): port kt3k's ulid module feat(ulid): port /x/ulid module Sep 4, 2023
@iuioiua
Copy link
Contributor

iuioiua commented Sep 5, 2023

Anything else pending for this PR?

@lino-levan
Copy link
Contributor Author

Nothing pending afaik. Waiting for any comments from @kt3k.

@iuioiua
Copy link
Contributor

iuioiua commented Sep 6, 2023

I've noticed a bug with monotonicUlid() in denoland/saaskit#514. For some reason, ULIDs aren't respecting their seedTime parameters. However, ulid() works fine.

@lino-levan
Copy link
Contributor Author

Could you write a reproduction? I suspect it may be a feature but I'm not too sure. @kt3k may know more about it.

@iuioiua
Copy link
Contributor

iuioiua commented Sep 6, 2023

Yeah, I'll get a repro snippet when I can.

@lino-levan
Copy link
Contributor Author

Alright I've been staring at this code for too long. I don't see this bug. I'd appreciate a fresh set of eyes.

@iuioiua
Copy link
Contributor

iuioiua commented Sep 10, 2023

Sorry, I haven't had the chance to look into this further. @kt3k, would std/ulid be considered an unstable module once merged? If so, I think it'd be worth merging as-is.

Copy link
Member

@kt3k kt3k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you for your efforts! @lino-levan

Note: Core team also looked in favor of including this.

@kt3k
Copy link
Member

kt3k commented Sep 12, 2023

would std/ulid be considered an unstable module once merged?

Yes, we are planning to keep ulid unstable (versioned as 0.x) for a while.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

proposal: std/ulid
3 participants