Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modeling single-partition data #323

Closed
thorhj opened this issue Nov 7, 2023 · 2 comments · Fixed by #325
Closed

Modeling single-partition data #323

thorhj opened this issue Nov 7, 2023 · 2 comments · Fixed by #325

Comments

@thorhj
Copy link

thorhj commented Nov 7, 2023

My table is modelling a schedule linking a user to a specific date. I have the following access patterns:

  1. Get all schedule items for one specific date
  2. Get all schedule items since a specific date, or schedules for a given month/year

Since I am never looking to fetch a singular schedule, I am simply adding all the schedule items to the same partition using "schedule" as the partition key and composing the sort key by date and user ID (only there to ensure uniqueness). For example:

PK SK
schedule date_2023-11-07#userid_42

I can model this with the following entity:

const Schedule = new Entity(
  {
    model: {
      entity: "schedule",
      service: "my-service",
      version: "1",
    },
    attributes: {
      userId: {
        field: "user_id",
        type: "string",
      },
      date: {
        field: "date",
        type: "string",
        required: true,
      },
    },
    indexes: {
      byTemplate: {
        pk: {
          composite: [],
          field: "pk",
          template: "schedule",
        },
        sk: {
          composite: ["date", "userId"],
          field: "sk",
        },
      },
    },
  }
);

Notice I am composing the pk from nothing and using a template to set a fixed value instead. This means I can query the data like so:

// Get schedules for a single date:
await Schedule.query.byTemplate({ date: "2023-11-07" }).go();

// Get schedules for 2023:
await Schedule.query.byTemplate({}).begins({ date: "2023" }).go();

// Get schedules since August 2023:
await Schedule.query.byTemplate({}).gte({ date: "2023-08" }).go();

This approach has a few downsides though:

  1. I have to use template for the partition key, meaning that I have to manually manage namespacing (which I haven't done in my example).
  2. If I want to query without a sort key I still have to pass in an empty object in Schedule.query.byTemplate({}).begins({ date: "2023" }). This is of course a small thing, but it would be nicer to just do Schedule.query.byTemplate().begins(...) when the partition key for the entity is "fixed".

If this is indeed the best way to model a single-partition entity (please let me know if there is a better way), then I would like to suggest a more convenient definition to the key schema type for fixed-partition entities:

indexes: {
  byTemplate: {
    pk: {
      value: "schedule",
      field: "pk"
    },
    sk: {
      composite: ["date", "userId"]
      field: "sk"
    }
  }
}

Here I am using value to say I want the value in the partition key to always be "schedule". You shouldn't be able to use value and composite together of course.

With this key definition the library would then be able to set a fixed partition key for the entity while still managing the namespacing. The typing could potentially be adjusted to allow undefined for the query key object (but this is potentially difficult to implement compared to the benefit).

What do you think?

@sam3d
Copy link

sam3d commented Nov 12, 2023

This sounds like #290

@tywalch tywalch linked a pull request Nov 12, 2023 that will close this issue
@tywalch
Copy link
Owner

tywalch commented Nov 12, 2023

@sam3d is correct here, this is the same need she described in her RFC. I added a PR this afternoon that will address this, and I'm hoping to merge it as soon as tomorrow morning 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants