Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[redis] support native binary storage #559

Open
1 task done
cjpearson opened this issue Jan 2, 2025 · 1 comment · May be fixed by #561
Open
1 task done

[redis] support native binary storage #559

cjpearson opened this issue Jan 2, 2025 · 1 comment · May be fixed by #561
Labels
enhancement New feature or request

Comments

@cjpearson
Copy link
Contributor

Describe the feature

Currently the redis driver does not implement getItemRaw/setItemRaw. It uses the default method of serializeRawData + setItem.

Since ioredis can binary data, it should be unnecessary to encode it as base64. Directly setting a Buffer would have less overhead and be more space-efficient. However, there would be compatibility issues with existing data. Perhaps it could be implemented as an option or separate driver?

https://github.com/redis/ioredis#handle-binary-data

Here's an example implementation. It would still need to be updated to handle non-Buffer values.

async getItemRaw(key) {
  const value = await getRedisClient().getBuffer(p(key));
  return value ?? null;
},
async setItemRaw(key, value, tOptions) {
  if (value instanceof Uint8Array) {
    value = Buffer.from(value, value.byteOffset, value.byteLength)
  }
  const ttl = tOptions?.ttl ?? opts.ttl;
  if (ttl) {
    await getRedisClient().setBuffer(p(key), value, "EX", ttl);
  } else {
    await getRedisClient().setBuffer(p(key), value);
  }
},

This could perhaps be done with #528 and I think that would avoid the breaking aspects. For example setItemRaw could continue to base64 encode and save as a string, but setItem(k, v, { type: 'bytes'}), could save as binary.

Additional information

  • Would you be willing to help implement this feature?
@cjpearson cjpearson added the enhancement New feature or request label Jan 2, 2025
@pi0 pi0 changed the title [redis] Unnecessary base64 encoding for binary data [redis] support native binary storage Jan 2, 2025
@pi0
Copy link
Member

pi0 commented Jan 2, 2025

Feel free to make a PR 👍🏼

cjpearson added a commit to cjpearson/unstorage that referenced this issue Jan 2, 2025
@cjpearson cjpearson linked a pull request Jan 2, 2025 that will close this issue
cjpearson added a commit to cjpearson/unstorage that referenced this issue Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants