Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: file chunking due to mix of nodeSize and chunkSize limitation #176

Merged
merged 1 commit into from
Nov 18, 2024

Conversation

clostao
Copy link
Member

@clostao clostao commented Nov 18, 2024

User description

It was setup wrongly when to use maxChunkSize and maxNodeSize this led to files not processing at all in auto-drive


PR Type

Bug fix, Tests


Description

  • Fixed incorrect usage of maxChunkSize and maxNodeSize by replacing DEFAULT_MAX_CHUNK_SIZE with DEFAULT_NODE_MAX_SIZE across the codebase.
  • Updated function parameters and logic to ensure correct handling of node and chunk sizes.
  • Adjusted tests to reflect changes in default node size handling.

Changes walkthrough 📝

Relevant files
Bug fix
chunker.ts
Correct default node size and chunk size handling               

packages/auto-dag-data/src/ipld/chunker.ts

  • Changed default max node size from DEFAULT_MAX_CHUNK_SIZE to
    DEFAULT_NODE_MAX_SIZE.
  • Updated function parameters to use DEFAULT_NODE_MAX_SIZE.
  • Adjusted logic to handle maxChunkSize separately.
  • +16/-14 
    nodes.ts
    Use consistent node size default in node creation               

    packages/auto-dag-data/src/ipld/nodes.ts

  • Replaced DEFAULT_MAX_CHUNK_SIZE with DEFAULT_NODE_MAX_SIZE for node
    size parameters.
  • Ensured consistent use of DEFAULT_NODE_MAX_SIZE across functions.
  • +11/-11 
    Tests
    nodes.spec.ts
    Update tests for new default node size                                     

    packages/auto-dag-data/tests/nodes.spec.ts

  • Updated test to use DEFAULT_NODE_MAX_SIZE instead of
    DEFAULT_MAX_CHUNK_SIZE.
  • Ensured test consistency with new default node size.
  • +2/-2     

    💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

    Copy link

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🧪 PR contains tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Consistency Check
    Ensure that the new default constants for node sizes are consistent and correctly replace the old chunk size constants throughout the codebase.

    Dependency Update
    Verify that the updated import statements and the use of new constants are correctly implemented without breaking existing functionalities.

    Copy link

    PR Code Suggestions ✨

    No code suggestions found for the PR.

    @clostao clostao merged commit 748b2aa into main Nov 18, 2024
    2 checks passed
    @clostao clostao deleted the fix/chunker-issue branch November 18, 2024 21:16
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants