Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: enable allowH2 by default and require Node.js >= 18.20.0 #734

Merged
merged 7 commits into from
Nov 30, 2024

Conversation

fengmk2
Copy link
Member

@fengmk2 fengmk2 commented Nov 29, 2024

Summary by CodeRabbit

Release Notes

  • New Features

    • Enhanced error handling across various services by introducing a centralized timeout error checking function.
    • HTTP/2 support enabled in the HTTP client configuration.
  • Bug Fixes

    • Corrected a typographical error in comments for better clarity.
  • Documentation

    • Updated Node.js version requirements in the project configuration.
  • Tests

    • Improved test cases for NpmChangesStream and TaskRepository to ensure accurate behavior and performance.
  • Chores

    • Updated Node.js version in CI workflow for more precise testing.

Copy link
Contributor

coderabbitai bot commented Nov 29, 2024

Walkthrough

The changes in this pull request include minor corrections in comments, updates to error handling mechanisms, configuration adjustments, and modifications to testing strategies across various files. Key updates involve increasing timeout values for HTTP requests, introducing a utility function for timeout error checks, and reorganizing import statements for clarity. Additionally, the Node.js version requirements have been updated, and new tests have been added to enhance coverage. Overall, the modifications focus on improving code maintainability and error handling without altering core functionalities.

Changes

File Path Change Summary
app/common/adapter/binary/AbstractBinary.ts Corrected a comment in listNodeABIVersions method.
app/common/adapter/changesStream/NpmChangesStream.ts Increased timeout in fetchChanges method from 10,000 ms to 60,000 ms.
app/core/service/ChangesStreamService.ts Updated error handling in executeTask method using isTimeoutError utility; reorganized import statements.
config/config.default.ts Added allowH2: true to config.httpclient.
test/common/adapter/changesStream/NpmChangesStream.test.ts Modified test for fetchChanges method to call without awaiting; added a skipped performance test case.
test/repository/TaskRepository.test.ts Increased timeout in setTimeout from 1 ms to 100 ms; modified assertion for updatedAt property.
.github/workflows/nodejs.yml Updated Node.js version in CI matrix from [18, 20, 22] to [18.20.0, 18, 20, 22].
app/common/ErrorUtil.ts Introduced TimeoutErrorNames array and isTimeoutError function for improved timeout error handling.
app/core/service/BinarySyncerService.ts Updated error handling in executeTask using isTimeoutError; reorganized imports.
app/port/schedule/SyncBinaryWorker.ts Modified error handling in subscribe method to use isTimeoutError.
package.json Updated minimum Node.js version from >= 16.13.0 to >= 18.20.0.
app/common/adapter/NPMRegistry.ts Updated error handling in getFullManifests method to utilize isTimeoutError.

Suggested labels

enhancement

🐇 In the code, we hop and play,
Fixing comments, come what may.
With timeout checks and tests anew,
Our functions shine, all bright and true!
So let’s celebrate this code delight,
With every change, our future's bright! 🌟


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between d0da939 and f9a3b36.

📒 Files selected for processing (1)
  • app/common/ErrorUtil.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/common/ErrorUtil.ts

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

codecov bot commented Nov 29, 2024

Codecov Report

Attention: Patch coverage is 73.21429% with 15 lines in your changes missing coverage. Please review.

Project coverage is 96.39%. Comparing base (b808ebc) to head (f9a3b36).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
app/common/ErrorUtil.ts 55.55% 12 Missing ⚠️
app/common/adapter/NPMRegistry.ts 75.00% 2 Missing ⚠️
app/port/schedule/SyncBinaryWorker.ts 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #734      +/-   ##
==========================================
- Coverage   96.50%   96.39%   -0.12%     
==========================================
  Files         188      189       +1     
  Lines       18880    18912      +32     
  Branches     2481     2480       -1     
==========================================
+ Hits        18221    18231      +10     
- Misses        659      681      +22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@fengmk2 fengmk2 changed the title feat: use urllib beta feat: enable allowH2 by default Nov 30, 2024
@fengmk2 fengmk2 marked this pull request as ready for review November 30, 2024 06:40
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (2)
config/config.default.ts (1)

189-191: Consider adding documentation for HTTP/2 configuration.

Since HTTP/2 support is now enabled by default, it would be helpful to add a comment explaining:

  • The benefits of HTTP/2
  • Any requirements or considerations for upstream servers
  • Any relevant configuration options

Here's a suggested documentation addition:

 config.httpclient = {
   useHttpClientNext: true,
+  // Enable HTTP/2 for improved performance through multiplexing and header compression
+  // Note: Requires upstream servers to support HTTP/2
   allowH2: true,
 };
app/core/service/ChangesStreamService.ts (1)

107-109: Enhance the comment to be more descriptive.

The error handling changes look good and properly handle HTTP/2 timeout scenarios. Consider making the comment more descriptive to explain why we're checking for these specific timeout cases.

-        // InformationalError: HTTP/2: "stream timeout after 60000"
+        // Handle various timeout scenarios:
+        // - BodyTimeoutError: Response body read timeout
+        // - HTTP/2 stream timeout: "stream timeout after 60000"
+        // These are treated as warnings rather than errors as they're typically transient
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between b808ebc and dab340c.

📒 Files selected for processing (5)
  • app/common/adapter/binary/AbstractBinary.ts (1 hunks)
  • app/common/adapter/changesStream/NpmChangesStream.ts (1 hunks)
  • app/core/service/ChangesStreamService.ts (1 hunks)
  • config/config.default.ts (1 hunks)
  • test/common/adapter/changesStream/NpmChangesStream.test.ts (3 hunks)
✅ Files skipped from review due to trivial changes (1)
  • app/common/adapter/binary/AbstractBinary.ts
🔇 Additional comments (4)
app/common/adapter/changesStream/NpmChangesStream.ts (1)

31-31: Consider documenting the rationale for the timeout increase

The timeout has been significantly increased from 10s to 60s. While this may help with slow responses, such a large increase could impact system resources if many requests are pending.

Let's check if there are related timeout issues:

Consider adding a comment explaining:

  • Why 60s was chosen as the timeout value
  • Any performance implications
  • Whether this was based on observed timeouts in production
✅ Verification successful

Let me try another search to find more context about the timeout changes:


Let me check the related files for timeout configurations:


The timeout increase is justified for changes stream stability

The timeout increase from 10s to 60s in NpmChangesStream is appropriate because:

  • The changes stream is a long-running HTTP connection that monitors package updates
  • The codebase shows error handling for various timeout scenarios including "stream timeout after 60000"
  • Other similar changes stream implementations (CnpmcoreChangesStream, CnpmjsorgChangesStream) use 30s timeout
  • The service has retry mechanisms and timeout handling for failed requests
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Search for timeout-related issues or discussions
# Look for timeout-related error logs or discussions
rg -i "timeout.*npm.*changes" -g "!*.{log,txt}"

Length of output: 131238


Script:

#!/bin/bash
# Search for timeout-related code and configuration in the NpmChangesStream class
ast-grep --pattern 'class NpmChangesStream {
  $$$
  timeout
  $$$
}'

# Also search for timeout configuration in related files
rg -l "timeout.*=.*\d+" --type ts

Length of output: 186


Script:

#!/bin/bash
# Check timeout configurations in related files
rg "timeout.*=.*\d+" --type ts -A 3 -B 3

# Also look for any comments or documentation about timeouts
rg -i "timeout" --type ts -g "!*.{d.ts,test.ts}" -A 2 -B 2

Length of output: 22092

test/common/adapter/changesStream/NpmChangesStream.test.ts (2)

62-65: LGTM: Stream handling follows best practices

The stream consumption using for-await-of is the correct approach for handling async iterables.


Line range hint 77-85: LGTM: Good test for chunk handling

The test effectively simulates and validates the handling of fragmented JSON data in the stream.

config/config.default.ts (1)

190-190: LGTM! Verify HTTP/2 support in dependencies.

The addition of allowH2: true enables HTTP/2 support which can improve performance through multiplexing and header compression.

Let's verify the HTTP client dependencies support HTTP/2 securely:

✅ Verification successful

HTTP/2 support is properly configured with secure defaults

The codebase shows that HTTP/2 is enabled alongside useHttpClientNext: true, which is the recommended configuration for secure HTTP/2 usage in Egg.js. The HTTP client being used is from the egg framework (v3.9.2), which has built-in support for HTTP/2 with secure defaults.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check package.json for HTTP client dependencies and their versions
fd -t f "package.json" -x cat {} | jq -r '.dependencies | with_entries(select(.key | contains("http")))'

# Look for any HTTP/2 related configurations or imports
rg -i "http.?2" --type ts

Length of output: 718


Script:

#!/bin/bash
# Let's check for HTTP client related imports and configurations
rg -i "httpclient|http.?agent" --type ts -A 2

# Also check for any egg-related dependencies that might handle HTTP
fd -t f "package.json" -x cat {} | jq -r '.dependencies | with_entries(select(.key | contains("egg")))'

Length of output: 137303


Script:

#!/bin/bash
# Let's check for HTTP/2 related code and configuration
rg -i "useHttpClientNext|http2|allowH2" --type ts -A 2

# Check package.json for HTTP client dependencies
fd -t f "package.json" -x cat {} | jq '.dependencies,.devDependencies'

Length of output: 2420

@fengmk2 fengmk2 changed the title feat: enable allowH2 by default feat: enable allowH2 by default and require Node.js >= 18 Nov 30, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Outside diff range and nitpick comments (5)
app/common/ErrorUtil.ts (2)

1-7: Add type safety to TimeoutErrorNames constant.

Consider adding type safety and making the array immutable to prevent accidental modifications.

-const TimeoutErrorNames = [
+const TimeoutErrorNames = [
   'HttpClientRequestTimeoutError',
   'HttpClientConnectTimeoutError',
   'ConnectionError',
   'ConnectTimeoutError',
   'BodyTimeoutError',
-];
+] as const;
+type TimeoutErrorName = typeof TimeoutErrorNames[number];

9-26: Add JSDoc documentation and improve type safety.

The function would benefit from comprehensive documentation and better type handling.

+/**
+ * Checks if an error is a timeout-related error by examining its name,
+ * aggregate errors, and error cause chain.
+ * @param err - The error to check
+ * @returns true if the error is timeout-related, false otherwise
+ */
-export function isTimeoutError(err: Error) {
+export function isTimeoutError(err: Error & { cause?: unknown }) {
app/port/schedule/SyncBinaryWorker.ts (1)

Line range hint 39-43: Consider including error stack trace for non-timeout errors.

While the error handling logic is good, for non-timeout errors, including the stack trace would be valuable for debugging.

Consider this enhancement:

       if (isTimeoutError(err)) {
         this.logger.warn(err);
       } else {
-        this.logger.error(err);
+        this.logger.error(err.stack || err);
       }
.github/workflows/nodejs.yml (1)

35-35: Consider optimizing the Node.js version matrix

The addition of 18.20.0 alongside 18 in the test matrix appears redundant since:

  1. Node.js 18 already covers version 18.20.0
  2. Running tests against both versions increases CI time without significant benefits

Consider simplifying the matrix to include only the major versions:

-        node-version: [18.20.0, 18, 20, 22]
+        node-version: [18, 20, 22]

Additionally, since the PR aims to require Node.js >= 18, the current matrix correctly enforces this requirement by not including versions below 18.

Also applies to: 86-86

app/core/service/BinarySyncerService.ts (1)

140-143: Consider structuring error handling for better maintainability

While the current changes improve error handling by adding error names and using appropriate logging levels, consider refactoring to handle different error types more systematically.

Consider this structure for better maintainability:

- task.error = `${err.name}: ${err.message}`;
- logs.push(`[${isoNow()}][${dir}] ❌ Synced "${binaryName}" fail, ${task.error}, log: ${logUrl}`);
- logs.push(`[${isoNow()}][${dir}] ❌❌❌❌❌ "${binaryName}" ❌❌❌❌❌`);
- if (isTimeoutError(err)) {
-   this.logger.warn('[BinarySyncerService.executeTask:fail] taskId: %s, targetName: %s, %s',
-     task.taskId, task.targetName, task.error);
-   this.logger.warn(err);
- } else {
-   this.logger.error('[BinarySyncerService.executeTask:fail] taskId: %s, targetName: %s, %s',
-     task.taskId, task.targetName, task.error);
-   this.logger.error(err);
- }
+ const errorDetails = {
+   message: `${err.name}: ${err.message}`,
+   logLevel: isTimeoutError(err) ? 'warn' : 'error',
+ };
+ task.error = errorDetails.message;
+ logs.push(`[${isoNow()}][${dir}] ❌ Synced "${binaryName}" fail, ${task.error}, log: ${logUrl}`);
+ logs.push(`[${isoNow()}][${dir}] ❌❌❌❌❌ "${binaryName}" ❌❌❌❌❌`);
+ this.logger[errorDetails.logLevel]('[BinarySyncerService.executeTask:fail] taskId: %s, targetName: %s, %s',
+   task.taskId, task.targetName, task.error);
+ this.logger[errorDetails.logLevel](err);

This refactoring:

  1. Consolidates error handling logic
  2. Reduces code duplication
  3. Makes it easier to add new error types in the future
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between c346f90 and 433e692.

📒 Files selected for processing (7)
  • .github/workflows/nodejs.yml (2 hunks)
  • app/common/ErrorUtil.ts (1 hunks)
  • app/core/service/BinarySyncerService.ts (2 hunks)
  • app/core/service/ChangesStreamService.ts (2 hunks)
  • app/port/schedule/SyncBinaryWorker.ts (2 hunks)
  • package.json (1 hunks)
  • tsconfig.json (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/core/service/ChangesStreamService.ts
🔇 Additional comments (7)
tsconfig.json (1)

5-5: LGTM! Verify Node.js version requirements.

The update to ES2022 target is appropriate as it aligns with the requirement of Node.js >= 18, which fully supports ES2022 features.

Let's verify the Node.js version requirements are properly set:

✅ Verification successful

Node.js version requirements are properly configured

The Node.js version requirements are correctly set up across the codebase:

  • package.json specifies "node": ">= 18.20.0"
  • CI workflows test against Node.js versions [18.20.0, 18, 20, 22]
  • This aligns perfectly with the ES2022 target in tsconfig.json
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check if Node.js version requirement is properly set in package.json
# and if any CI configurations need updating

# Check package.json engines field
echo "Checking package.json engines field:"
jq '.engines' package.json

# Check CI configuration files for Node.js version
echo -e "\nChecking CI configuration files:"
fd -e yml -e yaml . .github/workflows/ -x cat

Length of output: 6498

app/port/schedule/SyncBinaryWorker.ts (2)

5-5: LGTM! Good practice to centralize error handling.

The addition of the isTimeoutError utility import supports better error handling centralization.


39-39: Verify the implementation of isTimeoutError utility.

Let's ensure the isTimeoutError utility correctly handles all possible timeout scenarios.

✅ Verification successful

Implementation of isTimeoutError utility is robust and handles multiple timeout scenarios

The utility correctly handles:

  • Direct timeout errors through name matching against a comprehensive list: HttpClientRequestTimeoutError, HttpClientConnectTimeoutError, ConnectTimeoutError, BodyTimeoutError
  • Nested timeout errors within AggregateError
  • The implementation is used consistently across multiple services (BinarySyncerService, ChangesStreamService, SyncBinaryWorker)
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the implementation of isTimeoutError utility
# and the timeout error types it handles

# Check the implementation
echo "Checking isTimeoutError implementation:"
rg -A 10 "export function isTimeoutError"

# Look for all timeout-related error types in the codebase
echo -e "\nChecking all timeout-related error definitions:"
rg -i "timeout.*error" --type ts

Length of output: 2603

.github/workflows/nodejs.yml (1)

35-35: Verify Node.js version requirements are consistently defined

Since this PR aims to require Node.js >= 18, let's ensure this requirement is properly documented and enforced.

✅ Verification successful

Based on the shell script results, I can now generate the final response:

Node.js version requirements are consistently defined

The package.json correctly specifies ">= 18.20.0" as the Node.js engine requirement, which aligns with the Node.js versions being tested in the CI workflow (18.20.0, 18.x, 20.x, and 22.x). No additional Node.js version checks were found in the application code that would conflict with this requirement.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check Node.js version requirements across the codebase

# Check engine requirements in package.json
echo "Checking package.json for Node.js version requirements:"
jq '.engines?.node' package.json

# Look for any Node.js version checks in the code
echo -e "\nChecking for Node.js version checks in code:"
rg -i "node.*version|engines.*node" --type js --type ts --type json

Length of output: 110812

package.json (2)

149-149: Verify dependency compatibility with Node.js 18

The upgrade to Node.js 18 is a breaking change that requires careful verification of dependency compatibility.

Consider the following recommendations:

  1. Update the changelog to document this breaking change
  2. Consider bumping the major version number as per semver
  3. Update CI/CD pipelines to use Node.js 18
✅ Verification successful

Let me try a different approach to verify the dependency compatibility.


Based on the dependencies list and their versions, I can now generate a final response:

Node.js 18 upgrade appears safe to proceed

  • All major dependencies are using recent versions that are compatible with Node.js 18
  • Key dependencies like egg (^3.9.2), mysql2 (^3.9.4), and typescript (5.2.2) are on versions known to work well with Node.js 18
  • No dependencies explicitly specify Node.js engine requirements that would conflict with Node.js 18

However, since this is a breaking change:

  • Update CHANGELOG.md to document the Node.js version requirement change
  • Update CI configurations to use Node.js 18 (no current CI configs found in standard locations)
  • Consider bumping the major version number as per semver
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check dependencies for Node.js version requirements and potential conflicts

# Search for engines field in package-lock.json if it exists
echo "Checking package-lock.json for engine requirements..."
fd -t f "package-lock.json" -x jq -r '.packages | .[] | select(.engines != null) | "\(.name): \(.engines | tostring)"' {} \;

# Check if any dependencies explicitly require Node.js < 18
echo "Checking dependencies in node_modules..."
fd -t f "package.json" -d 2 node_modules -x jq -r 'select(.engines != null) | select(.engines.node != null) | "\(.name): \(.engines.node)"' {} \;

Length of output: 484


Script:

#!/bin/bash
# Let's check the package.json directly for dependencies and their versions
echo "Analyzing package.json dependencies..."
jq -r '.dependencies, .devDependencies | to_entries[] | .key + ": " + .value' package.json

# Let's also check if there are any Node.js version requirements in README or documentation
echo -e "\nChecking documentation for Node.js version mentions..."
rg -i "node.*?version|requires node" --type md

# Let's look for any CI configuration files that might need updates
echo -e "\nChecking CI configuration files..."
fd -e yml -e yaml -x cat {} \; | rg -i "node.*version|node:.*"

Length of output: 2025


149-149: Verify HTTP/2 related changes

The PR title mentions "enable allowH2 by default" but there's no direct configuration in package.json.

✅ Verification successful

Let me check the content of the files that contain HTTP/2 related configurations.


Based on the files examined, I can now generate the final response:

HTTP/2 configuration is properly implemented

The configuration for HTTP/2 is correctly set in config/config.default.ts with:

config.httpclient = {
  useHttpClientNext: true,
  allowH2: true,
};
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Look for HTTP/2 related configurations and changes

# Search for HTTP/2 related configurations
echo "Searching for HTTP/2 related configurations..."
rg -l "allowH2|http2|HTTP2|h2" --type ts

# Search for HTTP client configurations
echo "Searching for HTTP client setup..."
ast-grep --pattern 'new HttpClient($$$)' 

Length of output: 556


Script:

#!/bin/bash
# Check content of files with HTTP/2 configurations
echo "Content of config/config.default.ts:"
cat config/config.default.ts

echo -e "\nContent of test/port/controller/PackageVersionFileController/listFiles.test.ts:"
cat test/port/controller/PackageVersionFileController/listFiles.test.ts

Length of output: 45940

app/core/service/BinarySyncerService.ts (1)

12-12: LGTM! Import changes are well-organized and necessary

The added imports support the improved error handling and file operations functionality.

Also applies to: 18-20

tsconfig.json Outdated Show resolved Hide resolved
app/common/ErrorUtil.ts Outdated Show resolved Hide resolved
app/common/ErrorUtil.ts Outdated Show resolved Hide resolved
@fengmk2 fengmk2 changed the title feat: enable allowH2 by default and require Node.js >= 18 feat: enable allowH2 by default and require Node.js >= 18.20.0 Nov 30, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Outside diff range and nitpick comments (1)
app/common/ErrorUtil.ts (1)

10-27: Refactor for better maintainability.

Extract the common name-checking logic into a helper function to follow the DRY principle.

+function isTimeoutErrorName(name: string): boolean {
+  return TimeoutErrorNames.includes(name);
+}
+
 export function isTimeoutError(err: Error) {
-  if (TimeoutErrorNames.includes(err.name)) {
+  if (isTimeoutErrorName(err.name)) {
     return true;
   }
   if (err instanceof AggregateError && err.errors) {
     for (const subError of err.errors) {
-      if (TimeoutErrorNames.includes(subError.name)) {
+      if (subError instanceof Error && isTimeoutErrorName(subError.name)) {
         return true;
       }
     }
   }
   let currentError = err;
   while (currentError.cause) {
     if (currentError.cause instanceof Error) {
-      if (TimeoutErrorNames.includes(currentError.cause.name)) {
+      if (isTimeoutErrorName(currentError.cause.name)) {
         return true;
       }
       currentError = currentError.cause;
     } else {
       break;
     }
   }
   return false;
 }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 433e692 and d0da939.

📒 Files selected for processing (2)
  • app/common/ErrorUtil.ts (1 hunks)
  • app/common/adapter/NPMRegistry.ts (2 hunks)
🔇 Additional comments (4)
app/common/ErrorUtil.ts (2)

21-25: 🛠️ Refactor suggestion

Improve error cause handling.

The current implementation only checks one level of error cause.


1-8: Consider exporting TimeoutErrorNames constant.

The array could be useful in other parts of the codebase for consistent timeout error handling.

-const TimeoutErrorNames = [
+export const TimeoutErrorNames = [

Let's verify that all these error types exist in the codebase:

app/common/adapter/NPMRegistry.ts (2)

15-15: LGTM! Good practice to centralize error type checking.

The addition of isTimeoutError import improves code maintainability by centralizing error type checking logic.


56-59: Verify if 120s timeout is necessary and appropriate.

The timeout has been significantly increased to 120 seconds. While this might solve immediate timeout issues, it could mask underlying performance problems.

Let's analyze the codebase for context on this timeout value:

Consider:

  1. Adding monitoring for requests that take longer than the original timeout
  2. Implementing circuit breakers for better resilience
  3. Investigating if response streaming or pagination could help reduce response times
✅ Verification successful

Let me gather more context about the NPM registry adapter and its usage.


120s timeout for NPM registry requests is justified

The timeout increase is appropriate based on the codebase evidence:

  1. The method is specifically used for fetching full package manifests which can be large:
    • Code comment explicitly mentions examples: "@procore/core-icons" and "intraactive-sdk-ui" taking 44s
    • The endpoint includes cache=0 parameter to bypass CDN, which naturally increases response time
  2. The implementation includes proper error handling:
    • Has retry mechanism (3 attempts)
    • Specifically handles timeout errors differently from other errors
    • Includes exponential backoff (1-4s) between retries
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for similar timeout configurations and any related performance discussions
rg -A 2 "timeout.*120000" 
rg -A 2 "timeout.*12[0-9]000"

# Look for related comments or issues about timeouts
rg -l "timeout|performance issue" --type md

Length of output: 523


Script:

#!/bin/bash
# Check the NPM Registry adapter implementation for context
cat app/common/adapter/NPMRegistry.ts

# Look for any timeout related content in the docs
rg -A 5 "timeout" docs/sync-setup.md docs/elasticsearch-setup.md

# Search for NPM registry related error handling or issues
rg -B 2 -A 2 "NPMRegistry" app/

Length of output: 7741

app/common/ErrorUtil.ts Show resolved Hide resolved
app/common/adapter/NPMRegistry.ts Show resolved Hide resolved
app/common/adapter/NPMRegistry.ts Show resolved Hide resolved
@fengmk2
Copy link
Member Author

fengmk2 commented Nov 30, 2024

leoric 对 es2022 支持还有问题,会丢掉 date 字段

2024-11-30 15:39:08,772 ERROR 1896 [-/127.0.0.1/2f1c5490-aeee-11ef-9bd3-ab0973874f00/17.058ms PUT /-/user/org.couchdb.user:testuser] nodejs.ER_NO_DEFAULT_FOR_FIELDError: Field 'gmt_create' doesn't have a default value
    at Packet.asError (/Users/fengmk2/git/github.com/cnpm/cnpmcore/node_modules/_mysql2@3.11.5@mysql2/lib/packets/packet.js:740:17)
    at Query.execute (/Users/fengmk2/git/github.com/cnpm/cnpmcore/node_modules/_mysql2@3.11.5@mysql2/lib/commands/command.js:29:26)
    at PoolConnection.handlePacket (/Users/fengmk2/git/github.com/cnpm/cnpmcore/node_modules/_mysql2@3.11.5@mysql2/lib/base/connection.js:475:34)
    at PacketParser.onPacket (/Users/fengmk2/git/github.com/cnpm/cnpmcore/node_modules/_mysql2@3.11.5@mysql2/lib/base/connection.js:93:12)
    at PacketParser.executeStart (/Users/fengmk2/git/github.com/cnpm/cnpmcore/node_modules/_mysql2@3.11.5@mysql2/lib/packet_parser.js:75:16)
    at Socket.<anonymous> (/Users/fengmk2/git/github.com/cnpm/cnpmcore/node_modules/_mysql2@3.11.5@mysql2/lib/base/connection.js:100:25)
    at Socket.emit (node:events:518:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:561:12)
    at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
    at Socket.Readable.push (node:internal/streams/readable:392:5)
    at TCP.onStreamRead (node:internal/stream_base_commons:189:23)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
code: "ER_NO_DEFAULT_FOR_FIELD"
errno: 1364
sqlState: "HY000"
sqlMessage: "Field 'gmt_create' doesn't have a default value"
sql: "INSERT INTO `users` (`user_id`, `name`, `email`, `password_salt`, `password_integrity`, `ip`, `is_private`) VALUES ('674ac11c3116ab0768e8c4a1', 'testuser', 'testuser@example.com', 'a51413f259f4cc245c6f0aa97614a5ae8d2312f79fa5bf9272e0e83d3796', 'sha512-hw8oPVmmCUkHkmwTSBLHAy7VEKzZ73VLoAffCSIHQVV0Vo+Wd6v3vaK+KS4jYxezsX29fVKqgsMYb2yXhuquuw==', '127.0.0.1', true)"

@fengmk2 fengmk2 merged commit 9b01383 into master Nov 30, 2024
12 of 14 checks passed
@fengmk2 fengmk2 deleted the custom-httpclient branch November 30, 2024 07:53
fengmk2 pushed a commit that referenced this pull request Nov 30, 2024
[skip ci]

## [3.68.0](v3.67.1...v3.68.0) (2024-11-30)

### Features

* enable allowH2 by default and require Node.js >= 18.20.0 ([#734](#734)) ([9b01383](9b01383))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant