Sync commits

[pulltheflower]

• Fix mirror progress bug
• Fix table sync_versions version seq bug
• Add proxy config for multi sync
• Add url style config for duckdb reader
• Make minio bucket lookup configurable
• Server handle git lfs push when file larger than config size

What is this feature?

[Add a brief description of what the feature or update does.]

Why do we need this feature?

[Add a description of the problem the feature is trying to solve.]

Who is this feature for?

[Add information on what kind of user the feature is for.]

Which issue(s) does this PR fix?:

Fixes #

Special notes for your reviewer:

MR Summary:

The summary is added by @codegpt.

This Merge Request introduces several enhancements and bug fixes to improve the synchronization, configuration, and error handling of the system. Key updates include:

1. Enhanced LFS (Large File Storage) synchronization with Minio, including better error handling and progress tracking.
2. Improved Git HTTP handler to support LFS operations, including upload, download, lock management, and verification.
3. Added configuration options for S3 bucket lookup, proxy settings, and URL style configuration for DuckDB reader.
4. Fixed bugs related to mirror progress tracking and table synchronization.
5. Implemented server-side handling for Git LFS push operations for files exceeding a configurable size limit.

These changes aim to improve the system's robustness, configurability, and support for large file management in a distributed environment.

[starship-github]

Possible Issues And Suggestions:

• component/git_http_test.go
  • Comments:
    • The test logic change in TestGitHTTPComponent_LfsUpload might not cover all intended scenarios due to the removal of the condition checking if the object exists.
• Line 25 in builder/store/s3/minio.go
  • Comments:
    • Adding a new 'bucketLookupMapping' without validating input from 'cfg.S3.BucketLookup' could lead to unexpected behavior.
• Line 97 in common/config/config.go
  • Comments:
    • The new 'URLUploadMaxFileSize' config option is added but not used anywhere in the provided diffs.

MR Evaluation:

This feature is still under test, evaluation are given by AI and might be inaccurate.

After evaluation, the code changes in the Merge Request get score: 100.

Tips

CodeReview Commands (invoked as MR or PR comments)

• @codegpt /review to trigger an code review.
• @codegpt /evaluate to trigger code evaluation process.
• @codegpt /describe to regenerate the summary of the MR.
• @codegpt /secscan to scan security vulnerabilities for the MR or the Repository.
• @codegpt /help to get help.

CodeReview Discussion Chat

There are 2 ways to chat with Starship CodeReview:

• Review comments: Directly reply to a review comment made by StarShip.
  Example:
  • @codegpt How to fix this bug?
• Files and specific lines of code (under the "Files changed" tab):
  Tag @codegpt in a new review comment at the desired location with your query.
  Examples:
  • @codegpt generate unit testing code for this code snippet.

Note: Be mindful of the bot's finite context window.
It's strongly recommended to break down tasks such as reading entire modules into smaller chunks.
For a focused discussion, use review comments to chat about specific files and their changes, instead of using the MR/PR comments.

CodeReview Documentation and Community

• Visit our Documentation
  for detailed information on how to use Starship CodeReview.

About Us:

Visit the OpenCSG StarShip website for the Dashboard and detailed information on CodeReview, CodeGen, and other StarShip modules.

[pulltheflower]

Since this pr hasn't been updated in a long time, close it and resubmit #260

[starship-github]

The StarShip CodeReviewer was triggered but terminated because it encountered an issue: The MR state is not opened.

Tips

CodeReview Commands (invoked as MR or PR comments)

• @codegpt /review to trigger an code review.
• @codegpt /evaluate to trigger code evaluation process.
• @codegpt /describe to regenerate the summary of the MR.
• @codegpt /secscan to scan security vulnerabilities for the MR or the Repository.
• @codegpt /help to get help.

CodeReview Discussion Chat

There are 2 ways to chat with Starship CodeReview:

• Review comments: Directly reply to a review comment made by StarShip.
  Example:
  • @codegpt How to fix this bug?
• Files and specific lines of code (under the "Files changed" tab):
  Tag @codegpt in a new review comment at the desired location with your query.
  Examples:
  • @codegpt generate unit testing code for this code snippet.

Note: Be mindful of the bot's finite context window.
It's strongly recommended to break down tasks such as reading entire modules into smaller chunks.
For a focused discussion, use review comments to chat about specific files and their changes, instead of using the MR/PR comments.

CodeReview Documentation and Community

• Visit our Documentation
  for detailed information on how to use Starship CodeReview.

[starship-github]

The StarShip CodeReviewer was triggered but terminated because it encountered an issue: The MR state is not opened.

Tips

CodeReview Commands (invoked as MR or PR comments)

• @codegpt /review to trigger an code review.
• @codegpt /evaluate to trigger code evaluation process.
• @codegpt /describe to regenerate the summary of the MR.
• @codegpt /secscan to scan security vulnerabilities for the MR or the Repository.
• @codegpt /help to get help.

CodeReview Discussion Chat

There are 2 ways to chat with Starship CodeReview:

• Review comments: Directly reply to a review comment made by StarShip.
  Example:
  • @codegpt How to fix this bug?
• Files and specific lines of code (under the "Files changed" tab):
  Tag @codegpt in a new review comment at the desired location with your query.
  Examples:
  • @codegpt generate unit testing code for this code snippet.

Note: Be mindful of the bot's finite context window.
It's strongly recommended to break down tasks such as reading entire modules into smaller chunks.
For a focused discussion, use review comments to chat about specific files and their changes, instead of using the MR/PR comments.

CodeReview Documentation and Community

• Visit our Documentation
  for detailed information on how to use Starship CodeReview.