Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release version 3.0.0 to main branch #699

Merged
merged 107 commits into from
Dec 28, 2023
Merged

release version 3.0.0 to main branch #699

merged 107 commits into from
Dec 28, 2023

Conversation

sagewe
Copy link
Contributor

@sagewe sagewe commented Dec 21, 2023

Major Features and Improvements

Enhancements in the JVM Part:

  1. Core Component Reconstruction: The cluster-manager and node-manager components have been entirely rebuilt using Java, ensuring uniformity and enhanced performance.
  2. Transport Component Modification: The rollsite transport component has been removed and replaced with the more efficient osx component.
  3. Improved Process Management: Advanced logic has been implemented to manage processes more effectively, significantly reducing the risk of process leakage.
  4. Enhanced Data Storage Logic: Data storage mechanisms have been refined for better performance and reliability.
  5. Concurrency Control Improvements: We've upgraded the logic for concurrency control in the original components, leading to performance boosts.
  6. Visualization Component: A new visualization component has been added for convenient monitoring of computational information.
  7. Refined Logging: The logging system has been enhanced for more precise outputs, aiding in rapid anomaly detection.

Upgrades in the Python Part:

  1. Reconstruction of roll_pair and egg_pair: These components now support serialization and partition methods controlled by the caller. Serialization safety is uniformly managed by the caller.
  2. Automated Cleanup of Intermediate Tables: The issue of automatic cleaning for intermediate tables between federation and computing has been resolved, eliminating the need for extra operations by the caller.
  3. Unified Configuration Control: A flexible configuration system is introduced, supporting direct pass-through, configuration files, and environment variables to cater to diverse requirements.
  4. Client-Side PyPI Installation: Eggroll 3.0 supports easy installation via PyPI for clients.
  5. Optimized Log Configuration: Callers can now customize log formats according to their needs.
  6. Code Structure Refinement: The codebase has been streamlined for clarity, removing a substantial amount of redundant code.

Eggroll 3.0 brings comprehensive enhancements in system performance, usability, and reliability with these significant updates in both the JVM and Python parts.


Commit Authors

Commit Authors


Commits

Commits

Xiong-Li-github and others added 30 commits October 18, 2023 15:32
Signed-off-by: v_wbxiongli <740332065@qq.com>
Signed-off-by: v_wbxiongli <740332065@qq.com>
1. mv serdes and partitoner to client side
2. simplify and purge unused codes
3. better config support
4. better logging and tracing support

Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
…equirements

Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
…plify

introduce config utils to improve config set
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: v_wbxiongli <740332065@qq.com>
Signed-off-by: v_wbxiongli <740332065@qq.com>
Signed-off-by: v_wbxiongli <740332065@qq.com>
Signed-off-by: v_wbxiongli <740332065@qq.com>
Signed-off-by: v_wbxiongli <740332065@qq.com>
Signed-off-by: v_ndu <598671894@qq.com>
Signed-off-by: v_ndu <598671894@qq.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
chengtcc and others added 8 commits December 21, 2023 20:31
Signed-off-by: chengtcc <864261919@qq.com>
Signed-off-by: chengtcc <864261919@qq.com>
Signed-off-by: chengtcc <864261919@qq.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
@sagewe sagewe marked this pull request as ready for review December 27, 2023 11:31
@sagewe sagewe changed the title dev 3.0.0 rc release version 3.0.0 to main branch Dec 27, 2023
Signed-off-by: sagewe <wbwmat@gmail.com>
@dylan-fan dylan-fan merged commit 2ceb9d3 into main Dec 28, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants