Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add distributed capability for FLEX #149

Merged
merged 44 commits into from
Sep 19, 2024
Merged

Add distributed capability for FLEX #149

merged 44 commits into from
Sep 19, 2024

Conversation

mariogmarq
Copy link
Collaborator

This pull request provides FLEX with the distributed framework which allows to jointly train models with different devices. Code is self contained, adds gRPC as a dependency and provides a high coverage with integration and unit tests.

@mariogmarq mariogmarq requested a review from rbnuria September 19, 2024 09:11
Copy link

Coverage

Coverage Report
FileStmtsMissCoverMissing
flex/common
   utils.py751580%135–151, 189
flex/data
   dataset.py1553776%53–59, 201, 203, 215, 223–239, 307–308, 315, 323, 328, 335–336, 354–371, 398, 414–415
   dataset_pt_utils.py29293%48, 59
   fed_data_distribution.py182398%50–51, 397
   lazy_indexable.py79199%89
   pluggable_datasets.py62985%27, 41–49
flex/datasets
   federated_datasets.py452642%35, 39–71, 75–89, 102
   flexible_datasets.py11373%29, 46–47
flex/distributed
   __init__.py3233%18–19
   client.py11510410%30–238
   client_builder.py72720%18–252
   common.py161319%24–42
   server.py1701700%17–487
flex/distributed/aio
   __init__.py110%17
   server.py1511510%17–479
flex/distributed/proto
   tensor_pb2.py13931%12–25
   tensor_pb2_grpc.py14140%3–20
   transport_pb2.py50500%6–64
   transport_pb2_grpc.py32320%3–96
flex/pool
   aggregators.py50492%55–56, 145–146
   primitives_pt.py635021%46, 61–63, 104–122, 158–170, 197–206, 235–244
   primitives_tf.py42783%74–84
TOTAL192377560% 

Tests Skipped Failures Errors Time
193 4 💤 4 ❌ 3 🔥 2m 24s ⏱️

@rbnuria rbnuria merged commit 9422cb5 into main Sep 19, 2024
1 of 4 checks passed
@rbnuria rbnuria deleted the distributed branch September 19, 2024 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants