From 31a2fc605f09425844ac9f07e6ac9ff2e24b7868 Mon Sep 17 00:00:00 2001 From: joyce-yuan Date: Sat, 17 Aug 2024 05:36:52 +0000 Subject: [PATCH 1/6] three tables matching three figures in paper --- docs/index.md | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 1 deletion(-) diff --git a/docs/index.md b/docs/index.md index b01caa2..119569c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -21,4 +21,79 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | Swarm | 183.2(3.5) | 183.1(3.6)| 183.2(3.6)| 52.2(8.7) | 52.3(8.7)| 52.4(8.6)| | L2C | 167.0(25.4) | 158.8(30.6)| 152.8(35.2)| 37.6(7.4) | 36.6(7.4)| 35.8(7.7)| -**Table 1:** Performance overview (AUC) of various topologies with different number of collaborators. \ No newline at end of file +**Table 1:** Performance overview (AUC) of various topologies with different number of collaborators. + +**Table 2** Area Under Curve of Test Accuracy Varying Number of Users +| Num Users | DomainNet | | Camelyon17 | | Digit-Five | | +|-----------|----------------------|--------------------|---------------------|--------------------|----------------------|--------------------| +| | Within Domain | Random | Within Domain | Random | Within Domain | Random | +|-----------|----------------------|--------------------|---------------------|--------------------|----------------------|--------------------| +| 12 | 56.6267 | 50.1772 | 178.3622 | 145.6398 | 57.1168 | 68.9724 | +| 18 | 59.5647 | 54.2480 | 179.5941 | 145.9916 | 66.8201 | 69.8341 | +| 24 | 61.8006 | 54.3855 | 178.5976 | 149.2037 | 71.6536 | 72.5333 | +| 30 | 66.5896 | 58.4835 | 179.1761 | 153.0658 | 74.4239 | 72.6996 | +| 39 | 68.3743 | 59.6090 | 179.1404 | 149.8618 | 163.8116 | 163.9892 | +| 45 | 68.1241 | 59.7852 | 180.0231 | 147.4649 | 77.0248 | 73.0634 | + + +**Table 3*** Area Under Curve of Test Accuracy Varying Number of Domains + +| AUC DomainNet (48 users, 200 rounds) | +|---------------------------------------------------| +| Num Domains | Within Domain | Random | +|-------------|---------------------|---------------| +| 2 | 67.7514 | 58.7947 | +| 4 | 61.5723 | 50.2906 | +| 6 | 69.4671 | 47.7867 | + +| AUC Camelyon17 (30 users, 200 rounds) | +|--------------------------------------------------------------------------| +| Num Domains | Within Domain | Random | +|-------------|---------------------|---------------| +| 2 | 179.7901 | 172.9167 | +| 3 | 179.1761 | 153.0658 | +| 5 | 176.5059 | 139.4547 | + +| AUC Digit-Five (30 users, 200 rounds) | +|--------------------------------------------------------------------------| +| Num Domains | Within Domain | Random | +|-------------|---------------------|---------------| +| 2 | 71.8536 | 65.6555 | +| 3 | 74.4239 | 72.6996 | +| 5 | 77.3709 | 76.3041 | + +**Table 4** Test Accuracy and Standard Deviation Over Rounds + +| Rounds | DomainNet (39 users, 3 domains) | | | | | +|--------|-----------------------------------------------------|---------------|---------------|---------------|---------------| +| | Within Domain | | Random | | +|--------|---------------------------|------------------------|---------------------------|------------------------| +| | Mean | Std | Mean | Std | +| 100 | 0.3619 | 0.0635 | 0.3212 | 0.0625 | +| 200 | 0.4220 | 0.0563 | 0.3733 | 0.0791 | +| 300 | 0.4362 | 0.0498 | 0.4203 | 0.0537 | +| 400 | 0.4353 | 0.0687 | 0.4355 | 0.0585 | +| 500 | 0.4726 | 0.0502 | 0.4499 | 0.0496 | + + +| Rounds | Camelyon17 (39 users, 3 domains) | | | | | +|--------|-----------------------------------------------------|---------------|---------------|---------------|---------------| +| | Within Domain | | Random | | +|--------|---------------------------|------------------------|---------------------------|------------------------| +| | Mean | Std | Mean | Std | +| 40 | 0.9086 | 0.0255 | 0.7281 | 0.1405 | +| 80 | 0.9169 | 0.0251 | 0.7460 | 0.1196 | +| 120 | 0.9329 | 0.0195 | 0.7293 | 0.1520 | +| 160 | 0.9361 | 0.0239 | 0.8122 | 0.1346 | +| 200 | 0.9353 | 0.0251 | 0.7762 | 0.1516 | + +| Rounds | Digit Five (39 users, 3 domains) | | | | | +|--------|----------------------------------------------------|---------------|---------------|---------------|---------------| +| | Within Domain | | Random | | +|--------|---------------------------|-----------------------|---------------------------|-----------------------| +| | Mean | Std | Mean | Std | +| 20 | 0.7314 | 0.1290 | 0.6788 | 0.0839 | +| 40 | 0.8080 | 0.0974 | 0.8151 | 0.0549 | +| 60 | 0.8350 | 0.0937 | 0.8464 | 0.0558 | +| 80 | 0.8417 | 0.0928 | 0.8673 | 0.0454 | +| 100 | 0.8310 | 0.1030 | 0.8733 | 0.0502 | From dd3c3813394e020ea3cd73775485df77b67feeef Mon Sep 17 00:00:00 2001 From: joyce-yuan Date: Sat, 17 Aug 2024 05:52:47 +0000 Subject: [PATCH 2/6] fixed format --- docs/index.md | 60 +++++++++++++++++++++++++-------------------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/docs/index.md b/docs/index.md index 119569c..944cd51 100644 --- a/docs/index.md +++ b/docs/index.md @@ -24,38 +24,38 @@ The application currently uses MPI and GRPC (experimental) to enable communicati **Table 1:** Performance overview (AUC) of various topologies with different number of collaborators. **Table 2** Area Under Curve of Test Accuracy Varying Number of Users -| Num Users | DomainNet | | Camelyon17 | | Digit-Five | | -|-----------|----------------------|--------------------|---------------------|--------------------|----------------------|--------------------| -| | Within Domain | Random | Within Domain | Random | Within Domain | Random | -|-----------|----------------------|--------------------|---------------------|--------------------|----------------------|--------------------| -| 12 | 56.6267 | 50.1772 | 178.3622 | 145.6398 | 57.1168 | 68.9724 | -| 18 | 59.5647 | 54.2480 | 179.5941 | 145.9916 | 66.8201 | 69.8341 | -| 24 | 61.8006 | 54.3855 | 178.5976 | 149.2037 | 71.6536 | 72.5333 | -| 30 | 66.5896 | 58.4835 | 179.1761 | 153.0658 | 74.4239 | 72.6996 | -| 39 | 68.3743 | 59.6090 | 179.1404 | 149.8618 | 163.8116 | 163.9892 | -| 45 | 68.1241 | 59.7852 | 180.0231 | 147.4649 | 77.0248 | 73.0634 | +| Num Users | DomainNet | Camelyon17 | Digit-Five | +|-----------|----------------------|--------------------|---------------------|--------------------|----------------------|------------------| +| | Within Domain | Random | Within Domain | Random | Within Domain | Random | +|-----------|----------------------|--------------------|---------------------|--------------------|----------------------|------------------| +| 12 | 56.6267 | 50.1772 | 178.3622 | 145.6398 | 57.1168 | 68.9724 | +| 18 | 59.5647 | 54.2480 | 179.5941 | 145.9916 | 66.8201 | 69.8341 | +| 24 | 61.8006 | 54.3855 | 178.5976 | 149.2037 | 71.6536 | 72.5333 | +| 30 | 66.5896 | 58.4835 | 179.1761 | 153.0658 | 74.4239 | 72.6996 | +| 39 | 68.3743 | 59.6090 | 179.1404 | 149.8618 | 163.8116 | 163.9892 | +| 45 | 68.124 | 59.7852 | 180.0231 | 147.4649 | 77.0248 | 73.0634 | **Table 3*** Area Under Curve of Test Accuracy Varying Number of Domains -| AUC DomainNet (48 users, 200 rounds) | -|---------------------------------------------------| -| Num Domains | Within Domain | Random | -|-------------|---------------------|---------------| -| 2 | 67.7514 | 58.7947 | -| 4 | 61.5723 | 50.2906 | -| 6 | 69.4671 | 47.7867 | +| AUC DomainNet (48 users, 200 rounds) | +|---------------------------------------------| +| Num Domains | Within Domain | Random | +|-------------|---------------|---------------| +| 2 | 67.7514 | 58.7947 | +| 4 | 61.5723 | 50.2906 | +| 6 | 69.4671 | 47.7867 | -| AUC Camelyon17 (30 users, 200 rounds) | -|--------------------------------------------------------------------------| +| AUC Camelyon17 (30 users, 200 rounds) | +|---------------------------------------------------| | Num Domains | Within Domain | Random | |-------------|---------------------|---------------| | 2 | 179.7901 | 172.9167 | | 3 | 179.1761 | 153.0658 | | 5 | 176.5059 | 139.4547 | -| AUC Digit-Five (30 users, 200 rounds) | -|--------------------------------------------------------------------------| +| AUC Digit-Five (30 users, 200 rounds) | +|---------------------------------------------------| | Num Domains | Within Domain | Random | |-------------|---------------------|---------------| | 2 | 71.8536 | 65.6555 | @@ -64,9 +64,9 @@ The application currently uses MPI and GRPC (experimental) to enable communicati **Table 4** Test Accuracy and Standard Deviation Over Rounds -| Rounds | DomainNet (39 users, 3 domains) | | | | | -|--------|-----------------------------------------------------|---------------|---------------|---------------|---------------| -| | Within Domain | | Random | | +| DomainNet (39 users, 3 domains) | +|--------|----------------------------------------------------|----------------------------------------------------| +| Rounds | Within Domain | Random | |--------|---------------------------|------------------------|---------------------------|------------------------| | | Mean | Std | Mean | Std | | 100 | 0.3619 | 0.0635 | 0.3212 | 0.0625 | @@ -76,9 +76,9 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | 500 | 0.4726 | 0.0502 | 0.4499 | 0.0496 | -| Rounds | Camelyon17 (39 users, 3 domains) | | | | | -|--------|-----------------------------------------------------|---------------|---------------|---------------|---------------| -| | Within Domain | | Random | | +| | Camelyon17 (39 users, 3 domains) | +|--------|----------------------------------------------------|---------------|---------------|---------------|----| +| Rounds | Within Domain | Random | |--------|---------------------------|------------------------|---------------------------|------------------------| | | Mean | Std | Mean | Std | | 40 | 0.9086 | 0.0255 | 0.7281 | 0.1405 | @@ -87,9 +87,9 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | 160 | 0.9361 | 0.0239 | 0.8122 | 0.1346 | | 200 | 0.9353 | 0.0251 | 0.7762 | 0.1516 | -| Rounds | Digit Five (39 users, 3 domains) | | | | | -|--------|----------------------------------------------------|---------------|---------------|---------------|---------------| -| | Within Domain | | Random | | +| | Digit Five (39 users, 3 domains) | | +|--------|---------------------------------------------------|---------------------------------------------------| +| Rounds | Within Domain | Random | |--------|---------------------------|-----------------------|---------------------------|-----------------------| | | Mean | Std | Mean | Std | | 20 | 0.7314 | 0.1290 | 0.6788 | 0.0839 | From 6bb002ded6f64a80f3c19cc9310cdf5118cf55ec Mon Sep 17 00:00:00 2001 From: joyce-yuan Date: Sat, 17 Aug 2024 05:56:54 +0000 Subject: [PATCH 3/6] test --- docs/index.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/index.md b/docs/index.md index 944cd51..6e8266b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -64,9 +64,9 @@ The application currently uses MPI and GRPC (experimental) to enable communicati **Table 4** Test Accuracy and Standard Deviation Over Rounds -| DomainNet (39 users, 3 domains) | -|--------|----------------------------------------------------|----------------------------------------------------| -| Rounds | Within Domain | Random | +| Rounds | DomainNet (39 users, 3 domains) | | | | +|--------|----------------------------------------------------|---------------|---------------|---------------| +| | Within Domain | | Random | | |--------|---------------------------|------------------------|---------------------------|------------------------| | | Mean | Std | Mean | Std | | 100 | 0.3619 | 0.0635 | 0.3212 | 0.0625 | @@ -76,6 +76,7 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | 500 | 0.4726 | 0.0502 | 0.4499 | 0.0496 | + | | Camelyon17 (39 users, 3 domains) | |--------|----------------------------------------------------|---------------|---------------|---------------|----| | Rounds | Within Domain | Random | @@ -89,7 +90,7 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | | Digit Five (39 users, 3 domains) | | |--------|---------------------------------------------------|---------------------------------------------------| -| Rounds | Within Domain | Random | +| Rounds | Within Domain | | Random | |--------|---------------------------|-----------------------|---------------------------|-----------------------| | | Mean | Std | Mean | Std | | 20 | 0.7314 | 0.1290 | 0.6788 | 0.0839 | From d9dbefeae99931c4e29a76b1aedb33cbc3d3ccbe Mon Sep 17 00:00:00 2001 From: joyce-yuan Date: Sat, 17 Aug 2024 05:58:36 +0000 Subject: [PATCH 4/6] test --- docs/index.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/index.md b/docs/index.md index 6e8266b..d7a7b27 100644 --- a/docs/index.md +++ b/docs/index.md @@ -77,9 +77,9 @@ The application currently uses MPI and GRPC (experimental) to enable communicati -| | Camelyon17 (39 users, 3 domains) | -|--------|----------------------------------------------------|---------------|---------------|---------------|----| -| Rounds | Within Domain | Random | +| Rounds | Camelyon17 (39 users, 3 domains) | | | | +|--------|-----------------------------------------------------|---------------|---------------|---------------| +| | Within Domain | | Random | | |--------|---------------------------|------------------------|---------------------------|------------------------| | | Mean | Std | Mean | Std | | 40 | 0.9086 | 0.0255 | 0.7281 | 0.1405 | @@ -88,6 +88,7 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | 160 | 0.9361 | 0.0239 | 0.8122 | 0.1346 | | 200 | 0.9353 | 0.0251 | 0.7762 | 0.1516 | + | | Digit Five (39 users, 3 domains) | | |--------|---------------------------------------------------|---------------------------------------------------| | Rounds | Within Domain | | Random | From e865fda683ddca9b3a7c9901f0a55d5d0f730d81 Mon Sep 17 00:00:00 2001 From: joyce-yuan Date: Sat, 17 Aug 2024 06:00:45 +0000 Subject: [PATCH 5/6] test --- docs/index.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/index.md b/docs/index.md index d7a7b27..dd1ff91 100644 --- a/docs/index.md +++ b/docs/index.md @@ -24,7 +24,7 @@ The application currently uses MPI and GRPC (experimental) to enable communicati **Table 1:** Performance overview (AUC) of various topologies with different number of collaborators. **Table 2** Area Under Curve of Test Accuracy Varying Number of Users -| Num Users | DomainNet | Camelyon17 | Digit-Five | +| Num Users | DomainNet | | Camelyon17 | | Digit-Five | | |-----------|----------------------|--------------------|---------------------|--------------------|----------------------|------------------| | | Within Domain | Random | Within Domain | Random | Within Domain | Random | |-----------|----------------------|--------------------|---------------------|--------------------|----------------------|------------------| @@ -38,7 +38,7 @@ The application currently uses MPI and GRPC (experimental) to enable communicati **Table 3*** Area Under Curve of Test Accuracy Varying Number of Domains -| AUC DomainNet (48 users, 200 rounds) | +*AUC DomainNet (48 users, 200 rounds)* |---------------------------------------------| | Num Domains | Within Domain | Random | |-------------|---------------|---------------| @@ -46,7 +46,7 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | 4 | 61.5723 | 50.2906 | | 6 | 69.4671 | 47.7867 | -| AUC Camelyon17 (30 users, 200 rounds) | +*AUC Camelyon17 (30 users, 200 rounds)* |---------------------------------------------------| | Num Domains | Within Domain | Random | |-------------|---------------------|---------------| @@ -54,7 +54,7 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | 3 | 179.1761 | 153.0658 | | 5 | 176.5059 | 139.4547 | -| AUC Digit-Five (30 users, 200 rounds) | +*AUC Digit-Five (30 users, 200 rounds)* |---------------------------------------------------| | Num Domains | Within Domain | Random | |-------------|---------------------|---------------| @@ -89,9 +89,9 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | 200 | 0.9353 | 0.0251 | 0.7762 | 0.1516 | -| | Digit Five (39 users, 3 domains) | | -|--------|---------------------------------------------------|---------------------------------------------------| -| Rounds | Within Domain | | Random | +| Rounds | Digit Five (39 users, 3 domains) | | | | +|--------|----------------------------------------------------|---------------|---------------|---------------| +| | Within Domain | | Random | | |--------|---------------------------|-----------------------|---------------------------|-----------------------| | | Mean | Std | Mean | Std | | 20 | 0.7314 | 0.1290 | 0.6788 | 0.0839 | From ebfa1c12f41fa3418fbe6da1c438c9833c6c120d Mon Sep 17 00:00:00 2001 From: joyce-yuan Date: Sat, 17 Aug 2024 06:02:02 +0000 Subject: [PATCH 6/6] fixed table 3 --- docs/index.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/index.md b/docs/index.md index dd1ff91..0e4e780 100644 --- a/docs/index.md +++ b/docs/index.md @@ -36,10 +36,9 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | 45 | 68.124 | 59.7852 | 180.0231 | 147.4649 | 77.0248 | 73.0634 | -**Table 3*** Area Under Curve of Test Accuracy Varying Number of Domains +**Table 3** Area Under Curve of Test Accuracy Varying Number of Domains *AUC DomainNet (48 users, 200 rounds)* -|---------------------------------------------| | Num Domains | Within Domain | Random | |-------------|---------------|---------------| | 2 | 67.7514 | 58.7947 | @@ -47,7 +46,6 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | 6 | 69.4671 | 47.7867 | *AUC Camelyon17 (30 users, 200 rounds)* -|---------------------------------------------------| | Num Domains | Within Domain | Random | |-------------|---------------------|---------------| | 2 | 179.7901 | 172.9167 | @@ -55,7 +53,6 @@ The application currently uses MPI and GRPC (experimental) to enable communicati | 5 | 176.5059 | 139.4547 | *AUC Digit-Five (30 users, 200 rounds)* -|---------------------------------------------------| | Num Domains | Within Domain | Random | |-------------|---------------------|---------------| | 2 | 71.8536 | 65.6555 |