Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Federated Runtime + Workspace Builder (#876)
* Federated Runtime v1.0
experimental Aggregator and Collaborator in working condition
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added test, and mnist experimental workspaces working prototype
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Working on integration with Ishant's, and Keerti's code
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Second half of integration installment.
Remaning:
Integration of data.yaml file.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Removing unnecessary directory my_federation.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* 1. Removed unncessart files.
2. Fix `fx plan freeze` command not working issue.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added function for test workspace
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added include_exclude testcase workspace.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Default execution workspace is mnist.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* data.yaml changes are integrated with this update.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Running a federation with GaNDLF Documentation (#794)
* Initial commit of Running the federation with GaNDLF Documentation
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update docs/running_the_federation_with_gandlf.rst
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update docs/running_the_federation_with_gandlf.rst
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update docs/running_the_federation_with_gandlf.rst
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update docs/running_the_federation_with_gandlf.rst
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update docs/running_the_federation_with_gandlf.rst
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update docs/running_the_federation_with_gandlf.rst
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update docs/running_the_federation_with_gandlf.rst
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update docs/running_the_federation_with_gandlf.rst
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update docs/running_the_federation_with_gandlf.rst
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update docs/running_the_federation_with_gandlf.rst
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update README.md
Removed references to Intel's ownship, given it's now owned by the LF AI and Data.
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix Flake8 C419 for Ubuntu CI (#800)
C419 Unnecessary list comprehension passed to any()/all() prevents short-circuiting - rewrite as a generator
Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update README.md
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix warnings and issues in docs (#825)
* Fix warnings and issues in docs
Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com>
* fixup! Fix warnings and issues in docs
Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com>
---------
Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Add Logo (#827)
* Add Logo
* Update README.md
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Change OpenFL documentation font to improve accessibility (#809)
This replaces the font of OpenFL documents with Intel One Mono font for low vision developers.
Known issues:
1. The text font within the images has not been changed.
2. Some icons that do not exist in the new font cannot be displayed properly.
Fixes securefederatedai#799
Co-authored-by: Wang, Le <le3.wang@intel.com>
Signed-off-by: He, Dan H <dan.h.he@intel.com>
Signed-off-by: Jiang, Jiaqiu <jiaqiu.jiang@intel.com>
Signed-off-by: Li, Qingqing <qingqing.li@intel.com>
Signed-off-by: Wang, Le <le3.wang@intel.com>
Signed-off-by: Wu, Caili <caili.wu@intel.com>
Signed-off-by: He, Dan H <dan.h.he@intel.com>
Co-authored-by: He, Dan H <dan.h.he@intel.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update unit tests to improve code coverage (#821)
* Update ci config
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Add unit test for following files
- openfl/federated/plan/plan.py
- openfl/interface/aggregation_functions/core/adaptive_aggregation.py
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Add some test cases for databases module
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Fix bugs for databases module unittest
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Update unit tests for component module
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Restore workflow config and update some comments
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Enable save_ test case.
Add yaml under test dir for unit test usage.
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Remove plan to new dir.
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Remove plan to new dir.
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Add aggregator start test cases.
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Add 2 aggregator test cases.
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Add 1 aggregator test case.
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Format code.
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Refactor code.
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Add collaborator start test cases.
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Add 1 collaborator test case.
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Format with flake8
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
* Remove TODO comments
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
---------
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
Co-authored-by: Wang, Wenjie <wenjie2.wang@intel.com>
Co-authored-by: Lei5 Chen <lei5.chen@intel.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Add PyTorch linear regression example (#808)
This adds a new tutorial example on distributing a linear regression task over OpenFL cluster.
The model is defined by Pytorch which is able to run over both cpu (by default) and gpu. The dataset is generated by make_regression from sklearn.datasets with pre-defined parameters.
Fixes #797
Co-authored-by: Jiang, Jiaqiu <jiaqiu.jiang@intel.com>
Signed-off-by: He, Dan H <dan.h.he@intel.com>
Signed-off-by: Jiang, Jiaqiu <jiaqiu.jiang@intel.com>
Signed-off-by: Li, Qingqing <qingqing.li@intel.com>
Signed-off-by: Wang, Le <le3.wang@intel.com>
Signed-off-by: Wu, Caili <caili.wu@intel.com>
Signed-off-by: He, Dan H <dan.h.he@intel.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* This prints out the hash of the CSR to disk for both the aggregator and (#813)
* This prints out the hash of the CSR to disk for both the aggregator and
collaborator. The user then compares and approves this hash with the
hash printed out of the file to validate the CSR. In addition, a warning
message is pritned if certify is run in silent mode.
Fixes securefederatedai#692
Signed-off-by: Grant Baker <grant.baker@intel.com>
* Refactor read_csr function to use get_csr_hash
Signed-off-by: Grant Baker <grant.baker@intel.com>
* Ask to check hashes before prompt
---------
Signed-off-by: Grant Baker <grant.baker@intel.com>
Co-authored-by: Grant Baker <grant.baker@intel.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Improve workspace requirements import (#810)
Remove the dump_requirement_file operation in export_ method.
Fixes securefederatedai#767
Co-authored-by: Li, Qingqing <qingqing.li@intel.com>
Co-authored-by: Wu, Caili <caili.wu@intel.com>
Signed-off-by: He, Dan H <dan.h.he@intel.com>
Signed-off-by: Jiang, Jiaqiu <jiaqiu.jiang@intel.com>
Signed-off-by: Li, Qingqing <qingqing.li@intel.com>
Signed-off-by: Wang, Le <le3.wang@intel.com>
Signed-off-by: Wu, Caili <caili.wu@intel.com>
Signed-off-by: He, Dan H <dan.h.he@intel.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Issue 506 Added Example using FedProx (#818)
* created new ineractive_api dir to hold pytorch fedprox mnist example
corrected files
changed to FedProxOptimizer and ran set_old_weights for new FedProx Pytorch example
renamed FedProx notebook
used mode.parrameters() to get pytorch model weights
got weights using state_dict
changed old wieghts to list (for serialization) and fixed README
input wieghts before zero_grad
[Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api
This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer.
Fixes: #506
Signed-off-by: Klemme, Beverly <beverly.klemme@intel.com>
Signed-off-by: Baker, Grant <grant.baker@intel.com>
Signed-off-by: ELizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com>
Signed-off-by: Jillela, Emmanuel <emmanuel.jillela@intel.com>
Signed-off-by: Beverly Klemme <beverly.klemme@intel.com>
* [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api
This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer.
Fixes: securefederatedai#506
Signed-off-by: Klemme, Beverly <beverly.klemme@intel.com>
Signed-off-by: Baker, Grant <grant.baker@intel.com>
Signed-off-by: Elizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com>
Signed-off-by: Jillela, Emmanuel <emmanuel.jillela@intel.com>
Signed-off-by: Beverly Klemme <beverly.klemme@intel.com>
* Update README.md
Signed-off-by: Beverly Klemme <beverly.klemme@intel.com>
* addressed comments by psfoley: corrected words in the jupyter notebook metadata and added a link to the FedProx paper in the README.
Signed-off-by: Beverly Klemme <beverly.klemme@intel.com>
---------
Signed-off-by: Klemme, Beverly <beverly.klemme@intel.com>
Signed-off-by: Baker, Grant <grant.baker@intel.com>
Signed-off-by: ELizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com>
Signed-off-by: Jillela, Emmanuel <emmanuel.jillela@intel.com>
Signed-off-by: Beverly Klemme <beverly.klemme@intel.com>
Signed-off-by: Elizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* [Bug: 768] FX CLI: Separate create, cert gen commands (#807)
This change separates existing command
"fx collaborator.py generate-cert-request" command into two
commands.
"fx collaborator create -n {NAME} -d {DATA_PATH: optional}".
"fx collaborator generate-cert-request -n {NAME}".
Fixes #768
Signed-off-by: Emmanuel Jillela <emmanuel.jillela@intel.com>
Co-authored-by: Emmanuel Jillela <emmanuel.jillela@intel.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Add new tutorial example to OpenFL interactive API (#812)
* Add new tutorial example to OpenFL interactive API
This adds a new tutorial example on distributing a linear regression task over OpenFL cluster
The model is defined by scikit-learn which is able to run over both cpu (by default) and gpu. The dataset is 1-dimensional noisy data of sinusoid with pre-defined parameters.
Fixes #798
Co-authored-by: Beverly Klemme <beverly.klemme@intel.com>
Co-authored-by: Grant Baker <grant.baker@intel.com>
Signed-off-by: Yi CAO <yi2.cao@intel.com>
* reduced requirements.txt in workspace
Signed-off-by: Beverly Klemme <beverly.klemme@intel.com>
---------
Signed-off-by: Yi CAO <yi2.cao@intel.com>
Signed-off-by: Beverly Klemme <beverly.klemme@intel.com>
Co-authored-by: Yi CAO <yi2.cao@intel.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* build(deps): bump tensorflow in /openfl-workspace/tf_cnn_histology (#776)
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1)
---
updated-dependencies:
- dependency-name: tensorflow
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* build(deps): bump tensorflow (#777)
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1)
---
updated-dependencies:
- dependency-name: tensorflow
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update GaNDLF repo location and test CI with master branch
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update GaNDLF repo location and test CI with master branch
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update GaNDLF repo location and test CI with master branch
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix documentation links. Change path names and templates for CI
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix paths
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix paths
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix breaking tests
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Add compatible onnx version to requirements.txt file
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix wrong csv file name
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix wrong csv file name
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix wrong names in workflow file
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix wrong data path
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix lint in test_gandlf.py
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix lint errors
Signed-off-by: Patrick Foley <psfoley@gmail.com>
---------
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com>
Signed-off-by: He, Dan H <dan.h.he@intel.com>
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
Signed-off-by: Klemme, Beverly <beverly.klemme@intel.com>
Signed-off-by: Baker, Grant <grant.baker@intel.com>
Signed-off-by: ELizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com>
Signed-off-by: Jillela, Emmanuel <emmanuel.jillela@intel.com>
Signed-off-by: Beverly Klemme <beverly.klemme@intel.com>
Signed-off-by: Elizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com>
Signed-off-by: Emmanuel Jillela <emmanuel.jillela@intel.com>
Signed-off-by: Yi CAO <yi2.cao@intel.com>
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Co-authored-by: Prashant Shah <40899779+SprashAI@users.noreply.github.com>
Co-authored-by: akantak <aleksander.kantak@intel.com>
Co-authored-by: wangleflex <106506636+wangleflex@users.noreply.github.com>
Co-authored-by: He, Dan H <dan.h.he@intel.com>
Co-authored-by: xiaoranf <desolatefang@gmail.com>
Co-authored-by: Wang, Wenjie <wenjie2.wang@intel.com>
Co-authored-by: Lei5 Chen <lei5.chen@intel.com>
Co-authored-by: Beverly Klemme <35578090+bjklemme-intel@users.noreply.github.com>
Co-authored-by: Emmanuel Jillela <emmanuel.jillela@intel.com>
Co-authored-by: Yi CAO <yi2.cao@intel.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid (#841)
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Fix GaNDLF documentation links (#842)
* Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix links in GaNDLF Documentation
* Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix links in GaNDLF Documentation
Signed-off-by: Patrick Foley <psfoley@gmail.com>
---------
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Fix incorrectly formatted link in docs (#839)
Signed-off-by: Francis Storr <francis.storr@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added licensing information to all experimental workspaces
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Made changes to LocalRuntime, and tested
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Rename collaborator_name variable to name in
experimental/component/collaborator/collaborator.py file.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* build(deps): bump onnx in /openfl-workspace/gandlf_seg_test (#840)
Bumps [onnx](https://github.com/onnx/onnx) from 1.12 to 1.13.0.
- [Release notes](https://github.com/onnx/onnx/releases)
- [Changelog](https://github.com/onnx/onnx/blob/main/docs/Changelog.md)
- [Commits](https://github.com/onnx/onnx/compare/v1.12.0...v1.13.0)
---
updated-dependencies:
- dependency-name: onnx
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Selected collaborators functionality added, test workspace of the same
is work-progress.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Selected collaborator testcases included with this commit.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Testcase private attributes workspace added with this commit.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Instead of remove private attributes from checkpoint, changed the value
to string "Private attributes: Not Available."
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added / Modified types of funciton arguments, and their return types.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Aggregator shutdown related changes are done in this commit
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update testflow_subset_of_collaborators.py
Updated subset of collaborators testcase
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Modified subset of collaborators testcase.
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Add files via upload
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Accessibility updates (#861)
* Fix incorrectly formatted link in docs
Signed-off-by: Francis Storr <francis.storr@intel.com>
* Font styling, color contrast, other accessibility updates
This update:
1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples.
2. Adds colors (in `colors.css`)
3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future.
Closes #848
Signed-off-by: Francis Storr <francis.storr@intel.com>
---------
Signed-off-by: Francis Storr <francis.storr@intel.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp (#773)
* build(deps): bump tensorflow in /openfl-workspace/keras_nlp
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1)
---
updated-dependencies:
- dependency-name: tensorflow
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update RMSProp optimizer import
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* flake8
Signed-off-by: Patrick Foley <psfoley@gmail.com>
---------
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_mnist (#771)
* build(deps): bump tensorflow in /openfl-workspace/keras_cnn_mnist
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1)
---
updated-dependencies:
- dependency-name: tensorflow
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* revert experimental Adam to legacy (#863)
Signed-off-by: kta-intel <kevin.ta@intel.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: kta-intel <kevin.ta@intel.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kevin Ta <116312994+kta-intel@users.noreply.github.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_with_compression (#770)
* build(deps): bump tensorflow
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1)
---
updated-dependencies:
- dependency-name: tensorflow
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update Adam Optimizer import
Signed-off-by: Patrick Foley <psfoley@gmail.com>
---------
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* build(deps): bump tensorflow from 2.9.3 to 2.11.1 in /openfl-tutorials/interactive_api/Flax_CNN_CIFAR (#775)
* build(deps): bump tensorflow
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1)
---
updated-dependencies:
- dependency-name: tensorflow
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* Fixed breaking backages
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Add quiet flag back to pip install
Signed-off-by: Patrick Foley <psfoley@gmail.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* build(deps): bump tensorflow-cpu from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp_gramine_ready (#769)
* build(deps): bump tensorflow-cpu
Bumps [tensorflow-cpu](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1)
---
updated-dependencies:
- dependency-name: tensorflow-cpu
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* Use legacy RMSprop optimizer
Signed-off-by: Patrick Foley <psfoley@gmail.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Accessibility color contrast fixes (#864)
* Fix incorrectly formatted link in docs
Signed-off-by: Francis Storr <francis.storr@intel.com>
* Font styling, color contrast, other accessibility updates
This update:
1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples.
2. Adds colors (in `colors.css`)
3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future.
Closes #848
Signed-off-by: Francis Storr <francis.storr@intel.com>
* Color contrast updates for accessibility
Color contrast updates for accessibility
- update generic `a` element
- update color of links in the toggle-able read-the-docs panel
- update the color of the text in search results
- update the color of notes headers
Signed-off-by: Francis Storr <francis.storr@intel.com>
---------
Signed-off-by: Francis Storr <francis.storr@intel.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Testflow for verifying stdout redirection to Metaflow datastore (#758)
* implemented ray.wait
* reverted changes back after testing
* adding datastore cli test case
* removed unused variables
* removed stderr validation
* fixed lint suggestions
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Tweak link color so it’s not so aggressive (#865)
Signed-off-by: Francis Storr <francis.storr@intel.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* External Loop Functionality Added
External Loop Functionality Added
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* External Loop Functionality Added
External Loop Functionality Added
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* External Loop Functionality Added
External Loop Functionality Added
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added following tutorials, and testcase:
1. Vertical FL
2. Vertical FL Two Party
3. Internal Loop
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* plan.yaml modified in 301 watermarking, and subset of collaborators.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Modifying temp.sh for 2 collaborators.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update aggregator.py
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* FQDN switched to localhost
FQDN switched to localhost
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* While activating experimental OpenFL commands, install depedencies as well.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Install experimental requirements while activating experimental openfl commands.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update experimental-requirements.txt
Added ray 2.2.0 to experimental-requirements.txt file
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update requirements_workflow_interface.txt
Metaflow, and Ray dependencies are moved to openfl-workspace/experimental/experimental-requrirements.txt file.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Delete openfl/experimental/utilities/utils.py, and removed torch depency from openfl/experimental/utilities/resources.py
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update requirements_workflow_interface.txt
Updated experimental tutorials requirements.txt file
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Updating 201 GPUs tutorial
Updated Workflow_Interface_201_Exclusive_GPUs_with_Ray.ipynb for Aggregator, and Collaborator as stateful actor
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Modified 101, 102, 103, 301, 401 tutorials as per Aggregator, & Collaborator stateful collaborators
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Excluding 'experimental' directory from workspace templates
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update 101 tutorial data.yaml
Updated 101 tutorials data.yaml to correct a mistake by in index parameter of private attribute function.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update resources.py
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Modified get_number_of_gpus function
Modify get_number_of_gpus function
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* This commit brings following changes:
1. Removed torch dependency from "get_number_of_gpus"
2. Added keras mnist example notebook
3. Added keras mnist workspace example
4. Removed openfl-workspace/experimental/experimental-requirements.txt
5. "fx aggregator start" command will install libraries from experimental tutorials requirements_workflow_interface.txt file
6. Added testcase_datastore_cli, testcase_reference and testcase_reference_with_include_exclude
7. Updated 101 torch mnist plan.yaml and data.yaml
8. Removed aggregator start flow thread from cli/aggregator.py
9. At last changes made in federated runtime
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /tests/github/interactive_api_director/experiments/tensorflow_mnist/envoy (#772)
* build(deps): bump tensorflow
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1)
---
updated-dependencies:
- dependency-name: tensorflow
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update sd_requirements.txt
* revert to legacy SGD and install tensorflow==2.11 for workflow
Signed-off-by: kta-intel <kevin.ta@intel.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: kta-intel <kevin.ta@intel.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Patrick Foley <patrick.foley@intel.com>
Co-authored-by: kta-intel <kevin.ta@intel.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/tf_2dunet (#774)
* build(deps): bump tensorflow in /openfl-workspace/tf_2dunet
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1)
---
updated-dependencies:
- dependency-name: tensorflow
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update requirements.txt
to retrigger CI
* Update requirements.txt
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kevin Ta <116312994+kta-intel@users.noreply.github.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* This commit brings following changes:
1. Renamed Workflow_Interface_101_Keras_MNIST_with_GPU.ipynb to Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb
2. Updated documentation in Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb
3. protobuf bumped from 3.19.6 to 3.20.0 in setup.py
4. Incorporating review comments (WIP)
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Modified 104 Keras MNIST tutorial
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* This comming brings following changes:
1. Experimental Aggregator workflow testcases moved to test/github/experimental/workspace directory
2. Added dependecy related changes
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Removing unnecessary logs
Removing unnecessary logs
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update collaborator.py
Removing unnecessary logs
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Avoided deep copying the context by removing private attributes from context before checkpoint
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Incoporated review comments
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added dependecies to each tutorial notebooks
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* pinned tensorboardX (#870)
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* While taking aggregator step remove private attributes before taking checkpoint and set it back to ctx after checkpoint
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Removing unnecessary deepcopies from aggregator.py file
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Removing unnecessary comments
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update stream_redirect.py
Removing unnecessary imports
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added color escape sequences to datastore cli localruntime and workspace testcases
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update requirements_privacy_meter.txt
torch 1.13.1 requires torchvision 0.14.1 not 0.14.0
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update Workflow_Interface_VFL_Two_Party
Added actor related changed to Workflow_Interface_VFL_Two_Party
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Files added required to test experimental aggregator based workflow by GitHUB actions
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb
Removing GPU parameter from collaborator decorator.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update resources.py
Updated get_number_of_gpus function
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Updating import statement
In 104 keras workspace utils.py updating import SummaryWriter statement from torch to tensorflow
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* With this commit path of template can be explicitely given, user does not have to
choose from available example templates. Only applicable to experimental openfl.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Fixed flake8 issues and typos
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Resolved flake8 issues,
Added warning statements when data.yaml is not found or aggregator / collaborator private attributes are not found in data.yaml,
Tested by running all tutorials and tests
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Moving the logs after data.yaml warning, and removing unnecessary blank lines.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Adding N-level arguments functionality
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update workflow_interface.rst
Added section on how to save best and last model from Metaflow CLI
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Resolving last few TODO items.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Updated logs to provide exact information to user.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* This commit brings following updates:
1. Remove unnecessary spaces from Workflow_Interface_102_Aggregator_Validation.ipynb
2. Upgraded protobuf to 3.20.3 in setup.py as per tensorboard requirements
3. Modified 101-torch-cnn-mnist workspace for n-level arguments
4. Added a federated runtime workspace for 102-aggregator-validation
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* This commit brings following changes:
1. Added 501_pytorch_tinyimagenet_transfer_learning workspace for experimental workflow interface
2. Removed `feature_shape` from openfl/experimental/interface/cli/plan.py
3. Resolved flake8 errors in openfl-workspace/experimental directory
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update data.yaml
Correcting a mistake train dataset assigned to test and vice versa.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update flow.py
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update flow.py
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Adding functionality to convert jupyter notebook to workspace (in progress)
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Write flow.py, collaborator_private_attrs.py and aggregator_private_attrs.py automatically
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Find aggregator, and collaborator arguments
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Modifying arguments help
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Removed automatically generated Zone.Identifier file
Added Approach2 notebook
Modified creator.py
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* workspace_creator renamed to workspace_builder
finding values without help of any tags
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Finished exporting data.yaml, plan.yaml.
Workspace creator validated for 101 MNIST and 301 Watermarking with MNIST
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Removing round number constant argument from 301 MNIST WATERMARKING
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* * Created get_collaborator_kwargs function in localruntime class which takes collaborator name as argument and returns name of private attributes function and arguments required to call it.
* Changed workspace builder code used get_collaborator_kwargs function to build data.yaml file.
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Modified get_collaborator_args function
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Modified workspace_builder/build.py file to resolve a typo
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Removed unnecessary files from template_workspace directory
Moved print_tree to after generating python script
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added documentation and inline comments for workspace_builder
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added libraries required by WorkspaceBuilder
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update Tensorflow, gRPC, Protobuf dependencies (#868)
* Update Tensorflow to latest, finally update grpcio/protobuf
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Lint issue fix and missing tf reference
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* pyzmq version fixed
* fix taskrunner tests for windows
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* fix taskrunner test syntax for windows
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* adding user option to workspace pip install requirements for windows
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* fix windows CI test
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* testing virtual env for windows github actions
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* testing virtual env for windows github actions
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* testing virtual env for windows github actions
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* testing venv for windows
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* test venv for windows
* test venv for windows
* Added new KerasSerializer. Fixed other Interactive API experiments
* Update taskrunner.yml
* Update taskrunner.yml
* Update workspace.py
* Update workspace.py
* Update taskrunner.yml
* Remove get_model import from global namespace so dependencies are not loaded into memory unnecessarily (breaking windows build)
* Refactoring and cleaning up imports to support Windows install
* Fixed logger import paths
* Fix missing imports
* Fix native import
* Fix lint errors
* Fix keras optimizer patch. Remove irrelevant unit test
* Format logs in UTF-8 for windows
* Update interactive-kvasir.yml
* Consolidate github actions python versions to single file
* Update python versions
* Update python versions
* Update python versions
* Reduce # of DataLoader workers for Pytorch Kvasir CI test
* Fix Windows encoding
* Fix Windows encoding and limit rounds so Github Actions CI doesn't run out of memory
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix windows encoding
* Fix Windows encoding
---------
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
Co-authored-by: Mansi Sharma <77758170+mansishr@users.noreply.github.com>
Co-authored-by: Mansi Sharma <mansi.sharma@intel.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Add FL plan description to documentation (#872)
* Add plan description to documentation
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* fix indentation
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* Apply suggestions from code review
Co-authored-by: Patrick Foley <patrick.foley@intel.com>
---------
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
Co-authored-by: Patrick Foley <patrick.foley@intel.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Resolve Coverity Issues (#874)
* Fix coverity issues
* Resolve remaining coverity issues
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Migrate to Ubuntu 22.04 LTS release (supported through 2027) (#875)
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Fixed flake8 issues
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Fixed flake8 issues in test/experimental directory
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Accessibility updates (#861)
* Fix incorrectly formatted link in docs
Signed-off-by: Francis Storr <francis.storr@intel.com>
* Font styling, color contrast, other accessibility updates
This update:
1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples.
2. Adds colors (in `colors.css`)
3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future.
Closes #848
Signed-off-by: Francis Storr <francis.storr@intel.com>
---------
Signed-off-by: Francis Storr <francis.storr@intel.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833)
* fixing FedAvg averaging in order to be compatible with numpy v1.24+
Signed-off-by: kta-intel <kevin.ta@intel.com>
* uncommenting installations for consistency with other tutorials
Signed-off-by: kta-intel <kevin.ta@intel.com>
* fixing 301_MNIST_Watermarking tutorial FedAvg
Signed-off-by: kta-intel <kevin.ta@intel.com>
* fixing 301_MNIST_Watermarki
ng tutorial FedAvg
Signed-off-by: kta-intel <kevin.ta@intel.com>
* Switching to py38 kernel and clearing cell outputs
Signed-off-by: kta-intel <kevin.ta@intel.com>
---------
Signed-off-by: kta-intel <kevin.ta@intel.com>
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Testflow for verifying stdout redirection to Metaflow datastore (#758)
* implemented ray.wait
* reverted changes back after testing
* adding datastore cli test case
* removed unused variables
* removed stderr validation
* fixed lint suggestions
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Resolving DCO issue
fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833)
* fixing FedAvg averaging in order to be compatible with numpy v1.24+
Signed-off-by: kta-intel <kevin.ta@intel.com>
* uncommenting installations for consistency with other tutorials
Signed-off-by: kta-intel <kevin.ta@intel.com>
* fixing 301_MNIST_Watermarking tutorial FedAvg
Signed-off-by: kta-intel <kevin.ta@intel.com>
* fixing 301_MNIST_Watermarki
ng tutorial FedAvg
Signed-off-by: kta-intel <kevin.ta@intel.com>
* Switching to py38 kernel and clearing cell outputs
Signed-off-by: kta-intel <kevin.ta@intel.com>
---------
Signed-off-by: kta-intel <kevin.ta@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update EdenPipeline in the documentation (#877)
Signed-off-by: Amit Portnoy <1131991+amitport@users.noreply.github.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Fixed flake8 issues
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Added weighted_average aggregation function under openfl.experimental.interface.{keras,torch}.aggregation_funtions
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Tested all local-runtime & federated-runtime tutorials and testcases
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Resolving merge conflicts in experimental local-runtme tutorials 101, 103, 201, & 301
fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833)
* fixing FedAvg averaging in order to be compatible with numpy v1.24+
Signed-off-by: kta-intel <kevin.ta@intel.com>
* uncommenting installations for consistency with other tutorials
Signed-off-by: kta-intel <kevin.ta@intel.com>
* fixing 301_MNIST_Watermarking tutorial FedAvg
Signed-off-by: kta-intel <kevin.ta@intel.com>
* fixing 301_MNIST_Watermarki
ng tutorial FedAvg
Signed-off-by: kta-intel <kevin.ta@intel.com>
* Switching to py38 kernel and clearing cell outputs
Signed-off-by: kta-intel <kevin.ta@intel.com>
---------
Signed-off-by: kta-intel <kevin.ta@intel.com>
---------
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* WIP: CI Scans (#873)
* Initial scans commit for bandit, hadolint, trivy
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Address bandit scan results
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix Trivy action
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix linting
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Add Coverity Badge
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update Hadolint threshold to flag errors only
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Update Hadolint threshold to flag errors only
Signed-off-by: Patrick Foley <psfoley@gmail.com>
---------
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Update ROADMAP.md (#878)
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* This commit brings following changes:
1. Renaming workspace_builder module to workspace_export.
2. WorkspaceBuilder class renamed to WorkspaceExport.
3. Added WorkspaceExport.export function for user which wraps generate_requirements, generate_plan_yaml, generate_data_yaml functions.
4. Merged WorkspaceExport module to `fx workspace create` command by adding new following arguments:
- `--notebook` Path to jupyter notebook
- `--template_workspace_dir` Output path for generated workspace
5. WorkspaceExport removed from 101, and 103 tutorials.
6. setup.py changed accordingly.
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* This commit brings following changes:
1. output_dir renamed to output_workspace also the user must pass this parameter
(default=/tmp is now removed) or there will be an error.
2. template_workspace_path is not required anymore.
3. Added a new tutorials Workflow_Interface_1001_Workspace_Creation_from_JupyterNotebook.ipynb
(copy of Workflow_Interface_301_MNIST_Watermarking.ipynb) explaining how to use WorkspaceExport.
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* In Workflow_Interface_1001 tutorial list of collaborator are changed to 5 from 2
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Uncommeted pip install libraries
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Adding GPU for aggregator in 201, Global DP & Privacy Meter tutorials
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Modifed documentation in 1001 tutorial
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Modified documentation in 1001 tutorial
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* Fixing 'param' attribute not found issue. in 301, 1001 experimental tutrials and 301
mnist watermarking workspace.
Sol.
imagen library installs param depedency latest version
and param package got an upgrde recently. Pinned param to 1.13.0
after installing imagen which fixed the issue.
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
* VFL 2 party example
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* participant names to lowercase and changes to instructions
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
* Resolving missing packages from watermarking examples that break python 3.9+. Fix linting, spelling. Remove experimental aggregation functions
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Remove duplicate files in openfl.experimental
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Trigger CI
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Moving test to be conditional
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix inconsistent use of enumerate in loops
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fixing tests
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Accomodate conversion of names to lower case
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Missing torchvision dependency in tests
Signed-off-by: Patrick Foley <psfoley@gmail.com>
* Fix lint issues
Signed-off-by: Patrick Foley <psfoley@gmail.com>
---------
Signed-off-by: Parth Mandaliya <parthx.mandaliya@intel.com>
Signed-off-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
Signed-off-by: Patrick Foley <psfoley@gmail.com>
Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com>
Signed-off-by: He, Dan H <dan.h.he@intel.com>
Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com>
Signed-off-by: Klemme, Beverly <beverly.klemme@intel.com>
Signed-off-by: Baker, Grant <grant.baker@intel.com>
Signed-off-by: ELizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com>
Signed-off-by: Jillela, Emmanuel <emmanuel.jillela@intel.com>
Signed-off-by: Beverly Klemme <beverly.klemme@intel.com>
Signed-off-by: Elizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com>
Signed-off-by: Emmanuel Jillela <emmanuel.jillela@intel.com>
Signed-off-by: Yi CAO <yi2.cao@intel.com>
Signed-off-by: Francis Storr <francis.storr@intel.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: kta-intel <kevin.ta@intel.com>
Signed-off-by: Mansi Sharma <mansi.sharma@intel.com>
Signed-off-by: Amit Portnoy <1131991+amitport@users.noreply.github.com>
Co-authored-by: Patrick Foley <patrick.foley@intel.com>
Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu>
Co-authored-by: Prashant Shah <40899779+SprashAI@users.noreply.github.com>
Co-authored-by: akantak <aleksander.kantak@intel.com>
Co-authored-by: wangleflex <106506636+wangleflex@users.noreply.github.com>
Co-authored-by: He, Dan H <dan.h.he@intel.com>
Co-authored-by: xiaoranf <desolatefang@gmail.com>
Co-authored-by: Wang, Wenjie <wenjie2.wang@intel.com>
Co-authored-by: Lei5 Chen <lei5.chen@intel.com>
Co-authored-by: Beverly Klemme <35578090+bjklemme-intel@users.noreply.github.com>
Co-authored-by: Emmanuel Jillela <emmanuel.jillela@intel.com>
Co-authored-by: Yi CAO <yi2.cao@intel.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Francis Storr <francis.storr@intel.com>
Co-authored-by: Patrick Foley <psfoley@gmail.com>
Co-authored-by: Kevin Ta <116312994+kta-intel@users.noreply.github.com>
Co-authored-by: Keerti Prakash Talwar <115972088+KeertiX@users.noreply.github.com>
Co-authored-by: kta-intel <kevin.ta@intel.com>
Co-authored-by: porteratzo <44075849+porteratzo@users.noreply.github.com>
Co-authored-by: Parth Mandaliya <parth.mandaliya.007@gmail.com>
Co-authored-by: Mansi Sharma <77758170+mansishr@users.noreply.github.com>
Co-authored-by: Mansi Sharma <mansi.sharma@intel.com>
Co-authored-by: Amit Portnoy <1131991+amitport@users.noreply.github.com>- Loading branch information
1 parent
87ebf9f
commit be66d40