diff --git a/CHANGELOG.md b/CHANGELOG.md index 59c226e1..8be86b80 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,12 @@ # Changelog: + ### August 5, 2020 -* Experiment Class: Representation of an experiment as one or more configured System Models +##### [Experiments](https://github.com/cadCAD-org/cadCAD/blob/master/documentation/README.md#introduction) +* `cadCAD.configuration.Experiment` (Alpha) is in development and needed to be released to support the implementation of +web applications and proprietary feature extensions. It is intended to represent a unique identifier of an experiment of +one or more configured System Models. For this reason, `append_configs` is a method of +`cadCAD.configuration.Experiment`. As of now it does not support multi - system model simulation because configurations +are still appended globally despite `append_config` being a method of Experiment. ### June 22, 2020 * Bug Fix: Multiprocessing error for Windows @@ -8,26 +14,41 @@ ### June 19, 2020 ### New Features: -##### [Local Execution Mode](documentation/Simulation_Execution.md) -* Default parallelization of Monte-Carlo / Stochastic simulations -* **Backwards Compatible** with given legacy modes names - -##### [cadCAD Post-Processing Enhancements](https://github.com/cadCAD-org/cadCAD/blob/master/documentation/Simulation_Execution.md#execute-simulation-produce-system-event-dataset) -* Returns single dataset as three types depending on execution mode: - * Local Mode: - * 2d List -* Changes: - * Returning a single dataset was originally specified at the project’s inception instead of multiple per simulation -* [System Configuration Conversions](documentation/System_Configuration.md) - * System Configuration as List of Configuration Objects - * System Configuration as a Pandas DataFrame - * System Configuration as List of Dictionaries - -**Backwards compatibility:** -* Expandable state and policy update parameter space enables changes to the parameter space of updates while -supporting backwards compatibility -* Legacy execution modes supported +##### [Local Execution Mode (Default)](documentation/Simulation_Execution.md) +* Local Execution Mode (Default): Implicit parallelization of Monte-Carlo / Stochastic simulations (Automatically +selects Multi-Process / Threaded Mode if simulations are configure for a single run) + * **Backwards Compatibility:** `cadCAD.engine.ExecutionMode` accepts legacy execution modes from ver. `0.3.1` + +##### cadCAD Post-Processing Enhancements / Modifications +* [**Single Result Dataset**]((https://github.com/cadCAD-org/cadCAD/blob/master/documentation/Simulation_Execution.md#4-execute-simulation--produce-system-event-dataset)) as a 2 dimensional `list` + * Returns a single dataset instead of multiple datasets per Monte Carlo simulation as in `0.3.1`: + * New System Metrics as dataset attributes: + * **Simulation** (Alpha) is a unique identifier being developed to represent Experiments as stated above and + will be renamed accordingly + * **Subset** is a unique identifier of Monte Carlo simulations produced by parameter sweeps + * Note: Returning a single dataset was originally specified during the project’s inception instead of multiple per + simulation + +* The `configs` `list` has been temporarily flattened to contain single run System Model `Configuration` objects to +support elastic workloads. This functionality will be restored in a subsequent release by a class that returns +`configs`'s original representation in ver. `0.3.1`. + * The conversion utilities have been provided to restore its original representation of configurations with + runs >= 1 + * [System Configuration Conversions](documentation/System_Configuration.md) + * Configuration as List of Configuration Objects (as in ver. `0.3.1`) + * New: System Configuration as a Pandas DataFrame + * New: System Configuration as List of Dictionaries + + +##### Expandable state and policy update parameter space: +* Enables the development of feature enhancements that involve the use of additional parameters without requiring users +to modify their update parameters spaces when upgrading to newer versions. For this reason state / policy update +examples in documentation include an additional `**kwargs` parameter. + * [State Updates:](https://github.com/cadCAD-org/cadCAD/blob/master/documentation/README.md#state-update-functions) + * [Policy Updates:](https://github.com/cadCAD-org/cadCAD/blob/master/documentation/README.md#state-update-functions) + ### May 29, 2020 * Packaging: Add [Nix](https://nixos.org/) derivation and shell for local development and distribution of cadCAD package -using Nix. Nix is a powerful package manager for Linux and other Unix systems that makes package management reliable and reproducible, allowing you to share your development and build environments across different machines. +using Nix. Nix is a powerful package manager for Linux and other Unix systems that makes package management reliable and +reproducible, allowing you to share your development and build environments across different machines. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7d1356de..5ef7aa90 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -11,26 +11,35 @@ Pull Request (PR) presented as "->". General Template: -`user/branch -> BlockScience/staging` +`user:branch -> org:staging` Contributing a new feature: -`user/feature -> BlockScience/staging` +`user:feature -> org:staging` Contributing to an existing feature: -`user/feature -> BlockScience/feature` +`user:feature -> org:feature` ### General Advise for Forked Repositories: 1. `git pull fork staging` 2. `git checkout -b feature` (new feature) 3. Apply your awesomeness! (Commit Often) 4. `git push fork feature` -5. Apply a merge strategy you're comfortable with. -6. Submit PR from `user:staging` into `BlockScience:staging` +5. Apply a rebase/merge strategy you're comfortable with (Recommended Below). +6. Submit PR from `user:staging` into `org:staging` 7. PR is queued for review 8. PR Reviewed (Update necessary if rejected) 9. PR Approved (There may be circumstances delaying the merge.) -10. Your contribution merged into next feature release on `BlockScience:master` +10. Your contribution merged into next feature release on `org:master` + +### Recommended Strategy: [Rebase](https://git-scm.com/book/en/v2/Git-Branching-Rebasing) +1. Add cadCAD-org/cadCAD as remote within you forked project locally. +2. `git checkout remote/master` +3. `git pull remote master` +4. `git checkout your_branch` +5. `git rebase master` +6. Resolve merge conflicts (while leveraging rebase commands) +7. `git push fork your_branch` Thanks! :heart: diff --git a/README.md b/README.md index 7fbb7c15..221c3b58 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ through simulation, with support for Monte Carlo methods, A/B testing and parame #### Change Log: [ver. 0.4.21](CHANGELOG.md) -[Previous Stable Release (No Longer Supported)](https://github.com/cadCAD-org/cadCAD/tree/b9cc6b2e4af15d6361d60d6ec059246ab8fbf6da) +[Previous Stable Release (Deprecated / No Longer Supported)](https://github.com/cadCAD-org/cadCAD/tree/b9cc6b2e4af15d6361d60d6ec059246ab8fbf6da) ## 1. Installation: diff --git a/cadCAD/utils/sys_exec.py b/cadCAD/utils/sys_exec.py index 7c7559df..4895eef4 100644 --- a/cadCAD/utils/sys_exec.py +++ b/cadCAD/utils/sys_exec.py @@ -1,8 +1,8 @@ import warnings from pprint import pprint -from pyspark import RDD, Row -from pyspark.sql import DataFrame, SparkSession +from pyspark import RDD +from pyspark.sql import DataFrame, SparkSession, Row import pandas as pd # Distributed diff --git a/documentation/README.md b/documentation/README.md index d94aed94..97f46df9 100644 --- a/documentation/README.md +++ b/documentation/README.md @@ -9,10 +9,17 @@ system is described by a set of [State Variables](#State-Variables). The dynamic [Policy Functions](#Policy-Functions) and [State Update Functions](#State-Update-Functions), which are evaluated by cadCAD according to the definitions set by the user in [Partial State Update Blocks](#Partial-State-Update-Blocks). -A Simulation Configuration is comprised of a [System Model](#System-Model) and a set of -[Simulation Properties](#Simulation-Properties). +A Simulation Configuration is comprised of a [System Model](#System-Model) and a set of [Simulation Properties](#Simulation-Properties). -`Experiment`'s `append_configs`, stores a **Simulation Configuration** to be [Executed](Simulation_Execution.md) by cadCAD +### Experiments +`cadCAD.configuration.Experiment` (Alpha) is in development and needed to be released to support the development of web +applications and proprietary feature extensions to be [Executed](Simulation_Execution.md) by cadCAD. + +It is intended to represent a unique identifier of an experiment of one or more configured System Models / +Configurations. For this reason, `append_configs` is a method of `Experiment`. + +As of now it does not support multi - system model simulation because configurations are still appended globally despite +`append_config` being a method of `Experiment`. ```python from cadCAD.configuration import Experiment @@ -26,7 +33,7 @@ exp.append_configs( sim_configs = ... # Simulation Properties ) ``` -Parameters: +Parameters: `append_configs` * **user_id** : str - OPTIONAL: cadCAD Session User ID * **initial_state** : _dict_ - [State Variables](#State-Variables) and their initial values * **partial_state_update_blocks** : List[dict[dict]] - List of [Partial State Update Blocks](#Partial-State-Update-Blocks) diff --git a/documentation/System_Configuration.md b/documentation/System_Configuration.md index 3bb2bbab..2ce967a5 100644 --- a/documentation/System_Configuration.md +++ b/documentation/System_Configuration.md @@ -1,5 +1,4 @@ -# [Display System Model Configurations:](https://github.com/BlockScience/distroduce/blob/master/documentation/dist_exec_doc.ipynb) - +# Display System Model Configurations: ## Conversions ##### Note: The following applies as a result of simulation execution diff --git a/simulations/regression_tests/experiments/__init__.py b/simulations/regression_tests/experiments/__init__.py index 6d4cf830..d66d5f56 100644 --- a/simulations/regression_tests/experiments/__init__.py +++ b/simulations/regression_tests/experiments/__init__.py @@ -8,3 +8,4 @@ sweep_exp = Experiment() udo1_exp = Experiment() udo2_exp = Experiment() +poc = Experiment() \ No newline at end of file