-
-
Notifications
You must be signed in to change notification settings - Fork 2
Development Timeline
- Communicate on the expected results and discuss some details.
- Remain in constant touch with my mentors using Zoom and Google Docs.
- Settle the final design principle according to user requirements and feasibility.
- Set up the development environment and job management board for TODO list and weekly report.
Create the documentation and set up the infrastructure of the R package. Update the wiki page and the project board. Research on the other similar fairness packages and come up with a list of some metrics need to be implemented first. Process two toy datasets (COMPAS and Adult) and merge it to mlr3 infrastructure.
Review the codes of mlr3 Metric Class and mlr3measures. Set up the design principle of fairness metrics. Create MetricFairness and defined the first Metric (Groupwise False Positive Bias) and its documentation.
Implement popular fairness metrics by connecting to the fairness package or AI fairness 360. Design and adjust the visualization and metrics of other packages. Implement visualizations for auditing.
Define a clean API for fairness auditing in mlr3. Create the corresponding documentations and demos for better usage.
Midterm Evaluations:
The delivered results should be the measure class of sub-group level prediction performances. Which contain the fairness metrics that are helpful for the users to evaluate the fairness of datasets. And the visualizations and API with documentation and demos for auditing.
Implement debiasing strategies as pre or post processing PipeOperators in the style of the mlr3 pipelines package. One of the possible approaches would be inherited from AIF360. Might research and reimplement other published debiasing algorithms as needed if there is extra time.
Create an introduction vignette and demos for debiasing algorithms to showcase the new package. If the progress is left behind, that time could be used to catch up the unfinished progress. Otherwise this time could be used to add extra features as required or checking the coding style and design principles.
Finishing the package, create the pull request and release it on github. Check the code style and design principles.
Final Evaluations:
The delivered results should be the debiasing algorithms in mlr3 PipeOperators. With clear documentation and showcases. Some examples about how to use the mlr3 ecosystem to detect the fairness problems of models or datasets, and use debiasing algorithms to correct such biases.