diff --git a/README.md b/README.md index f6aec83..e21da5d 100644 --- a/README.md +++ b/README.md @@ -33,13 +33,28 @@ Corrections or suggestions? Please file a [GitHub issue](https://github.com/jpha ### Lecture 1 Additional Software Tools -* Python [PiML-Toolbox](https://github.com/SelfExplainML/PiML-Toolbox) -* Python [explainable boosting machine (EBM)/GA2M](https://github.com/interpretml/interpret) -* R [`gam`](https://cran.r-project.org/web/packages/gam/index.html) -* `h2o` [penalized GLM](https://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/glm.html) (R and Python) -* Monotonic gradient boosting machine (GBM): [`h2o`](https://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/algo-params/monotone_constraints.html) and [`xgboost`](https://xgboost.readthedocs.io/en/latest/tutorials/monotonic.html) (R and Python) -* R [`rpart`](https://cran.r-project.org/web/packages/rpart/index.html) -* Python [`skope-rules`](https://github.com/scikit-learn-contrib/skope-rules) +* **Python**: + * [causalml](https://oreil.ly/XsiMk) + * [interpret](https://github.com/interpretml/interpret) + * [imodels](https://oreil.ly/coPjR) + * [PiML-Toolbox](https://github.com/SelfExplainML/PiML-Toolbox) + * [sklearn-expertsys](https://oreil.ly/igFz6) + * [skope-rules](https://github.com/scikit-learn-contrib/skope-rules) + * [tensorflow/lattice](https://oreil.ly/Z9iCS) + +* **R**: + * [arules](https://oreil.ly/bBv9s) + * [elasticnet](https://oreil.ly/pBOBN) + * [gam](https://cran.r-project.org/web/packages/gam/index.html) + * [glmnet](https://oreil.ly/rMzEl) + * [quantreg](https://oreil.ly/qBWk9) + * [rpart](https://cran.r-project.org/web/packages/rpart/index.html) + * [RuleFit](https://oreil.ly/K-qc4) + +* **Python, R or other**: + * [h2o-3](https://oreil.ly/PPUk5) + * [Rudin Group code](https://oreil.ly/QmRFF) + * [xgboost](https://github.com/dmlc/xgboost) ### Lecture 1 Additional Software Examples @@ -77,35 +92,63 @@ Corrections or suggestions? Please file a [GitHub issue](https://github.com/jpha ### Lecture 2 Class Materials * [Lecture Notes](tex/lecture_2.pdf) -* Software Examples: +* [Assignment 2](assignments/tex/assignment_2.pdf) +* Reading: [_Machine Learning for High-Risk Applications_](https://pages.dataiku.com/oreilly-responsible-ai), Chapter 2 (pp. 50 - 80) and Chapter 6 (pp. 208 - 230) + +### Lecture 2 Additional Software Tools + +* **Python**: + * [allennlp](https://github.com/allenai/allennlp) + * [alibi](https://github.com/SeldonIO/alibi) + * [anchor](https://oreil.ly/K3UuW) + * [DiCE](https://oreil.ly/-lwV4) + * [interpret](https://github.com/interpretml/interpret) + * [lime](https://oreil.ly/j5Cqj) + * [shap](https://github.com/slundberg/shap) + * [PiML-Toolbox](https://github.com/SelfExplainML/PiML-Toolbox) + * [tf-explain](https://github.com/sicara/tf-explain) + +* **R**: + * [ALEPlot](https://oreil.ly/OSfUT) + * [DALEX](https://cran.r-project.org/web/packages/DALEX/index.html) + * [ICEbox](https://oreil.ly/6nl1W) + * [iml](https://cran.r-project.org/web/packages/iml/index.html) + * [Model Oriented](https://oreil.ly/7wUMp) + * [pdp](https://oreil.ly/PasMQ) + * [shapFlex](https://oreil.ly/RADtC) + * [vip](https://oreil.ly/YcD2_) + +* **Python, R or other**: + * [h2o-3](https://oreil.ly/GtGvK) + +### Lecture 2 Additional Software Examples: * [Global and Local Explanations of a Constrained Model](https://nbviewer.jupyter.org/github/jphall663/GWU_rml/blob/master/lecture_2.ipynb) * [Building from Penalized GLM to Monotonic GBM](https://nbviewer.org/github/jphall663/interpretable_machine_learning_with_python/blob/master/glm_mgbm_gbm.ipynb?flush_cache=true) * [Monotonic XGBoost models, partial dependence, individual conditional expectation plots, and Shapley explanations](https://nbviewer.org/github/jphall663/interpretable_machine_learning_with_python/blob/master/xgboost_pdp_ice.ipynb) * [Decision tree surrogates, LOCO, and ensembles of explanations](https://nbviewer.org/github/jphall663/interpretable_machine_learning_with_python/blob/master/dt_surrogate_loco.ipynb) -* [Assignment 2](https://raw.githubusercontent.com/jphall663/GWU_rml/master/assignments/tex/assignment_2.pdf) - -### Lecture 2 Suggested Software - -* Python: - * [`allennlp`](https://github.com/allenai/allennlp) - * [`alibi`](https://github.com/SeldonIO/alibi) - * [`interpret`](https://github.com/interpretml/interpret) - * [`shap`](https://github.com/slundberg/shap) - * [`PiML-Toolbox`](https://github.com/SelfExplainML/PiML-Toolbox) - * [`tf-explain`](https://github.com/sicara/tf-explain) -* R: - * [`DALEX`](https://cran.r-project.org/web/packages/DALEX/index.html) - * [`iml`](https://cran.r-project.org/web/packages/iml/index.html) -### Lecture 2 Suggested Reading +### Lecture 2 Additional Reading * **Introduction and Background**: + * [*On the Art and Science of Explainable Machine Learning*](https://oreil.ly/myVr8) * [*Proposed Guidelines for the Responsible Use of Explainable Machine Learning*](https://arxiv.org/pdf/1906.03533.pdf) - * [**Responsible Artificial Intelligence**](https://www.springer.com/gp/book/9783030303709) - Chapter 4 * **Post-hoc Explanation Techniques**: + * [_A Unified Approach to Interpreting Model Predictions_](https://papers.nips.cc/paper_files/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf) + * [_Anchors: High-Precision Model-Agnostic Explanations_](https://homes.cs.washington.edu/~marcotcr/aaai18.pdf) + * **Elements of Statistical Learning** - [Section 10.13](https://hastie.su.domains/ElemStatLearn/printings/ESLII_print12_toc.pdf) + * [_Extracting Tree-Structured Representations of Trained Networks_](https://proceedings.neurips.cc/paper/1995/file/45f31d16b1058d586fc3be7207b58053-Paper.pdf) + * [_Interpretability via Model Extraction_](https://arxiv.org/pdf/1706.09773.pdf) * **Interpretable Machine Learning** - [Chapter 6](https://christophm.github.io/interpretable-ml-book/agnostic.html) and [Chapter 7](https://christophm.github.io/interpretable-ml-book/example-based.html) + * [_Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation_](https://arxiv.org/pdf/1309.6392.pdf) * [*Towards Better Understanding of Gradient-based Attribution Methods for Deep Neural Networks*](https://arxiv.org/pdf/1711.06104.pdf) + * [_Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models_](https://arxiv.org/pdf/1612.08468.pdf) + * [_“Why Should I Trust You?” Explaining the Predictions of Any Classifier_](https://www.kdd.org/kdd2016/papers/files/rfp0573-ribeiroA.pdf) + +* **Problems with Psot-hoc Explanation**: + * [*General Pitfalls of Model-Agnostic Interpretation Methods*](link:https://oreil.ly/On9uS) + * [_Limitations of Interpretable Machine Learning Methods_](https://oreil.ly/VHMWh) + * [*When Not to Trust Your Explanations*](https://oreil.ly/9Oxa6) ***