Update README.md

README.md

@@ -33,13 +33,28 @@ Corrections or suggestions? Please file a [GitHub issue](https://github.com/jpha
 
 ### Lecture 1 Additional Software Tools
 
-* Python [PiML-Toolbox](https://github.com/SelfExplainML/PiML-Toolbox)
-* Python [explainable boosting machine (EBM)/GA2M](https://github.com/interpretml/interpret)
-* R [`gam`](https://cran.r-project.org/web/packages/gam/index.html)
-* `h2o` [penalized GLM](https://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/glm.html) (R and Python)
-* Monotonic gradient boosting machine (GBM): [`h2o`](https://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/algo-params/monotone_constraints.html) and [`xgboost`](https://xgboost.readthedocs.io/en/latest/tutorials/monotonic.html) (R and Python)
-* R [`rpart`](https://cran.r-project.org/web/packages/rpart/index.html)
-* Python [`skope-rules`](https://github.com/scikit-learn-contrib/skope-rules)
+* **Python**:
+  * [causalml](https://oreil.ly/XsiMk)
+  * [interpret](https://github.com/interpretml/interpret)
+  * [imodels](https://oreil.ly/coPjR)
+  * [PiML-Toolbox](https://github.com/SelfExplainML/PiML-Toolbox)
+  * [sklearn-expertsys](https://oreil.ly/igFz6)
+  * [skope-rules](https://github.com/scikit-learn-contrib/skope-rules)
+  * [tensorflow/lattice](https://oreil.ly/Z9iCS)
+
+* **R**:
+  * [arules](https://oreil.ly/bBv9s)
+  * [elasticnet](https://oreil.ly/pBOBN)
+  * [gam](https://cran.r-project.org/web/packages/gam/index.html)
+  * [glmnet](https://oreil.ly/rMzEl)
+  * [quantreg](https://oreil.ly/qBWk9)
+  * [rpart](https://cran.r-project.org/web/packages/rpart/index.html)
+  * [RuleFit](https://oreil.ly/K-qc4)
+
+* **Python, R or other**: 
+  * [h2o-3](https://oreil.ly/PPUk5)
+  * [Rudin Group code](https://oreil.ly/QmRFF)
+  * [xgboost](https://github.com/dmlc/xgboost)
 
 ### Lecture 1 Additional Software Examples
 
@@ -77,35 +92,63 @@ Corrections or suggestions? Please file a [GitHub issue](https://github.com/jpha
 ### Lecture 2 Class Materials
 
 * [Lecture Notes](tex/lecture_2.pdf) 
-* Software Examples: 
+* [Assignment 2](assignments/tex/assignment_2.pdf) 
+* Reading: [_Machine Learning for High-Risk Applications_](https://pages.dataiku.com/oreilly-responsible-ai), Chapter 2 (pp. 50 - 80) and Chapter 6 (pp. 208 - 230) 
+
+### Lecture 2 Additional Software Tools
+
+* **Python**:
+  * [allennlp](https://github.com/allenai/allennlp)
+  * [alibi](https://github.com/SeldonIO/alibi)
+  * [anchor](https://oreil.ly/K3UuW)
+  * [DiCE](https://oreil.ly/-lwV4)
+  * [interpret](https://github.com/interpretml/interpret)
+  * [lime](https://oreil.ly/j5Cqj)
+  * [shap](https://github.com/slundberg/shap)
+  * [PiML-Toolbox](https://github.com/SelfExplainML/PiML-Toolbox)
+  * [tf-explain](https://github.com/sicara/tf-explain)
+
+* **R**:
+  * [ALEPlot](https://oreil.ly/OSfUT)
+  * [DALEX](https://cran.r-project.org/web/packages/DALEX/index.html)
+  * [ICEbox](https://oreil.ly/6nl1W)
+  * [iml](https://cran.r-project.org/web/packages/iml/index.html)
+  * [Model Oriented](https://oreil.ly/7wUMp)
+  * [pdp](https://oreil.ly/PasMQ)
+  * [shapFlex](https://oreil.ly/RADtC)
+  * [vip](https://oreil.ly/YcD2_)
+
+* **Python, R or other**:
+  * [h2o-3](https://oreil.ly/GtGvK)
+
+### Lecture 2 Additional Software Examples: 
   * [Global and Local Explanations of a Constrained Model](https://nbviewer.jupyter.org/github/jphall663/GWU_rml/blob/master/lecture_2.ipynb) 
   * [Building from Penalized GLM to Monotonic GBM](https://nbviewer.org/github/jphall663/interpretable_machine_learning_with_python/blob/master/glm_mgbm_gbm.ipynb?flush_cache=true)
   * [Monotonic XGBoost models, partial dependence, individual conditional expectation plots, and Shapley explanations](https://nbviewer.org/github/jphall663/interpretable_machine_learning_with_python/blob/master/xgboost_pdp_ice.ipynb)
   * [Decision tree surrogates, LOCO, and ensembles of explanations](https://nbviewer.org/github/jphall663/interpretable_machine_learning_with_python/blob/master/dt_surrogate_loco.ipynb)
-* [Assignment 2](https://raw.githubusercontent.com/jphall663/GWU_rml/master/assignments/tex/assignment_2.pdf) 
-
-### Lecture 2 Suggested Software
-
-* Python:
-  * [`allennlp`](https://github.com/allenai/allennlp)
-  * [`alibi`](https://github.com/SeldonIO/alibi)
-  * [`interpret`](https://github.com/interpretml/interpret)
-  * [`shap`](https://github.com/slundberg/shap)
-  * [`PiML-Toolbox`](https://github.com/SelfExplainML/PiML-Toolbox)
-  * [`tf-explain`](https://github.com/sicara/tf-explain)
-* R:
-  * [`DALEX`](https://cran.r-project.org/web/packages/DALEX/index.html)
-  * [`iml`](https://cran.r-project.org/web/packages/iml/index.html)
 
-### Lecture 2 Suggested Reading
+### Lecture 2 Additional Reading
 
 * **Introduction and Background**: 
+  * [*On the Art and Science of Explainable Machine Learning*](https://oreil.ly/myVr8)
   * [*Proposed Guidelines for the Responsible Use of Explainable Machine Learning*](https://arxiv.org/pdf/1906.03533.pdf)
-  * [**Responsible Artificial Intelligence**](https://www.springer.com/gp/book/9783030303709) - Chapter 4
 
 * **Post-hoc Explanation Techniques**:
+  * [_A Unified Approach to Interpreting Model Predictions_](https://papers.nips.cc/paper_files/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf)
+  * [_Anchors: High-Precision Model-Agnostic Explanations_](https://homes.cs.washington.edu/~marcotcr/aaai18.pdf)
+  * **Elements of Statistical Learning** - [Section 10.13](https://hastie.su.domains/ElemStatLearn/printings/ESLII_print12_toc.pdf)
+  * [_Extracting Tree-Structured Representations of Trained Networks_](https://proceedings.neurips.cc/paper/1995/file/45f31d16b1058d586fc3be7207b58053-Paper.pdf)
+  * [_Interpretability via Model Extraction_](https://arxiv.org/pdf/1706.09773.pdf)
   * **Interpretable Machine Learning** - [Chapter 6](https://christophm.github.io/interpretable-ml-book/agnostic.html) and [Chapter 7](https://christophm.github.io/interpretable-ml-book/example-based.html)
+  * [_Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation_](https://arxiv.org/pdf/1309.6392.pdf)
   * [*Towards Better Understanding of Gradient-based Attribution Methods for Deep Neural Networks*](https://arxiv.org/pdf/1711.06104.pdf)
+  * [_Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models_](https://arxiv.org/pdf/1612.08468.pdf)
+  * [_“Why Should I Trust You?” Explaining the Predictions of Any Classifier_](https://www.kdd.org/kdd2016/papers/files/rfp0573-ribeiroA.pdf)
+
+* **Problems with Psot-hoc Explanation**:
+  * [*General Pitfalls of Model-Agnostic Interpretation Methods*](link:https://oreil.ly/On9uS)
+  * [_Limitations of Interpretable Machine Learning Methods_](https://oreil.ly/VHMWh)
+  * [*When Not to Trust Your Explanations*](https://oreil.ly/9Oxa6)
 
 ***