Update README.md

README.md

@@ -80,53 +80,13 @@ Corrections or suggestions? Please file a [GitHub issue](https://github.com/jpha
 ### Lecture 4 Class Materials
 
 * [Lecture Notes](tex/lecture_4.pdf)
+* Software Examples:
+  * [Attacks for Red-teaming](https://colab.research.google.com/drive/1X1t1wqqVk8dlz1ubb0VBcLP_KFdP3HsE?usp=sharing)
+  * [Data Poisoning](https://colab.research.google.com/drive/13hs11eJAEsX3ZAHA6oH1Lmi4I-dH7d1G?usp=sharing)
+  * [Backdoor Attack](https://colab.research.google.com/drive/1QRCSW42L6wDs6ML9VgQu-xAbpdny4Mq1?usp=sharing)  
 * [Assignment 4](assignments/tex/assignment_4.pdf)
 * Reading: [_Machine Learning for High-Risk Applications_](https://pages.dataiku.com/oreilly-responsible-ai), Chapter 5 and Chapter 11
-
-### Lecture 4 Additional Software Tools
-
-* [adversarial-robustness-toolbox](https://oreil.ly/5eXYi)
-* [counterfit](https://oreil.ly/4WM4P)
-* [cleverhans](https://github.com/tensorflow/cleverhans)
-* [foolbox](https://github.com/bethgelab/foolbox)
-* [ml_privacy_meter](https://oreil.ly/HuHxf)
-* [NIST de-identification tools](https://oreil.ly/M8xhr)
-* [robustness](https://github.com/MadryLab/robustness)
-* [tensorflow/privacy](https://oreil.ly/hkurv)
-
-#### Lecture 4 Additional Software Examples
-
-* [Attacking a Machine Learning Model](https://nbviewer.jupyter.org/github/jphall663/GWU_rml/blob/master/lecture_4.ipynb)
-* _Machine Learning for High-risk Applications_: [Use Cases](https://oreil.ly/machine-learning-high-risk-apps-code) (Chapter 11)
-
-### Lecture 4 Additional Reading
-
-* **Introduction and Background**:
-
-  * [*A Marauder’s Map of Security and Privacy in Machine Learning*](https://arxiv.org/pdf/1811.01134.pdf)
-  * [BIML Interactive Machine Learning Risk Framework](https://berryvilleiml.com/interactive/)
-  * [FTC's "Start with Security" guidelines](https://oreil.ly/jmeja)
-  * [Mitre Adversarial Threat Matrix](https://github.com/mitre/advmlthreatmatrix)
-  * [NIST Computer Security Resource Center](https://oreil.ly/pncXb)
-  * [*The Security of Machine Learning*](https://people.eecs.berkeley.edu/~adj/publications/paper-files/SecML-MLJ2010.pdf)
-  * [*Proposals for model vulnerability and security*](https://www.oreilly.com/content/proposals-for-model-vulnerability-and-security/)
-
-* **Machine Learning Attacks and Countermeasures**:
-
-  * [*Membership Inference Attacks Against Machine Learning Models*](https://arxiv.org/pdf/1610.05820.pdf)
-  * [*Stealing Machine Learning Models via Prediction APIs*](https://arxiv.org/pdf/1609.02943.pdf)
-  * [*Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures*](https://dl.acm.org/doi/pdf/10.1145/2810103.2813677)
-  * [*Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifiers*](https://arxiv.org/pdf/1306.4447.pdf)
-  * [Robust ML](https://www.robust-ml.org/)  
-  * [*Sponge Examples: Energy-latency Attacks on Neural Networks*](https://arxiv.org/pdf/2006.03463.pdf)
-
-* **Examples of Real-world Attacks**:
-
-  * [Fraudsters Cloned Company Director’s Voice In $35 Million Heist, Police Find](https://www.forbes.com/sites/thomasbrewster/2021/10/14/huge-bank-fraud-uses-deep-fake-voice-tech-to-steal-millions/?sh=7f3ba4bd7559)
-  * [ISIS 'still evading detection on Facebook', report says](https://www.bbc.com/news/technology-53389657)
-  * [Researchers bypass airport and payment facial recognition systems using masks](https://www.engadget.com/2019-12-16-facial-recognition-fooled-masks.html)
-  * [Slight Street Sign Modifications Can Completely Fool Machine Learning Algorithms](https://spectrum.ieee.org/cars-that-think/transportation/sensors/slight-street-sign-modifications-can-fool-machine-learning-algorithms)
-  * [These students figured out their tests were graded by AI — and the easy way to cheat](https://www.theverge.com/2020/9/2/21419012/edgenuity-online-class-ai-grading-keyword-mashing-students-school-cheating-algorithm-glitch)
+* [Lecture 4 Additional Materials](additional_materials/am4.md)
 
 ***