- data privacy
- metric appropriate?
- data leakage?
- train-test-split reasonable?
- Interpretability
- fairness
- security
- reverse engineering
- integrity - adversaries (Adversarial Reprogramming of Neural Networks)
- Adversarial training
- Defensive distillation
- " It is clear that testing of naturally occurring inputs is sufficient for traditional machine learning applications, but verification of unusual inputs is necessary for security guarantees. We should verify, but so far we only know how to test."
- use
cleverhans
to test their models against standardized, state-of-the-art attacks - Poisoning training sets
- The IIA’s Artificial Intelligence Auditing Framework
- How to lie with Data Science – Towards Data Science