📒
📒
📒
📒
Machine & Deep Learning Compendium
Search…
The Machine & Deep Learning Compendium
Types Of Machine Learning
Data Science
Data Science Tools
Management
Data Science Management
Calculus
Probability & Statistics
Probability
Feature Types
Features
Calibration
Multi Label Classification
Distribution
Distribution Transformation
Information Theory
Game Theory
Datasets
Dataset Confidence
Normalization & Scaling
Regularization
Validation
Meta Learning
Evaluation Metrics
Benchmarking
Hyper Parameter Optimization
Multi CPU Processing
Algorithms 101
Training Strategies
Classic Machine Learning
Label Algorithms
Clustering Algorithms
Anomaly Detection
Decision Trees
Active Learning Algorithms
Linear Separator Algorithms
Ensembles
Reinforcement Learning
Incremental Learning
Dimensionality Reduction Methods
Genetic Algorithms & Genetic Programming
Learning Classifier Systems
Recommender Systems
Timeseries
Fourier Transform
Digital Signal Processing (DSP)
Propensity Score Matching
Natural Language Processing
Graphs
Deep Learning
Experimental Design
Product
Business Domains For Data Science
MLOPs Engineering
Model Formats
Data Engineering
Powered By GitBook
Validation

DATASETS RELIABILITY & CORRECTNESS

1. Clever Hans effect - in relations to cues left in the dataset that models find, instead of actually solving the defined task!
  • Ablating, i.e. removing, part of a model and observing the impact this has on performance is a common method for verifying that the part in question is useful. If performance doesn't go down, then the part is useless and should be removed. Carrying this method over to datasets, it should become common practice to perform dataset ablations, as well, for example:
  • Provide only incomplete input (as done in the reviewed paper): This verifies that the complete input is required. If not, the dataset contains cues that allow taking shortcuts.
  • Shuffle the input: This verifies the importance of word (or sentence) order. If a bag-of-words/sentences gives similar results, even though the task requires sequential reasoning, then the model has not learned sequential reasoning and the dataset contains cues that allow the model to "solve" the task without it.
  • Assign random labels: How much does performance drop if ten percent of instances are relabeled randomly? How much with all random labels? If scores don't change much, the model probably didn't learning anything interesting about the task.
  • Randomly replace content words: How much does performance drop if all noun phrases and/or verb phrases are replaced with random noun phrases and verbs? If not much, the dataset may provide unintended non-content cues, such as sentence length or distribution of function words.

UNIT / DATA TESTS

  1. 1.
    A great :P unit test and logging post on medium - it’s actually mine :)
  2. 2.
    A mind blowing lecture about unit testing your data using Voluptuous & engrade & TDDA lecture
  3. 3.
    ​Great expectations, article, “TDDA” for Unit tests and CI, Youtube​
  4. 4.
    ​DataProfiler git​
  5. 5.
    ​Unit tests in python​
  6. 6.
    ​Unit tests in python - youtube​
  7. 7.
    ​Unit tests asserts​
  8. 8.
    ​Auger - automatic unit tests, has a blog post inside, doesn't work with py 3+
  9. 9.
    ​A rather naive unit tests article aimed for DS​
  10. 10.
    A good pytest tutorial​
  11. 11.
    ​Mock, mock 2​

REGULATION FOR AI

  1. 1.
    ​Preparing for EU regulations by MonaLabs
  2. 2.
    ​EU regulation DOC​
  3. 3.
    ​EIOPA - regulation for insurance companies.
  4. 4.
    Ethics and regulations in Israel
    1. 1.
      ​First Report by the intelligence committee headed by prof. Itzik ben israel and prof. evyatar matanya
    2. 2.
      ​Second report by AI and data science committee ​
    3. 3.
      Third by meizam leumi for AI systems in ethics and regulation in israel, lecture​

FAIRNESS, ACCOUNTABILITY & TRANSPARENCY ML

  1. 1.
    FATML website - The past few years have seen growing recognition that machine learning raises novel challenges for ensuring non-discrimination, due process, and understandability in decision-making. In particular, policymakers, regulators, and advocates have expressed fears about the potentially discriminatory impact of machine learning, with many calling for further technical research into the dangers of inadvertently encoding bias into automated decisions.
At the same time, there is increasing alarm that the complexity of machine learning may reduce the justification for consequential decisions to “the algorithm made me do it.”
  1. 1.
    ​Principles and best practices, projects​
  2. 2.
    ​FAccT - A computer science conference with a cross-disciplinary focus that brings together researchers and practitioners interested in fairness, accountability, and transparency in socio-technical systems.
  3. 3.
    ​Paper - there is no fairness, enforcing fairness can improve accuracy​
  4. 4.
    ​Google on responsible ai practices see also PAIR
  5. 5.
    ​Bengio on ai​
  6. 6.
    ​Poisoning attacks on fairness - Research in adversarial machine learning has shown how the performance of machine learning models can be seriously compromised by injecting even a small fraction of poisoning points into the training data. We empirically show that our attack is effective not only in the white-box setting, in which the attacker has full access to the target model, but also in a more challenging black-box scenario in which the attacks are optimized against a substitute model and then transferred to the target model
  7. 7.
    A series of articles about Bias & Fairness by Johnathan Hui
    1. 1.
      ​In Clinical research - Selection , Sample , Time , Attrition , Survivorship, reporting, funding, citation, Volunteer , self-selection , non-response, pre-screening , healthy person, membership, ascertainment, performance, berkson admission, neyman, measurement, observer, expectation, response, self reporting, social desirability, recall, acquiescence agreement, leading, courtesy, attention verification, lead time, immortal time, misclassification, chronological, detection, spectrum, cofounder, susceptibility, collider, simpson, ommited, allocation, channeling.
    2. 2.
      ​AI - known cases in Vision, NLP - sentiment, embedding, language models, historical, compass, recommender, datasets.
    3. 3.
      ​address AI Bias with Fairness criteria and tools - per population, predictive parity, calibration by group
    4. 4.
      ​Caveats and limitations of AI Fairness Approaches - sample bias, label bias, miscalibration outcome test, redlining, etc.
    5. 5.
      ​AI Fairness Approaches - statistical fairness, equalizing acceptance rate, error rate, etc.

FAIRNESS TOOLS

  1. 1.
    ​PII tools, by gensim​
  2. 2.
    ​Fair-learn A Python package to assess and improve fairness of machine learning models.
    ​
  3. 3.
    ​Sk-lego​
  1. 1.
    Regression
    ​
  2. 2.
    classification
    ​
  3. 3.
    ​
    ​
  4. 4.
    ​
    ​
  5. 5.
    ​
    ​
  6. 6.
    information filter
    ​
M. Zafar et al. (2017), Fairness Constraints: Mechanisms for Fair Classification
M. Hardt, E. Price and N. Srebro (2016), Equality of Opportunity in Supervised Learning

​

INTERPRETABLE / EXPLAINABLE AI (XAI)

  1. 1.
    ​A curated document about XAI research resources. ​
  2. 2.
    Interpretability and Explainability in Machine Learning course / slides. Understanding, evaluating, rule based, prototype based, risk scores, generalized additive models, explaining black box, visualizing, feature importance, actionable explanations, casual models, human in the loop, connection with debugging.
  3. 3.
    ​Explainable Machine Learning: Understanding the Limits & Pushing the Boundaries a tutorial by Hima Lakkaraju (tutorial VIDEO, youtube, twitter)
    ​
  4. 4.
    ​Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead by Cinthia rudin
    1. 1.
      A great talk on the topic by Shir Meir Lador
  5. 5.
    ​explainML tutorial​
  6. 6.
    ​When not to trust explanations :)​
  7. 7.
    From the above image: Paper: Principles and practice of explainable models - a really good review for everything XAI - “a survey to help industry practitioners (but also data scientists more broadly) understand the field of explainable machine learning better and apply the right tools. Our latter sections build a narrative around a putative data scientist, and discuss how she might go about explaining her models by asking the right questions. From an organization viewpoint, after motivating the area broadly, we discuss the main developments, including the principles that allow us to study transparent models vs opaque models, as well as model-specific or model-agnostic post-hoc explainability approaches. We also briefly reflect on deep learning models, and conclude with a discussion about future research directions.”
  8. 8.
    ​Book: interpretable machine learning, christoph mulner​
  9. 9.
    (great) Interpretability overview, transparent (simultability, decomposability, algorithmic transparency) post-hoc interpretability (text explanation, visual local, explanation by example,), evaluation, utility.
  10. 10.
    ​Medium: the great debate​
  1. 1.
    ​Paper: pitfalls to avoid when interpreting ML models “y. A growing number of techniques provide model interpretations, but can lead to wrong conclusions if applied incorrectly. We illustrate pitfalls of ML model interpretation such as bad model generalization, dependent features, feature interactions or unjustified causal interpretations. Our paper addresses ML practitioners by raising awareness of pitfalls and pointing out solutions for correct model interpretation, as well as ML researchers by discussing open issues for further research.” - mulner et al.
  1. 1.
    *** whitening a black box. This is very good, includes eli5, lime, shap, many others.
  2. 2.
    Book: exploratory model analysis
  3. 3.
    ​Alibi-explain - White-box and black-box ML model explanation library. Alibi is an open source Python library aimed at machine learning model inspection and interpretation. The focus of the library is to provide high-quality implementations of black-box, white-box, local and global explanation methods for classification and regression models.
  1. 1.
    ​Hands on explainable ai youtube, git​
  2. 2.
    ​Explainable methods are not always consistent and do not agree with each other, this article has a make-sense explanation and flow for using shap and its many plots.
  1. 1.
    Intro to shap and lime, part 1, part 2​
  2. 2.
    Lime
    1. 1.
      ​*** how lime works behind the scenes​
    2. 2.
      ​LIME to interpret models NLP and IMAGE, github- In the experiments in our research paper, we demonstrate that both machine learning experts and lay users greatly benefit from explanations similar to Figures 5 and 6 and are able to choose which models generalize better, improve models by changing them, and get crucial insights into the models' behavior.
  3. 3.
    Anchor
    1. 1.
      ​Anchor from the authors of Lime, - An anchor explanation is a rule that sufficiently “anchors” the prediction locally – such that changes to the rest of the feature values of the instance do not matter. In other words, for instances on which the anchor holds, the prediction is (almost) always the same.
  4. 4.
    Shap:
    1. 1.
      Medium Intro to lime and shap​
    2. 2.
      **** In depth SHAP​
    3. 3.
      ​Github​
    4. 4.
      ​Country happiness using shap​
    5. 5.
      ​Stackoverflow example, predicting tags, pandas keras etc​
    6. 6.
      ​Intro to shapely and shap​
    7. 7.
      ​Fiddler on shap​
  5. 5.
    Shapash
    1. 1.
      ​shapash git - ​a web app (lime and shap). ​
    2. 2.
      ​making models understandable by everyone - Yann Golhen
    3. 3.
      ​using shapash for confidence on XAI. - francesco marini using 3 new metrics
      1. 1.
        Consistency - do different explainability methods give, on average, similar explanations?
      2. 2.
        Stability - for similar instances, are the explanations similar?
      3. 3.
        Compacity - do fewer features drive the model?
  6. 6.
    Partial Shap
    1. 1.
      ​Which Of Your Features Are Overfitting? by Samuele Mazzanti - "Discover “ParShap”: an advanced method to detect which columns make your model underperform on new data" implemented in pingouin-stats.
  7. 7.
    SHAP advanced
    1. 1.
      ​Official shap tutorial on their plots, you can never read this too many times.​
    2. 2.
      ​What are shap values on kaggle - whatever you do start with this
    3. 3.
      ​Shap values on kaggle #2 - continue with this
    4. 4.
      How to calculate Shap values per class based on this graph
  1. 1.
    Shap intro, part 2 with many algo examples and an explanation about the four plots.
  2. 2.
    ​A thorough post about the many ways of explaining a model, from regression, to bayes, to trees, forests, lime, beta, feature selection/elimination​
  3. 3.
    ​Trusting models​
  4. 4.
    ​3. Interpret using uncertainty​
  5. 5.
    ​Keras-vis for cnns, 3 methods, activation maximization, saliency and class activation maps
  6. 6.
    ​The notebook! Blog​
  7. 7.
    ​More resources!​
  8. 8.
    ​Visualizing the impact of feature attribution baseline - Path attribution methods are a gradient-based way of explaining deep models. These methods require choosing a hyperparameter known as the baseline input. What does this hyperparameter mean, and how important is it? In this article, we investigate these questions using image classification networks as a case study. We discuss several different ways to choose a baseline input and the assumptions that are implicit in each baseline. Although we focus here on path attribution methods, our discussion of baselines is closely connected with the concept of missingness in the feature space - a concept that is critical to interpretability research.
  9. 9.
    WHAT IF TOOL - GOOGLE, notebook, walkthrough​
  10. 10.
    ​Language interpretability tool (LIT) - The Language Interpretability Tool (LIT) is an open-source platform for visualization and understanding of NLP models.
  11. 11.
    ​Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead - “trying to \textit{explain} black box models, rather than creating models that are \textit{interpretable} in the first place, is likely to perpetuate bad practices and can potentially cause catastrophic harm to society. There is a way forward -- it is to design models that are inherently interpretable. This manuscript clarifies the chasm between explaining black boxes and using inherently interpretable models, outlines several key reasons why explainable black boxes should be avoided in high-stakes decisions, identifies challenges to interpretable machine learning, and provides several example applications where interpretable models could potentially replace black box models in criminal justice, healthcare, and computer vision.”
  12. 12.
    ​Using genetic algorithms​
  13. 13.
    ​ Google’s what-if tool from PAIR​
  14. 14.
    ​Boruta (medium) was designed to automatically perform feature selection on a dataset using randomized features, i.e., measuring valid features against their shadow/noisy counterparts.
  15. 15.
    ​InterpretML by Microsoft, git.
  16. 16.
    ​Connecting Interpretability and Robustness in Decision Trees through Separation, git​

WHY WE SHOULDN’T TRUST MODELS

  1. 1.
    ​Clever Hans effect for NLP​
    1. 1.
      Datasets need more love
    2. 2.
      Datasets ablation and public beta
    3. 3.
      Inter-prediction agreement
  2. 2.
    Behavioral testing and CHECKLIST
    1. 1.
      ​Blog, Youtube, paper, git​
    2. 2.
      ​Yonatan hadar on the subject in hebrew​

BIAS

  1. 1.
    arize.ai on model bias.

DEBIASING MODELS

  1. 1.
    ​Adversarial removal of demographic features - “We show that demographic information of authors is encoded in -- and can be recovered from -- the intermediate representations learned by text-based neural classifiers. The implication is that decisions of classifiers trained on textual data are not agnostic to -- and likely condition on -- demographic attributes. “ “we explore several techniques to improve the effectiveness of the adversarial component. Our main conclusion is a cautionary one: do not rely on the adversarial training to achieve invariant representation to sensitive features.”
  2. 2.
    ​Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection (paper) , github, presentation by Shauli et al. - removing biased information such as gender from an embedding space using nullspace projection. The objective is this: give a representation of text, for example BERT embeddings of many resumes/CVs, we want to achieve a state where a certain quality, for example a gender representation of the person who wrote this resume is not encoded in X. they used the light version definition for “not encoded”, i.e., you cant predict the quality from the representation with a higher than random score, using a linear model. I.e., every linear model you will train, will not be able to predict the person’s gender out of the embedding space and will reach a 50% accuracy. This is done by an iterative process that includes. 1. Linear model training to predict the quality of the concept from the representation. 2. Performing ‘projection to null space’ for the linear classifier, this is an acceptable linear algebra calculation that has a meaning of zeroing the representation from the projection on the separation place that the linear model is representing, making the model useless. I.e., it will always predict the zero vector. This is done iteratively on the neutralized output, i.e., in the second iteration we look for an alternative way to predict the gender out of X, until we reach 50% accuracy (or some other metric you want to measure) at this point we have neutralized all the linear directions in the embedding space, that were predictive to the gender of the author.
For a matrix W, the null space is a sub-space of all X such that WX=0, i.e., W maps X to the zero vector, this is a linear projection of the zero vector into a subspace. For example you can take a 3d vectors and calculate its projection on XY.
  1. 1.
    Can we extinct predictive samples? Its an open question, Maybe we can use influence functions?
​Understanding Black-box Predictions via Influence Functions - How can we explain the predictions of a blackbox model? In this paper, we use influence functions — a classic technique from robust statistics — to trace a model’s prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction.
We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually indistinguishable training-set attacks.
  1. 1.
    ​Removing ‘gender bias using pair mean pca​
  2. 2.
    ​Bias detector by intuit - Based on first and last name/zip code the package analyzes the probability of the user belonging to different genders/races. Then, the model predictions per gender/race are compared using various bias metrics.
  3. 3.
    ​
  4. 4.
    ​

​

PRIVACY

DIFFERENTIAL PRIVACY

  1. 1.
    ​Differential privacy has emerged as a major area of research in the effort to prevent the identification of individuals and private data. It is a mathematical definition for the privacy loss that results to individuals when their private information is used to create AI products. It works by injecting noise into a dataset, during a machine learning training process, or into the output of a machine learning model, without introducing significant adverse effects on data analysis or model performance. It achieves this by calibrating the noise level to the sensitivity of the algorithm. The result is a differentially private dataset or model that cannot be reverse engineered by an attacker, while still providing useful information. Uses BOTLON & EPSILON
  2. 2.
    ​youtube​

ANONYMIZATION

DE-ANONYMIZATION

  1. 1.
    GPT2 - Of language datasets ​
    ​
Previous
Regularization
Next
Meta Learning
Last modified 3mo ago
Copy link
Outline
DATASETS RELIABILITY & CORRECTNESS
UNIT / DATA TESTS
REGULATION FOR AI
FAIRNESS, ACCOUNTABILITY & TRANSPARENCY ML
INTERPRETABLE / EXPLAINABLE AI (XAI)
WHY WE SHOULDN’T TRUST MODELS
BIAS
DEBIASING MODELS
PRIVACY
DIFFERENTIAL PRIVACY
ANONYMIZATION
DE-ANONYMIZATION