Skip to content

deel-ai/xplique

Repository files navigation

Xplique


🦊 Xplique (pronounced \ɛks.plik\) is a Python toolkit dedicated to explainability. The goal of this library is to gather the state of the art of Explainable AI to help you understand your complex neural network models. Originally built for Tensorflow's model it also works for PyTorch models partially.
📘 Explore Xplique docs | Explore Xplique tutorials 🔥

Attributions · Concept · Feature Visualization · Metrics . Example-based

Important

With the release of Keras 3.X since TensorFlow 2.16, some methods may not function as expected. We are actively working on a fix. In the meantime, we recommend using TensorFlow 2.15 or earlier versions for optimal compatibility.

The library is composed of several modules, the Attributions Methods module implements various methods (e.g Saliency, Grad-CAM, Integrated-Gradients...), with explanations, examples and links to official papers. The Feature Visualization module allows to see how neural networks build their understanding of images by finding inputs that maximize neurons, channels, layers or compositions of these elements. The Concepts module allows you to extract human concepts from a model and to test their usefulness with respect to a class. Finally, the Metrics module covers the current metrics used in explainability. Used in conjunction with the Attribution Methods module, it allows you to test the different methods or evaluate the explanations of a model.


Note

We are proud to announce the release of the Example-based module! This module is dedicated to methods that explain a model by retrieving relevant examples from a dataset. It includes methods that belong to different families: similar examples, contrastive (counter-factuals and semi-factuals) examples, and prototypes (as concepts based methods have a dedicated sections).

🔥 Tutorials

We propose some Hands-on tutorials to get familiar with the library and its api:

You can find a certain number of other practical tutorials just here. This section is actively developed and more contents will be included. We will try to cover all the possible usage of the library, feel free to contact us if you have any suggestions or recommendations towards tutorials you would like to see.

🚀 Quick Start

Xplique requires a version of python higher than 3.7 and several libraries including Tensorflow and Numpy. Installation can be done using Pypi:

pip install xplique

Now that Xplique is installed, here are basic examples of what you can do with the available modules.

Attributions Methods Let's start with a simple example, by computing Grad-CAM for several images (or a complete dataset) on a trained model.
from xplique.attributions import GradCAM

# load images, labels and model
# ...

explainer = GradCAM(model)
explanations = explainer.explain(images, labels)
# or just `explainer(images, labels)`

All attributions methods share a common API described in the attributions API documentation.

Attributions Metrics

In order to measure if the explanations provided by our method are faithful (it reflects well the functioning of the model) we can use a fidelity metric such as Deletion

from xplique.attributions import GradCAM
from xplique.metrics import Deletion

# load images, labels and model
# ...

explainer = GradCAM(model)
explanations = explainer(inputs, labels)
metric = Deletion(model, inputs, labels)

score_grad_cam = metric(explanations)

All attributions metrics share a common API. You can find out more about it here.

Concepts Extraction

CAV

Concerning the concept-based methods, we can for example extract a concept vector from a layer of a model. In order to do this, we use two datasets, one containing inputs containing the concept: positive_samples, the other containing other entries which do not contain the concept: negative_samples.

from xplique.concepts import Cav

# load a model, samples that contain a concept
# (positive) and samples who don't (negative)
# ...

extractor = Cav(model, 'mixed3')
concept_vector = extractor(positive_samples,
                           negative_samples)

More information on CAV here and on TCAV here.

CRAFT

Use Craft to investigate a single class.

from xplique.concepts import CraftTf as Craft

# Cut the model in two parts: g and h
# Create a Craft concept extractor from these 2 models
craft = Craft(input_to_latent_model = g,
              latent_to_logit_model = h)

# Use Craft to compute the concepts for a specific class
craft.fit(images_preprocessed, class_id=rabbit_class_id)

# Compute Sobol indices to understand which concept matters
importances = craft.estimate_importance()

# Display those concepts by showing the 10 best crops for each concept
craft.plot_concepts_crops(nb_crops=10)

More information in the CRAFT documentation.

Feature Visualization

Finally, in order to find an image that maximizes a neuron and at the same time a layer, we build two objectives that we combine together. We then call the optimizer which returns our images

from xplique.features_visualizations import Objective
from xplique.features_visualizations import optimize

# load a model...

neuron_obj = Objective.neuron(model, "logits", 200)
channel_obj = Objective.layer(model, "mixed3", 10)

obj = neuron_obj + 2.0 * channel_obj
images, obj_names = optimize(obj)

Want to know more ? Check the Feature Viz documentation

PyTorch with Xplique

Even though the library was mainly designed to be a Tensorflow toolbox we have been working on a very practical wrapper to facilitate the integration of your PyTorch models into Xplique's framework!

import torch

from xplique.wrappers import TorchWrapper
from xplique.attributions import Saliency
from xplique.metrics import Deletion

# load images, targets and model
# ...

device = 'cuda' if torch.cuda.is_available() else 'cpu'
wrapped_model = TorchWrapper(torch_model, device)

explainer = Saliency(wrapped_model)
explanations = explainer(inputs, targets)

metric = Deletion(wrapped_model, inputs, targets)
score_saliency = metric(explanations)

Want to know more ? Check the PyTorch documentation

📦 What's Included

There are 4 modules in Xplique, Attribution methods, Attribution metrics, Concepts, and Feature visualization. In particular, the attribution methods module supports a huge diversity of tasks:Classification, Regression, Object Detection, and Semantic Segmentation. For diverse data types: Images, Time Series, and Tabular data. The methods compatible with such task are highlighted in the following table:

Table of attributions available
Attribution Method Type of Model Source Images Time Series and Tabular Data Tutorial
Deconvolution TF Paper C✔️ OD❌ SS❌ C✔️ R✔️ Open In Colab
Grad-CAM TF Paper C✔️ OD❌ SS❌ Open In Colab
Grad-CAM++ TF Paper C✔️ OD❌ SS❌ Open In Colab
Gradient Input TF, PyTorch** Paper C✔️ OD✔️ SS✔️ C✔️ R✔️ Open In Colab
Guided Backprop TF Paper C✔️ OD❌ SS❌ C✔️ R✔️ Open In Colab
Integrated Gradients TF, PyTorch** Paper C✔️ OD✔️ SS✔️ C✔️ R✔️ Open In Colab
Kernel SHAP TF, PyTorch**, Callable* Paper C✔️ OD✔️ SS✔️ C✔️ R✔️ Open In Colab
Lime TF, PyTorch**, Callable* Paper C✔️ OD✔️ SS✔️ C✔️ R✔️ Open In Colab
Occlusion TF, PyTorch**, Callable* Paper C✔️ OD✔️ SS✔️ C✔️ R✔️ Open In Colab
Rise TF, PyTorch**, Callable* Paper C✔️ OD✔️ SS✔️ C✔️ R✔️ Open In Colab
Saliency TF, PyTorch** Paper C✔️ OD✔️ SS✔️ C✔️ R✔️ Open In Colab
SmoothGrad TF, PyTorch** Paper C✔️ OD✔️ SS✔️ C✔️ R✔️ Open In Colab
SquareGrad TF, PyTorch** Paper C✔️ OD✔️ SS✔️ C✔️ R✔️ Open In Colab
VarGrad TF, PyTorch** Paper C✔️ OD✔️ SS✔️ C✔️ R✔️ Open In Colab
Sobol Attribution TF, PyTorch** Paper C✔️ OD✔️ SS✔️ 🔵 Open In Colab
Hsic Attribution TF, PyTorch** Paper C✔️ OD✔️ SS✔️ 🔵 Open In Colab
FORGrad enhancement TF, PyTorch** Paper C✔️ OD✔️ SS✔️ Open In Colab

TF : Tensorflow compatible

C : Classification | R : Regression | OD : Object Detection | SS : Semantic Segmentation (SS)

* : See the Callable documentation

** : See the Xplique for PyTorch documentation, and the PyTorch models: Getting started notebook.

✔️ : Supported by Xplique | ❌ : Not applicable | 🔵 : Work in Progress

Table of attribution's metric available
Attribution Metrics Type of Model Property Source
MuFidelity TF, PyTorch** Fidelity Paper
Deletion TF, PyTorch** Fidelity Paper
Insertion TF, PyTorch** Fidelity Paper
Average Stability TF, PyTorch** Stability Paper
MeGe TF, PyTorch** Representativity Paper
ReCo TF, PyTorch** Consistency Paper
(WIP) e-robustness

TF : Tensorflow compatible

** : See the Xplique for PyTorch documentation, and the PyTorch models: Getting started notebook.

Table of concept methods available
Concepts method Type of Model Source Tutorial
Concept Activation Vector (CAV) TF Paper
Testing CAV (TCAV) TF Paper
CRAFT Tensorflow TF Paper Open In Colab
CRAFT PyTorch PyTorch** Paper Open In Colab
(WIP) Robust TCAV
(WIP) Automatic Concept Extraction (ACE)

TF : Tensorflow compatible

** : See the Xplique for Pytorch documentation, and the PyTorch's model: Getting started Open In Colab notebook

Table of Feature Visualization methods available
Feature Visualization (Paper) Type of Model Details
Neurons TF Optimizes for specific neurons
Layer TF Optimizes for specific layers
Channel TF Optimizes for specific channels
Direction TF Optimizes for specific vector
Fourrier Preconditioning TF Optimize in Fourier basis (see preconditioning)
Objective combination TF Allows to combine objectives
MaCo TF Fixed Magnitude optimisation, see Paper

TF : Tensorflow compatible

Even though we are only at the early stages, we have also recently added an Example-based methods module. Do not hesitate to give us feedback! Currently, the methods available are summarized in the following table:

Table of example-based methods available
Method Family Documentation Tutorial
SimilarExamples Similar Examples SimilarExamples Open In Colab
Cole Similar Examples Cole Open In Colab
NaiveCounterFactuals Counter Factuals NaiveCounterFactuals Open In Colab
LabelAwareCounterFactuals Counter Factuals LabelAwareCounterFactuals Open In Colab
KLEORSimMiss Semi Factuals KLEOR Open In Colab
KLEORGlobalSim Semi Factuals KLEOR Open In Colab
ProtoGreedy Prototypes ProtoGreedy Open In Colab
ProtoDash Prototypes ProtoDash Open In Colab
MMDCritic Prototypes MMDCritic Open In Colab

👍 Contributing

Feel free to propose your ideas or come and contribute with us on the Xplique toolbox! We have a specific document where we describe in a simple way how to make your first pull request: just here.

👀 See Also

This library is one approach of many to explain your model. We don't expect it to be the perfect solution, we create it to explore one point in the space of possibilities.

Other interesting tools to explain your model:
  • Lucid the wonderful library specialized in feature visualization from OpenAI.
  • Captum the PyTorch library for Interpretability research
  • Tf-explain that implement multiples attribution methods and propose callbacks API for tensorflow.
  • Alibi Explain for model inspection and interpretation
  • SHAP a very popular library to compute local explanations using the classic Shapley values from game theory and their related extensions
To learn more about Explainable AI in general:
More from the DEEL project:
  • deel-lip a Python library for training k-Lipschitz neural networks on TF.
  • deel-torchlip a Python library for training k-Lipschitz neural networks on PyTorch.
  • Influenciae Python toolkit dedicated to computing influence values for the discovery of potentially problematic samples in a dataset.
  • LARD Landing Approach Runway Detection (LARD) is a dataset of aerial front view images of runways designed for aircraft landing phase
  • PUNCC Puncc (Predictive uncertainty calibration and conformalization) is an open-source Python library that integrates a collection of state-of-the-art conformal prediction algorithms and related techniques for regression and classification problems
  • OODEEL OODeel is a library that performs post-hoc deep OOD detection on already trained neural network image classifiers. The philosophy of the library is to favor quality over quantity and to foster easy adoption
  • DEEL White paper a summary of the DEEL team on the challenges of certifiable AI and the role of data quality, representativity and explainability for this purpose.

🙏 Acknowledgments

DEEL Logo
This project received funding from the French ”Investing for the Future – PIA3” program within the Artificial and Natural Intelligence Toulouse Institute (ANITI). The authors gratefully acknowledge the support of the DEEL project.

👨‍🎓 Creators

This library was started as a side-project by Thomas FEL who is currently a graduate student at the Artificial and Natural Intelligence Toulouse Institute under the direction of Thomas SERRE. His thesis work focuses on explainability for deep neural networks.

He then received help from some members of the DEEL team to enhance the library namely from Lucas Hervier and Antonin Poché.

🗞️ Citation

If you use Xplique as part of your workflow in a scientific publication, please consider citing the 🗞️ Xplique official paper:

@article{fel2022xplique,
  title={Xplique: A Deep Learning Explainability Toolbox},
  author={Fel, Thomas and Hervier, Lucas and Vigouroux, David and Poche, Antonin and Plakoo, Justin and Cadene, Remi and Chalvidal, Mathieu and Colin, Julien and Boissin, Thibaut and Bethune, Louis and Picard, Agustin and Nicodeme, Claire 
          and Gardes, Laurent and Flandin, Gregory and Serre, Thomas},
  journal={Workshop on Explainable Artificial Intelligence for Computer Vision (CVPR)},
  year={2022}
}

📝 License

The package is released under MIT license.