innsight is an R package that interprets the behavior and explains individual predictions of modern neural networks. Many methods for explaining individual predictions already exist, but hardly any of them are implemented or available in R. Most of these so-called feature attribution methods are only implemented in Python and thus difficult to access or use for the R community. In this sense, the package innsight provides a common interface for various methods for the interpretability of neural networks and can therefore be considered as an R analogue to iNNvestigate or Captum for Python.
This package implements several model-specific interpretability (feature attribution) methods based on neural networks in R, e.g.,
Example results for these methods on ImageNet with pretrained network VGG19 (see Example 3: ImageNet with keras for details):
The package innsight aims to be as flexible as possible and independent of a specific deep learning package in which the passed network has been learned. Basically, a neural network of the libraries torch, keras and neuralnet can be passed, which is internally converted into a torch model with special insights needed for interpretation. But it is also possible to pass an arbitrary net in form of a named list (see vignette for details).
The package can be installed directly from CRAN and the development version from GitHub with the following commands (successful installation of
devtools is required)
# Stable version (v0.1.1) install.packages("innsight") # Development version devtools::install_github("bips-hb/innsight")
Internally, any passed model is converted to a torch model, thus the correct functionality of this package relies on a complete and correct installation of torch. For this reason, the following command must be run manually to install the missing libraries LibTorch and LibLantern:
You have a trained neural network
model and your model input data
data. Now you want to interpret individual data points or the overall behavior by using the methods from the package innsight, then stick to the following pseudo code:
# --------------- Step 0: Train your model ----------------- # 'model' has to be an instance of either torch::nn_sequential, # keras::keras_model_sequential, keras::keras_model or neuralnet::neuralnet model = ... # -------------- Step 1: Convert your model ---------------- # For keras and neuralnet converter <- Converter$new(model) # For a torch model the argument 'input_dim' is required converter <- Converter$new(model, input_dim = model_input_dim) # -------------- Step 2: Apply method ---------------------- # Apply global method result <- Method$new(converter) # no data argument is needed # Apply local methods result <- Method$new(converter, data) # -------------- Step 3: Get and plot results -------------- # Get the results as an array res <- get_result(result) # Plot individual results plot(result) # Plot a boxplot of all given data points in argument 'data' boxplot(result) # Interactive plots can also be created for both methods plot(result, as_plotly = TRUE)
If you would like to contribute, please open an issue or submit a pull request.
This package becomes even more alive and valuable if people are using it for their analyses. Therefore, don’t hesitate to write me (email@example.com) or create a feature request if you are missing something for your analyses or have great ideas for extending this package. Currently, we are working on the following: