This class analyzes a passed neural network and stores its internal structure and the individual layers by converting the entire network into an nn_module. With the help of this converter, many methods for interpreting the behavior of neural networks are provided, which give a better understanding of the whole model or individual predictions. You can use models from the following libraries:

Furthermore, a model can be passed as a list (see details for more information).

Usage,NULL

Details

In order to better understand and analyze the prediction of a neural network, the preactivation or other information of the individual layers, which are not stored in an ordinary forward pass, are often required. For this reason, a given neural network is converted into a torch-based neural network, which provides all the necessary information for an interpretation. The converted torch model is stored in the field model and is an instance of innsight::ConvertedModel. However, before the torch model is created, all relevant details of the passed model are extracted into a named list. This list can be saved in complete form in the model_dict field with the argument save_model_as_list, but this may consume a lot of memory for large networks and is not done by default. Also, this named list can again be used as a passed model for the class Converter, which will be described in more detail in the section 'Implemented Libraries'.

Implemented Methods

An object of the Converter class can be applied to the following methods:

  • Layerwise Relevance Propagation (LRP), Bach et al. (2015)

  • Deep Learning Important Features (DeepLift), Shrikumar et al. (2017)

  • SmoothGrad including 'SmoothGrad x Input', Smilkov et al. (2017)

  • Vanilla Gradient including 'Gradient x Input'

  • ConnectionWeights, Olden et al. (2004)

Implemented Libraries

The converter is implemented for models from the libraries nn_sequential, neuralnet and keras. But you can also write a wrapper for other libraries because a model can be passed as a named list with the following components:

  • $input_dim
    An integer vector with the model input dimension, e.g. for a dense layer with 5 input features use c(5) or for a 1D-convolutional layer with signal length 50 and 4 channels use c(4,50).

  • $input_names (optional)
    A list with the names for each input dimension, e.g. for a dense layer with 3 input features use list(c("X1", "X2", "X3")) or for a 1D-convolutional layer with signal length 5 and 2 channels use list(c("C1", "C2"), c("L1","L2","L3","L4","L5")). By default (NULL) the names are generated.

  • $output_dim (optional)
    An integer vector with the model output dimension analogous to $input_dim. This value does not need to be specified. But if it is set, the calculated value will be compared with it to avoid errors during converting.

  • $output_names (optional)
    A list with the names for each output dimension analogous to $input_names. By default (NULL) the names are generated.

  • $layers
    A list with the respective layers of the model. Each layer is represented as another list that requires the following entries depending on the type:

    • Dense Layer:

      • $type: 'Dense'

      • $weight: The weight matrix of the dense layer with shape (dim_out, dim_in).

      • $bias: The bias vector of the dense layer with length dim_out.

      • activation_name: The name of the activation function for this dense layer, e.g. 'relu', 'tanh' or 'softmax'.

      • dim_in (optional): The input dimension of this layer. This value is not necessary, but helpful to check the format of the weight matrix.

      • dim_out (optional): The output dimension of this layer. This value is not necessary, but helpful to check the format of the weight matrix.

    • Convolutional Layers:

      • $type: 'Conv1D' or 'Conv2D'

      • $weight: The weight array of the convolutional layer with shape (out_channels, in_channels, kernel_length) for 1D or (out_channels, in_channels, kernel_height, kernel_width) for 2D.

      • $bias: The bias vector of the layer with length out_channels.

      • $activation_name: The name of the activation function for this layer, e.g. 'relu', 'tanh' or 'softmax'.

      • $dim_in (optional): The input dimension of this layer according to the format (in_channels, in_length) for 1D or (in_channels, in_height, in_width) for 2D.

      • $dim_out (optional): The output dimension of this layer according to the format (out_channels, out_length) for 1D or (out_channels, out_height, out_width) for 2D.

      • $stride (optional): The stride of the convolution (single integer for 1D and tuple of two integers for 2D). If this value is not specified, the default values (1D: 1 and 2D: c(1,1)) are used.

      • $padding (optional): Zero-padding added to the sides of the input before convolution. For 1D-convolution a tuple of the form (pad_left, pad_right) and for 2D-convolution (pad_left, pad_right, pad_top, pad_bottom) is required. If this value is not specified, the default values (1D: c(0,0) and 2D: c(0,0,0,0)) are used.

      • $dilation (optional): Spacing between kernel elements (single integer for 1D and tuple of two integers for 2D). If this value is not specified, the default values (1D: 1 and 2D: c(1,1)) are used.

    • Pooling Layers:

      • $type: 'MaxPooling1D', 'MaxPooling2D', 'AveragePooling1D' or 'AveragePooling2D'

      • $kernel_size: The size of the pooling window as an integer value for 1D-pooling and an tuple of two integers for 2D-pooling.

      • $strides (optional): The stride of the pooling window (single integer for 1D and tuple of two integers for 2D). If this value is not specified (NULL), the value of kernel_size will be used.

      • dim_in (optional): The input dimension of this layer. This value is not necessary, but helpful to check the correctness of the converted model.

      • dim_out (optional): The output dimension of this layer. This value is not necessary, but helpful to check the correctness of the converted model.

    • Flatten Layer:

      • $type: 'Flatten'

      • $dim_in (optional): The input dimension of this layer without the batch dimension.

      • $dim_out (optional): The output dimension of this layer without the batch dimension.

Note: This package works internally only with the data format 'channels first', i.e. all input dimensions and weight matrices must be adapted accordingly.

References

  • J. D. Olden et al. (2004) An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecological Modelling 178, p. 389–397

  • S. Bach et al. (2015) On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10, p. 1-46

  • A. Shrikumar et al. (2017) Learning important features through propagating activation differences. ICML 2017, p. 4844-4866

  • D. Smilkov et al. (2017) SmoothGrad: removing noise by adding noise. CoRR, abs/1706.03825

Public fields

model

The converted neural network based on the torch module ConvertedModel.

model_dict

The model stored in a named list (see details for more information). By default, the entry model_dict$layers is deleted because it may require a lot of memory for large networks. However, with the argument save_model_as_list this can be saved anyway.

Methods


Method new()

Create a new Converter for a given neural network.

Usage

Converter$new(
  model,
  input_dim = NULL,
  input_names = NULL,
  output_names = NULL,
  dtype = "float",
  save_model_as_list = FALSE
)

Arguments

model

A trained neural network for classification or regression tasks to be interpreted. Only models from the following types or packages are allowed: nn_sequential, keras_model, keras_model_sequential, neuralnet or a named list (see details).

input_dim

An integer vector with the model input dimension excluding the batch dimension, e.g. for a dense layer with 5 input features use c(5) or for a 1D convolutional layer with signal length 50 and 4 channels use c(4, 50).
Note: This argument is only necessary for torch::nn_sequential, for all others it is automatically extracted from the passed model. In addition, the input dimension input_dim has to be in the format channels first.

input_names

(Optional) A list with the names for each input dimension, e.g. for a dense layer with 3 input features use list(c("X1", "X2", "X3")) or for a 1D convolutional layer with signal length 5 and 2 channels use list(c("C1", "C2"), c("L1","L2","L3","L4","L5")).
Note: This argument is optional and otherwise the names are generated automatically. But if this argument is set, all found input names in the passed model will be disregarded.

output_names

(Optional) A list with the names for the output, e.g. for a model with 3 outputs use list(c("Y1", "Y2", "Y3")).
Note: This argument is optional and otherwise the names are generated automatically. But if this argument is set, all found output names in the passed model will be disregarded.

dtype

The data type for the calculations. Use either 'float' for torch::torch_float or 'double' for torch::torch_double.

save_model_as_list

This logical value specifies whether the passed model should be stored as a list (as it is described in the details also as an alternative input for a model). This list can take a lot of memory for large networks, so by default the model is not stored as a list (FALSE).

Returns

A new instance of the R6 class 'Converter'.


Method clone()

The objects of this class are cloneable with this method.

Usage

Converter$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

#----------------------- Example 1: Torch ----------------------------------
library(torch)

model <- nn_sequential(
  nn_linear(5, 10),
  nn_relu(),
  nn_linear(10, 2, bias = FALSE),
  nn_softmax(dim = 2)
)
data <- torch_randn(25, 5)

# Convert the model (for torch models is 'input_dim' required!)
converter <- Converter$new(model, input_dim = c(5))

# Get the converted model
converted_model <- converter$model

# Test it with the original model
mean(abs(converted_model(data) - model(data)))
#> torch_tensor
#> 0
#> [ CPUFloatType{} ][ grad_fn = <MeanBackward0> ]


#----------------------- Example 2: Neuralnet ------------------------------
library(neuralnet)
data(iris)

# Train a neural network
nn <- neuralnet((Species == "setosa") ~ Petal.Length + Petal.Width,
  iris,
  linear.output = FALSE,
  hidden = c(3, 2), act.fct = "tanh", rep = 1
)

# Convert the model
converter <- Converter$new(nn)

# Print all the layers
converter$model$modules_list
#> $Dense_1
#> An `nn_module` containing 0 parameters.
#> 
#> ── Modules ─────────────────────────────────────────────────────────────────────
#> • activation_f: <nn_tanh> #0 parameters
#> 
#> $Dense_2
#> An `nn_module` containing 0 parameters.
#> 
#> ── Modules ─────────────────────────────────────────────────────────────────────
#> • activation_f: <nn_tanh> #0 parameters
#> 
#> $Dense_3
#> An `nn_module` containing 0 parameters.
#> 
#> ── Modules ─────────────────────────────────────────────────────────────────────
#> • activation_f: <nn_tanh> #0 parameters
#> 

#----------------------- Example 3: Keras ----------------------------------
library(keras)

if (is_keras_available()) {
  # Define a keras model
  model <- keras_model_sequential()
  model %>%
    layer_conv_2d(
      input_shape = c(32, 32, 3), kernel_size = 8, filters = 8,
      activation = "relu", padding = "same"
    ) %>%
    layer_conv_2d(
      kernel_size = 8, filters = 4,
      activation = "tanh", padding = "same"
    ) %>%
    layer_conv_2d(
      kernel_size = 4, filters = 2,
      activation = "relu", padding = "same"
    ) %>%
    layer_flatten() %>%
    layer_dense(units = 64, activation = "relu") %>%
    layer_dense(units = 1, activation = "sigmoid")

  # Convert this model and save model as list
  converter <- Converter$new(model, save_model_as_list = TRUE)

  # Print the converted model as a named list
  str(converter$model_dict)
}
#> List of 5
#>  $ layers      :List of 6
#>   ..$ Conv2D_1 :List of 9
#>   .. ..$ weight         : num [1:8, 1:3, 1:8, 1:8] 0.0752 0.082 -0.0231 0.0839 0.0832 ...
#>   .. ..$ bias           : num [1:8] 0 0 0 0 0 0 0 0
#>   .. ..$ activation_name: chr "relu"
#>   .. ..$ dim_in         : int [1:3] 3 32 32
#>   .. ..$ dim_out        : int [1:3] 8 32 32
#>   .. ..$ stride         : int [1:2] 1 1
#>   .. ..$ padding        : int [1:4] 3 4 3 4
#>   .. ..$ dilation       : int [1:2] 1 1
#>   .. ..$ type           : chr "Conv2D"
#>   ..$ Conv2D_2 :List of 9
#>   .. ..$ weight         : num [1:4, 1:8, 1:8, 1:8] -0.000306 -0.033413 -0.084426 -0.027486 -0.052241 ...
#>   .. ..$ bias           : num [1:4] 0 0 0 0
#>   .. ..$ activation_name: chr "tanh"
#>   .. ..$ dim_in         : int [1:3] 8 32 32
#>   .. ..$ dim_out        : int [1:3] 4 32 32
#>   .. ..$ stride         : int [1:2] 1 1
#>   .. ..$ padding        : int [1:4] 3 4 3 4
#>   .. ..$ dilation       : int [1:2] 1 1
#>   .. ..$ type           : chr "Conv2D"
#>   ..$ Conv2D_3 :List of 9
#>   .. ..$ weight         : num [1:2, 1:4, 1:4, 1:4] 0.228 -0.18 0.234 0.191 -0.241 ...
#>   .. ..$ bias           : num [1:2] 0 0
#>   .. ..$ activation_name: chr "relu"
#>   .. ..$ dim_in         : int [1:3] 4 32 32
#>   .. ..$ dim_out        : int [1:3] 2 32 32
#>   .. ..$ stride         : int [1:2] 1 1
#>   .. ..$ padding        : int [1:4] 1 2 1 2
#>   .. ..$ dilation       : int [1:2] 1 1
#>   .. ..$ type           : chr "Conv2D"
#>   ..$ Flatten_4:List of 3
#>   .. ..$ type   : chr "Flatten"
#>   .. ..$ dim_in : int [1:3] 2 32 32
#>   .. ..$ dim_out: int 2048
#>   ..$ Dense_5  :List of 6
#>   .. ..$ type           : chr "Dense"
#>   .. ..$ weight         : num [1:64, 1:2048] -0.01569 0.00692 0.01315 -0.04365 0.0068 ...
#>   .. ..$ bias           : num [1:64] 0 0 0 0 0 0 0 0 0 0 ...
#>   .. ..$ activation_name: chr "relu"
#>   .. ..$ dim_in         : int 2048
#>   .. ..$ dim_out        : int 64
#>   ..$ Dense_6  :List of 6
#>   .. ..$ type           : chr "Dense"
#>   .. ..$ weight         : num [1, 1:64] -0.1821 0.2344 -0.0345 -0.3018 0.0544 ...
#>   .. ..$ bias           : num 0
#>   .. ..$ activation_name: chr "sigmoid"
#>   .. ..$ dim_in         : int 64
#>   .. ..$ dim_out        : int 1
#>  $ input_dim   : int [1:3] 3 32 32
#>  $ output_dim  : int 1
#>  $ input_names :List of 3
#>   ..$ : chr [1:3] "C1" "C2" "C3"
#>   ..$ : chr [1:32] "H1" "H2" "H3" "H4" ...
#>   ..$ : chr [1:32] "W1" "W2" "W3" "W4" ...
#>  $ output_names:List of 1
#>   ..$ : chr "Y1"

#----------------------- Example 4: List  ----------------------------------

# Define a model

model <- list()
model$input_dim <- 5
model$input_names <- list(c("Feat1", "Feat2", "Feat3", "Feat4", "Feat5"))
model$output_dim <- 2
model$output_names <- list(c("Cat", "no-Cat"))
model$layers$Layer_1 <-
  list(
    type = "Dense",
    weight = matrix(rnorm(5 * 20), 20, 5),
    bias = rnorm(20),
    activation_name = "tanh",
    dim_in = 5,
    dim_out = 20
  )
model$layers$Layer_2 <-
  list(
    type = "Dense",
    weight = matrix(rnorm(20 * 2), 2, 20),
    bias = rnorm(2),
    activation_name = "softmax"#,
    #dim_in = 20, # These values are optional, but
    #dim_out = 2  # useful for internal checks
  )

# Convert the model
converter <- Converter$new(model)

# Get the model as a torch::nn_module
torch_model <- converter$model

# You can use it as a normal torch model
x <- torch::torch_randn(3, 5)
torch_model(x)
#> torch_tensor
#>  0.9743  0.0257
#>  0.0040  0.9960
#>  0.7592  0.2408
#> [ CPUFloatType{3,2} ]