This class analyzes a passed neural network and stores its internal
structure and the individual layers by converting the entire network into an
nn_module. With the help of this converter, many
methods for interpreting the behavior of neural networks are provided, which
give a better understanding of the whole model or individual predictions.
You can use models from the following libraries:
torch (nn_sequential)
Furthermore, a model can be passed as a list (see details for more information).
In order to better understand and analyze the prediction of a neural
network, the preactivation or other information of the individual layers,
which are not stored in an ordinary forward pass, are often required. For
this reason, a given neural network is converted into a torch-based neural
network, which provides all the necessary information for an interpretation.
The converted torch model is stored in the field model and is an instance
of innsight::ConvertedModel.
However, before the torch model is created, all relevant details of the
passed model are extracted into a named list. This list can be saved
in complete form in the model_dict field with the argument
save_model_as_list, but this may consume a lot of memory for large
networks and is not done by default. Also, this named list can again be
used as a passed model for the class Converter, which will be described
in more detail in the section 'Implemented Libraries'.
An object of the Converter class can be applied to the following methods:
Layerwise Relevance Propagation (LRP), Bach et al. (2015)
Deep Learning Important Features (DeepLift), Shrikumar et al. (2017)
SmoothGrad including 'SmoothGrad x Input', Smilkov et al. (2017)
Vanilla Gradient including 'Gradient x Input'
ConnectionWeights, Olden et al. (2004)
The converter is implemented for models from the libraries
nn_sequential,
neuralnet and keras. But you
can also write a wrapper for other libraries because a model can be passed
as a named list with the following components:
$input_dim
An integer vector with the model input dimension, e.g. for
a dense layer with 5 input features use c(5) or for a 1D-convolutional
layer with signal length 50 and 4 channels use c(4,50).
$input_names (optional)
A list with the names for each input dimension, e.g. for
a dense layer with 3 input features use list(c("X1", "X2", "X3")) or for a
1D-convolutional layer with signal length 5 and 2 channels use
list(c("C1", "C2"), c("L1","L2","L3","L4","L5")). By default (NULL)
the names are generated.
$output_dim (optional)
An integer vector with the model output dimension analogous to $input_dim.
This value does not need to be specified. But if it is set, the calculated
value will be compared with it to avoid errors during converting.
$output_names (optional)
A list with the names for each output dimension analogous to $input_names.
By default (NULL) the names are generated.
$layers
A list with the respective layers of the model. Each layer is represented as
another list that requires the following entries depending on the type:
Dense Layer:
$type: 'Dense'
$weight: The weight matrix of the dense layer with shape
(dim_out, dim_in).
$bias: The bias vector of the dense layer with length
dim_out.
activation_name: The name of the activation function for this
dense layer, e.g. 'relu', 'tanh' or 'softmax'.
dim_in (optional): The input dimension of this layer. This
value is not necessary, but helpful to check the format of the weight
matrix.
dim_out (optional): The output dimension of this layer. This
value is not necessary, but helpful to check the format of the weight
matrix.
Convolutional Layers:
$type: 'Conv1D' or 'Conv2D'
$weight: The weight array of the convolutional layer with shape
(out_channels, in_channels, kernel_length) for 1D or
(out_channels, in_channels, kernel_height, kernel_width) for
2D.
$bias: The bias vector of the layer with length out_channels.
$activation_name: The name of the activation function for this
layer, e.g. 'relu', 'tanh' or 'softmax'.
$dim_in (optional): The input dimension of this layer according
to the format (in_channels, in_length) for 1D or
(in_channels, in_height, in_width) for 2D.
$dim_out (optional): The output dimension of this layer
according to the format (out_channels, out_length) for 1D or
(out_channels, out_height, out_width) for 2D.
$stride (optional): The stride of the convolution (single
integer for 1D and tuple of two integers for 2D). If this value is not
specified, the default values (1D: 1 and 2D: c(1,1)) are used.
$padding (optional): Zero-padding added to the sides of the
input before convolution. For 1D-convolution a tuple of the form
(pad_left, pad_right) and for 2D-convolution
(pad_left, pad_right, pad_top, pad_bottom) is required. If this
value is not specified, the default values (1D: c(0,0) and 2D:
c(0,0,0,0)) are used.
$dilation (optional): Spacing between kernel elements (single
integer for 1D and tuple of two integers for 2D). If this value is
not specified, the default values (1D: 1 and 2D: c(1,1)) are used.
Pooling Layers:
$type: 'MaxPooling1D', 'MaxPooling2D', 'AveragePooling1D'
or 'AveragePooling2D'
$kernel_size: The size of the pooling window as an integer
value for 1D-pooling and an tuple of two integers for 2D-pooling.
$strides (optional): The stride of the pooling window (single
integer for 1D and tuple of two integers for 2D). If this value is not
specified (NULL), the value of kernel_size will be used.
dim_in (optional): The input dimension of this layer. This
value is not necessary, but helpful to check the correctness of the
converted model.
dim_out (optional): The output dimension of this layer. This
value is not necessary, but helpful to check the correctness of the
converted model.
Flatten Layer:
$type: 'Flatten'
$dim_in (optional): The input dimension of this layer
without the batch dimension.
$dim_out (optional): The output dimension of this layer
without the batch dimension.
Note: This package works internally only with the data format 'channels first', i.e. all input dimensions and weight matrices must be adapted accordingly.
J. D. Olden et al. (2004) An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecological Modelling 178, p. 389–397
S. Bach et al. (2015) On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10, p. 1-46
A. Shrikumar et al. (2017) Learning important features through propagating activation differences. ICML 2017, p. 4844-4866
D. Smilkov et al. (2017) SmoothGrad: removing noise by adding noise. CoRR, abs/1706.03825
modelThe converted neural network based on the torch module ConvertedModel.
model_dictThe model stored in a named list (see details for more
information). By default, the entry model_dict$layers is deleted
because it may require a lot of memory for large networks. However, with
the argument save_model_as_list this can be saved anyway.
new()Create a new Converter for a given neural network.
Converter$new(
model,
input_dim = NULL,
input_names = NULL,
output_names = NULL,
dtype = "float",
save_model_as_list = FALSE
)modelA trained neural network for classification or regression
tasks to be interpreted. Only models from the following types or
packages are allowed: nn_sequential,
keras_model,
keras_model_sequential,
neuralnet or a named list (see details).
input_dimAn integer vector with the model input dimension
excluding the batch dimension, e.g. for a dense layer with 5 input
features use c(5) or for a 1D convolutional layer with signal
length 50 and 4 channels use c(4, 50).
Note: This argument is only necessary for torch::nn_sequential,
for all others it is automatically extracted from the passed model.
In addition, the input dimension input_dim has to be in the format
channels first.
input_names(Optional) A list with the names for each input
dimension, e.g. for a dense layer with 3 input features use
list(c("X1", "X2", "X3")) or for a 1D convolutional layer with
signal length 5 and 2 channels use
list(c("C1", "C2"), c("L1","L2","L3","L4","L5")).
Note: This argument is optional and otherwise the names are
generated automatically. But if this argument is set, all found
input names in the passed model will be disregarded.
output_names(Optional) A list with the names for the output,
e.g. for a model with 3 outputs use list(c("Y1", "Y2", "Y3")).
Note: This argument is optional and otherwise the names are
generated automatically. But if this argument is set, all found
output names in the passed model will be disregarded.
dtypeThe data type for the calculations. Use
either 'float' for torch::torch_float or 'double' for
torch::torch_double.
save_model_as_listThis logical value specifies whether the
passed model should be stored as a list (as it is described in the
details also as an alternative input for a model). This list can take
a lot of memory for large networks, so by default the model is not
stored as a list (FALSE).
#----------------------- Example 1: Torch ----------------------------------
library(torch)
model <- nn_sequential(
nn_linear(5, 10),
nn_relu(),
nn_linear(10, 2, bias = FALSE),
nn_softmax(dim = 2)
)
data <- torch_randn(25, 5)
# Convert the model (for torch models is 'input_dim' required!)
converter <- Converter$new(model, input_dim = c(5))
# Get the converted model
converted_model <- converter$model
# Test it with the original model
mean(abs(converted_model(data) - model(data)))
#> torch_tensor
#> 0
#> [ CPUFloatType{} ][ grad_fn = <MeanBackward0> ]
#----------------------- Example 2: Neuralnet ------------------------------
library(neuralnet)
data(iris)
# Train a neural network
nn <- neuralnet((Species == "setosa") ~ Petal.Length + Petal.Width,
iris,
linear.output = FALSE,
hidden = c(3, 2), act.fct = "tanh", rep = 1
)
# Convert the model
converter <- Converter$new(nn)
# Print all the layers
converter$model$modules_list
#> $Dense_1
#> An `nn_module` containing 0 parameters.
#>
#> ── Modules ─────────────────────────────────────────────────────────────────────
#> • activation_f: <nn_tanh> #0 parameters
#>
#> $Dense_2
#> An `nn_module` containing 0 parameters.
#>
#> ── Modules ─────────────────────────────────────────────────────────────────────
#> • activation_f: <nn_tanh> #0 parameters
#>
#> $Dense_3
#> An `nn_module` containing 0 parameters.
#>
#> ── Modules ─────────────────────────────────────────────────────────────────────
#> • activation_f: <nn_tanh> #0 parameters
#>
#----------------------- Example 3: Keras ----------------------------------
library(keras)
if (is_keras_available()) {
# Define a keras model
model <- keras_model_sequential()
model %>%
layer_conv_2d(
input_shape = c(32, 32, 3), kernel_size = 8, filters = 8,
activation = "relu", padding = "same"
) %>%
layer_conv_2d(
kernel_size = 8, filters = 4,
activation = "tanh", padding = "same"
) %>%
layer_conv_2d(
kernel_size = 4, filters = 2,
activation = "relu", padding = "same"
) %>%
layer_flatten() %>%
layer_dense(units = 64, activation = "relu") %>%
layer_dense(units = 1, activation = "sigmoid")
# Convert this model and save model as list
converter <- Converter$new(model, save_model_as_list = TRUE)
# Print the converted model as a named list
str(converter$model_dict)
}
#> List of 5
#> $ layers :List of 6
#> ..$ Conv2D_1 :List of 9
#> .. ..$ weight : num [1:8, 1:3, 1:8, 1:8] 0.0752 0.082 -0.0231 0.0839 0.0832 ...
#> .. ..$ bias : num [1:8] 0 0 0 0 0 0 0 0
#> .. ..$ activation_name: chr "relu"
#> .. ..$ dim_in : int [1:3] 3 32 32
#> .. ..$ dim_out : int [1:3] 8 32 32
#> .. ..$ stride : int [1:2] 1 1
#> .. ..$ padding : int [1:4] 3 4 3 4
#> .. ..$ dilation : int [1:2] 1 1
#> .. ..$ type : chr "Conv2D"
#> ..$ Conv2D_2 :List of 9
#> .. ..$ weight : num [1:4, 1:8, 1:8, 1:8] -0.000306 -0.033413 -0.084426 -0.027486 -0.052241 ...
#> .. ..$ bias : num [1:4] 0 0 0 0
#> .. ..$ activation_name: chr "tanh"
#> .. ..$ dim_in : int [1:3] 8 32 32
#> .. ..$ dim_out : int [1:3] 4 32 32
#> .. ..$ stride : int [1:2] 1 1
#> .. ..$ padding : int [1:4] 3 4 3 4
#> .. ..$ dilation : int [1:2] 1 1
#> .. ..$ type : chr "Conv2D"
#> ..$ Conv2D_3 :List of 9
#> .. ..$ weight : num [1:2, 1:4, 1:4, 1:4] 0.228 -0.18 0.234 0.191 -0.241 ...
#> .. ..$ bias : num [1:2] 0 0
#> .. ..$ activation_name: chr "relu"
#> .. ..$ dim_in : int [1:3] 4 32 32
#> .. ..$ dim_out : int [1:3] 2 32 32
#> .. ..$ stride : int [1:2] 1 1
#> .. ..$ padding : int [1:4] 1 2 1 2
#> .. ..$ dilation : int [1:2] 1 1
#> .. ..$ type : chr "Conv2D"
#> ..$ Flatten_4:List of 3
#> .. ..$ type : chr "Flatten"
#> .. ..$ dim_in : int [1:3] 2 32 32
#> .. ..$ dim_out: int 2048
#> ..$ Dense_5 :List of 6
#> .. ..$ type : chr "Dense"
#> .. ..$ weight : num [1:64, 1:2048] -0.01569 0.00692 0.01315 -0.04365 0.0068 ...
#> .. ..$ bias : num [1:64] 0 0 0 0 0 0 0 0 0 0 ...
#> .. ..$ activation_name: chr "relu"
#> .. ..$ dim_in : int 2048
#> .. ..$ dim_out : int 64
#> ..$ Dense_6 :List of 6
#> .. ..$ type : chr "Dense"
#> .. ..$ weight : num [1, 1:64] -0.1821 0.2344 -0.0345 -0.3018 0.0544 ...
#> .. ..$ bias : num 0
#> .. ..$ activation_name: chr "sigmoid"
#> .. ..$ dim_in : int 64
#> .. ..$ dim_out : int 1
#> $ input_dim : int [1:3] 3 32 32
#> $ output_dim : int 1
#> $ input_names :List of 3
#> ..$ : chr [1:3] "C1" "C2" "C3"
#> ..$ : chr [1:32] "H1" "H2" "H3" "H4" ...
#> ..$ : chr [1:32] "W1" "W2" "W3" "W4" ...
#> $ output_names:List of 1
#> ..$ : chr "Y1"
#----------------------- Example 4: List ----------------------------------
# Define a model
model <- list()
model$input_dim <- 5
model$input_names <- list(c("Feat1", "Feat2", "Feat3", "Feat4", "Feat5"))
model$output_dim <- 2
model$output_names <- list(c("Cat", "no-Cat"))
model$layers$Layer_1 <-
list(
type = "Dense",
weight = matrix(rnorm(5 * 20), 20, 5),
bias = rnorm(20),
activation_name = "tanh",
dim_in = 5,
dim_out = 20
)
model$layers$Layer_2 <-
list(
type = "Dense",
weight = matrix(rnorm(20 * 2), 2, 20),
bias = rnorm(2),
activation_name = "softmax"#,
#dim_in = 20, # These values are optional, but
#dim_out = 2 # useful for internal checks
)
# Convert the model
converter <- Converter$new(model)
# Get the model as a torch::nn_module
torch_model <- converter$model
# You can use it as a normal torch model
x <- torch::torch_randn(3, 5)
torch_model(x)
#> torch_tensor
#> 0.9743 0.0257
#> 0.0040 0.9960
#> 0.7592 0.2408
#> [ CPUFloatType{3,2} ]