This class analyzes a passed neural network and stores its internal structure and the individual layers by converting the entire network into an nn_module. With the help of this converter, many methods for interpreting the behavior of neural networks are provided, which give a better understanding of the whole model or individual predictions. You can use models from the following libraries:

• torch (nn_sequential)

• keras (keras_model, keras_model_sequential),

• neuralnet

Furthermore, a model can be passed as a list (see details for more information).

## Details

In order to better understand and analyze the prediction of a neural network, the preactivation or other information of the individual layers, which are not stored in an ordinary forward pass, are often required. For this reason, a given neural network is converted into a torch-based neural network, which provides all the necessary information for an interpretation. The converted torch model is stored in the field model and is an instance of innsight::ConvertedModel. However, before the torch model is created, all relevant details of the passed model are extracted into a named list. This list can be saved in complete form in the model_dict field with the argument save_model_as_list, but this may consume a lot of memory for large networks and is not done by default. Also, this named list can again be used as a passed model for the class Converter, which will be described in more detail in the section 'Implemented Libraries'.

### Implemented Methods

An object of the Converter class can be applied to the following methods:

• Layerwise Relevance Propagation (LRP), Bach et al. (2015)

• Deep Learning Important Features (DeepLift), Shrikumar et al. (2017)

• ConnectionWeights, Olden et al. (2004)

### Implemented Libraries

The converter is implemented for models from the libraries nn_sequential, neuralnet and keras. But you can also write a wrapper for other libraries because a model can be passed as a named list with the following components:

• $input_dim An integer vector with the model input dimension, e.g. for a dense layer with 5 input features use c(5) or for a 1D-convolutional layer with signal length 50 and 4 channels use c(4,50). • $input_names (optional)
A list with the names for each input dimension, e.g. for a dense layer with 3 input features use list(c("X1", "X2", "X3")) or for a 1D-convolutional layer with signal length 5 and 2 channels use list(c("C1", "C2"), c("L1","L2","L3","L4","L5")). By default (NULL) the names are generated.

• $output_dim (optional) An integer vector with the model output dimension analogous to $input_dim. This value does not need to be specified. But if it is set, the calculated value will be compared with it to avoid errors during converting.

• $output_names (optional) A list with the names for each output dimension analogous to $input_names. By default (NULL) the names are generated.

• $layers A list with the respective layers of the model. Each layer is represented as another list that requires the following entries depending on the type: • Dense Layer: • $type: 'Dense'

• $weight: The weight matrix of the dense layer with shape (dim_out, dim_in). • $bias: The bias vector of the dense layer with length dim_out.

• activation_name: The name of the activation function for this dense layer, e.g. 'relu', 'tanh' or 'softmax'.

• dim_in (optional): The input dimension of this layer. This value is not necessary, but helpful to check the format of the weight matrix.

• dim_out (optional): The output dimension of this layer. This value is not necessary, but helpful to check the format of the weight matrix.

• Convolutional Layers:

• $type: 'Conv1D' or 'Conv2D' • $weight: The weight array of the convolutional layer with shape (out_channels, in_channels, kernel_length) for 1D or (out_channels, in_channels, kernel_height, kernel_width) for 2D.

• $bias: The bias vector of the layer with length out_channels. • $activation_name: The name of the activation function for this layer, e.g. 'relu', 'tanh' or 'softmax'.

• $dim_in (optional): The input dimension of this layer according to the format (in_channels, in_length) for 1D or (in_channels, in_height, in_width) for 2D. • $dim_out (optional): The output dimension of this layer according to the format (out_channels, out_length) for 1D or (out_channels, out_height, out_width) for 2D.

• $stride (optional): The stride of the convolution (single integer for 1D and tuple of two integers for 2D). If this value is not specified, the default values (1D: 1 and 2D: c(1,1)) are used. • $padding (optional): Zero-padding added to the sides of the input before convolution. For 1D-convolution a tuple of the form (pad_left, pad_right) and for 2D-convolution (pad_left, pad_right, pad_top, pad_bottom) is required. If this value is not specified, the default values (1D: c(0,0) and 2D: c(0,0,0,0)) are used.

• $dilation (optional): Spacing between kernel elements (single integer for 1D and tuple of two integers for 2D). If this value is not specified, the default values (1D: 1 and 2D: c(1,1)) are used. • Pooling Layers: • $type: 'MaxPooling1D', 'MaxPooling2D', 'AveragePooling1D' or 'AveragePooling2D'

• $kernel_size: The size of the pooling window as an integer value for 1D-pooling and an tuple of two integers for 2D-pooling. • $strides (optional): The stride of the pooling window (single integer for 1D and tuple of two integers for 2D). If this value is not specified (NULL), the value of kernel_size will be used.

• dim_in (optional): The input dimension of this layer. This value is not necessary, but helpful to check the correctness of the converted model.

• dim_out (optional): The output dimension of this layer. This value is not necessary, but helpful to check the correctness of the converted model.

• Flatten Layer:

• $type: 'Flatten' • $dim_in (optional): The input dimension of this layer without the batch dimension.

## Methods

### Method new()

Create a new Converter for a given neural network.

#### Arguments

deep

Whether to make a deep clone.

## Examples

#----------------------- Example 1: Torch ----------------------------------
library(torch)

model <- nn_sequential(
nn_linear(5, 10),
nn_relu(),
nn_linear(10, 2, bias = FALSE),
nn_softmax(dim = 2)
)
data <- torch_randn(25, 5)

# Convert the model (for torch models is 'input_dim' required!)
converter <- Converter$new(model, input_dim = c(5)) # Get the converted model converted_model <- converter$model

# Test it with the original model
mean(abs(converted_model(data) - model(data)))
#> torch_tensor
#> 0
#> [ CPUFloatType{} ][ grad_fn = <MeanBackward0> ]

#----------------------- Example 2: Neuralnet ------------------------------
library(neuralnet)
data(iris)

# Train a neural network
nn <- neuralnet((Species == "setosa") ~ Petal.Length + Petal.Width,
iris,
linear.output = FALSE,
hidden = c(3, 2), act.fct = "tanh", rep = 1
)

# Convert the model
converter <- Converter$new(nn) # Print all the layers converter$model$modules_list #>$Dense_1
#> An nn_module containing 0 parameters.
#>
#> ── Modules ─────────────────────────────────────────────────────────────────────
#> • activation_f: <nn_tanh> #0 parameters
#>
#> $Dense_2 #> An nn_module containing 0 parameters. #> #> ── Modules ───────────────────────────────────────────────────────────────────── #> • activation_f: <nn_tanh> #0 parameters #> #>$Dense_3
#> An nn_module containing 0 parameters.
#>
#> ── Modules ─────────────────────────────────────────────────────────────────────
#> • activation_f: <nn_tanh> #0 parameters
#>

#----------------------- Example 3: Keras ----------------------------------
library(keras)

if (is_keras_available()) {
# Define a keras model
model <- keras_model_sequential()
model %>%
layer_conv_2d(
input_shape = c(32, 32, 3), kernel_size = 8, filters = 8,
activation = "relu", padding = "same"
) %>%
layer_conv_2d(
kernel_size = 8, filters = 4,
activation = "tanh", padding = "same"
) %>%
layer_conv_2d(
kernel_size = 4, filters = 2,
activation = "relu", padding = "same"
) %>%
layer_flatten() %>%
layer_dense(units = 64, activation = "relu") %>%
layer_dense(units = 1, activation = "sigmoid")

# Convert this model and save model as list
converter <- Converter$new(model, save_model_as_list = TRUE) # Print the converted model as a named list str(converter$model_dict)
}
#> List of 5
#>  $layers :List of 6 #> ..$ Conv2D_1 :List of 9
#>   .. ..$weight : num [1:8, 1:3, 1:8, 1:8] -0.04437 -0.03568 -0.06902 0.02998 -0.00233 ... #> .. ..$ bias           : num [1:8] 0 0 0 0 0 0 0 0
#>   .. ..$activation_name: chr "relu" #> .. ..$ dim_in         : int [1:3] 3 32 32
#>   .. ..$dim_out : int [1:3] 8 32 32 #> .. ..$ stride         : int [1:2] 1 1
#>   .. ..$padding : int [1:4] 3 4 3 4 #> .. ..$ dilation       : int [1:2] 1 1
#>   .. ..$type : chr "Conv2D" #> ..$ Conv2D_2 :List of 9
#>   .. ..$weight : num [1:4, 1:8, 1:8, 1:8] 0.053 0.0784 0.0277 0.0562 0.0327 ... #> .. ..$ bias           : num [1:4] 0 0 0 0
#>   .. ..$activation_name: chr "tanh" #> .. ..$ dim_in         : int [1:3] 8 32 32
#>   .. ..$dim_out : int [1:3] 4 32 32 #> .. ..$ stride         : int [1:2] 1 1
#>   .. ..$padding : int [1:4] 3 4 3 4 #> .. ..$ dilation       : int [1:2] 1 1
#>   .. ..$type : chr "Conv2D" #> ..$ Conv2D_3 :List of 9
#>   .. ..$weight : num [1:2, 1:4, 1:4, 1:4] -0.0434 0.0746 0.053 -0.2068 0.2008 ... #> .. ..$ bias           : num [1:2] 0 0
#>   .. ..$activation_name: chr "relu" #> .. ..$ dim_in         : int [1:3] 4 32 32
#>   .. ..$dim_out : int [1:3] 2 32 32 #> .. ..$ stride         : int [1:2] 1 1
#>   .. ..$padding : int [1:4] 1 2 1 2 #> .. ..$ dilation       : int [1:2] 1 1
#>   .. ..$type : chr "Conv2D" #> ..$ Flatten_4:List of 3
#>   .. ..$type : chr "Flatten" #> .. ..$ dim_in : int [1:3] 2 32 32
#>   .. ..$dim_out: int 2048 #> ..$ Dense_5  :List of 6
#>   .. ..$type : chr "Dense" #> .. ..$ weight         : num [1:64, 1:2048] -0.026 0.0397 0.0103 -0.0224 0.0132 ...
#>   .. ..$bias : num [1:64] 0 0 0 0 0 0 0 0 0 0 ... #> .. ..$ activation_name: chr "relu"
#>   .. ..$dim_in : int 2048 #> .. ..$ dim_out        : int 64
#>   ..$Dense_6 :List of 6 #> .. ..$ type           : chr "Dense"
#>   .. ..$weight : num [1, 1:64] -0.2862 0.2483 0.2071 0.088 -0.0365 ... #> .. ..$ bias           : num 0
#>   .. ..$activation_name: chr "sigmoid" #> .. ..$ dim_in         : int 64
#>   .. ..$dim_out : int 1 #>$ input_dim   : int [1:3] 3 32 32
#>  $output_dim : int 1 #>$ input_names :List of 3
#>   ..$: chr [1:3] "C1" "C2" "C3" #> ..$ : chr [1:32] "H1" "H2" "H3" "H4" ...
#>   ..$: chr [1:32] "W1" "W2" "W3" "W4" ... #>$ output_names:List of 1
#>   ..$: chr "Y1" #----------------------- Example 4: List ---------------------------------- # Define a model model <- list() model$input_dim <- 5
model$input_names <- list(c("Feat1", "Feat2", "Feat3", "Feat4", "Feat5")) model$output_dim <- 2
model$output_names <- list(c("Cat", "no-Cat")) model$layers$Layer_1 <- list( type = "Dense", weight = matrix(rnorm(5 * 20), 20, 5), bias = rnorm(20), activation_name = "tanh", dim_in = 5, dim_out = 20 ) model$layers$Layer_2 <- list( type = "Dense", weight = matrix(rnorm(20 * 2), 2, 20), bias = rnorm(2), activation_name = "softmax"#, #dim_in = 20, # These values are optional, but #dim_out = 2 # useful for internal checks ) # Convert the model converter <- Converter$new(model)

# Get the model as a torch::nn_module
torch_model <- converter\$model

# You can use it as a normal torch model
x <- torch::torch_randn(3, 5)
torch_model(x)
#> torch_tensor
#>  0.9841  0.0159
#>  0.0363  0.9637
#>  0.9993  0.0007
#> [ CPUFloatType{3,2} ]