15.14 Layers: the building blocks of deep learning

  • Deep-learning model: directed, acyclic graph of layers
    • Most common is a linear stack of layers, mapping a single input to a single output
  • Layer: a data-processing module that takes as input 1+ tensors (e.g. vectors) and that outputs one or more tensors
    • Different layer types for different tensor formats/data processing types (e.g., densely connected layers, recurrent layers etc.)
    • Layer’s state: the layer’s weights which together contain the network’s knowledge
    • Layers = LEGO bricks of deep learning (this metaphor is made explicit by frameworks like Keras)
  • Building deep-learning models in Keras: Clip together compatible layers to form useful data-transformation pipelines
  • Layer compatibility = every layer will only accept input tensors of a certain shape and will return output tensors of a certain shape
    • Keras takes care of layer compatibility (dynamically built to match shape of incoming layer)
# Example code: DL model
# Network architecture 
  model <- keras_model_sequential() %>% # linear stack of layers
    layer_dense(units = 64, # Output space dim.
                activation = "relu", # Activation function (linear)
                input_shape = dim(train_data)[[2]]) %>% # Input Dim.
    layer_dense(units = 64, 
                activation = "relu") %>%
    layer_dense(units = 1)

# "relu": Rectified linear unit activation function