8.14 Layers: the building blocks of deep learning

  • Deep-learning model: directed, acyclic graph of layers
    • Most common is a linear stack of layers, mapping a single input to a single output
  • Layer: a data-processing module that takes as input 1+ tensors (e.g. vectors) and that outputs one or more tensors
    • Different layer types for different tensor formats/data processing types (e.g., densely connected layers, recurrent layers etc.)
    • Layer’s state: the layer’s weights which together contain the network’s knowledge
    • Layers = LEGO bricks of deep learning (this metaphor is made explicit by frameworks like Keras)
  • Building deep-learning models in Keras: Clip together compatible layers to form useful data-transformation pipelines
  • Layer compatibility = every layer will only accept input tensors of a certain shape and will return output tensors of a certain shape
    • Keras takes care of layer compatibility (dynamically built to match shape of incoming layer)