A Trick for Backpropagation of Linear Transformations
Linear transformations such as sums, matrix products, dot products, Hadamard products, and many more can often be represented using an einsum (short for Einstein summation). This post explains a simple trick to backpropagate through any einsum, regardless of what operations it represents. Example Einsum For example, an einsum for matrix multiplication can be written like so: import numpy as np A = np.arange(2 * 3).reshape(2, 3) # A = [ # [0, 1, 2], # [3, 4, 5] # ] B = np.arange(3 * 4).reshap