Paying attention to feature distribution alignment (pun intended)
Intro Yes, I’m making a joke of the tendency to put the words “attention” and “alignment” in any ML paper 😎. Now let’s see how this provocative title is related to our adventures in the land of polynomial features. The Legendre polynomial basis serverd us well in recent posts about polynomial features. One interesting thing we saw in the series is that its orthogonality is, in some sense informativeness. This is because it orthogonal bases produce features, and hence each basis function, in so