This is a really cool dive into the interpretability of intermediate-level neural network activations. I really like the way that they've identified a core debate in the ML conversation, and created a relatively simple test-space to better understand it and (hopefully) move the discussion forward.

Very nice. Was a bit confused by the jump to splines and fitting splines to butterflies, more explanation would be appreciated

They write about neurons having “meaning” without a strict definition. What’s the quantitative meaning of meaning?

This isn't immediately clear on whether it's discussing biological neurons or artificial neurons. I suppose it's the latter.