Software-based automatic differentiation is flawed

Software efforts in automatic differentiation are built on the observation that every computation is implemented as a series of elementary operations, and that symbolic partial derivatives can be supplied for each of these operations. Any computation can then be traversed (either forwards or backwards) utilizing the chain rule to accumulate derivatives. These frameworks have no mechanism for simplifying these expressions before evaluating them. As we illustrate below, the resulting errors tend to be unbounded.