Neural networks using the backpropagation algorithm were biologically “unrealistic in almost every respect” he said. For one thing, neurons mostly send information in one direction.

Backpropagation has since become one of the most widely used algorithms in the field of artificial intelligence. After the publication of the backpropagation algorithm, it quickly became a popular ...

Here, we propose a hardware implementation of the backpropagation algorithm that progressively updates each layer using in situ stochastic gradient descent, avoiding this storage requirement.