D · Deep nets (refresh) · 9/38
Dashed line, solid bend. Switch tanh / ReLU / sigmoid.
The line is affine. The bend is why depth works.
One affine + nonlinearity. Stack these → MLP.