Baby Neural nets

https://github.com/thoppe/baby_neural_nets

TRAVIS HOPPE, @metasemantic

me working on deep learning

Neural Networks and target shapes

Can we watch the network learn?
What does changing the network topology look like?

Shallow vs Deep

Multi-Layer Perceptron learning a circle
Brightness equivalent to NN's uncertainty

Shallow, 200

Deep, [10]x15

Activation functions

ReLU

tanh

Concavity and Topology

Letter X (high concavity)

Bullseye (interesting topology)

Multi color patterns

Only show most common color

Source image

Deep MLP [10]x40

Multi color patterns

Straight lines difficult for tanh

Source image

Deep MLP [10]x40

Multi color patterns

Many colors but simple regions

Source image

Deep MLP [10]x40

Details are hard

Pattern doesn't converge after hours of iterations
(needs more complex topology?)

Source image

Deep MLP [10]x40

What's next?

Look at CNN (convolutional neural network)

Combine uncertainty information in color plots

"Morph" from one shape to another as network problem shifts

Thanks, you!

Say hello: @metasemantic