Artificial Neural Networks
Artificial neural networks
An artificial neuron.The basic behavior of an artificial neural network is determined by the dot product (weighted sum) operator in each neuron.
Dot product video.
Statistical properties of the dot product.
The central limit theorem.If you store 1 <vector,scalar> association in a weighed sum the weight vector will point in the same direction as the input vector. Store 2 <vector,scalar> associations and both input vectors will point some angle away from the weight vector. As a result the scalar output will be more sensitive to small changes in the input vectors. A reduction in an initial mild error correction capacity.
If you store too many associations you can only get approximations to the scalar values you want. They will basically be contamintated with Gaussian noise.
The Weighted Sum:
A frozen neural network:
The Walsh Hadamard transform:
The Walsh Hadamard transform (short):
Activation weight switching (Beyond ReLU):
Zero curvature initialisation of neural networks:
SwitchNet4 neural network: