Artificial Neural Networks

Artificial neural networks

An artificial neuron.
The basic behavior of an artificial neural network is determined by the dot product (weighted sum) operator in each neuron.
This also has a geometric interpretation.
The dot product of A and B is the magnitude (vector length) of vector A by the magnitude of vector B by the cosine of the angle between them.



 Dot product video.
To understand information storage in a weighted sum it helps to know the central limit theorem.


Statistical properties of the dot product.


The central limit theorem.
If you store 1 <vector,scalar> association in a weighed sum the weight vector will point in the same direction as the input vector. Store 2 <vector,scalar> associations and both input vectors will point some angle away from the weight vector. As a result the scalar output will be more sensitive to small changes in the input vectors. A reduction in an initial mild error correction capacity.
If you store too many associations you can only get approximations to the scalar values you want. They will basically be contamintated with Gaussian noise.

Comments

  1. Further information.

    The Weighted Sum:
    https://archive.org/details/the-weighted-sum

    A frozen neural network:
    https://archive.org/details/afrozenneuralnetwork

    The Walsh Hadamard transform:
    https://archive.org/details/whtebook-archive

    The Walsh Hadamard transform (short):
    https://archive.org/details/short-wht

    Activation weight switching (Beyond ReLU):
    https://archive.org/details/activation-weight-switching

    Zero curvature initialisation of neural networks:
    https://archive.org/details/zero-curvatue

    SwitchNet4 neural network:
    https://discourse.processing.org/t/switch-net-4-neural-network/33220





    ReplyDelete

Post a Comment

Popular posts from this blog

Switch Net

Switch Net 4 (Switch Net N) Combine multiple low width neural layers with a fast transform.

Random Projections for Neural Networks