[Tensorflow] A Single Neuron

1. What is Deep Learning?

Some of the most impressive advances in artificial intelligence in recent years have been in the field of deep learning. Natural language translation, image recognition, and game playing are all tasks where deep learning models have neared or even exceeded humal-level performance.

 

So what is deep learning? Deep learning is an approach to machine laerning characterized by deep stacks of computations. The depth of computation is what has enabled deep learning model to disentangle the kinds of complex hierarchical patterns found in the most challenging real-world datasets.

 

Through their power and sociability neural networks have become the defining model of deep learning. Nueral networks are composed of neurons, where each neuron individually performs only a simple computation. The power of a neural network comes instead from the complexity of the connections these neurons can form.

 

2. The Linear Unit

The input is x. Its connection to the neuron has a weight which is \(w\). Whenever a value flows through a connection, you multiply the value by the connection's weight. For the input \(x\), what reaches the neuron is \(w \times x\). A neural networks learns by modifying its weights.

 

The \(b\) is a special kind of weight we call the bias. The bias doesn't have any input data associated with it; instead, we put a 1 in the diagram so the value that reaches the neuron is just \(b\). The bias enables the neuron to modify the output independently of its input.

 

The \(y\) is the value the neuron ultimately outputs. To get the output, the neuron sums up all the values it receives through its connections. THis neuron's activation is \(y = wx + b\).

 

3. The Linear Unit as a Model

Though individual neurons will usually only function as part of a larger network, it's often useful to start with a single neuron model as a baseline. Single neuron models are linear models.

 

 

4. Multiple Inputs

We can just add up more input connections to the neuron, one for each additional features. To find the output, we would multiply each input to its connection weight and then add them all together.

 

 

The formular for this neuron would be \(y = w_0 x_0 + w_1 x_1 + w_2 x_2 + b\). A linear unit with two inputs will fit a plane, and a unit with more inputs than that will fit a hyper plane.

 

5. Linear Unit in Keras

The easiest way to create a model in Keras is through Keras. Sequential, which creates a neural network as a stack of layers. We can create models like those above using dense layer.

 

We could define a linear model accepting three input features and producing a single output.

 

from tensorflow import keras
from tensorflow.keras import layers 

# Create a network with 1 linear layers 
model = keras.Sequential([
        layers.Dense(units=1, input_shape = [3]) 
        ])

 

With the first argument, units, we define how many output we want. In this case, we are just predicting 1 output, we'll use units=1. With the second argument, input_shape, we tell Keras the dimensions of the inputs. Setting input_shape=[3] ensures the model will acceppt three features as input.

 

 

Source from :  https://www.kaggle.com/learn