[Tensorflow] Deep Neural Networks

1. Layers

Neural networks typically organize their neurons into layers. When we collect together linear units having a common set of inputs we get a dense layer.

 

 

We could think of each layer in a neural network as performing some kind of relatively simple transform its input in more complex ways. In a well-trained neural netwrok, each layer is a transformation getting us a little bit closer to a solution.

 

A layer in Keras is a very general kind of thing. A layer can be, essentially, any kind of data transformation. Many layers, like the convultional and recurrent layers, trasform data through use of neurons and differ primarilly in the pattern of connections they form. Other though are used for feature engineering or just simple arithmetic.

 

2. The Activation Function

It turns out, however, that two dense layers with nothing in between are no better than a single dense by itself. Dense layers by themselves can never move us out of the world of line and planes. What we need is something nonlinear. What we need are acrivation functions.

 

An activation functions are simply some function we apply to each of layer's output. The most coommon rectifier function is max(0, x).

 

 

The rectifier function has a graph that's a line with the negative part "rectified" to zero. Applying the function to the outputs of a nueron will put a bend in the data, moving us away from simple lines.

 

When we attach the rectifier to a linear unit, we get a rectifier linear unit or ReLU. Applying a ReLU activation to a linear unit means the output becomes \(max(0, wx+b)\).

 

3. Stacking Dense Layers

Now that we have some non-linearity, let's see how we can stack layers to get complex data transformation.

 

 

The layers before the output layer are sometimes called hidden since we never see their outputs directly. Now notice that the final layer is a linear unit. That makes this network appropriate to a regression task, where we are trying to predict some arbitrary numeric value. Other tasks might require an activation function on the output.

 

4. Building Sequential Models

The Sequential model we've been using will connect together a list of layers in order from first to last : The first layers get the input, the last layer produces output.

 

from tensorflow import keras 
from tensorflow.keras import layers 

model = keras.Sequentail([
        # The hidden ReLU layers 
        layers.Dense(units=4, activation='relu', input_shape=[2]),
        layers.Dense(units=3, activation='relu'),
        # The linear output layer 
        layers.Dense(units=1)
        ])

 

Be sure to pass all the layers together in a list, like[layer, layer, layer, ...], instead of as separate arguments. To add an activation function to a layer, just give its name in the activation argument.

 

 

Source from : https://www.kaggle.com/learn