Network¶

Here there can be found two relevant classes in the library: NetworkDescriptor and Network. These classes are directly linked between them and both have a principal and more abstract class and then one class for each network type that inherit from them.

NetworkDescriptor and its subclasses are the ones responsible of establishing the structure and atributes that the network will have. All these classes have functions for a random initialization (to initialize individuals in the genetic algorithm) and functions to make changes and make easier the mutations. This would be the genotype in the evolutionary algorithm.

NetworkDescriptor and its subclasses are otherwise the phenotype, they are the network itself constructed by using the TensorFlow library. That is why they only have one function, to turn the received descriptor into a network that contains all that information.

class deatf.network.NetworkDescriptor(number_hidden_layers=None, input_dim=None, output_dim=None, max_num_layers=None, max_num_neurons=None, init_functions=[], act_functions=[], dropout=False, dropout_probs=[], batch_norm=False)¶

Bases: object

This class implements the descriptor of a generic network. Subclasses of this are the evolved ones. Parameters described in this class are the ones that are in all the types of networks

Parameters

number_hidden_layers – Number of hidden layers in the network.
input_dim – Dimension of the input data (can have one or more dimensions).
output_dim – Expected output of the network (similarly, can have one or more dimensions).
init_functions – Weight initialization functions. Can have different ranges, depending on the subclass.
act_functions – Activation functions to be applied after each layer.
batch_norm – A boolean that indicates if batch normalization is applied after each layer in the network.
dropout – A boolean that indicates if dropout is applied after each layer in the network.
dropout_probs – The different probabilities of the dropout after each layer.

remove_layer(layer_pos)¶

Removes the layer in the position received by parameter from the network.

Parameters: layer_pos – Position of the layer that is going to be removed.

remove_random_layer()¶

Selects and removes a random layer from the network.

Returns: True if the action is done or False if it can not be done.

change_activation(layer_pos, new_act_fn)¶

Exchanges the activation function in the selected layer for a new one passed by parameter.

Parameters

layer_pos – Position of the layer whose activation function wants to be changed.
new_act_fn – New activation fucntion that is going to be asigned.

Returns

True if the action is done or False if it can not be done (in this case it will always be done).

change_weight_init(layer_pos, new_weight_fn)¶

Exchanges the weight initialization function in the selected layer for a new one passed by parameter.

Parameters

layer_pos – Position of the layer whose activation function wants to be changed.
new_act_fn – New activation fucntion that is going to be asigned.

Returns

True if the action is done or False if it can not be done (in this case it will always be done).

change_dropout()¶

Change the dropout conditional. If dropout layers are used, quit them; otherwise, add them.

Returns: True if the action is done or False if it can not be done (in this case it will always be done).

change_dropout_prob()¶

The dropout probability for each layer is changed by a new random one (between 0 and 1).

Returns: True if the action is done or False if it can not be done (in this case it will always be done).

change_batch_norm()¶

Change the batch normalization conditional. If batch normalization is used, quit it; otherwise, add it.

Returns: True if the action is done or False if it can not be done (in this case it will always be done).

print_components()¶

Codifies all the components of the network in a string.

Returns: A string with all the components of the network.

class deatf.network.MLPDescriptor(number_hidden_layers=None, input_dim=None, output_dim=None, dims=[], init_functions=[], act_functions=[], dropout=False, dropout_probs=[], batch_norm=False)¶

Bases: deatf.network.NetworkDescriptor

Descriptor of a Multi Layer Perceptron, formed by hidden layers and with the possibility of selecting the number of neurons in each layer and including dropout and batch normalization.

Parameters

number_hidden_layers – Number of hidden layers in the network.
input_dim – Dimension of the input data (can have one or more dimensions).
output_dim – Expected output of the network (similarly, can have one or more dimensions).
init_functions – Weight initialization functions. Can have different ranges, depending on the subclass.
act_functions – Activation functions to be applied after each layer.
batch_norm – A boolean that indicates if batch normalization is applied after each layer in the network.
dropout – A boolean that indicates if dropout is applied after each layer in the network.
dropout_probs – The different probabilities of the dropout after each layer.
dims – Number of neurons in each layer.

random_init(input_size, output_size, max_num_layers, max_num_neurons, max_stride, max_filter, dropout, batch_norm)¶

Given the input, the output and some limits for the parametes of the network, a random initialization of the object (the network descriptor) is done.

Parameters

input_size – Input size of the network (it will be flattened in order to fit in the MLP).
output_size – Output size of the network (it will be flattened in order to fit in the MLP).
max_num_layers – Maximum number of layers that can be in the network.
max_num_neurons – Maximum number fo units that can be in each recurrent layer of the network.
max_stride – Maximum stride possible (used as 2).
max_filter – Maximum filter size possible (used as 3).
dropout – A boolean value that indicates if the networks can have dropout.
batch_norm – A boolean value that indicates if the networks can have batch normalization.

add_layer(layer_pos, lay_dims, init_function, act_function, drop_prob)¶

Adds a layer in the given position with all the characteristics indicated by parameters.

Parameters

layer_pos – Position of the layer.
lay_dims – Number of neurons in the layer.
init_function – Function for initializing the weights of the layer.
act_function – Activation function to be applied.
drop_prob – Probability of dropout.
batch_norm – Whether batch normalization is applied after the layer.

Returns

True if the action is done or False if it can not be done.

remove_layer(layer_pos)¶

Removes the layer in the position received by parameter from the network.

Parameters: layer_pos – Position of the layer that is going to be removed.
Returns: True if the action is done or False if it can not be done.

change_layer_dimension(layer_pos, new_dim)¶

Changes the dimension of the layer in position given by parameter.

Parameters

layer_pos – Position of the layer that will be changed.
new_dim – Dimension that will be changed to.

Returns

True if the action is done or False if it can not be done.

codify_components()¶

Codifies all the components of the network in a list of tuples, where the first element of the tuple is a description of the value that is in the second element of the tuple.

Returns: List with all the components of the network.

class deatf.network.CNNDescriptor(number_hidden_layers=None, input_dim=None, output_dim=None, layer_types=[], max_filter=None, max_stride=None, filters=[], strides=[], init_functions=[], act_functions=[], batch_norm=False)¶

Bases: deatf.network.NetworkDescriptor

Descriptor of a Convolutional Neural Network, formed by convolutional and pooling layers and with the possibility of selecting the filters and strides in each layer and including batch normalization, but not including dropout.

Parameters

number_hidden_layers – Number of hidden layers in the network.
input_dim – Dimension of the input data (can have one or more dimensions).
output_dim – Expected output of the network (similarly, can have one or more dimensions).
init_functions – Weight initialization functions. Can have different ranges, depending on the subclass.
act_functions – Activation functions to be applied after each layer.
batch_norm – A boolean that indicates if batch normalization is applied after each layer in the network.
dropout – A boolean that indicates if dropout is applied after each layer in the network.
dropout_probs – The different probabilities of the dropout after each layer.
layers – List of indeces that reference the type of layer.
filters – List of filters sizes in each layer.
strides – List of strides sizes in each layer.
max_filter – Maximum size that filters can have.
max_stride – Maximum size that stride can have.

random_init(input_size, output_size, max_num_layers, max_num_neurons, max_stride, max_filter, dropout, batch_norm)¶

Given the input, the output and some limits for the parametes of the network, a random initialization of the object (the network descriptor) is done.

Parameters

input_size – Input size of the network.
output_size – Output size of the network.
max_num_layers – Maximum number of layers that can be in the network.
max_num_neurons – Maximum number fo units that can be in each recurrent layer of the network.
max_stride – Maximum stride possible (used as 2).
max_filter – Maximum filter size possible (used as 3).
dropout – A boolean value that indicates if the networks can have dropout.
batch_norm – A boolean value that indicates if the networks can have batch normalization.

add_layer(layer_pos, lay_type, filter_size, filter_channel, stride_size, act_function, init_function)¶

Adds a layer in the given position with all the characteristics indicated by parameters.

Parameters

layer_pos – Position of the layer.
lay_type – Type of operation (0, 1: pooling, 2 convolutional).
filter_size – Filter size.
stride_size – Stride size.
act_function – Activation function.
init_function – Initialization function.

Returns

True if the action is done or False if it can not be done.

remove_layer(layer_pos)¶

Removes the layer in the position received by parameter from the network.

Parameters: layer_pos – Position of the layer that is going to be removed.
Returns: True if the action is done or False if it can not be done.

change_filters(layer_pos, new_filter_size, new_channel)¶

Changes the filter of the layer in the position received by parameter for a new filter specified with the size and channels received also by parameters.

Parameters

layer_pos – Position of the filter to be changed.
new_filter_size – Height and width of the filter (only square filters are allowed).
new_channel – Number of output channels.

Returns

True if the action is done or False if it can not be done.

change_stride(layer_pos, new_stride_size)¶

Changes the stride of the layer in the position received by parameter for a new stried specified with the data received also by parameters.

Parameters

layer_pos – Position of the filter to be changed.
new_stride_size – Stride assigned to that layer.

Returns

True if the action is done or False if it can not be done.

codify_components()¶

Codifies all the components of the network in a list of tuples, where the first element of the tuple is a description of the value that is in the second element of the tuple.

Returns: List with all the components of the network.

class deatf.network.TCNNDescriptor(number_hidden_layers=None, input_dim=None, output_dim=None, max_filter=None, max_stride=None, filters=[], strides=[], init_functions=[], act_functions=[], batch_norm=False)¶

Bases: deatf.network.NetworkDescriptor

Descriptor of a Transposed Convolutional Neural Network, formed by transposed convolutional layers and with the possibility of selecting the filters and strides in each layer and including batch normalization, but not including dropout.

Parameters

number_hidden_layers – Number of hidden layers in the network.
input_dim – Dimension of the input data (can have one or more dimensions).
output_dim – Expected output of the network (similarly, can have one or more dimensions).
init_functions – Weight initialization functions. Can have different ranges, depending on the subclass.
act_functions – Activation functions to be applied after each layer.
batch_norm – A boolean that indicates if batch normalization is applied after each layer in the network.
dropout – A boolean that indicates if dropout is applied after each layer in the network.
dropout_probs – The different probabilities of the dropout after each layer.
filters – List of filters sizes in each layer.
strides – List of strides sizes in each layer.
max_filter – Maximum size that filters can have.
max_stride – Maximum size that stride can have.

random_init(input_size, output_size, max_num_layers, max_num_neurons, max_stride, max_filter, dropout, batch_norm)¶

Given the input, the output and some limits for the parametes of the network, a random initialization of the object (the network descriptor) is done.

Parameters

input_size – Input size of the network.
output_size – Output size of the network.
max_num_layers – Maximum number of layers that can be in the network.
max_num_neurons – Maximum number fo units that can be in each recurrent layer of the network.
max_stride – Maximum stride possible (used as 2).
max_filter – Maximum filter size possible (used as 3).
dropout – A boolean value that indicates if the networks can have dropout.
batch_norm – A boolean value that indicates if the networks can have batch normalization.

add_layer(layer_pos, filter_size, filter_channel, stride_size, act_function, init_function)¶

Adds a layer in the given position with all the characteristics indicated by parameters.

Parameters

layer_pos – Position of the layer.
filter_size – Filter size.
stride_size – Stride size.
act_function – Activation function.
init_function – Initialization function.

Returns

True if the action is done or False if it can not be done.

remove_layer(layer_pos)¶

Removes the layer in the position received by parameter from the network.

Parameters: layer_pos – Position of the layer that is going to be removed.
Returns: True if the action is done or False if it can not be done.

change_filters(layer_pos, new_filter_size, new_channel)¶

Changes the filter of the layer in the position received by parameter for a new filter specified with the size and channels received also by parameters.

Parameters

layer_pos – Position of the filter to be changed.
new_filter_size – Height and width of the filter (only square filters are allowed).
new_channel – Number of output channels.

Returns

True if the action is done or False if it can not be done.

change_stride(layer_pos, new_stride_size)¶

Changes the stride of the layer in the position received by parameter for a new stried specified with the data received also by parameters.

Parameters

layer_pos – Position of the stride to be changed.
new_stride_size – Stride assigned to that layer.

Returns

True if the action is done or False if it can not be done.

codify_components()¶

Codifies all the components of the network in a list of tuples, where the first element of the tuple is a description of the value that is in the second element of the tuple.

Returns: List with all the components of the network.

class deatf.network.RNNDescriptor(number_hidden_layers=None, input_dim=None, output_dim=None, rnn_layers=[], bidirectional=[], units_in_layer=[], max_units=None, init_functions=[], act_functions=[], dropout=False, dropout_prob=[], batch_norm=False)¶

Bases: deatf.network.NetworkDescriptor

Descriptor of a Recurrent Neural Network, formed by recurrent layers and with the possibility of selecting the type and number of unit in the recurrent layer and allowing bidirectional layers.

Parameters

number_hidden_layers – Number of hidden layers in the network.
input_dim – Dimension of the input data (can have one or more dimensions).
output_dim – Expected output of the network (similarly, can have one or more dimensions).
init_functions – Weight initialization functions. Can have different ranges, depending on the subclass.
act_functions – Activation functions to be applied after each layer.
batch_norm – A boolean that indicates if batch normalization is applied after each layer in the network.
dropout – A boolean that indicates if dropout is applied after each layer in the network.
dropout_probs – The different probabilities of the dropout after each layer.
rnn_layers – List of the types of recurrent layers.
units_in_layer – List of number of units in each layer.
max_units – Maximum numbers of units that layers can have.
bidirectional – A boolean value to indicate if bidirectional layers are allowed.

random_init(input_size, output_size, max_num_layers, max_num_neurons, max_stride, max_filter, dropout, batch_norm)¶

Given the input, the output and some limits for the parametes of the network, a random initialization of the object (the network descriptor) is done.

Parameters

input_size – Input size of the network.
output_size – Output size of the network.
max_num_layers – Maximum number of layers that can be in the network.
max_num_neurons – Maximum number fo units that can be in each recurrent layer of the network.
max_stride – Maximum stride possible (used as 2).
max_filter – Maximum filter size possible (used as 3).
dropout – A boolean value that indicates if the networks can have dropout.
batch_norm – A boolean value that indicates if the networks can have batch normalization.

add_layer(layer_pos, rnn_type, units_in_layer, bidirectional, act_function, init_function)¶

Adds a layer in the given position with all the characteristics indicated by parameters.

Parameters

layer_pos – Position of the layer.
lay_params – Type of recurrent layer, how many units, etc.

Returns

True if the action is done or False if it can not be done.

remove_layer(layer_pos)¶

Removes the layer in the position received by parameter from the network.

Parameters: layer_pos – Position of the layer that is going to be removed.
Returns: True if the action is done or False if it can not be done.

change_layer_type(layer_pos, layer_type)¶

Changes the type of the layer in the position received by parameter for a new one selected randomly.

Parameters: layer_pos – Position of the filter to be changed.
Returns: True if the action is done or False if it can not be done.

change_units(layer_pos, new_units)¶

Changes the number of units of the layer in the position received by parameter for a new number specified with the data received also by parameters.

Parameters

layer_pos – Position of the filter to be changed.
new_units – Number of units assigned to that layer.

Returns

True if the action is done or False if it can not be done.

change_bidirectional(layer_pos)¶

Changes the layer in the position received by parameter for a bidirectional if is not already; otherwise, for a non bidirectional one.

Parameters: layer_pos – Position of the filter to be changed.
Returns: True if the action is done or False if it can not be done.

codify_components()¶

Codifies all the components of the network in a list of tuples, where the first element of the tuple is a description of the value that is in the second element of the tuple.

Returns: List with all the components of the network.

class deatf.network.Network(network_descriptor)¶

Bases: object

This class contains the TensorFlow definition of the networks (i.e., the “implementation” of the descriptors).

Parameters: network_descriptor – The descriptor with the information to generate this class (the network).

class deatf.network.MLP(network_descriptor)¶

Bases: deatf.network.Network

TensorFlow definition of the Multi Layer Perceptron.

Parameters: network_descriptor – Descriptor of the MLP.

building(x)¶

Given a TensorFlow layer, this functions continues adding more layers of a MLP.

Parameters: x – A layer from TensorFlow.
Returns: The layer received from parameter with the MLP concatenated to it.

class deatf.network.CNN(network_descriptor)¶

Bases: deatf.network.Network

TensorFlow definition of the Convolutional Neural Network.

Parameters: network_descriptor – Descriptor of the CNN.

building(x)¶

Given a TensorFlow layer, this functions continues adding more layers of a CNN.

Parameters: x – A layer from TensorFlow.
Returns: The layer received from parameter with the CNN concatenated to it.

class deatf.network.TCNN(network_descriptor)¶

Bases: deatf.network.Network

TensorFlow definition of the Transposed Convolutional Neural Network.

Parameters: network_descriptor – Descriptor of the TCNN.

building(x)¶

Given a TensorFlow layer, this functions continues adding more layers of a TCNN.

Parameters: x – A layer from TensorFlow.
Returns: The layer received from parameter with the TCNN concatenated to it.

class deatf.network.RNN(network_descriptor)¶

Bases: deatf.network.Network

TensorFlow definition of the Recurrent Neural Network.

Parameters: network_descriptor – Descriptor of the RNN.

building(x)¶

Given a TensorFlow layer, this functions continues adding more layers of a RNN.

Parameters: x – A layer from TensorFlow.
Returns: The layer received from parameter with the RNN concatenated to it.

deatf.network.calculate_CNN_shape(input_shape, filters, strides, desired_layer)¶

Given the input shape and the filters and strides that have been applied to that input by using convolutional and pooling layers, it return the output shape in the desired layer.

Parameters

input_shape – Shape of the input given to the convolutional layers.
filters – List with the filters that convolutional and pooling layers use.
strides – List with the strides that convolutional and pooling layers use.
desired_layer – Position of the layer where the output shape is wanted (if -1 is passed, it calculates until the last layer).

Returns

The output shape after applying those operations.

deatf.network.calculate_TCNN_shape(input_shape, filters, strides, desired_layer)¶

Given the input shape and the filters and strides that have been applied to that input by using transposed convolutional layers, it return the output shape in the desired layer.

Parameters

input_shape – Shape of the input given to the transposed convolutional layers.
filters – List with the filters that transposed convolutional layers use.
strides – List with the strides that transposed convolutional layers use.
desired_layer – Position of the layer where the output shape is wanted (if -1 is passed, it calculates until the last layer).

Returns

The output shape after applying those operations.