Neural network with backward propagation returns the same outputs for large vectors

5

I have a problem with the neural network of backward propagation, it is written in Python , here I leave the GitHub link (in English), with a vector of 4096 entries, 2 hidden layer neurons and 197 output neurons.

I put the network to train and once finished the training I do the classification with other vectors of 4096 test entries. It turns out that these vectors have different values of inputs and they would have to return different output values and it is not like that, they always return the same output values (vectors of 197 elements with equal values).

Testing with smaller vectors (for example 10) returns different outputs, but when increasing the size of the vector, the outputs are similar to match.

Someone has an idea of how to solve this problem. I need to process vectors of 4096 elements, thanks.

Here I leave an example code with 6 inputs of 50 elements for an object that is represented by the output [1,0,0,0,0,0,0,0,0,0] . Then I classify 5 vectors with random elements:

import neural-network as red
ERROR = 0.01
from random import randint, choice
training_sets=[]
inputs = [[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0,0,0,0,0,2,2,2,0,0,0,0,0,0,0,0],
    [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,1,2,2,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],
    [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],
    [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,1,1,1,1,2,1,0,0,0,0,0,0,0,0,0,0],
    [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0,0,0,0,0,0,0,0],
    [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,1,1,2,1,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]]
for i in inputs:
    training_sets.append([i,[1,0,0,0,0,0,0,0,0,0]]) # Entrada y salida de la red neuronal para un objeto
r = red.NeuralNetwork(len(training_sets[0][0]), 2,len(training_sets[0][1]))
i=0
while r.calculate_total_error(training_sets)>=ERROR:
   training_inputs, training_outputs = choice(training_sets)
   r.train(training_inputs, training_outputs) #Entrenar la red Neuronal
   print(str(i), r.calculate_total_error(training_sets))
   i=i+1 

vectorPrueba=[] #Vector de prueba a clasificar
for j in range(0, 5):
  for i in range(0, 50):
     vectorPrueba.append(randint(0,2)) #Completo con valores 0,1 y 2 al azar
  salida = r.feed_forward(vectorPrueba)
  vectorPrueba = []
  print salida #Devuelve la salida de cada vector

After finishing the execution, the 5 output vectors that are returned to me are the following:

  

[0.9854057188339234, 0.016461838771138552, 0.017103414498752866, 0.016732153465498976, 0.01674068815079351, 0.016485508202722498, 0.016624436927601596, 0.01697716105043288, 0.01710824193364747, 0.016851276127026776]

     

[0.9854057188329309, 0.01646183877244117, 0.017103414499968748, 0.01673215346622041, 0.016740688151502595, 0.016485508203670583, 0.01662443692859318, 0.016977161051286285, 0.0171082419345771, 0.016851276128414153]

     

[0.9854057188334164, 0.016461838771695332, 0.017103414499274005, 0.01673215346557889, 0.016740688150914253, 0.01648550820299679, 0.01662443692790807, 0.016977161050612977, 0.017108241933953106, 0.016851276127679213]

     

[0.9854057188341665, 0.016461838770744055, 0.017103414498399316, 0.01673215346509605, 0.016740688150327837, 0.01648550820233476, 0.016624436927245422, 0.016977161050000102, 0.017108241933327245, 0.016851276126710966]

     

[0.9854057188340861, 0.01646183877084542, 0.017103414498492647, 0.016732153465146303, 0.01674068815039041, 0.016485508202404575, 0.01662443692731513, 0.016977161050064624, 0.01710824193339344, 0.016851276126814193]

     

[0.9854057188330729, 0.016461838772193094, 0.017103414499723312, 0.016732153465959795, 0.0167406881513114, 0.016485508203417144, 0.01662443692831659, 0.016977161051037217, 0.017108241934303412, 0.016851276128148848]

As you can see, the outputs are very similar. If I use 4096 vectors the outputs are the same, that's what I mean.

    
asked by Nico Rossello 12.09.2016 в 23:26
source

1 answer

2

I do not know which tool you use, I use neurolab ( link ), when something similar happens to me, I usually increase the neurons in the layer hidden, another thing is that the activation function, in neurolab needs inputs from 0 to 1 or -1 to 1 depending on the activation function, so you need to normalize the inputs. When they do not normalize, what you describe happens.

    
answered by 14.09.2016 в 18:12