python code for simple neural network to solve c = a1-a2 not working

Question

I'm new to python and neural networks so I'd appreciate any assistance. trying to figure out how to make this simple NN that computes c = a1 - a2 but not sure where to start as there is no need for bias terms, if a=[(1,0),(0,0),(0,1)] then how can I compute c for each tuple? or find weights for the tensor?

training_data = np.array([[1,0],[0,0],[0,1]], "float32")
target_data = np.array([[1],[0],[-1]], "float32")

print("input : " + str(training_data))
print("output : " + str(target_data))

model = models.Sequential()
model.add(layers.core.Dense(16, input_dim=2, activation='relu'))
model.add(layers.core.Dense(1, activation='sigmoid'))
model.compile(loss='mean_squared_error',
    optimizer='adam',
    metrics=['accuracy'])
model.fit(training_data, target_data, epochs=100)
scores = model.evaluate(training_data, target_data)
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))
print (model.predict(training_data).round())

this is what I have but its accuracy (~33%) is very low and not sure what I am missing

Answer 1

Multiple issues here.

1. You are using a sigmoid output function, but your targets include negative numbers. Your model cannot work because sigmoid restricts the output to [0, 1]. In general, you have to think about your data range and what an appropriate output function can be. Here, just use no activation in the output layer.
2. Accuracy is meaningless for regression tasks as it will only count a match in case of an exact equality. Accuracy is only used for classification tasks. Thus, disregard accuracy completely and only look at the squared error/loss.
3. You are simply not training long enough. I increased to 1000 steps.
4. Note that your target function is linear, so you don't even need a hidden layer (but it doesn't hurt here, either).

With this code:

training_data = np.array([[1,0],[0,0],[0,1]], "float32")
target_data = np.array([[1],[0],[-1]], "float32")

print("input : " + str(training_data))
print("output : " + str(target_data))

model = models.Sequential()
model.add(layers.Dense(16, input_dim=2, activation='relu'))
model.add(layers.Dense(1))
model.compile(loss='mean_squared_error',
    optimizer='adam',
    metrics=['accuracy'])
model.fit(training_data, target_data, epochs=1000)
scores = model.evaluate(training_data, target_data)
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))
print (model.predict(training_data))

I get outputs [[ 1.0008523 ], [-0.00198349], [-0.99906766]].

NB the comments on your question are incorrect, you should expect a neural network to do very well on a small training set -- if it's not, something is likely wrong with your model and/or training process.