Artificial Intelligence

Understanding AI via neural networks

A frustration I had when looking at explanations of neural networks  (NNs) was there is typically no explanation of what an artificial  neuron does, but not what one actually is.  How is a neuron instantiated? 

Well, the logic and mathematical machinery of NNs applies to many and various instantiations.   A neuron could be a piece of card bluetak'd to a whiteboard, and links to other neurons could be shown with pieces of string.  We could write on each card the identities of the neurons that send data to it, and we could write on  the weights for reach of those inputs.  We could then manually update the inputs, maybe by writing on the whiteboard on the left, and move on to updating the weighted sums in the various layers, also using a whiteboard marker.

Another instantiation, maybe a more common one, is that the NN itself is an object, an occurrence of an NN class.  The various neurons don't need to have a very defined existence, they can be represented simply by tables of weights, weighted sums and activations ( the outputs that are allowed to be only between 0 and 1). 

I have  piece of python code that makes an NN exactly like this. 

import numpy as np

class NeuralNetwork:

   def __init__(self, input_size, hidden_size, output_size):

       # Initialize weights and biases    randomly

       hidden_layer = [0][0]

       hidden_layer_activation = [0][0]

       self.weights1 = np.random.randn(input_size, hidden_size)

       self.bias1 = np.zeros(hidden_size)

       self.weights2 = np.random.randn(hidden_size, output_size)

       self.bias2 = np.zeros(output_size)

   def sigmoid(self, x):

       return 1 / (1 + np.exp(-x))

   def sigmoid_derivative(self, x):

       return self.sigmoid(x) * (1 - self.sigmoid(x))

   def forward_propagation(self, X):

       # Calculate hidden layer activations

       hidden_layer = np.dot(X, self.weights1) + self.bias1

       hidden_layer_activation = self.sigmoid(hidden_layer)

       # Calculate output layer activations

       output_layer = np.dot(hidden_layer_activation, self.weights2) + self.bias2

       output_layer_activation = self.sigmoid(output_layer)

       return output_layer_activation

   def backward_propagation(self, X, y, output):

       # Calculate output layer error

       hidden_layer = [0][0]

       output_error = output - y

       output_delta = output_error * self.sigmoid_derivative(output)

       # Calculate hidden layer error

       hidden_layer_error  = np.dot(output_delta, self.weights2.T)

       hidden_layer_delta = hidden_layer_error * self.sigmoid_derivative(hidden_layer)

       hidden_layer_activation = self.sigmoid(hidden_layer)

       # Update weights and biases

       self.weights2 -= np.dot(hidden_layer_activation.T, output_delta) * learning_rate

       self.bias2 -= np.sum(output_delta, axis=0) * learning_rate

       self.weights1 -= np.dot(X.T, hidden_layer_delta) * learning_rate

       self.bias1 -= np.sum(hidden_layer_delta, axis=0) * learning_rate

   def train(self, X, y, learning_rate, epochs):

       for epoch in range(epochs):

           output = self.forward_propagation(X)

           self.backward_propagation(X, y,output)

           error = np.mean(np.square(output - y))

           print(f"Epoch {epoch+1}: Error = {error}")

# Example usage

input_size = 2

hidden_size = 4

output_size = 1

learning_rate = 0.1

epochs = 1000

# Create a neural network instance

nn = NeuralNetwork(input_size, hidden_size, output_size)

# Sample data

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])

y = np.array([[0], [1], [1], [0]])

# Train the neural network

nn.train(X, y, learning_rate, epochs)

# Make predictions

new_input = np.array([[0.5, 0.7]])

prediction = nn.forward_propagation(new_input)

print("Prediction:", prediction)