How to write this simple Neural Network in Mojo?

[rishiraj]

This is my simple Neural Network using pure Numpy written in Python. What will be its Mojo equivalent?

%%python
import numpy as np

class NeuralNetwork:
    def __init__(self, input_size, hidden_size, output_size):
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.output_size = output_size

        # Initialize weights and biases
        self.W1 = np.random.randn(self.input_size, self.hidden_size)
        self.b1 = np.zeros((1, self.hidden_size))
        self.W2 = np.random.randn(self.hidden_size, self.output_size)
        self.b2 = np.zeros((1, self.output_size))

    def forward(self, X):
        # Perform forward propagation
        self.z1 = np.dot(X, self.W1) + self.b1
        self.a1 = np.tanh(self.z1)
        self.z2 = np.dot(self.a1, self.W2) + self.b2
        self.a2 = self.sigmoid(self.z2)
        return self.a2

    def backward(self, X, y, learning_rate):
        m = X.shape[0]

        # Compute gradients
        dZ2 = self.a2 - y
        dW2 = (1 / m) * np.dot(self.a1.T, dZ2)
        db2 = (1 / m) * np.sum(dZ2, axis=0)
        dZ1 = np.dot(dZ2, self.W2.T) * (1 - np.power(self.a1, 2))
        dW1 = (1 / m) * np.dot(X.T, dZ1)
        db1 = (1 / m) * np.sum(dZ1, axis=0)

        # Update weights and biases
        self.W2 -= learning_rate * dW2
        self.b2 -= learning_rate * db2
        self.W1 -= learning_rate * dW1
        self.b1 -= learning_rate * db1

    def train(self, X, y, num_epochs, learning_rate):
        for epoch in range(num_epochs):
            # Forward propagation
            output = self.forward(X)

            # Backward propagation
            self.backward(X, y, learning_rate)

            # Print the loss every 100 epochs
            if epoch % 100 == 0:
                loss = self.compute_loss(y, output)
                print(f"Epoch {epoch}, Loss: {loss}")

    @staticmethod
    def sigmoid(x):
        return 1 / (1 + np.exp(-x))

    def compute_loss(self, y, output):
        return np.mean(-y * np.log(output) - (1 - y) * np.log(1 - output))

[mahmoudajawad]

This seemed like an interesting challenge to see how far I can go with mojo, and so far I was able to get to first line of using numpy, and I'm failing to see how I can convert numpy object to mojo:

from PythonInterface import Python

struct NeuralNetwork:
    var input_size: Int
    var hidden_size: Int
    var output_size: Int
    
    var W1: FloatLiteral
    var W2: FloatLiteral
    var b1: FloatLiteral
    var b2: FloatLiteral
    
    fn __init__(inout self, input_size: Int, hidden_size: Int, output_size: Int) raises:
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.output_size = output_size

        # Initialize weights and biases
        let np = Python.import_module("numpy")
        self.W1 = np.random.randn(self.input_size, self.hidden_size)

# Rest lines are not yet changed beyond setting function types

I have few questions to mojo team:

1. Is re-importing numpy in every function the only way to access the lib?
2. How to convert return of numpy.random.randn to something mojo can understand?

[beratcmn]

I'm curious if you find any solutions to this

[qkaTlehdrnf]

I think the point will be fully conversion to gpu-available code

[erfanzar]

Check this code out https://github.com/erfanzar/EasyDeL/blob/mojo-beta/lib/mojo/EasyDel/matrix_/matrix_struct.%F0%9F%94%A5

[erfanzar]

Copy and paste matrix and set row and cols for that for example you have input with shape 1,50 and you want to have 20 outputs its be like

Matrix1 = MatrixF32(1,50)

Matrix2 = MatrixF32(50,20)
Then matmul these two matrixs

[yashpadale]

https://github.com/yashpadale/ArtificialNeuralNetwork/blob/main/ArtifiicalNeuralNetwork.mojo
check out this link it creates a simple neural network in mojo