3.0 WIP Neural network framework

Instructions

Quick startup guide: https://scratch.mit.edu/projects/1114225395/ !!IMPORTANT!! (Long version of instructions below) How to use: IMPORTANT NOTE: Avoid using any of these symbols in function arguments when calling any of the functions (because they will confuse my poorly coded semi-OOP system): ".;/" (period, semicolon, and the forward slash. Quotation marks are not included; they are used for delimitation.) Avoid using spaces as well. Avoid duplicate names, i.e. creating a neural network with the name "joe" and then creating another one with the same name, "joe". Instead, try doing, "joe1" and "joe2". function CreateNN <Name: string> <Optimizer: string> <Cost Function: string> <Learning Rate: number> - This essentially creates a blank template for a neural network with the name you specified. You will need to add layers and then call the connect layers function later. function Create Layer NNName: <Name: string> <LayerType: string> <NumNeurons: number> <ActivationFunction: string> <WeightMin: number> <WeightMax: number> - Adds a layer of type <LayerType> (which can be "input", "hidden", or "output") to the neural network <Name>. Adds <NumNeurons> neurons to the layer, with the activation function <ActivationFunction> (see possible activation functions below) for the entire layer. Additionally, it defines the <WeightMin> and <WeightMax> parameters, which will act as the "limits" when generating random weights and biases. function Connect Layers for <Name: string> - Connects all the layers together for the neural network <Name>. If you've ever seen a neural network diagram, you can think of this as "creating the strings" that connect each neuron. function Forward pass for <Name: string> - MAKE SURE TO ADD THE INPUTS IN THE LIST NAMED "inputs". What this does is pretty self-explanatory, it takes the vector of inputs and it runs it through the neural network --> outputs. THE OUTPUTS WILL BE LOCATED IN THE "outputs" list. function Save <name: string> - Sets the variable named "return" to a serialized, compact version of neural network <name>. function Load <data: string> - Takes in a serialized string (created from the save function) and loads it into the test list. function Compute Cost for <Name: string> - Uses the target outputs list and the target outputs list to compute a cost value. The target outputs list is pretty self-explanatory. It holds the target outputs (what the model should have printed) given the inputs in the list inputs. - (Has to be done after setting the target outputs & doing forward propagation) Computes the cost value, storing it in the variable named "Cost". The only cost function supported right now is MeanSquaredError (I'm not that great at neural networks, so I probably won't be adding more any time soon). - This is mainly for printing out the cost, i.e. if you were trying to graph it. function Calculate gradients for <Name: string> <(optional) clip_gradients: boolean, enter 0-1> <(optional) min: number> <(optional) max: number> - Also HAS to be done after forward propagation. There must be target outputs in the target outputs list, otherwise it will throw an error. - This calculates the partial derivatives given the target outputs in the target outputs list and what your model outputted in the outputs list. It does not do any actual optimization yet. - Optional clip gradients parameter if you want to apply gradient clipping (clipping the gradients between a certain range, i.e. -1.0 to 1.0) function Optimize for <Name: string> !!EXAMPLE CODE IN NOTES & CREDITS B/C I'M REACHING CHARACTER LIMIT!! Possible activation functions: - ReLU - Sigmoid - TanH - None --> If you want to add more, it's actually pretty easy. Look for the "activation function" function definition, then make a new if statement for the activation function you want. Follow the format provided by previous if statements. Finally, add the new activation function to the "possible activation functions" list. Aight, sorry for the spaghetti code if you uh ever try and see inside...