This is a part 2 of a project I was making, here we have a NN that's about as modular as PyTorch, hence the name ScratchTorch. These 3 Neural Network's have a size of 2-4-1, meaning 2 inputs, 4 hidden neurons, and 1 output, and each network can learn its own thing. The first network learns XOR, second one learns AND, and the third one learns OR XOR Truth Table: 0, 0 → 0 0, 1 → 1 1, 0 → 1 1, 1 → 0 AND Truth Table: 0, 0 → 0 0, 1 → 0 1, 0 → 0 1, 1 → 1 OR Truth Table: 0, 0 → 0 0, 1 → 1 1, 0 → 1 1, 1 → 1 Architecture: 2 inputs → 4 hidden neurons (tanh) → 1 output (sigmoid), trained with SGD at a learning rate of 0.1. V4: - Combined Add Activation and Add Layer - Fixed 1 output problem. - Fixed Exclamation mark. V3.9: - Added AdamW - Thats it. V3.8: - Fixed deadly bug. - Added comments. - Cleaned code. - Brought back MSE V3.7: - LOOKING FOR THIS DUMB BUG TO SMASH IT >:( V3.6: - Optimized If Else Statements. - Still looking at jumpy error, I think it has to do with not using BCE :) V3.5: - Added Init Methods - Added Xavier. - Added He. - Added Random. - Added Zeros. - Added Ones. - CURRENTLY debugging jumpy error :( V3: - Added Adam. - Added Momentum. - Added RMSProp. - Removed Sanity. V2.1: - Added Learning Rate decay. - Added Xavier initialization. - CURRENTLY adding Adam, Momentum, RMSprop V2: - Created new NETWORK_OFFSETS list. - Allowed creation and running multiple networks. V1: - Made layers connect to each other (2-4, 4-1). - Made it so the user can add specific activation functions per layer. - Optimized the code and math. - Fixed backpropagation to correctly compute hidden deltas. - Added new activation functions: ReLU, Tanh, Linear. - One network Alpha (careful, it sucks): https://scratch.mit.edu/projects/1287598642/ Made by ME #neuralnetwork #machinelearning #AI yippppppppe
