Works both OFFLINE and ONLINE [!] [The Type of AI where you can say anything!] >>>Number of predictions is number of text to generate LSTMs would work as memory, but this model is constrained by limits, so RNN instead (seq2seqRNN) A testing Recurrent Neural Network made for text prediction of next word from prompt of any length. (15 token max, but can be batch, trained 2x0.4 epoch) (* No this is not fine-tuned perfectly yet, or trained ideally) - use turbowarp if necessary (much faster gen) - ###v0.8 IS STILL IN TESTING, MODEL IS bit broken### No, this is not an RNN engine, rather it is only made for (maybe) future versions of GeminiAI (if it outpeforms), you could train it fine, but it may break, and lead to issues not correct tuning will cause issue, (you can't simply throw another dataset and make it tuned perfectly, you need to modify temperature var iteratively) ... it needs a ton of fine tuning in order to replicate good results from any other dataset, in my opinion it is better for you to make your own NN. If you do think of using a part this (which is a mess), credit. I will do code cleanup later. ---- GeminAI Gen 1.9 (older model), learns its sequential context from each word, then applies it across output length, which is not quite "text prediction", but more of "repharsing", newer model learns to place its repharsing capability per token while making more relevent to input Model - It is a small RNN model trained on changing words not generating, on scratch it is not possible to do such. -- notes -- This is only trained a half epoch(v.8), more means it will fit better overall, but it is enough for now. too much can cause overfitting also. outputgen is the prediciton of the words, sometimes it gets repeated, that will be fixed later === Wikipedia - https://en.wikipedia.org/wiki/Recurrent_neural_network Dataset - SCSD by @AO-85757 --- ver logs --- v.1+ 1 - find largest outputted by removal of repetition 2 - compute if next word predicted same, change model consts 3 - log model processes v.2 4 - new dot product process v.3 5 - trained by sections, not averaged, but input more limited (but capture more context) v.4 6 - majorly improved context capturing processes in input 7 - better learning rate, less overfit, faster v.5 8 - fixed range issue- now correctly interprets input (or atleast better), tuned its generation a bit v.5+ 9 - added more prev token input to gen text v.6 10 - fix a bunch of bugs and issues with training alg, model doesn't overfit during training, big improvement in generation of model also (not best fine-tuned yet, also training is a bit not yet) 11 (b) - decreased random range (because it gives alot incoherent stuff when further) v.7 12 - better referring to input (for now it causes repeating starting words sometimes), words not jumble randomly, fixed even more training errors, trained better, tuned better, .4 epoch - 1 hour 13 (b) - fixed repetition, temp issue v.8indev 14dev1 - fixed issues with calculating derivatives of weights, fixed overfitting, once for all!, fixed training slightly broken 15dev2 - traning more restricted, less random initialization 16dev3 - increased output range for text, less dependent on input 17dev4 - output is now not random as text generates the same thing over again (only weight based) 18dev5 - trained with more weight applying per matrix data (but still avg'ed to avoid underfitting) 19dev5+ - smaller starting range, higher iterations (more creative prompts) (add new tokenizer thing) find per last space, iterations from the space check if the joined letter exists in vocab (not "a" continue iterate) 20dev6 - changed response structure, sounds better on complex prompts now, (b) - changed indexing of res 21dev7 - filtered random jumble of words btw gemini is my own model referring on scratch, not google(test) - probability version - Text Gen - Gemini Development Framework Series (GDFSfTG) retrain- new fix awaiting..., the first part is interpretedtext
> Gemini 2.0 is coming.
