THIS IS NOT THE OFFICIAL MODELS THAT ARE BEING RELEASED, CHANGE ARE STILL BEING MADE, MODELS ARE STILL BEING FINE TUNED (MythicGPT 4 mini and 4 Reasoning) Mythic 4 mini was retrained and the new benchmark scores are now updated in the project. ___________________________________________ These are the results that each of the Mythic GPT models get on a benchmark that I made myself with some help from Claude to generate a bunch of test sentences. The benchmark consists of 2421 questions on many different conversational topics and math topics, each in separate categories (11 categories total). Each of the categories consist of 25% easy questions, 45% medium questions, and 30% difficult questions. Each of the models answered each of the 2421 questions, and the answers were graded by Claude based on accuracy and comprehensiveness of each response. RESPONCE SPEEDS: I did not include this in the project because it is based off your hardware, but these are the speeds at which the models run locally on my computer outside of scratch. Go to any published model, and just calculate a ratio of how fast 4 mini and 4 Reasoning will be on your computer. Tokens/sec ___Time per reply ___ Params ------------------- ---------- -------------- ------- MythicGPT 2.0 Turbo -|- 15,760 -|- 0.5 ms -|- ~50K MythicGPT 2.1 Pro -|- 8,729 -|- 0.8 ms -|- ~180K MythicGPT 3 -|- 6,952 -|- 1.1 ms -|- ~470K MythicGPT 4 mini -|- 1,319 (4 mini v2) -|- 5.6 (v2 updated) ms -|- ~424K MythicGPT 4 reasoning -|- 725 -|- 16.4 ms -|- ~1.75M
