The AI’s name is Aria. Refer to it as such. Aria is given 15 seconds to complete each level, she starts off with no information on her surroundings and random movement, but she will eventually recognize how to complete each level efficiently based on data she collects while exploring. The levels get gradually harder, and I had to engineer her AI to handle every possible situation. This project took less than a month despite its complexity. How it works: Keep in mind that everything stated is by no means perfect, this is just the approach I took after evaluating my options. Secondly, this is much easier said than done in terms of code. Just the amateur text processing system located in the “speech” sprite took a good deal of tinkering and debugging to get it to finally work. Nonetheless, I’m proud of my work, so without further ado, here’s the breakdown: Firstly, it’s important to understand how this kind of AI works, which is by trial and error. If Aria were to have specific commands saying: “Once you start, move forward X steps and then jump forward over the pit. Change X by 5 every new generation until the desired outcome is reached.” That would not be AI. That is simply put; a definitive outcome. The simulation would have been the same no matter what. To bypass this, you have to start with random movement. This, atleast, is not definitive, but it is still problematic. Now it’s just up to random RNG for Aria to complete the level, and that’s not intelligence. It’s dumb luck. So how do we make it so Aria can know what to do under certain circumstances without explicitly telling her exactly what to do? The answer is a reward system. Aria starts each movement with 0 points, and one long string of numbers detailing her exact conditions. (Edge sensors, wall sensors, X, Y, etc.) She then checks a list of her previous conditions to see if it contains her current string of conditions, which I will from this point forward call her “status.” Doing things chronologically, starting at the very beginning, this list has no values, so she checks another list of conditions, except this one is for negative conditions, except this one also starts empty. So, since none of them contain her status, she executes a random movement. (Move right, move left, or jump.) Each of these movements are assigned an action ID. (1,2, and 3 respectively.) After moving, she checks to see if she is touching any checkpoints, which she almost inevitably will be. You can see them for yourself if you set the “ghost” effect to 0 in the “checkpoints” sprite. I highly recommend doing this to help you follow along. The blue ones that are most present in this level are positive, and the red ones and neutral. There also green ones that appear later in the simulation, and those perform the opposite function of the blue ones, thus, they’re labeled “negative.” If Aria is touching a positive checkpoint, she will check her previous action. If the ID is 1, or right, she earns 10 points. If not, she loses 10 points. This is to encourage her to prioritize moving right over the other directions. Reminder, this is encouragement, not a strict rule, and doing this will still result a different simulation each time, and it is not definitive, as I’ll talk about in a second. As for neutral checkpoints, they subtract 10 points no matter what. These are placed mainly as barriers, since they don’t give her a positive outcome no matter what, Aria learns to avoid them. If she’s touching multiple checkpoints, all their points are combined for the total. This is what I was talking about, with the “encouragement” not being definitive. Aria will also be awarded points if she jumps near an edge, (+5) and loses points if she hits a wall. (-5) Because the points given for these 2 actions are half of the normal 10, they never have the final say over checkpoints. Even just one of them. For example, 5 - 10 equals a negative number, even though only one checkpoint awarded points, but if the checkpoint hadn’t existed, we would have gotten 5, which is positive. Simply put, checkpoints take priority. The final point total is extremely important, and you’ll see why in a second. However, there are 2 different ways to go about this next step. I chose the latter for it’s superior efficiency, but I’ll cover both in this explanation. You can find the continuation in the “notes and credits” tab, as I have hit the word limit for this textbox. Thanks, scratch.
Explanation Continued: First option: after Aria finishes counting her points, she puts all the information neatly into a row across 3 lists. (Action performed, points awarded, and status.) She then moves on to her next movement, and repeats the process until her conditions match a value in the status list. She then checks the points she was awarded for the action she performed last time she was in this position. If the points are 0 or below, she moves the “action” and “condition” values to 2 other lists called: “failed actions” and “failed conditions” and deletes the “points earned” value. Aria then starts the movement process from the top, checking for a matching status again. (The “failed actions/condition” lists are the “negative lists” I mentioned way back at the beginning of this explanation.) Second option: she immediately checks the points values, and puts the set of values in their respective places. (Still deleting the “points awarded” value.) You can see why I chose this option over the other, though keep in mind, this would be a much longer explanation if I hadn’t covered the basis in the previous option. Moving forward, next time Aria checks for matching values in the status list, (assuming none match,) and she moves on to checking the “failed conditions” list, she’ll see she loses points for X action. For example, if she moved right last time and lost points, she’ll see that “Action ID: 1” does not benefit her in this scenario, so she’ll use one of the other 2 actions. (Left, jump, or 2, 3.) That leaves a problem though, what if she lost points for jumping AND moving right? What if she lost points for doing all 3? The solution to this is simple, you simply check every single value in the “failed conditions” list for a matching status. If you find one, you take note of it’s “failed action ID” in a list (which I will refer to as “list A”), and continue looking for more matches. Once you hit the bottom of the list, you then are faced with the problem of figuring out which movements haven’t been tried yet, assuming they haven’t all been tried that is. This next step fixes both problems. Start from the bottom, (ID: 1) and move your way up (ID: 3) searching for an empty slot. Does “list A” contain a 1? If so, does it have any 2s? 3s? If “list A” contains all 3 IDs, meaning every action has been performed, and no Aria lost points every time, then perform a random action. This solution isn’t perfect, and I have no doubt someone with more expertise could find a better way to solve this problem, but for this simulation, this solution works perfectly fine, and the conditions for this last resort to occur are rarely ever met anyway. Last thing to touch on- if a row of values ever has a positive value, it will never be moved to the “negative lists,” and instead Aria will perform the action listed in the row of values. This is why Aria will eventually end up taking a very similar path every time if you were to let her play any one level on repeat for a long time. Again, she learned the most optimal path. By no means was she forced into it, and her final iteration of her “final path” will vary from simulation to simulation. Thank you for reading this, and I hope you explore how AI works on your own too. I would like to see a lot more of these “AI learn to (whatever)” projects on scratch, because as far s I’m aware, this is the only one, and I know many people on scratch are well capable of making something similar to this with half as much effort as I had to use. Since you made it this far, please consider liking and favoriting this project, it means a lot. If you have any questions, comment on this project and I’ll see if I can help. Once again: thank you, and enjoy the simulation!
