After a few hours of tinkering with run length encoding I've managed to translate a 120x90 15fps mp4 (~10mb) running in Scratch. I'm going to double the resolution and experiment with this more but this is a big accomplishment to get working in the first place.
@kasidid2 on YouTube for the original video If this gets enough attention I'll open source this on my Github. Audio timing is really off, can't seem to figure it out as it seems like a Scratch 3 issue.