Playing arbitrary audio

Instructions

This is a remastered version extending the work that the original creator did (given below): https://scratch.mit.edu/projects/791629208/ The project basically uses sine waves to try and play any sound from lists, without needing to upload it through the sounds tab. Which means you can pause and rewind it! This version uses caching for less cpu usage once sounds are already played. Plus, it fixes a major glitch in the original project that made audios distorted. # Introduction The Sound blocks are undoubtedly very cool. But wait, you can only play specific sounds you have put before? That's pretty boring. What if you can play any sound, stored in something like a list? This demo shows what may be possible on Scratch in terms of sound reproduction. The music that this project is creating is made solely by a list (which is freqs). It produces a recognizable sound (although with a lot of artifacts). # How it works ## The limitations As we have discussed earlier, we can't put custom sounds on Scratch, so instead we need to somehow recreate sounds from sounds. You might have seen those MIDI players on Scratch that plays music from Scratch's Music extension, however those usually aren't enough to recreate arbitrary sounds. This demo uses something similar, however instead of instruments, it uses sine waves to reconstruct the sound. A sine wave (or a pure tone) is the simplest construct of sound. You may have heard it before—the beep when a speech is censored is a 1 kHz sine wave tone. It is as simple as it sounds, with each tone occupying just a single frequency. With such a simple sound, you may wonder how will you convert normal sounds into them. ## Fourier Transform In the early 19th century (that’s 18xx), this guy called Joseph Fourier found something that turns a function of time into a function of frequency (and phase for those pedantic people). This way, we can turn something like an audio signal into their frequencies (or, in this context, the pitch). Fourier transform (and it's derivatives) are used everywhere. Physics, audio processing, image compression, etc. ## Playback To play back the sound, It use 128 clones to play the tones. While Scratch can't overlay the same sound, it can overlay multiple sound, even on the same sprite. The clones adjust the volume of the tones according to the `current` variable, which represents the current frequencies at this time. (I'm glad audio volumes aren't global.) Currently this is set using a timer, however with some tweaks it can be seeked to a specific point in time. # Results The results of the synthesized sound contains a lot of artifacts. This may be because the spectrogram has a low resolution; about 128 tones. The gamma parameter may also have a role here; it limits the dynamic range of the sound.