Beatbox to Real Drums
My idea is to create an app in which people can record their organic beatbox and translate it into a drum loop with real drum samples.
As an electronic music producer, I often have ideas for drum beats in my head, but when it comes to translating the ideas in my head into a digital audio workstation (DAW) I often lose a lot of the idea. It’s like trying to remember a dream and it’s just out of your grasp of memory. But what if there was an app where I could open it, record the drum beat in my head, and the app would automatically pick up and translate it from a library of sounds to the closest match of the sounds of the voice being recorded, and be able to drop that file into your DAW?
So what is the idea?
An app that translates a drum beat made by voice (known as ‘beatboxing’) into a real drum beat that can be used in music production software, or digital audio workstation (DAW).
What problem is this solving?
The problem that this project solves is the issue musicians have in losing their idea for a drum beat. It is very difficult as a musician to keep an idea alive in your mind before you can translate it accurately into your DAW.
What will it do?
This app would record sound from the users mouth, match the sounds to a library of drum samples to find the closest match of the sounds being made, and then create a midi file that can be uploaded to the user’s DAW.
How will it work?
It will require a lot of hard work.
- This app will need access to the user’s microphone and will pick up a beatbox audio recording.
- The app would need to process the audio file to find key sound signatures to match what might sound like a bass drum, snare, or hi-hat.
- The app would then need to process the recorded audio and match the recorded sounds to any matches to the drum sample library which would be a component of the app.
- After the beat is processed, the idea would be that the musician captures the rough idea into a MIDI file that can be used in the user’s DAW. If it can be translated into a MIDI file, that enables the user to adjust the samples used for each component of the drum kit, change the positions of the notes, augment, add, or subtract notes, etc. My thought is that if it were an app, it would be pretty easy to upload into the user’s computer, where typically DAW environments exist.
What challenges I anticipate:
So many challenges to consider for this idea, especially for someone with limited coding experience. At this point it is just an idea, but there are plenty of challenges to overcome to make this app successful.
Here are the components that I will need to research:
- Creating an actual usable app
- Recording audio from user’s devices
- Have app include a small library of drum samples
- Analysis of audio
- Matching analysis of audio to drum sample library
- Translating beatbox audio to drum sample MIDI file
- How to create a MIDI file on mobile devices
Subtasks and considerations:
- Before recording the beatbox audio, can the user choose their BPM (beats per minute)?
- Would there be a metronome sound while the user is recording?
- After recording the beatbox audio, would the beat that is translated be quantized (matching the sound to the closest up or down beat)?
- How will the matching technology work? After the beatbox audio is recorded, how can the app have enough intelligence to match the sounds to real drum samples?
Key components of my project:
What research have I done and/or still need to do?
I have done preliminary research on this topic to see if such a technology exists, and I have not been able to find anything that matches what I am trying to accomplish. I have found apps that have the individual components I want to include, but nothing that includes the full scope I am trying to accomplish. See below for apps I will be researching in more detail to understand the technology.
What resources, skills, and technologies are needed to realize it?
In terms of skills needed for this project, it will require a ton of learning on my end, especially with coding, recording audio technology, machine learning, audio matching, the creation of MIDI files on mobile, and much more.
Technologies needed for this app to be successful:
- Audio recording, upload and display of audio (Like Apple voice memos)
- Analyze the audio and match parts of it to individual sounds (like Celebrity Voice Changers)
- Ability to use a library of drum samples (Like Garage Band for Mobile)
- Ability to translate beatbox to a drumbeat as a MIDI file to be used on the user’s DAW (like TIZE — Beat Maker for Mobile, or n-Track Studio 9 for Mobile)
What kind of fidelity do you want to achieve?
In terms of the fidelity of this project, as long as the app could get around 80% of your original idea in your head to a usable midi file for DAW, I would consider the experience to be a success.
Where do I am lacking in skill and need assistance:
There are plenty of areas I need help with, and hopefully after my research I will be more confident in how to accomplish what I have set out to do. But it would be extremely helpful to have insights into a few things:
- How to actually code an app that records, stores and has viewable audio recordings.
- How to match audio a user would record with a library of existing sounds.
- How to create MIDI files on an app.
- How to upload a library of sounds onto an app.
Project Timeline:
11/9–11/16: Research, User Interviews, Deconstruct User Flow
11/16–11/23: Sketching, Digital Prototype
11/23–11/30: Code and Build
11/30–12/7: Polish and Present
