[WIP] Beats Detect & Replace

[doremi]

My journey to create a stroke sound track continues… This is Work In Progress.

Following my experiments with Kerkersklave's Beatmeter Generator and Reaper posted here:
viewtopic.php?p=319704#p319704

I actually got nowhere using tips provided by fellow creators. Either I lack brain cells to understand solutions I'm being proposed, or I lack brain cells to find out if the added sound is before, on, or after the music beat/metronome.

So, I told to myself: Why not doing it from scratch? So I studied the WAV file format, and loaded it into memory to see how complicated beats are. Here's the graph of Kerkersklave's click.wav file. It's the actual data samples imported into an Excel file.

WaveFileOpenDebug-clickwav.gif (15.57 KiB) Viewed 4914 times

It looks exactly like waves displayed in video and audio softwares, but I have the raw data to work with. Now I need to find a way to detect the beat peak, apply the same principle to the whole music track, and replace all beats with the sample beat. Wish me luck!

PS: I'm not sure but this could perhaps be interesting to those creating an estim track too.

[doremi]

(This post is left blank in case I release the app.)

[Batman314]

I'm not sure if you've looked into this, but it looks like you're just recreating drum replacement software like this. https://youtu.be/fL_Vum5krbE

There is a free version of Studio One that probably has the drum replacement capabilities in this video. There's also other free programs for drum replacement. You can also do it in most audio editing software.

-Batman

[doremi]

it looks like you're just recreating drum replacement software

Exactly!

I'll try this Studio One feature to see if it works with beats mixed with vocals. At least, it's free to try.
As they say, Thanks Batman!

[doremi]

There is a free version of Studio One that probably has the drum replacement capabilities in this video.

In the end, this proposed solution was a total failure. The free version called Studio One Prime doesn't have the Audio Bend toolbar, so can't be done.

So, I'm back to square 2 (cause my square 1 is already done).

[masperturbator]

You're trying to find beats (something like transient detection), calculate bpm (tempo) by using the distance between detected beats over time, and generate a second WAV file that is a pattern made by sequencing and looping smaller WAV files according to the detected tempo of the input file (a song.)

I would bet that Github will have some beat detection software that can do the tempo part.

Then, the parts you're missing are specific to what a beat meter is. You will need to be able to input a time signature (measure) so that your one-beat WAV file can be made into patterns that will loop, because no beat meter (beyond 1 stroke per tempo beat) is logically possible without the meter author (a program in this case) having some concept of a bar.

[nebworx]

If I rephrase to see if I understood...
You want to detect the beats of a sound track (music with voices + other noises potentially), which is different from detecting the BPM. Then, you want to be able to replace the original beats by a sound of your choice.

Is this right? In this case, it's far from obvious, it's even a research topic for signal processing researchers. Deezer has recently provided a library (Spleeter: https://research.deezer.com/projects/spleeter.html) quite innovative in this field to extract drums (for instance). I wanted to play with this to make beatmeters, but it doesn't work very well... I extracted the drums with Spleeter and used a beat detection library (ADTlib https://github.com/CarlSouthall/ADTLib). It's far from working...

[doremi]

If I rephrase to see if I understood...
You want to detect the beats of a sound track (music with voices + other noises potentially), which is different from detecting the BPM. Then, you want to be able to replace the original beats by a sound of your choice.

Exactly!

Extracting/isolating a drum from noise is certainly interesting for musicians (what Deezer seems to be experimenting with?), but I don't care for that sound for this project, only its precise position (so it can be replaced by a new sound in an empty track). I downloaded a paper on beat detection. It's Ph.D. level with complicated math, mentioning harmonics, way way over my head.

Seeing the harmonics in the above Excel file, I thought I could simplify things by concentrating on the top envelope shape, and assume that an up-down bump for a specific duration/height should be considered a beat, thus adding some little configurable parameters to the algorithm making it more precise/convenient.

[eXquisite]

If you want to easily export timestamps of the beats you can try using Audacity. It has a feature under "Analyze" -> "Beat finder..." that will mark all the beats that you can quickly export to txt file. You would need to find a way to add new beats in the exported timestamps anyway, but I thought it might be close to something you are trying to achieve.

[doremi]

If you want to easily export timestamps of the beats you can try using Audacity. It has a feature under "Analyze" -> "Beat finder..." that will mark all the beats that you can quickly export to txt file. You would need to find a way to add new beats in the exported timestamps anyway, but I thought it might be close to something you are trying to achieve.

I'm not familiar with Audacity. About these marks, can they be transfered into another Audacity track, an empty one? and then systematically add the stroke beat to each mark? That would be exactly what I want to achieve. And if so, I'll test if the detection works when beats occur during lyrics.


EDIT:
The beat detection is a little hit&miss, but it's pretty close. I tried different thresholds, but I'm not sure of its effect other than the number of generated label markers. Thanks for the suggestion.

I see a possibility to add VST pluggins. Some are made just for beat detection...

[IrishNachos]

For Vegas Pro I've found using the Vegasaur plugin works pretty good for beat detection. It's not perfect but pretty, pretty, pretty good.

IN

[doremi]

That Vegasaur is not too $much, but not free, so I continue searching. Thanks for the suggestion!

(some day I'll figure out how to add a beat meter).

I'm on this too.

[masa1mei3]

I imagine this will be really helpful for everyone. Thanks for taking on this project doremi!