I was learning to play jazz on the guitar
Guitar Tab Overlayer is a web app for guitarists. It provides an interesting way to look at music for guitar.
Musical notation can be expressed in tableture as well as staff notation. A lot of guitar books show both at the same time with the staff notation on top and the tableture on the bottom. This is really nice because you have more information. The musical notation indicates the pitch and flow and structure of the music, whereas the tableture tells you how to play it. When I was practicing with a book, I noticed that I was trying to look at two things at the same time and I thought really I could be looking at one thing.
I also had the feeling that by looking at two things, I wasn't really getting twice the information. Also, because I am an intermediate trained musician, I am able to read music and understand what I'm playing. So really for me, I saw that taking the musical note and just a number for the tableture was enough for me because the string on which to play the note is determined already by the note and the fret number. I can figure this out fast enough, it's a solution. So I had the idea to just put the tab numbers over the note heads. Also the note heads convey duration such as whole notes or half notes and quarter and 8th notes being filled notehead and open. To me, this was unnecessary because I already know the rhythm.
If we look at the structure of the source code, It's a Python code base with a simple web app, served in a docker container. But the real important stuff is separated into these files. Draw tab glyphs on music extract music extract staffs extract tabs
So the glue or the high level operation is draw tab glyphs on Music so it's going to take two images an image of just the tab part tab which are part, and an image of just the musical notation part. It's going to get the glyphs from the tableture and a glyph is just a rectangle of pixels and their XY position. So convert an image to a list of glyphs. Similarly for the music image with the music, it will be converted to notes and these are just the positions of the notes. Then it is simply a pasting of one small images into a larger image. So that's the high level one.
so the high level operation needs two images. So how are those images created? Well, the user will select a cropping rectangle. After uploading an image containing a whole page, which can include multiple lines of music, the user will select the cropping rectangle and then the tool will convert that into two rectangles.
I will come back to that. Blue 42 tag placeholder it.
So how does the image processing work for the tableture? It takes an image and it will respond with a list of Glyphs, one for each note in that line of the music. I had considered using a machine learning approach, but I found that I was able to get pretty far just with basic image filtering, as we can see here.
Blue 42. Tag code section copying. As we can see here, there is a sequence of operations. So we have an image. This is a NumPy array of the pixels. We will inverse the colors, do a color threshold. We will try to remove lines. We're trying to isolate the note heads.
Blue 42 tag image. Blue 42 Tag showing lines with glyphs to extract blue 42 screenshot. Blue 42 tag resume section. Hut
So we want to remove the lines to try to make it so that we have a region of black and surrounded by white. So the black blobs should indicate where the glyphs are located. And here we just have just sort of a smattering of operations to try to get it down to something where the next part of the operation goes. So once we have preprocessed the image, then how the detection works is to find the contours and then get their locations so they can then be taken out. The Glyphs can then be taken out from that location.
One interesting thing I came up with was this P add that you see here. P stands for pipeline and it's a simple wrapper that allows me to it's syntactic sugar that allows me to have a pipeline where the input becomes an output and then the next input. But here these keys allow me to access every step of that pipeline as well as always being able to have access to the last output of the pipeline, which is what will go to the next stage.