Thursday, 17 November 2016

Visual Metronome & Embedded Video / Audio Concept

Concept is to be able to play along to the written score, at the set tempo. We’ve had a bpm (beats per minute) property on a score for a while but it’s not been used for anything meaningful until now.

The experience for the user is intended to be pressing a play button of some sort, and you get a click track or visual count down for the number of beats in the bar, as determined by the time signature property on the score, then a click track plus a bouncing red ball that falls down on to each beat note, at the correct tempo, making an arc between each beat. Future revisions could have the ball bounce on each note in the score at the correct time too.

Definitely not trying to implement synthetic playback, there’s enough attempts at that out there using different approaches like sampling drummers and so on. In theory we probably could extend the functionality in a future revision but it’s not intended here. What is intended here is this lays the foundation for embedding a video or audio track of the author playing the score, which is synchronised to the written score.

As the score plays back it might be useful to be able to stop / start at different points, so having a set of controls like a video player will be useful, e.g. a volume slider for the click track,  or even a mute click track check box, we’ll also need a start / stop button, plus a location slider, i.e. slide all the way to the left takes you to the start, to the right is the end so you can position anywhere.

As the location slider moves, so the score should reflect the location in real time, so the user can see where they are at. As audio / video is added in later phases of the development, the slider should move the video in real time too.

Modern interfaces have the controls overlaid on the window and they fade out when not in use, and any mouse movement or keystroke makes them reappear. Might push that complexity to a second phase, and just start with a bar along the bottom of the window that has the controls on it, only downside is it uses dedicated space on the screen, rather than allowing maximum possible for the score to be seen.

We need to think about how to fit the score in whatever size window the user has decided, i.e. the window has standard controls for minimise, maximise, close and can be dragged to whatever size wanted. The piece of the score that surrounds the current beat needs to be visible at all times, so we must scroll the score as it plays, both horizontally and vertically but it would seem strange to not have the whole line visible, might not include horizontal scrolling in first pass to encourage that good habit. 

A zoom control might be more appropriate here, especially for reuse in later versions where an author may be looking to export a regular video of the score being played with the score embedded.

So MVP is a first-class window, bounces a red ball off the beat notes after an intro countdown, start / stop control at the bottom, this and features below all bound to the licensed version of Drum Score Editor. 

Future List
  • Add a control that allows back and forth through the score.
  • Add an audio click track to help play along, with mute and volume options.
  • Add ability to record audio while playing along, add recorded audio into the score playback, with volume option
  • Add ability to record video while playing along, add recorded video into the score playback, with volume option, it be either / or for audio and video track, i.e. only one media format embedded
  • Investigate placing a watermark in an export of the resultant video with name of who bought it and a copyright statement

Sunday, 13 November 2016

A cutting tale

When Drum Score Editor was first imagined, it was to be a drummers score editor, not an adapted music editor. One of the things I grew up with was the concept of dot and cut. I self-taught myself reading and writing music for the guitar from the age of about 10, and about the same time started to learn the side drum.

Written music wasn't a big thing back then, by today's standards it was pretty rough and ready the written scores. I was taught the opposite of a dot is a cut, and that a cut shortens the duration of a note by half it's value.

I was also aware that when you do that it counted the same as if you'd written a note that was the one with another tail or flag on the stem. But a cut note was special, it was a cut note, not a smaller note, in duration terms.

And so despite knowing there's a vaguely inconsistent thing there, I maintained the ability to mark a note as cut in Drum Score Editor. In fact in earlier versions, where the beaming of tails wasn't as complete I believed it was a necessary thing for drummers somehow. This has changed.

By having this complexity in the model inside the software it makes moving things to more advanced levels even more difficult. So from the versions after 2.4, there is no concept of a cut note, simply a cut action, which halves the note value, i.e. a semiquaver, when cut becomes a demisemiquaver. This removes a lot of code complexity when applying all the logic needed to auto beam notes.

The main factors in automatically joining note flags or tails, also known as beaming just to confuse further, are the note value, it's position within the bar, the time signature in effect at the time, whether the note is a beat note, a primary subdivision of the beat (as we tend to single beam those), if there's an irregular group in effect, or multiples thereof. Adding in an artificial "is this note cut" really took this to beyond my abilities to code up!

The one side effect for me in the code was beautify, which relied on knowing the note was cut, to be able to judge the distance along the bar to place the note. It served as a hint because although dotting a note adds half it's value and cutting takes half off the next note, we as humans don't like to see it written as the mathematical representation of that. We like interpretation, and Drum Score Editor does just that. It uses 1.666 rather than 1.5 as the multiplier and shortens the distance for the following cut note. Try it, if you write a score manually, looks too clipped if you follow the mathematical interpretation!