Thursday, 17 November 2016

Visual Metronome & Embedded Video / Audio Concept

Concept is to be able to play along to the written score, at the set tempo. We’ve had a bpm (beats per minute) property on a score for a while but it’s not been used for anything meaningful until now.

The experience for the user is intended to be pressing a play button of some sort, and you get a click track or visual count down for the number of beats in the bar, as determined by the time signature property on the score, then a click track plus a bouncing red ball that falls down on to each beat note, at the correct tempo, making an arc between each beat. Future revisions could have the ball bounce on each note in the score at the correct time too.

Definitely not trying to implement synthetic playback, there’s enough attempts at that out there using different approaches like sampling drummers and so on. In theory we probably could extend the functionality in a future revision but it’s not intended here. What is intended here is this lays the foundation for embedding a video or audio track of the author playing the score, which is synchronised to the written score.

As the score plays back it might be useful to be able to stop / start at different points, so having a set of controls like a video player will be useful, e.g. a volume slider for the click track,  or even a mute click track check box, we’ll also need a start / stop button, plus a location slider, i.e. slide all the way to the left takes you to the start, to the right is the end so you can position anywhere.

As the location slider moves, so the score should reflect the location in real time, so the user can see where they are at. As audio / video is added in later phases of the development, the slider should move the video in real time too.

Modern interfaces have the controls overlaid on the window and they fade out when not in use, and any mouse movement or keystroke makes them reappear. Might push that complexity to a second phase, and just start with a bar along the bottom of the window that has the controls on it, only downside is it uses dedicated space on the screen, rather than allowing maximum possible for the score to be seen.

We need to think about how to fit the score in whatever size window the user has decided, i.e. the window has standard controls for minimise, maximise, close and can be dragged to whatever size wanted. The piece of the score that surrounds the current beat needs to be visible at all times, so we must scroll the score as it plays, both horizontally and vertically but it would seem strange to not have the whole line visible, might not include horizontal scrolling in first pass to encourage that good habit. 

A zoom control might be more appropriate here, especially for reuse in later versions where an author may be looking to export a regular video of the score being played with the score embedded.

So MVP is a first-class window, bounces a red ball off the beat notes after an intro countdown, start / stop control at the bottom, this and features below all bound to the licensed version of Drum Score Editor. 

Future List
  • Add a control that allows back and forth through the score.
  • Add an audio click track to help play along, with mute and volume options.
  • Add ability to record audio while playing along, add recorded audio into the score playback, with volume option
  • Add ability to record video while playing along, add recorded video into the score playback, with volume option, it be either / or for audio and video track, i.e. only one media format embedded
  • Investigate placing a watermark in an export of the resultant video with name of who bought it and a copyright statement

No comments:

Post a Comment