Sunday, 18 December 2011

Java2D Stroke Rendering

Summary: converting all use of strokes to rectangles! Why? It seems Java2D strokes render around the co-ordinate system rather than from it.

This may seem intuitive to some, e.g. if you have a stroke thickness set to 4.0, then it renders 2.0 of that to the left of the x co-ordinate and 2.0 to the right. I searched as much documentation as I could to try and see if this was the case but couldn't see it anywhere. I hope this post helps some other newb at some point and prevents a waste of coding time.

I guess this isn't a problem if your project considers this and makes allowances if say for example the user is nudging a widget to the left of the drawing area. The code could ensures they don't go too far left by checking for the x co-ord equal to zero plus half the stroke thickness. I'm hoping my code will be more readable that that when I come back to it in a few years time - consider how to make sure the right width is returned for a widget, ensuring it doesn't go too far right in the drawing area and so on.

Maybe it's just the way I'm wired, I think it's more appropriate to know that x is the left edge of your widget not x - (strokeWidth / 2.0f).

Thursday, 8 December 2011

2 Years On

Looking back it's 2 years since I set out in earnest to learn how to write a GUI app in Java. Not just any GUI app, but a full drum music manuscript editing app.

Looking back through the posts I have learned so much, and had assumed I knew more than I did. I've not only had to finesse the nuances of the language but also learn and understand many of the class libraries that come with Java, and they're not all consistent in their approach. Some are just incomplete.

I've also learned more about the music theory itself, plus things like printers point sizes and all the competing factions for metric versus the many different interpretations. I've also learned about the many different music score related projects out there like Lilypond and MusicXML.

Overall I've confirmed my suspicions that it's really hard for someone to be a part time developer on a project like this - there's so many aspects to it from the file handling to mouse gestures to the music engraving or artwork. And when you've got a lot to learn about the patterns used in GUI software development and language specific approaches and limitations, you end up going down a few ratholes before realising the wrong approach was taken and a few backward steps need to be taken, coding reworked and then you might progress. Then repeat several times!

It's been very satisfying to get proficient again in a programming language, to get back to working that level of technical detail. The app itself is close to being useful, a good push this winter should see a prototype editing and rendering tool, plus hopefully an input method using something other than the computer keyboard :)


Tuesday, 29 November 2011

Beams & Ligatures

Feels like the principle of iterating through the array of elements to be drawn now needs to become more comprehensive.

When drawing a note, you draw it’s stem and then any tails needed, except if the next note in the array has it’s ligatures (aka beams) extended back to this notes tail.

Also you don’t draw a notes tails if it is to extend back to the previous note but you do need to know what type of note the previous one is and where it’s stem is.

So a note needs to know the properties of the note to it’s right, and to it’s left in order to draw itself.

Now this light bulb has gone on, the approach of adding context through parameters to the paint() method could get quite clumsy. It may be better to allow each note access to read the manuscript it’s part of.

Following this approach would mean a note would fix up to the array of notes, find out it’s index in the array, then use that to access it’s neighbours. This could be quite expensive in terms of CPU cycles to perform array lookups as it’s a serial scan through an array to find it’s index.

Should we worry about CPU cost in this day and age? How much will performance be dragged down by if there’s 200 elements in the array?

A more performant approach would be to provide the references to the next and previous notes, and staff elements (you need to know if your neighbor is a bar line as that changes your drawing behavior too). This is what’s been implemented in the code today, for now, using some funky referencing while performing a single scan of the staff elements array.

My remaining concern is what else will we discover needs to be provided at paint() time in the way of context that will continue to extend complexity here? I wonder how long it will take me to get up to speed on this piece of code in 3 years time between this blog, documentation and source comments!

Saturday, 26 November 2011

Ties, Binds and Slurs

Not necessarily what you’re thinking! Supporting these in our data model, controller and view in the component is trickier than suspected.

Recent experience has taught me that traditional design approaches such as figuring out the data path requirements is a good way to approach figuring out how to implement functionality in the component. Firstly, we need to figure out which object is responsible for drawing these. Of course, it’s view in the component that will trigger this.

Our options seem to be:

  • Store an array of ties in the data model and have the view do the drawing at it’s level. If we only have a central list, this would mean the controller needs to create actions to build and amend the list, and also ensure if notes were deleted that were referenced in the list that we don’t break the data integrity in the model. The controller could use a keystroke on a selection to mean check the array, if tie exists then delete it, if not then create it. 
  • Have each note store a reference to the note at the other end of a tie it’s part of, and the second note in a tie is responsible for drawing the tie. The controller could work by taking a keystroke on a selection to mean toggle any tie between the first and last notes in the selection.

Another consideration is a tie could actually impact any future playback feature we build in and we want to think to the future, creating a good base now. If it’s an irregular grouping we need to know at the note level if it’s in such a grouping, eg ply 3 in the time of 2. I imagine playback will be executed by iterating the array of notes so the ability to ask the note if it’s the head of a tie binding could be useful. If it’s not an attribute of the note itself, it means an array scan of any central tie array for each note we process.

Doesn’t sound like a good idea from that perspective. Given we need to store an attribute which is unique to a tie, i.e. if it’s an irregular grouping, what’s the count,, e.g. 3, meaning 3 in the time of 2, 7 in the time of 6 and so on, it sounds like we need a class anyway. The tie class would store the head and tail notes of the class plus it’s irregular grouping factor. It’s probably appropriate that the tie knows how to draw itself. Whereever the drawing is triggered from, either at the component view level or while iterating through each individual notes draw routines, it’s good to separate that capability I think. Next let’s consider a note may be part of more than one tie, it’s not seen often but it’s possible, so need to cater for it. This suggests the concept of an array of ties would be a good way to store this. Given we now have a class, then a list of them isn’t too hard to implement, the question that remains for me is do we have a master list of ties, or do tie lists exist only within the notes themselves. In the latter case the note class would have a list of ties that the note is part of. While drawing the note would iterate this list, figure out if it’s the tail in the tie partnership and if so tell the tie to draw itself. So certainly each note needs a link to the ties it’s part of – that allows it to delete the tie should it be deleted itself and we protect ourselves against orphaned ties – there’s a concept!

Lastly, the note, with it's list of ties which link it to it's partner note, needs to cater for when the tie crosses between staff lines. It will be able to detect this by checking the y co-ords of the staff lines each note connects to but it won't know if they're more than one line apart. We somehow need to either get it a reference to the list of staff lines or get the controller to prevent ties being created across more than two lines. This could be close to impossible, given the user could drag a tied note down a couple of lines.

May need to find that reference to the staff array. It could be passed through the paint() calls, as today's thinking also says we need to either pass the list of notes through or at least a reference to the prior note in the array, when a note paints itself. This is so it can figure out where to draw beams back to. Oh, apparently beams are called ligatures too - hadn't heard that before but then again it's only the RSPBA exams I took on theory, nothing more comprehensive!

And something else, what I've been calling accidentals are better referred to as grace notes, as accidentals only seem to affect pitch - not something us simplistic drummers can do on a kevlar high tension surface...

Monday, 14 November 2011

To scale or not to scale ...

Doing well here, bar lines and notes of all different values but I've not really adopted a scaling model. I.E. if the user makes the staff line longer, do I make the notes wider? If they make it taller, do the notes get taller?

I'm thinking the right way is perhaps to offer a "point" model like fonts. You can set the notes to 9-point, 12-point etc - whatever that translates to in music notation rather than characters.

I deliberately didn't want to implement the component around a text font implementation. I wanted to be able to have total control over look and feel, which is ironic if I'm going back to treating this like a font problem. Not sure I'd have had the fine grained control over how to display that the note is a buzz not a tap using the systems underlying fonts though. Danger is I'm reinventing a round circular object, I think I'll call it a wheel!

Saturday, 12 November 2011

Data Model Optimisation

Going really well building out capability within the JMusicComponent, however as features are added, it becomes clear that a more experienced GUI programmer would have used a different data model. I can see now that adopting almost a traditional database approach would work, now that the access paths to the data become clearer to me.

I'm not stopping development now to rewrite the data model and access methods - don't get me wrong, I really feel like I should as when I was being taught programming, the most efficient was of doing things was a must, or the programme would potentially be useless, however in todays age of comparatively extremely powerful processors, I'm surprised at how responsive the programme still is despite lot's of serially scanning arrays on many keystrokes.

Because of that, and a statement I made this week that I'd have a working prototype up and running this winter, I must prioritise adding function rather than optimising what already works. I'm looking to get some basic, but enough functionality up to start using the keyboard drumpad idea as an input device, ie go straight from drumstroke to score. I think the way this code is structured it will really lend itself to multiple different solutions with it that really differentiate.

That needs proving, so onwards!!

Saturday, 5 November 2011

Something to show at last

Just to remind myself from time to time that progress is being made!

Selection Shmelection

I think I've got it now. I was getting too bogged down in how to integrate menus into managing the contents of the component. Ignoring all that now - as long as I use actions as the basis of all change then I can figure out menu integration with those later - just concentrate on the component itself and it's keyboard and mouse interface. May have to include a state table to manage whether the actions are disabled or not at any given point - parked!

I have a new found respect for properly trained software engineers who write GUI components. Wishing I'd actually taken some formal training on this, rather than trying to learn by just reading loads of articles on the web. There's actually so much to take in to do this properly, and as none of my formal programming training was in GUI / desktop / UI etc the right way to structure and approach the multitude of problems is very confusing to try and figure out.

I've still got to figure out Undo Manager integration with the component, that dreaded drag and drop API, CCP (copy cut paste), then there's the whole zoom scaling problem which Java provides broken support for too - you'd think if you scaled the drawing it would scale the mouse co-ord for you too, right? Any clues on that one mightily appreciated - when the time comes to tackle that again!

Onwards ...

Thursday, 8 September 2011


So here's how I'm thinking about this. A selection is an object, which contains a list of the staff elements which have been selected. We don't select staff areas, i.e. the container for the actual stave.

The model should manage the selection object, in response to the controller interpreting gestures, e.g.
  • click and hold mouse button gesture with no shift key down means we need to create a selection object; 
  • dragging the mouse while the mouse key remains held creates a visual rectangle and as that rectangle is drawn, any staff elements which fall within it have their selected property set and they are included in the list in the selection object. Only elements which are completely contained within the selection rectangle are included in the selection object;
  • when the mouse key is released, and a selection object exists, the selection object is complete, nothing actually needs to happen as a result of this gesture though;
  • click and hold mouse button with shift key down means start a new visual rectangle for including objects but any objects that become contained within the rectangle get appended to the existing selection object;
The ability to append to a selection means the selected objects need to be represented visually in a manner other than the mouse drag created selection rectangle. The elements could for example be rendered in a different way, e.g. colour blue if selected, or perhaps slightly enlarged (don't know how difficult that is). For now let's decide that unselected elements are black, selected ones are blue.

Keyboard gestures also drive selection actions.
  • Shift and arrow keys draw the same selection rectangle, ie work like the click and hold gesture, with shift down, i.e. append to existing selection else create new. 
  • If you want to unselect the selection a mouse click does that or a keyboard navigation gesture does.
  • Any new input of course will replace a selection, i.e. select 3 notes then enter a new one mean the 3 get deleted and the one new one gets placed at the start of the selection.
  • Navigation gestures need examination. Does down arrow mean go down a line and forget any selection, or does it mean move the selection down? I think it means the former, let's think about a text component example, if you select a bunch of text then hit down arrow the caret moves down and the selection is forgotten.
Probably need to integrate with the cut and paste API, which in turn integrates with that horrible java drag'n'drop API.

Time to cogitate ...

Dynamic X

In the spirit of maintaining the journal, that's the code working to ensure all elements (bar lines, notes, rests etc) placed on the line retain their relative distance apart when the manuscript area is resized, i.e. a dynamic X co-ordinate.

Onwards ...

Wednesday, 7 September 2011

No layout manager and componentResized

Swing does confuse me. It's probably me, not Swing I hasten to add. So I have no layout manager because none of the preset ones work for music staff elements, and writing one looks incredibly complex, especially if you want to reference all the components in the array that you want to lay out.

So I though no problem, no layout manager, implement the ComponentListener interface and when we're resized go figure out what that means to the staff elements contained. Works well, except for when the object is created - as it's created and it's size initially set, it's not technically resizing so no resize event. Means you end up with a component full of mush or empty on first paint, you then need to move it or lose focus and regain - all of those cause a resize event - inconsistent nonsense to me!

So it's been resolved by overriding setBounds(x,y,width,height), which now calls the super, then lays out the contents the way we want. Tracing shows we call setBounds before a repaint so it now all works the way it should. Waste of a couple of hours figuring that out though - or an investment of a couple of hours, that's the right way to think about it isn't it :)

Tuesday, 6 September 2011

Embarrassed to say ...

this is the first post this year. Meaning I've done very little to take this project forward in 9 months. The good news is the break has done my sanity some good and it hasn't taken long to get back up to speed with where I was.

The JMusicComponent is now managing multiple JStaffAreas (each containing a single staff line for pipe band drumming), and there's an array of JStaffElements sitting nicely on the line's they have an affinity to (rather than being contained within - that's important given the way notes interconnect with slurs - think this will be much easier to work with).

The to do list now looks like this:
  • ensure placement of notes and bar lines on a staff line are maintained as percentage of total line length, so that when the user increases (or decreases) the length of the line, the notes space themselves out appropriately
  • Implement UI gestures to trigger actions for creating and deleting staff lines
  • Implement the concept of selection of notes, and actions targeted upon those selections
Onwards ....