Automated violin fingerings - insights
As a pet project of mine, taking advantage of my background in Artificial Intelligence, I decided to devise a computer program to calculate violin fingerings. By doing this, I have gained some insights which I thought I would share.
A computer program to calculate fingerings based on musical context (ex. timbre) was out of question, so I focused on a criterion that would be easier for the computer to evaluate: fingering efficiency. Namely, I developed an algorithm were fingerings that required the less amount of shifting, changing strings, etc. were considered better, as they are easier to execute.
Computer programming, because you have to translate everyday concepts into what is basically a mathematical model, has a way of revealing hidden corners of a problem, such as, for example, hidden assumptions or inconsistencies. For the purpose of the algorithm, I considered three standard hand postures as a basis - first finger adjacent to second, second adjacent to third and third adjacent to fourth. These correspond to the fundamental half-whole-whole tone, whole-half-whole tone, and whole-whole-half tone hand postures. I did not consider finger extensions (ex. the whole-whole-whole tone hand posture which can be uncomfortable for some hands in first position). This is actually a simplified version of what I ended up coding, but enough for the purpose of this blog entry!
Upon modeling what we consider first, second, third, etc. position, I quickly found out that we are actually simplifying the amount of positions that are really possible on the violin. In fact, for example, the position starting on F on the D string is not really the same position as the one starting on F#, although we usually call both second position. We can see that applying a standard hand posture on the first puts the 4th finger on B-flat, whereas on the second it goes up to B-natural. For all my years of playing I never considered these different positions, but alas, when I needed to formalize the concept of positions for a computer algorithm, I understood how they are actually different. In fact, for the algorithm to work well, it works not with first, second, etc. position, but rather with E-position, F-position, F#-position (all of these on the D string), etc.
Since developing the program, I have been using it to finger orchestral passages automatically (which I use as a basis for my fingerings, correcting for musical reasons when necessary). It's a nice to have (a standard 10 note passage can result in literally billions of different possible fingerings that the algorithm sorts out in a few seconds). However, developing the program has fundamentally changed how I think about positions, and I have started to think more and more about positions in terms of what note the first finger is anchored to, like in the program. Ex. now I need to shift from E-position to G#-position, not first to third. What at first was a surprising approach actually feels very logical now and I am more aware of where exactly I am in the fingerboard now. All because "G#-position" is more exact than "third position", which could also refer to "G-position" (considering the D-string here).
Other interesting insight is how often the program outputs fingerings in what we usually call half-position. Turns out violists got this right; violinists tend to underuse it because it's easy to just extend the fingers. However, a lot of passages really are easier fingered directly in half position, with much less tension in the hand. The fingerings suggested by the program have, if nothing else, at least motivated me to do a thorough study of half position!
For added historical context: Our system of positions is actually, in part, derived from the musical notation system. At first (we can confirm this in the treatise of Leopold Mozart) only two positions were considered - whole (which we would now call first, third, fifth, etc.) and half (which we would now call second, fourth, sixth, etc.) The whole position basically had notes on lines in the G-clef fingered with the 1st and 3rd finger, and notes on spaces fingered with the 2nd and 4th. In half position, this was reversed. This meant that choosing the position was basically a matter of matching the hand to two different notation patterns. These two positions then diversified into the ones we know today, which also take into account the position on the fingerboard. However, they still reflect the underlying logic of the whole and half position. They are not entirely derived from actual physical positioning, but rather the "musical" positioning as dictated by the way music is notated. This is why first, second, third, etc. positions assume different physical positions depending on what scale the music is in.
TweetReplies
Are interfaces possible with other music notation systems? Perhaps something that can work with NWCTXT in JavaScript or Lua?
Can that program interface with different music notation programs?
Would you prefer to share this program with us? ??
As a student of the Doeflien method, a lot of what you call positions the Doeflien's call "Attitudes" and they define four of them they are largely based on the location of the half steps in the major key. First attitude is with the second and third finger where the half step happens. Second is where the third and fourth fingers make the half step. Third is where the first and second fingers make the half step. Fourth is where the first finger and nut make the half step.
These attitudes align with key signatures. First attitude is used where the open string or third finger is the same note as the key signature. Second attitude is where the first finger or fourth finger is the same as the key signature. Third attitude is where the second finger is the same as the key signature. Fourth is the odd attitude in that it is only occasionally used in first position.
The attitudes come into value when moving to higher positions. Say you want to play B-Flat in second position - put your first finger on the B-Flat on the G-string and play the second attitude on the G and D strings then first attitude on the A and E strings. Over time the progression of attitude to attitude becomes second nature and when you see the key signature you automatically align your hand accordingly. Yes, we Doeflien trained folks use half and second positions a lot more than others.
Bottom line, there is no one-way to play the instrument, just different solutions to the problem of how to read and play western notated music.
Thank you for your excellent work!
"However, a lot of passages really are easier fingered directly in half position, with much less tension in the hand."
You make an interesting point in the order of historical and musical shifting. The model of all possible digital patterns for left hand fingerings is a valuable tool and it is particularly interesting that you have included reference to the Mozart treatise. I think you have more to explore in this concept of 'tension in the hand'. Sympathetic vibration utilized in musical context may be a primary factor in determining fingerings, e.g. open strings in Bach. Representing a model of efficiency based on left hand tension may use a 12-tone model analysis of the musical passage with cross-referencing to the natural resonant spaces on the violin within that passage, i.e. each string representing a just-tempered harmonic series and specific temperament of intervals within scalar passages, etc. Viewing the resistance frequency at the left hand finger 'stopping point' as a component in an electrical circuit seeking the path of least resistance with a 1 second range of musical material creating harmonic resonance in the immediate acoustical space of the instrument.
This is fantastic, thanks! I have an idea that I'd love to discuss with you if you would send me a message through my website.
Great job! Are you publishing your notated parts somewhere?
I was surprised to see so much interest in the program itself! It can definitely be integrated or interfaced with other notation systems, there is nothing platform specific about the code. It takes in a sequence of notes and outputs a sequence of fingerings (actually, right now it returns for each note, what is the string that it should be fingered on, with what finger, and the "position" in the sense that I mentioned in the original post, eg. that the first finger should be "positioned" on B-flat). The main limitation right now is that the program, being a search algorithm, is fundamentally limited by the length of the input - essentially meaning that too many notes will make it take a very long time to run. Through various optimizations it can comfortably finger phrases of 20-25 notes (or more, depending on what notes). It can be further optimized (I am working on multithreaded code), but the limitation will still be there, just perhaps at 23-28 notes, for example. This means that you cannot just input a whole piece into it, you have to divide it into phrases. There is some degree of automatism that could be introduced into this (use rests to divide phrases, or ties) but which I did not program because I use it manually, inputing phrases that interest me for my practice. As of now I am not publishing the resulting fingerings anywhere, partly because they are tailored to my personal preferences as I will explain below.
The program in its current form takes in Lilypond-type input and outputs a Lilypond .ly file which is then compiled by Lilypond into an actual score. This was done for ease of development as Lilypond is text-based, which makes it really easy to use for processing. In addition, Lilypond produces a beautiful typeset score from my output, and I do not have to worry about typesetting issues in the fingering code. I have typeset literally hundreds of pages of music in Lilypond, so I am really used to it and it was convenient for me to use.
As the program was developed for personal use, it is in its current from probably hard to use (no graphical interface for starters). I guess programmers and specially people used to Lilypond would have no trouble using it. Another consequence of being developed for personal use is that I have parametrized it to my fingering preferences. There are some parameters that control whether, for example, the algorithm prefers small shifts instead of one large shift covering the same distance, or the large over the small ones (in my case I prefer the larger one, but I know plenty of people who prefer small ones). It is easy to expose these parameters to the user, but it might not be evident what to put in them. It wouldn't be hard to have the program "learn" the user preferences by having a configuration screen where he is presented with various alternatives for eg scale fingering and has can choose the preferred one, but there isn't one as of now.
Given the interest, I might open source the code (or deploy it to some kind of website? I would do it if it cost me nothing, but the program is processor intensive, so I do not know if it would be suitable to reside in the resource-limited webhosting environments that free webhosts provide). It would probably be beneficial to be used by different people because 1) they can give feedback, resulting in algorithm improvement and 2) if a lot of people use it and input real musical examples, the program can store the inputs and results and use that to get faster over time, using a technique similar to what chess programs use to speed up calculations in the endgame.
Phew! That was regarding the program and development.
I am happy that George identified a method with a similarly different approach to positions, I actually searched for different approaches when developing but found none, but am not surprised at all that they exist. I will see if I can come across a copy of this method and look at Doeflien's approach. Do you find it advantageous to study positions in this way, rather than the traditional approach? (Perhaps a hard question to answer, but I am really interested on your opinion.)
Regarding the use of eg resonance with open strings as suggested by Kevin, or even stopped-string resonance and/or afterlength resonance, in order to produce musical fingerings, it is evidently possible as you suggested. These and other musical characteristics that can be easily quantified (ex. principles such as "stay as much in low positions as possible to optimize volume" or "finger a single musical phrase in so few strings as possible for even timbre") can be easily integrated into an artificial intelligence program such as mine. Where it gets difficult is that this kind of musical optimization is rarely used for a whole piece or even musical phrase. The difficulty then, of course, is where to apply this logic. That's a whole different level of difficulty than just figuring out the most efficient fingerings based on effort metrics.
David, one way to speed it up would be to use a moving window approach -- you don't need to be concerned about how every note fingering relates to every other, but only those that are as near each other as you choose. Computing time increases exponentially with input length the way you are doing it now I suspect.
A moving window approach would be locally but not globally optimal. This is a common trend in artificial intelligence search/planning. The truth is that the problem space itself (ie the number of different possible fingerings) does grow exponentially with input. I think the advantage of the program over me fingering manually is precisely that I can only really consider a limited amount of information for fingerings (eg a context of about 6-8 notes realistically - this is why fingering often is derived from standard solutions, rather than actually considering ALL of the possible alternatives), while the program goes much further to 20-25 (considering literally billions of solutions in some cases, although of course through standard search optimizations such as alpha-beta pruning and partial solution caching, the program does not actually need to check them one by one to find the optimal one). Of course most of the time the solution is a "obvious" one, but I have found plenty of surprises, and non-obvious/conventional solutions such as a thorough use of, for example, half and even positions, and also the clever use of open strings to avoid shifting. Note that I did not program these optimization "strategies", they just surface naturally by considering all possible alternatives and choosing the best one as regards efficiency.
Very often the last two notes do make a difference in how to finger the first two ones, because in order to avoid extra shifts in the middle, I have to already be in a suitable position when starting the phrase. In fact, because of this "global optimization", it was even hard to find a meaningful example for this post because there are few examples in the orchestral works I was considering that require a solution more complex than two or three shifts! (Which I found by feeding excerpt after excerpt in the program).
In my day-to-day use of the program, of course I am sort of using a moving window approach. Since at excerpts longer than 25 notes can start to take more than 10 seconds to compute, when I have excerpts longer than that, I divide them up (when possible, dividing into musical phrases). Luckily, this kind of division is often evident and basically corresponds to how I would divide the work logically If I was fingering it manually.
Sounds interesting. There are standard ways to try and change complexity from exponential, which is worst as you know. Using a window helps but just minimally because it doesn't change the complexity. Yet the last note in the piece doesn't care how the first note was fingered :) Also maybe something like Azure ML could help.
If it was me I would code it up in something and run it on the back end of a server, if I wanted to make it available yet protect the code.
Hello David. That's a great article you have there.
I'm actually working on something similar based on the MuseScore plugin:
https://musescore.org/en/project/fiddlefingering
Unfamiliar with violin fingering notation, I made it look more like guitar tabs:
http://corpuschristiirishsessions.com/files/misc/tab-sheets/set-list-tabs/Tam%20Lin%20-%20Fiddle.pdf
This is as far as I could get for the violin.
-
After that, I made a version for the mandolin showing all the possible positions on the fretboard:
http://corpuschristiirishsessions.com/files/misc/tab-sheets/set-list-tabs/Tam%20Lin%20-%20Mandolin-20.pdf
Now I started seeing patterns!
This helped me practice different fingering positions on the mandolin before trying them on the violin.
What you mentioned in your article is exactly what I saw in this particular tune: the F#-position on the D string worked the best.
--
I didn't make much progress beyond the simple violin fingering notation. So, it was very interesting to hear someone working on that. Not only that, but using artificial intelligence.
What I can imagine is your computer algorithm looking for the most efficient path as shown in the tabs. Even with the tab fret positions, I'd still have to figure out what fingers to use manually.
I hope this helps your algorithm, or maybe if you can interface it with MuseScore.
This sounds very interesting!
Would it be possible to input other parameters apart from tension in the hand, such as reducing the amount of string crossings, or the number of shifts in a piece, to make this adaptable to the individual preferences of the players?
Many thanks,
Henriette de Vrijer
This article has been archived and is no longer accepting comments.
Violinist.com is made possible by...
Dimitri Musafia, Master Maker of Violin and Viola Cases
Thomastik-Infeld's Dynamo Strings
Violinist.com Summer Music Programs Directory
ARIA International Summer Academy
Johnson String Instrument/Carriage House Violins
Laurie's Books
Discover the best of Violinist.com in these collections of editor Laurie Niles' exclusive interviews.
Violinist.com Interviews Volume 1, with introduction by Hilary Hahn
Violinist.com Interviews Volume 2, with introduction by Rachel Barton Pine
December 17, 2016 at 05:34 PM · I find this really interesting. Just yesterday I was working with a student on Kreutzer Etude No. 17, using the Galamian edition. Toward the end are some double-stops, and she was really struggling with the fingerings. Specifically it was a G flat (on the D string) and a D flat (on the G string), fingered 2-3. I rewrote it as an F# and C# and said, "Does the fingering seem strange if the notes are written that way?" Her response, "Not at all, that makes sense now!"
So our perception of the notation is a huge factor.
I often find myself telling students that "this would actually be very easy if you weren't reading the music and just doing what was easiest to do on the violin!" Sometimes the most "violinistic" fingerings just don't translate well to the page.