<\/p>\n
Trained musicians,\u00a0while listening to music performances,\u00a0are able\u00a0to transcribe them into Common Western music notation. We are studying computational methods to automate this process by\u00a0parsing<\/strong> an input sequence of timed music events\u00a0(a\u00a0MIDI file) into\u00a0a structured music score in a format such as XML\/MEI.\u00a0Our approach\u00a0is\u00a0based on two main ingredients.<\/p>\n First, we use quantitative language theoretical<\/strong><\/a>\u00a0models and techniques, in order to<\/p>\n Second, we created an abstract Intermediate Representation<\/strong> of music scores, hierarchical, that can be built from the (parse-trees) returned by the above parsing algorithms, and exported into XML score formats. The design of this IR is a research topic per se, described below.<\/p>\n This method enables to study of theoretical foundations for the problem of music transcription, and it instantiates into several case studies, such as the following.<\/p>\n Monophonic transcription<\/strong><\/p>\n In the case of melodies with at most one note sounding at a time, we were able to\u00a0improve the level of complexity and details of transcriptions, in particular for handling ornaments and rests,\u00a0as illustrated in these <\/span>examples<\/a>.<\/span><\/p>\n The above examples were taken from a dataset<\/a> made of about 300 extracts from the classical repertoire, with increasing rhythmic and melodic complexity, provided as XML (scores) and MIDI (performance) files.\u00a0We have built this dataset for the evaluation and training of our tools. It shall also be useful to the community as a landmark for the evaluation of the transcription of complex melodies.<\/span><\/p>\n Bassline transcription<\/strong><\/p>\n In collaboration with John Xavier Riley from C4DM<\/a> at Queen Mary University London, we are working on an approach for end-to-end transcription<\/strong> of jazz basslines, following a 2-steps workflow: a MIDI file is first extracted from audio recordings of jazz standards, with source separation, pitch, and onset estimation techniques, and\u00a0our transcription tools are then applied as a back-end to provide a music score. Significant improvements have been made to the technics developed in order to deal with the swing<\/strong> and the particular pitch-spelling issues (see below) corresponding to the case of jazz.\u00a0<\/span>Some preliminary results may be found here<\/a> and here<\/a>.<\/p>\n Drum transcription<\/strong><\/p>\n The output of electronic drumkits can be recorded into MIDI files.\u00a0Google Magenta provided a dataset<\/a> of more than 13 hours of MIDI recordings of drummers\u00a0on a Roland TD-11<\/a>\u00a0kit. After some successful first experiments presented in these international workshop<\/a> and conference,<\/a>\u00a0w<\/span>e are now\u00a0conducting a effort for the \u00a0transcription of this dataset into drum scores, based on our tools.\u00a0This work, in the context of the thesis of\u00a0Lydia Rodriguez-de la Nava, benefits from the expertise of\u00a0Martin Digard, a professional drummer holding a Master’s degree in Natural Language Processing from INALCO. It is also the first real-case application of our transcription approach to a polyphonic instrument.<\/span><\/p>\n Piano transcription<\/strong><\/p>\n Scaling up from monophonic to polyphonic instruments is a difficult step in the context of transcription. This is one of the main topics of the\u00a0thesis of Lydia Rodriguez-de la Nava. She is developing voice-separation algorithms for this purpose and integrating them into our transcription framework and models. This work is helped by the use of the dataset ASAP,<\/a> made of linked piano scores and MIDI performances, published in\u00a0collaboration with\u00a0Francesco Foscarin and Andrew Mc Leod.<\/span><\/p>\n We are developing an abstract\u00a0Intermediate Representation<\/strong> of music scores, for various\u00a0Music Information Retrieval problems such as transcription or music score analysis.\u00a0<\/span>It enables us to deal with various score file formats (XML or plain text) in input or output, to reason on and apply transformations to music content without the hassle of decoding these formats.<\/p>\n The main originality of our model is its tree structure<\/strong>, even at low levels (e.g.<\/em> for the description of rhythms).\u00a0A salient feature of music notation is indeed its hierarchical nature: events are grouped into bars, tuplets, and under beams, and durations are defined proportionally and composed with ties and dots.\u00a0<\/span><\/p>\n This model is used in particular in our transcription procedures, for post-processing transformations, by term rewriting<\/strong> of the scores obtained.\u00a0It\u00a0was also used for training automata models\u00a0on corpora of digital music scores. \u00a0<\/span><\/p>\n During her thesis, Lydia Rodriguez-de la Nava is developing algorithms for the separation of polyphonic music content into\u00a0voices,\u00a0differentiating the different melodies, or chords,\u00a0that we are able to perceive by listening.\u00a0This problem is crucial in the context of music transcription,\u00a0to produce easy-to-read scores, where\u00a0the lines of melodies, accompaniments, bass lines, etc.<\/em>,\u00a0are clearly represented.<\/p>\n She\u00a0proposes a voice separation algorithm,\u00a0which is based on principles of perception\u00a0and is flexible with respect to The height of music notes (pitch) is represented in the MIDI standard<\/a> by the corresponding number of piano keys (from 1 to 88, one semitone for each).\u00a0In CW music notation, however, it is described by a note name (the distance between names is one tone or one semitone) and an optional\u00a0modifier, which is a positive or negative number of semitones (represented by sharp or flat symbols). Therefore, several notations are possible for each piano key (for instance C#<\/em> or Db<\/em> for 73). Choosing appropriate ones is not obvious as it depends on context (direction, tonal context…).<\/span><\/p>\n We have been proposing two\u00a0approaches for the joint estimation of pitch spelling<\/strong> and key signatures<\/strong> from MIDI files.<\/p>\n We are studying metrics of similarity between melodies, defined as\u00a0edit distances<\/strong> for character strings (ED) and labeled trees (TED). In particular, with Mathieu Giraud (Algomus<\/a> team, Lille), we have conducted an in-depth <\/span>theoretical study<\/a> on the computability of an ED\u00a0introduced in 1990 by Mongeau and Sankoff<\/a> and\u00a0widespread in the\u00a0Music Information Retrieval community.<\/span><\/p>\n Moreover, we are developing solutions for comparing digital music score files. This cannot be achieved reliably by running a simple Unix diff procedure on the two (XML) text files, due to the ambiguity and verbosity of the score formats (without mentioning incompatibilities between different formats, different versions of the same format and files produced by different software). W<\/span>ith Francesco Foscarin, we have proposed a procedure<\/span>\u00a0based on an ad-hoc similarity metric combining several ED and TED, and much more involved than for the case of text files (Unix diff utility), because of the complex structure of music scores. This approach has been presented\u00a0in this conference<\/a>\u00a0and this conference<\/a>.\u00a0<\/span>It is currently used in a c<\/span>ase study on Crowdsourced Correction of\u00a0Optical Music Recognition output for a musicological collection with IReMus lab.<\/span><\/p>\n All the above research topics find application in the constitution, management, and search and retrieval of databases of music scores, for cultural heritage preservation and study.<\/p>\n <\/p>\n <\/p>","protected":false},"excerpt":{"rendered":" Research Interests Symbolic\u00a0Music information processing Quantitative extensions of formal language models Structured representations of\u00a0Music notation Hierarchical representations of digital music scores \u2028Prior languages of music notation style Search and Retrieval in symbolic musical content Similarity metrics and edit-distances Research Topics Automated Music Transcription Trained musicians,\u00a0while listening to music performances,\u00a0are able\u00a0to…<\/p>\n\n
Music Score Model<\/h4>\n
Voice separation<\/h4>\n
\nthe genre of music and the instrument. It works, roughly, by searching a shortest path in a graph whose vertices are partitions of events into voices at every date. Some rules are defined in order to associate a cost to every such vertex, and another cost to transitions between vertices (at successive dates). This algorithm is evaluated on datasets of the Music 21<\/a> toolkit and on our dataset ASAP<\/a>.<\/p>\nPitch Spelling<\/h4>\n
\n
Melodic similarity evaluation<\/h4>\n
Digital music score collections<\/h4>\n