This Lesson contains technical information regarding the data formats and the interface design for CosmoNote, although you don't need to be a programmer to understand its essence. If you have some notion of how a programming language works, the proposed activity will be more straightforward for you.
CosmoNote as a client-side web application that displays SVG (scalable vector graphics) images synchronized with audio signals and allows custom annotations from users. This is how it works:
The process of uploading Music Data to CosmoNote follows a process where the team has to prepare a series of files, formatted in a specific way.
A standard MIDI file (.mid) is first converted to the JSON format using a Python script that extracts the instructions for note numbers, note onsets, velocity values, and pedal information. The JSON file will facilitate the visualization of these data on a web page.
Stereo audio files (.wav) are recorded for each MIDI file with the Bosendorfer piano sampled at 44.1kHz with 24 bits per sample. The recording process brings two issues to be solved: a) The perceived loudness across pieces can be different, and b) the size the original files, approximately 15 MB per minute is too big for the web.
We use the FFMPEG-normalize tool and obtain compressed 160kbps EBU-23dB normalized mp3 files that are loaded quicker and are compatible with most browsers.
Music features are computed using algorithms that extract acoustic features from the audio and other descriptors from MIDI files (more on this in future Lessons). These are saved as csv files that have time samples that correspond to
CosmoNote uses a CouchDB database in a server managed by IRCAM.
CouchDB was chosen because it works well with web applications by supporting HTTP for requests (thus avoiding database drivers) and using JSON for data storage. To get all the data from the server to the client, CosmoNote uses PouchDB, which makes it run as a local application that runs directly in the user's browser.
The data visualization is done entirely using D3.js, a highly customizable, SVG-based visualization library for creating graphs using a wide variety of data. D3 takes the input JSON data and displays programatically, giving the user full control of what they see (more on next Lesson: Using CosmoNote).
Beyond visuals, D3 also handles any type of user interaction that the underlying browser supports, an important consideration for asking our citizen scientists to create annotations.