Supporting music composition with interactive paper

Overview & Synopsis

This thesis focuses on the design of interactive paper interfaces for supporting musical creation. Music composition has been deeply influenced by the computational power brought by computers but despite the use of software to create new sounds or work with symbolic notation, composers still use paper in their creative process. Interactive paper creates new opportunities for combining expression on paper and computation. However, designing for highly individual creative practitioners who use personal musical representations is challenging. In this thesis, I argue that composers need personal and adaptable structures on paper in which they can express and explore musical ideas. I first present three field studies (Chapter 3) with contemporary composers that examined the use of paper and the computer during the composition process and how linking the two media supports exploration of musical ideas. I then describe a participatory design study that investigates the use of formal musical representations (Chapter 4) for creating new paper interfaces that extend computer-aided composition tools. I introduce Paper Substrates (Chapter 5), interactive paper components that provide modular structures for interacting with personal representations of computer-based musical data. I detail tools that we created to develop paper applications with the Paper Substrates approach. Several examples illustrate the creation of personal structures and musical content that can still be interpreted by computer-aided composition software. I then describe a structured observation study with 12 composers who used Polyphony to compose a short electroacoustic piece (Chapter 6). Polyphony is a unified user interface that integrates interactive paper and electronic user interfaces for composing music. The study allowed us to systematically observe and compare their compositional processes. Finally, I report on a research and creation project with the composer Philippe Leroux during the composition of his piece Quid sit musicus (Chapter 7). Several work sessions with the composer and a musical assistant lead to the design of new paper- based interfaces for generating composition material, synthesizing sounds and controlling the spatialization from handwritten gestures from calligraphic gestures over an old manuscript.