Interactive Audio Sculpting: Plugin Customization and UI Affordances in Immersive Environments

Overview & Synopsis

WAM Jam Party is a collaborative WebXR platform for creating 3D musical installations using Web Audio Modules (WAM) plugins. Expanding on earlier work, this paper introduces major improvements focused on usability, expressiveness, and interaction design. A new 3D GUI editor allows users to attach interactive, custom graphical interfaces to existing WAM plugins without modifying their source code, enabling representations such as synths with distinctive textures, labels, 3D knobs etc. or physics-based instruments like a guitar with a body shaped like a real acoustic guitar. The design offers a "3D Plugin Shop" and floating menus with preview thumbnail images as alternative means to select and assemble audio components. The system allows dynamic code injection for advanced behaviors, including animated parameter modulators, piano roll sequencers or an animated drum kit. User studies report higher engagement, faster onboarding, and creative workflows. This study also gives some practical insights for building collaborative audio tools in immersive web-based environments.