Archive 2022: AR Performance Capture: Playing Music with an Augmented Self

A live performance experiment combining augmented reality with hardware synthesis. An iPhone captures a real-time AR overlay while the Moog DFAM synthesizer performs underneath — blurring the line between physical instrument and digital layer.

The setup routes three synthesizers — Moog Matriarch, Moog DFAM, and ASM Hydrasynth — through a Focusrite interface into Logic Pro. A custom Unity application built with Apple ARKit renders the augmented overlay directly onto the iPhone's camera feed during the performance.

Process

The experiment succeeded as a real-time capture, though the audio required post-production. The original iPhone recording lacked the multi-track fidelity of the studio session, so the audio was re-recorded in Logic Pro and synchronized with the visual capture. The result is a hybrid artifact — part live performance, part studio production.