Binaural Piano VST Plugin

This project was created as a contribution to the AES MATLAB Plugin Student Competition 2018.

This plugin is a demonstration that shows how MATLAB can be used to generate VST plugins for Digital Audio Workstation (DAW) software. The plugin converts MIDI note messages from a MIDI device and head orientation data from an Inertial Measurement Unit (IMU) based low-cost head tracker into a binaural piano signal, where low tones come from the left and high tones from the right, just like an ordinary piano.

Instead of using recorded waveform sample libraries of piano tones and measured HRTF data we focused on synthetic sound generation based on appropriate piano and HRTF models, respectively. The advantage of this approach is a slim plugin demonstrator without any dependencies on huge waveform libraries. For the sake of simplicity, the piano sound generation is reduced to a simple and low complexity Karplus-Strong based string excitation model and can be switched to pure tone organ sound. The HRTF model is given in closed-form.

The VST plugin itself is designed with MATLAB's Audio System Toolbox which uses the powerful MATLAB Coder to generate standalone machine code from plain MATLAB code. However, it does not offer the full functionality of MATLAB. For instance, a sophisticated graphical user interface, full MIDI support or MEX function calls cannot be realized with the MATLAB Coder (at the moment, at least). Therefore, we combined the advantages of MATLAB interpretation and standalone machine code by introducing a MATLAB helper app which communicates with the VST plugin via UDP. The helper app is a relay to forward MIDI and head tracker data to the VST plugin and offers a simple software keyboard.

Usage instructions can be found in "README.md".