DEV: IVA as web project?

Libraries and frameworks

Tone.js

Tone.js is a Web Audio framework for creating interactive music in the browser. The architecture of Tone.js aims to be familiar to both musicians and audio programmers creating web-based audio applications. On the high-level, Tone offers common DAW (digital audio workstation) features like a global transport for synchronizing and scheduling events as well as prebuilt synths and effects. Additionally, Tone provides high-performance building blocks to create your own synthesizers, effects, and complex control signals.

p5.js

• Could be a familiar starting point for other contributors.

Computer Vision

• Great examples of using computer vision in a browser: https://kylemcdonald.github.io/cv-examples/
• Does not work with Firefox on my computer, but seems to be related to an outdated p5 library: https://github.com/processing/p5.js/issues/3210
• Examples show great potential for usage as instruments
• Some realtime examples result in low framerates on my computer (but is already almost 8 years old).
• Libraries
  • ml5.js: Friendly Machine Learning for the Web
  • OpenCV.js: This is a JavaScript binding that exposes OpenCV library to the web.
  • clmtrackr: is a javascript library for fitting facial models to faces in videos or images
  • oflow.js: optical flow detection in JavaScript
  • tracking.js: The tracking.js library brings different computer vision algorithms and techniques into the browser environment.
  • JSFeat: Computer Vision library

How to transfer current instruments

Stimberg

• Optical flow
  • Example with p5: https://editor.p5js.org/kylemcdonald/sketches/rJg3gPc3Q
    • Works with good performance, but seems not as fine-grained as Simon's instrument, but this could possibly be enhanced
  • In conjunction with p5js and Tone.js should be doable

Traber

• Since it is based on simple brightness calculations, it should be easily transferable to the web