Cybernetic Texture — for robotic arm, MPU6050, a box of objects and Max/MSP
Year: 2026–2027 (in progress)
Master’s thesis piece, Wuhan Conservatory of Music, supervised by Feng Jian
The arm plucks, stirs, strikes and grabs. When it plucks the hanging accelerometer, the sensor swings freely. The swing depends on the force of the pluck, the length of the cord and air resistance, so the data that reach Max/MSP decay in a regular cycle but are always slightly disturbed. When the arm stirs, strikes or grabs the objects in the box, they collide with one another in chains that cannot be fully predicted. Random modules in the Max patch add a third layer of uncertainty.
I use “cybernetic” to mean feedback rather than full control. The sensor data are analyzed and processed in Max/MSP in real time and take part in the next round of sound generation. This closes a loop of action, feedback, processing and regeneration, in place of a one-way trigger and response.
The sound has three materials: the noise of the arm’s joints, string-like timbres, and the collisions in the box. The idea draws on two images from physics, string theory and the motion of celestial bodies: the path of the swinging sensor shapes the sound design of the objects in the box.
(2'58"-3'31") dancing ≈ non-linear ≈ positive feedback
Capture Uncapturable — for sign language and MediaPipe
(Python, Max/MSP)
Year: 2025
Duration: 6′30″
Presented:
Accepted, Student Consortium, NIME 2026 (New Interfaces for Musical Expression), London (Jun 2026)
Selected, ICMC 2026 (International Computer Music Conference), Hamburg (May 2026)
Electronic Music Marathon Showcase, 3rd Summit on Music Intelligence (SOMI), Central Conservatory of Music (Apr 2026)
Duration: 6′30″
Presented:
Accepted, Student Consortium, NIME 2026 (New Interfaces for Musical Expression), London (Jun 2026)
Selected, ICMC 2026 (International Computer Music Conference), Hamburg (May 2026)
Electronic Music Marathon Showcase, 3rd Summit on Music Intelligence (SOMI), Central Conservatory of Music (Apr 2026)
The central image is the sign for "strobe camera" in Chinese Sign Language. The hands become a camera that freezes what we cannot normally see: magma moving and rock breaking inside the earth, and water, forest and rain on its surface. The other signs in the piece are fire, sea, rain, landform and Danxia landform. All of them come from the National Common Sign Language vocabularies for physics and geography, and some are adapted for the piece.
I perform the signs myself. MediaPipe tracks the landmarks of my hands from a camera, and a Python bridge sends them over OSC into Max/MSP. There the data is tested against thresholds I set by hand: the distances between fingertips, how far the hand is open, the orientation of the palm, and the direction and speed of movement. These are close to the features that distinguish one sign from another in the language itself. When the data crosses a threshold it triggers a sound process, and the same data keeps shaping that process while it runs.
There is no trained classifier and no translation. The system responds to a small vocabulary of signs that I chose and defined, so the vocabulary is a compositional decision. The difficult part is onset: signing touches nothing, so there is no key or string to give an attack, and I have to decide for myself the moment a sign begins.
Sign language is not used here to illustrate or translate the music; the signing is what plays it. The aim is that everyone in the room, hearing or not, can listen to the gesture and see the sound.