LEFT · I–VII
Choose the root degree
Move your left palm horizontally across the on-screen rail. Hold a new lane briefly until the interface confirms it.
First performance tutorial
How to Play Gesture Synth
This Gesture Synth tutorial explains how to use Gesture Synth from camera setup to your first progression: position both hands, choose roots and chords, then shape volume and filter.
The short version
Two hands control three musical signals
Move your left palm across the I–VII rail to choose a root. Count the extended fingers on your right hand to choose a chord type. Then move that right palm up and down for volume or left and right for the filter.
Camera setup
Open the camera and start the instrument
- 01
Open Gesture Synth
Use the playable Gesture Synth homepage. This guide does not request camera access.
- 02
Press Start camera
Allow video access when the browser asks. Gesture Synth requests the camera only; it does not request microphone audio.
- 03
Wait for hand tracking
Keep both hands separated and fully inside the camera stage while the tracker loads.
Hand controls
How to Play Gesture Synth: Two-Hand Controls
RIGHT · 0–5
Choose the chord type
- 0 fingers: Mute
- 1 finger: Major
- 2 fingers: Minor
- 3 fingers: Dominant 7
- 4 fingers: Major 7
- 5 fingers: Sus4
RIGHT · X/Y
Shape the sound
Raise your right palm for more volume and lower it for less. Move your right palm to the right for a brighter, more open filter or left for a darker tone.
First chord
Play a clear C major chord
- Leave the key selector on C.
- Move your left palm to lane I and hold it briefly.
- Face your right palm toward the camera, tuck the thumb, and extend only the index finger.
- Raise your right palm until the volume meter and C chord are active.
The live tutorial follows this same sequence and advances only after the current signal is recognized.
Beginner routine
How to Play Gesture Synth: Beginner Practice Routine
Keep the key on C for all three exercises. Work slowly enough that each left-hand lane and right-hand shape settles before you move on.
Exercise 1
Hold one stable C major chord
- Hold the left hand in lane I and show one extended right-hand finger.
- Keep C major sounding steadily for several seconds without changing the chord.
- Move the right palm up and down to control volume, then left and right to explore the filter while keeping the same finger shape.
Exercise 2
Switch between two Major chords
Practice the left-hand move first. Keep one right-hand finger extended throughout so the quality stays Major.
ChordsC ↔ G
Left handI ↔ V
Right hand1 ↔ 1
Exercise 3
Loop three four-chord progressions
Change the left-hand lane first, let it settle, then confirm the right-hand finger count. Begin with C → G → Am → F, then use the two variations to practice different root orders.
ChordsC → G → Am → F
Left handI → V → VI → IV
Right hand1 → 1 → 2 → 1
ChordsC → F → G → C
Left handI → IV → V → I
Right hand1 → 1 → 1 → 1
ChordsAm → F → C → G
Left handVI → IV → I → V
Right hand2 → 1 → 1 → 1
When all three loops feel steady, use the complete Gesture Synth chord reference to transpose the same hand logic to another key or try another quality.
Mobile and browsers
Mobile and browser tips
- For the primary experience, use a current Chrome or Edge browser on desktop or Android. Other browsers need camera, Web Audio, WebAssembly, and hand-tracking support.
- On a phone, keep the device steady and both hands inside the portrait stage. Tracking can vary with device performance, available memory, lighting, and distance.
- Camera access requires HTTPS, or localhost during development. Press Start camera and allow video access when asked; Gesture Synth does not request microphone audio.
Recognition fixes
Common hand-tracking mistakes
The wrong finger count appears
Face the whole palm toward the camera. Keep every fingertip visible and clearly fold fingers that should not count, especially the thumb.
One hand keeps disappearing
Move farther from the camera, separate your hands, and stay in the comfortable center of the frame.
The result changes while you move
Pause briefly on each root lane and hand shape. Fast movement or a busy background can make the current frame uncertain.
The chord is still muted
Both hands must be visible. A closed right fist means 0 fingers and intentionally selects Mute; raise the right palm after choosing a non-muted shape.
Tracking is uncertain
Use brighter, even light and keep the camera around chest or shoulder height. Avoid strong backlight behind your hands.
You need the complete map
Use the Gesture Synth chord guide for root degrees, finger types, and examples.
Setup questions
Gesture Synth setup and practice questions
Do I need a special webcam?
No specific webcam model is required. An integrated or external camera can work when the browser can access it and can keep both hands clear in the frame.
Can Gesture Synth control volume and filter at the same time?
Yes. The right palm’s vertical position controls volume while its horizontal position controls the filter. Those two expression controls update while the right-hand finger shape continues to select the chord quality.
Why is there no sound after the camera starts?
Make sure both hands are visible, the left hand has settled on an I–VII lane, and the right hand shows at least one extended finger instead of a closed fist. Then raise the right palm above the lowest volume position. If camera or audio startup fails, use the retry action shown by the instrument.
Are camera frames uploaded?
The instrument passes video frames to the MediaPipe hand tracker running in your browser; the application does not upload those camera frames. The page still downloads its hand-tracking assets and may make ordinary analytics requests.
What should I practice after C–G–Am–F?
Try C → F → G → C to keep one Major finger shape while changing roots. Then try Am → F → C → G, which begins with two fingers for Minor before returning to one-finger Major chords.