Low-latency, low-jitter MIDI/OSC transmission from Ableton Live to an iPad app
Abstract
An iPad controller for Ableton Live only works on stage if the connection is fast and steady. More than 10 ms between a touch and the sound is already noticeable, and visual feedback on the iPad should arrive within 30 ms. There were no dedicated tools to measure this, so I built a test system and used it to optimise the connection step by step.
The system had three parts: an iPad app with test scenarios, such as 8 knobs turned at once with multitouch; a Mac app that relayed every message between the iPad and Ableton Live and showed the latency and jitter of each connection in a live dashboard; and a Python MIDI Remote Script inside Ableton Live.
With every change measured, the median input latency from the iPad to Ableton Live dropped from 27.8 ms to 4.98 ms. Together with a typical audio interface latency of 4 ms, that is 8.98 ms from touch to sound, within the 10 ms target. The visual round trip measured 9.97 ms, well within 30 ms.
The reusable parts (networking, latency measurement and controls) went into a Swift package for future controller apps.
Results
- Median input latency, iPad to Ableton Live
- 27.8 → 4.98 ms
- Touch to sound, with 4 ms audio latency
- 8.98 ms
- Target: under 10 ms
- Visual round trip
- 9.97 ms
- Target: under 30 ms
- Run loop pulse. Ableton’s Remote Scripts only handled incoming OSC messages in a run loop that ticked about every 100 ms. That caused most of the delay and the sawtooth below. A listener on the song position still only got down to 80 ms. Remote Scripts prioritise MIDI, so the Mac app sent a MIDI CC message every 0.1 ms as a pulse, and every pulse made the script work through its OSC queue. The run loop went from about 100 ms to under 1 ms.
- OSC bundles. Fast knob turns sent bursts of messages that seemed to hit a limit of packets per second and showed up as waves in the latency. Packing several OSC messages into one bundle cut the number of packets and removed the waves.
- 128 steps per turn. The iPad’s touchscreen produced far more values than anyone can hear. Quantising each knob to 128 steps per turn, the resolution of a MIDI CC, meant fewer messages to send.
- Mac to Ableton Live
- iPad to Mac
Before: no optimisation, median 27.8 ms
After: all three steps, median 4.98 ms
Input latency of 50 consecutive packets while turning 8 knobs at once. Both panels use the same scale.
Show the data
| Packet | Before (ms) | After (ms) | ||
|---|---|---|---|---|
| Mac to Ableton | iPad to Mac | Mac to Ableton | iPad to Mac | |
| 1 | 3.53 | 2.27 | 6.04 | 2.86 |
| 2 | 0.41 | 2.03 | 2.26 | 1.41 |
| 3 | 0.48 | 1.94 | 1.53 | 1.82 |
| 4 | 0.57 | 1.86 | 1.56 | 1.51 |
| 5 | 40.11 | 1.36 | 3.21 | 1.75 |
| 6 | 40.37 | 2.48 | 4.55 | 0.80 |
| 7 | 40.79 | 2.11 | 0.62 | 1.08 |
| 8 | 37.70 | 1.69 | 1.85 | 12.16 |
| 9 | 38.42 | 4.63 | 0.99 | 1.98 |
| 10 | 34.30 | 4.68 | 1.45 | 1.82 |
| 11 | 34.37 | 4.59 | 3.21 | 1.81 |
| 12 | 31.58 | 3.29 | 2.34 | 2.44 |
| 13 | 31.87 | 2.99 | 0.95 | 1.31 |
| 14 | 28.21 | 2.63 | 3.80 | 1.07 |
| 15 | 28.41 | 2.42 | 0.98 | 1.48 |
| 16 | 23.54 | 3.15 | 4.36 | 1.52 |
| 17 | 23.85 | 2.83 | 0.91 | 2.15 |
| 18 | 23.88 | 2.81 | 5.28 | 3.12 |
| 19 | 20.83 | 1.77 | 3.73 | 1.15 |
| 20 | 20.98 | 2.24 | 4.65 | 1.17 |
| 21 | 16.99 | 2.11 | 1.12 | 1.36 |
| 22 | 17.25 | 2.25 | 7.11 | 2.48 |
| 23 | 13.41 | 1.94 | 8.56 | 3.55 |
| 24 | 9.51 | 1.58 | 6.05 | 1.36 |
| 25 | 9.93 | 2.67 | 3.12 | 1.61 |
| 26 | 5.71 | 2.78 | 3.83 | 1.73 |
| 27 | 2.30 | 2.33 | 4.26 | 1.80 |
| 28 | 39.21 | 2.58 | 8.99 | 2.96 |
| 29 | 36.55 | 4.46 | 9.44 | 2.53 |
| 30 | 32.61 | 4.56 | 2.21 | 1.00 |
| 31 | 28.74 | 4.27 | 4.97 | 1.04 |
| 32 | 29.19 | 3.81 | 5.29 | 1.56 |
| 33 | 25.57 | 3.28 | 1.98 | 0.91 |
| 34 | 21.69 | 3.96 | 0.95 | 1.07 |
| 35 | 22.23 | 3.44 | 2.64 | 2.36 |
| 36 | 18.28 | 3.01 | 9.90 | 1.49 |
| 37 | 10.41 | 3.35 | 10.07 | 1.34 |
| 38 | 10.38 | 3.39 | 6.41 | 2.00 |
| 39 | 8.04 | 1.66 | 3.51 | 1.35 |
| 40 | 1.93 | 3.63 | 3.40 | 1.32 |
| 41 | 43.93 | 4.82 | 4.89 | 1.53 |
| 42 | 45.50 | 4.19 | 1.37 | 1.43 |
| 43 | 42.44 | 3.50 | 2.39 | 1.01 |
| 44 | 38.61 | 3.34 | 9.99 | 0.92 |
| 45 | 34.64 | 3.18 | 3.70 | 1.00 |
| 46 | 30.93 | 2.76 | 5.73 | 1.36 |
| 47 | 31.09 | 2.49 | 9.21 | 1.00 |
| 48 | 27.07 | 2.51 | 1.79 | 1.46 |
| 49 | 22.82 | 2.97 | 4.13 | 3.19 |
| 50 | 18.88 | 2.40 | 0.96 | 1.20 |
Choosing the transport
Before optimising anything, I compared ways to connect the iPad to the Mac inside the test system:
- Inter-Device MIDI and Bluetooth MIDI (CoreMIDI): stable, but every new connection needed manual setup in macOS.
- Peertalk: 2.5 ms round trip, but over 10 ms under load, no longer maintained and incompatible with iOS 16.
- Multipeer Connectivity: never under 30 ms round trip, and it switched transports on its own, which caused jitter over 10 ms.
- SwiftyOSC: 50 to 100 ms, because it ran its networking on the main thread.
- MIDI 2.0: promising, but Ableton Live did not support it yet.
- SwiftNIO with OSCCore won: under 1.5 ms round trip between iPad and Mac, and the lowest latency on the Mac to Ableton Live connection as well.