Skip to main content

handle-ws-midi-message-optimization

WebSocket MIDI message handling: current state and optimization roadmap​

This page reflects the current code paths and what remains to finish the planned optimization.

Current scheduling path (WASM)​

  • We no longer use one setTimeout per note. On WASM targets, we batch closures and schedule them with a requestAnimationFrame-backed min-heap scheduler.
  • This reduces timer overhead and clusters work per frame.
impl<'c> HandleWebsocketMidiMessage for WasmHandleMidiMessage<'c> {
    fn handle(&self, message: &MidiMessageOutputDto, state: Rc<AppState>) {
        if let Some(output) = handle_ws_midi_message(&message, state) {
            RAF_SCHEDULER.with(|r| r.schedule_batch(output));
        }
    }
}

The scheduler drains due events each animation frame within a lookahead window and skips events that are far too late.

pub fn schedule_batch(&self, batch: Vec<(f64, HandleWsMidiMessageClosure)>) {
    if batch.is_empty() { return; }
    let base = now_ms();
    for (delay_ms, f) in batch.into_iter() {
        self.heap.borrow_mut().push(Scheduled::new(base + delay_ms.max(0.0), f));
    }
    self.ensure_tick();
}

Legacy handler still used for building closures​

The current handler used in production path is handle_ws_midi_message, which still performs the older time math (including the 1000ms fallback). The scheduling is improved (rAF), but timing calculation improvements are not yet wired in.

let now = chrono::Utc::now().timestamp_millis() as f64;
let mut message_time = midi_message.get_time().parse::<f64>().unwrap_or_default();
if message_time < now { message_time += current_audio_state.server_time_offset as f64; }
let mut t = message_time - current_audio_state.server_time_offset as f64
    + pianorhythm_shared::GLOBAL_TIME_OFFSET as f64 - now;

Optimized timing calculation (implemented)​

We implemented a monotonic-aware calculation using a wall-clock snapshot plus a monotonic processing delta. Note the Instant type is from the instant crate for WASM compatibility.

pub struct TimingContext {
    pub message_received_at: instant::Instant,
    pub message_wall_time: f64,
    pub server_time_offset: f64,
}

pub fn calculate_note_timing_optimized(
    message_time: f64, server_time_offset: f64, timing_context: &TimingContext,
) -> f64 {
    let server_adjusted_time = message_time + server_time_offset;
    let processing_delay = timing_context.message_received_at.elapsed().as_millis() as f64;
    let base_delay = server_adjusted_time - timing_context.message_wall_time;

Optimized message handler (implemented but not yet wired)​

The optimized handler batches notes and clamps total delay to reasonable bounds. Differences vs the earlier draft:

  • ScheduledNote.socket_id_hash is Option<u32>
  • base_time is set to the wall-clock snapshot
#[derive(Debug, Clone)]
pub struct ScheduledNote {
    pub delay_ms: f64,
    pub note_data: MidiDto,
    pub note_source: NoteSourceType,
    pub socket_id_hash: Option<u32>,
}
 
if !batched_notes.is_empty() {
    return Some(BatchedNoteSchedule {
        notes: batched_notes,
        base_time: timing_context.message_wall_time,
    });
}

Optimized batch scheduling hook (stubbed)​

The trait method exists but the optimized path is not invoked yet and the scheduler hook is a stub for now.

fn schedule_note_batch_optimized(&self, batch: BatchedNoteSchedule) {
    // schedule_midi_batch_global(notes_with_timing);
}

Current MIDI parsing calls​

Currently, Note On uses the dedicated synth_ws_socket_note_on call. Note Off and some controls still go through parse_midi_data.

MidiDtoType::NoteOn if buffer_data.has_noteOn() => {
    let v = buffer_data.get_noteOn();
    let event = PianoRhythmWebSocketMidiNoteOn { /* fields */ };
    pianorhythm_synth::synth_ws_socket_note_on(event, socket_id_hash);
}
 
MidiDtoType::NoteOff if buffer_data.has_noteOff() => {
    let v = buffer_data.get_noteOff();
    _ = pianorhythm_synth::parse_midi_data(&[ /* NOTE_OFF bytes */ ],
        socket_id_hash, Some(note_source.to_u8()), None);
}

What’s done vs pending​

  • Done
    • rAF-based scheduler for closure batches (WASM)
    • Monotonic-aware timing calc helper and batched handler are implemented
  • Pending
    • Wire WasmHandleMidiMessage::handle to use handle_ws_midi_message_optimized
    • Implement schedule_note_batch_optimized to use the audio-context-synchronized scheduler
    • Migrate Note Off and other events to dedicated from_socket_* style helpers if desired

Next steps to finish the optimization​

  1. Switch routing to the optimized handler
  • Replace handle_ws_midi_message with handle_ws_midi_message_optimized in the WASM path
  1. Implement optimized scheduling
  • Convert BatchedNoteSchedule into (delay_ms, ScheduledNote) and schedule with audio-context time or extend RAF scheduler to accept batches of notes directly
  1. Validation
  • Add unit tests for calculate_note_timing_optimized and batching limits
  • Bench rAF scheduler under bursty loads; verify DROP_TOO_LATE_MS safeguards
  • End-to-end latency comparison vs legacy path

Notes on time sync and buffering​

  • EnhancedTimeSync and AdaptiveFlushTimer live in the Note Buffer engine and complement this path by stabilizing server offset and controlling flush cadence. This page focuses on the WS-to-audio scheduling hop; see ./note-buffer-optimizations.mdx for deeper details.
Last updated on by Oak