Back to Projects
HealthTech · MedTech Enterprise

Real-Time Voice-to-EHR for Surgical Documentation

Surgeons needed hands-free dictation in the OR that landed as a structured operative note they could edit by voice.

OpenAI Realtime API gRPC Streaming Structured OpNotes Voice Edits
Business Impact
Hands-free dictation to a structured OpNote, with voice edits in the OR

The Problem

Operative notes have to be captured while the surgeon’s hands are busy. A dictation dump is not enough: the note needs structure, and corrections need to happen by voice, not a keyboard in the OR.

The Architecture

flowchart LR
  subgraph capture [OR Audio]
      Mic[Microphone]
  end

  subgraph engine [gRPC Service]
      Live[Live transcription]
      Format[OpNote formatting]
      Edit[Voice edit actions]
  end

  subgraph model [Model]
      RT[OpenAI Realtime API]
  end

  subgraph out [Output]
      Note[Structured OpNote]
  end

  Mic --> Live
  Live --> RT
  RT --> Live
  Live --> Format
  Format --> Note
  Edit --> RT
  Edit --> Note

Three Voice Flows

A stateless gRPC service sits in front of OpenAI Realtime:

  1. Live transcription — streaming deltas while the surgeon talks
  2. OpNote formatting — turn a transcript into a structured operative note
  3. Voice edits — navigate and change the note without leaving voice

Tech Stack

  • Python — service and handlers
  • gRPC — bidirectional streaming to the client
  • OpenAI Realtime API — transcription and voice actions
  • Azure — deploy and storage

The Impact

Documentation stays in the OR: dictate, structure, edit by voice.