Build a Scribe Agent

This guide walks through building a Scribe Agent end-to-end on the Community Hub. By the end you’ll have a Scribe Agent that you and other clinicians can download to Isa and use from Patient Chat. A Scribe Agent bundles four things into one downloadable package: an ASR model for transcription, an optional diarization model for telling speakers apart, an extraction LLM that turns the transcript into structured fields, and one or more Extraction Templates that define what those fields are. You can build from scratch or duplicate an existing Scribe Agent on the Community Hub and adapt it.

​

Prerequisites

• A Community Hub account
• A clear idea of what the Scribe Agent should produce — the documentation format (e.g. SOAP note, discharge summary, referral letter) and the fields it should fill

​

Open the build form

Sign in to the Community Hub and start a new Scribe Agent from the Build menu in the top navigation. The form opens with the Scribe Agent’s fields ready to fill in.

​

Name your Scribe Agent

Names are how other clinicians find your Scribe Agent in the Isa Hub — prefer something concrete like GP Consultation Scribe over My Scribe Agent.

​

Describe the intended use

The Intended Use field is what other clinicians read when they find your Scribe Agent in the Isa Hub. One or two sentences on what the Scribe Agent transcribes and what shape of note it produces is enough.

​

Pick a category and add keywords

• Category — Admin or Clinical.
• Keywords — comma-separated tags that help others find your Scribe Agent when searching. Add a few that match the specialty, the documentation format, or the use case (e.g. dermatology, SOAP, follow-up).

​

Pick an ASR model

The ASR model is what turns the audio into text. Set your device — iPhone, iPad, or Mac — and the selector recommends models that fit: pick from the Isaree-recommended set or search the open-weight models on Hugging Face. Each model carries a Runs / Needs X GB badge, the same fit treatment as Agent models. For why the estimates look conservative, see Choose a model.

​

Pick a diarization model

The diarization model separates the speakers — yours from the patient’s — so the extraction step knows which words came from whom. It’s optional: leave it on None if the consultation is dictation only or if speaker attribution doesn’t matter for your template.

​

Pick an extraction LLM

The extraction LLM is the on-device model that turns the transcript into your template’s structured fields. It’s required — every Scribe Agent ships one. Extraction is independent of the Primary Agent: whichever model the clinician talks to in Isa, extraction runs on the model you pick here. Pick a model that fits the device your audience uses — see Choose a model. At recording time, the clinician can route extraction to a cloud provider instead — see Understand where the pipeline runs.

​

Create Extraction Templates

Extraction Templates are where you define the information to be extracted from the conversation. One Scribe Agent can hold several templates, and the clinician picks which one to use before the consultation. A single GP Consultation Scribe could ship with a SOAP note, a referral letter, and a sick note — same transcription, three different structured outputs. Add a template for each shape of note this Scribe Agent should support. Each template has three parts:

• Template name — what shows up when you pick a template in Patient Chat. Use the documentation format the user will recognize (e.g. SOAP Note, Discharge Summary).
• Extraction prompt — the instructions sent to the model that does the extraction. Tell it what role to take, what to look for in the transcript, and how to handle missing information. This is where most of the quality work happens.
• Output schema — the structured fields the extraction fills in. Build it visually by adding one field at a time, or paste a JSON schema directly.

In the visual builder, each field has:

• Field name — the key in the structured output. Use snake_case (chief_complaint, not Chief Complaint).
• Type — String, Number, Integer, Boolean, Enum, or Object (nested).
• Description — a short hint that helps the extraction model decide what belongs in this field.
• List of these — tick to make the field an array: a list of whatever type you picked, including objects.

Schemas aren’t limited to flat fields. Nest objects and arrays to model structured sections — a medication list whose entries each carry a name, dose, and frequency, or a vitals object grouping related measurements. The editor validates the schema live as you type and won’t let you save one that would break the Scribe Agent. It’s fully localized in English and German.

​

Build the Scribe Agent

Once the required fields — Agent Name, Intended Use, ASR Model, and Extraction LLM — are filled in, build the Scribe Agent. It now exists on the Community Hub and is ready to download from the Isa Hub.

​

Understand where the pipeline runs

Everything you pick on this form — the ASR model, the diarization model, and the extraction LLM — runs on the user’s device. A clinician who keeps those defaults runs the whole pipeline on-device: neither the audio nor the transcript leaves the device. At recording time, the clinician can route individual steps to the cloud instead, using their own API keys: transcription (and diarization) via ElevenLabs, extraction via OpenAI or Aki.io. Those switches live in the recording flow in Isa — see Scribe in Isa — not on this form.

​

Next