Switch your speech AI engine without changing what the app promises

A familiar API can hide an unfamiliar experience

Imagine a caption appearing while somebody speaks. A word changes, a sentence settles, and the app lets you save it. To a developer, those are events from a speech recognition engine. To the person using the app, they are promises about what can still change and what is ready to rely on.

On October 29, developer Mattt introduced AnySpeechRecognizer alongside AnyDecisionModel. The speech package presents an Apple-style analyzer interface with a choice of engines, including Apple, Parakeet and Qwen3-ASR. The announcement describes open models running on device. This is an independent Swift package, not an Apple announcement adding every engine to the operating system.

The release is a useful prompt for a more specific design question: if an app changes recognition engines, which behaviors must remain understandable? A common API helps connect software. It cannot by itself make interim text, timing, languages, model downloads and failure states identical.

This article focuses on that migration problem rather than comparing recognition scores. We have not tested these engines or measured their speed. The examples below are proposed acceptance checks for a small app, using audio you are authorized to process.

Start with the transcript's job

A transcript used for search has different needs from captions used during a conversation. Search may tolerate waiting until a complete clip has been processed. Live captions need a clear relationship between what is being said now and what is still being revised. A note-taking app might care most about finding an important moment later.

Describe the job in a sentence that does not mention a model. For example: a student records their own practice explanation and wants a searchable record with links back to the audio. That sentence creates requirements for timing and storage without assuming that every word must appear immediately.

Now write the failure that would make the tool misleading. Perhaps a search result jumps to the wrong moment, or a tentative word is saved as if it were final. Perhaps a failed session leaves an apparently complete transcript. These are different from an ordinary misspelled word, and they deserve separate checks.

Avoid starting with a list of every feature a library supports. Start with what your reader, listener or creator will do next. A small app that clearly supports one job can be more useful than one that displays every available output without explaining its status.

Source: Mattt: Introducing AnySpeechRecognizer and AnyDecisionModel ↗ · Mattt: AnySpeechRecognizer repository, backend differences and result lifecycle ↗ · Apple Developer: Bring advanced speech-to-text to your app with SpeechAnalyzer ↗

Use final and interim text differently

The AnySpeechRecognizer repository documents a distinction between volatile results and final results. A newer volatile result can replace earlier tentative text, while final results are stable. Its backend table also documents differences in streaming behavior, including Qwen3-ASR producing final results per segment rather than the same interim behavior as the other listed engines.

Keep that source-derived distinction small and concrete in your design. Use one visual treatment for text that may change and another for text that has settled. Explain the difference in ordinary language. Do not make users infer finality from a subtle animation or a color they may not notice.

Consider a hypothetical dictation field. If the engine revises an unfinished phrase, the field should replace the pending portion rather than append both versions. Appending everything creates a transcript that duplicates speech the person never repeated. The text can look impressively active while becoming less faithful to the recording.

A useful acceptance check is to pause midway through a sentence, continue it and watch what the app stores. Compare the saved text with the displayed final text. The test is about your application's handling of events; it is not a claim that one particular engine always makes a particular revision.

If the selected engine does not provide an equivalent interim experience, make the change visible. A clear message that transcription will appear in completed segments is better than a fake live effect created by revealing already completed words one at a time.

Preserve the recording's timeline

The package documentation describes result time ranges on the source recording's timeline. That detail becomes important when a transcript is more than a block of text. A person clicking a sentence expects to hear the relevant audio, not whichever offset happened to exist in an intermediate chunk.

Make a short test recording with three deliberately distinct spoken landmarks. Leave silence between them. Write down their approximate positions from ordinary playback, then check whether selecting the resulting transcript segments takes you to the corresponding places.

Include a test that starts partway through a file if your workflow supports it. Decide whether the displayed times refer to the original file or the extracted clip. Both conventions can be understandable; mixing them without explanation creates confusing links and exported notes.

Do not claim word-level precision when an engine only provides a broader segment range. A highlight covering an entire phrase may be honest and useful. A precise-looking cursor moving word by word can imply information that your data does not contain.

If you export text to another tool, include enough context to interpret the timestamps. A file identifier, a time format and the meaning of the starting point are often more valuable than extra decimal places. Design the export for a person who will not remember how the recording was processed.

Treat cancellation as a user action

Stopping a transcription session should have a defined meaning. Is the app preserving completed results, discarding the entire attempt, or asking what to do? The package repository documents cancellation and cleanup behavior, but your interface still needs to explain what the person will retain.

Create a hypothetical session that is long enough to cancel halfway through. Stop it after some final text has appeared. Then navigate away and return. Look for unfinished indicators, accidental duplicate sessions and a transcript presented without any sign that processing ended early.

Distinguish cancellation from a processing error. Someone who intentionally stopped should not receive a frightening message implying the recording was corrupted. Someone whose session failed should not receive the same cheerful completion state as a successful run.

Write the saved record's state explicitly. Suggested states might include processing, complete, stopped and failed. The exact labels are your choice. The key is that an exported or reopened transcript should carry its own status rather than depending on a transient message that disappeared.

This matters even in a personal prototype. You might return to a note days later and assume that its last sentence was the end of the recording. A small status label can prevent an ordinary interruption from becoming a misleading source.

Separate capture, recognition and storage

Apple's SpeechAnalyzer session at WWDC25 explains an architecture for providing audio to analysis and receiving transcription results. AnySpeechRecognizer's documentation also makes clear that the package does not itself capture audio. That means microphone capture and the rest of your app remain separate responsibilities.

Draw three boxes in your own project notes: obtaining audio, recognizing speech and retaining results. Add the model download step separately. This simple exercise helps you describe what on-device processing actually means in your application.

An app can run recognition locally and still send recordings, diagnostics or transcripts somewhere else. Conversely, a model download does not mean that a user's subsequent recording was uploaded. Review each connection and storage action rather than treating local as a complete privacy description.

For the first test, use your own non-sensitive recording. State where it is stored and how to delete it. If you later process other people's speech, obtain appropriate permission and explain the intended use before recording. A useful transcript does not remove the need for that conversation.

Do not add a wearable recorder or new headset just to try this method. An existing microphone and a small authorized audio file are sufficient to examine your app's event handling. Better input hardware cannot repair a duplicate-result bug or clarify a missing completion state.

Plan for the model that is not ready

The moment before recognition begins is part of the experience. A model might need to be downloaded, a supported engine may not be available in a configuration, or the application might lack permission to obtain the input. These are planning possibilities to verify for your chosen setup, not observations from our testing.

Keep readiness separate from processing. A person should be able to tell whether the app is preparing an engine, listening to audio or working through an existing file. A single spinner labeled working makes those states unnecessarily hard to diagnose.

If the app offers an engine choice, say what happens when the selected engine cannot start. An automatic fallback might be convenient, but it should not quietly change where data is processed or which language behavior is expected. Ask for an appropriate choice or disclose the fallback before it affects the recording.

Record the exact package version and model identifier used by a session. Put this information in an inspectable details view or development log, not in the middle of a reader's transcript. It helps a developer reproduce a problem while keeping the main interface focused on the user's task.

A first-run check should include preparing the model, closing the app and returning later. Decide which resources are retained and how the next launch explains readiness. This proposed check is often more representative than repeatedly testing a warm development session.

Score the errors your task cannot afford

A transcript can have many harmless punctuation differences and one harmful factual substitution. For your acceptance set, identify the words whose meaning matters most. They could be a person's name, an address, a quantity or a negative such as do not.

Use several short recordings rather than one polished monologue. Include ordinary pacing, a correction spoken aloud and a sentence with a meaningful negation. If you need multiple speakers, ask participating people for permission and keep the recordings within the declared purpose.

Review the text against the audio yourself. Mark uncertain passages without guessing. A reference transcript that contains your own assumptions will make the evaluation look more certain than it is. Retain the uncertainty when comparing engines.

Keep the scoring categories separate: content accuracy, finality handling, time navigation, completion status and recovery from interruption. A model can be strong at recognition while your app fails to save results correctly. A single combined score would hide that difference.

If you change the prompt, preprocessing or recording format between runs, document it. Otherwise, the engine name becomes a convenient explanation for differences caused elsewhere. You are trying to understand the entire recognition route, not declare a winner from an uncontrolled demo.

Make one engine switch reviewable

For a first migration, choose a small authorized recording and two engines your actual development environment supports. Keep the input, user task and output rules unchanged. Save each result with its configuration, then review the same critical passages and interface states.

Write down what remained equivalent, what changed visibly and what you could not verify. An honest outcome could be that both produce useful searchable notes but only one offers the interim behavior your current interface expects. That is a design finding rather than an embarrassing failure.

Do not hide a missing capability behind a familiar type name. If the app cannot provide reliable word timing with a selected backend, offer phrase navigation. If it cannot process the file, explain the failure and leave the source recording available according to your retention rules.

The value of an interchangeable speech engine is the freedom to choose the right implementation for the job. Preserve that freedom by keeping the application's promises explicit. Then changing a backend becomes a reviewable product decision instead of a surprise that the person reading the transcript has to discover.