Tofu-1

Conversations happen in sound

Picture a woman calling to report a tree has fallen on her property during a storm. She is standing outside in the rain, with poor cell signal, her son keeps asking her something, and halfway through reading her address she stops to answer him.

Anyone on the other end of that call would follow what happened without thinking about it. They would hear that she turned away, wait for her to come back, ignore the rain, and pick up the address where she left off.

Most voice systems in use today cannot do this reliably. They are cascades: they turn speech into text, pass the text to a language model, and turn the reply back into speech. Each step discards part of the call: who was speaking, who they were speaking to, whether they hesitated, and whether they had finished.

Tofu-1 works differently. It listens to the audio itself, so what the sound carries stays available to the model. It listens and speaks at the same time, so a caller can interrupt or correct it as they would a person. And it can keep working on a request while it talks, so the call does not stall while it looks something up.

For a bank, a health system, or a government agency, a misheard call is a fraud claim not filed, a prescription not refilled, or a benefit not approved. The sections below take each of these abilities in turn, with recordings and figures.

Hearing the whole scene

Tofu-1 understands audio, not only words. It tells one speaker from another, recognizes who a phrase is meant for, and separates speech from the other sounds on a call: a crying child, a television, a car on the highway, a hospital intercom, and keypad tones. It also hears how something is said, including hesitation, frustration, and urgency.

Go back to the woman standing in the rain. When she turns to answer her son, her words still reach the microphone. A transcript would record them accurately and miss the event that matters most: her attention left the agent. A system that treats every phrase as part of the request may answer words meant for the child. A system that treats overlapping voices as noise loses track of what is happening on the call.

The TV Test

A television in the room is one of the hardest cases. An advertisement says “Call now.” A newscaster asks a question. A character says “Yes.” Each phrase is clear, intelligible speech, and none of it is meant for the agent on the phone.

We call this the TV Test: can the model follow the caller while ignoring speech that belongs to a different scene? Noise removal does not solve it, because the television is not noise. It is speech from the wrong source, and it is often louder than the caller.

To pass, the model has to keep track of the person it is helping as new sound arrives, and recognize that other speech can belong to another conversation entirely. Volume cannot settle the question. A loud television can be irrelevant, and a quiet correction from the caller can be the most important thing said on the call.

There is a limit. If the background makes the caller impossible to understand, Tofu-1 should say so and ask them to repeat themselves or move somewhere quieter. Ignoring the television must never become a reason to guess at what the caller said.

Generating the whole scene

Because it models the sound of a call and not only the words, Tofu-1 can also produce that sound: a second speaker, the space around them, and the noise of the line. Listen to this 10-second clip.

Listening while speaking

On a phone call, people talk over each other constantly. They say “mm-hmm” to show they are following, cut in to fix a digit in a member ID, and start answering before a question is finished. Tofu-1 is full duplex: it listens and speaks at the same time, and what it hears can change what it is saying.

The hard part is judgment. While the agent reads back a policy number, a short “right” means keep going. “No, it's 4, not 9” means stop. The model has to tell the two apart while it is still mid-sentence, and it has to keep a television from taking the floor while letting a caller's quiet correction through.

Thinking while talking

High-stakes calls involve real work: verifying a caller's identity, checking the status of a claim, looking up benefits, and confirming a payment. A person on the phone keeps talking with the caller while that happens. Tofu-1 does the same. Conversation and reasoning run side by side, so the model stays responsive while the work continues.

Running both at once has a catch: the caller can change the request while the work is underway. If a patient corrects an appointment date in the middle of a lookup, the answer has to reflect the new date. And “I'm checking” and “Your payment went through” are different promises. Tofu-1 does not say something is done until it is.

What comes next

Tofu-1 is the first release in Bland's family of speech-to-speech models, and requests are now open for its research preview. We are looking for teams that handle high-stakes calls in banking, healthcare, insurance, and government, and want to test where the model works, where it breaks, and what it should become.