Google Made Flash 4x Faster and Changed Our Latency Budget

We had shelved the feature in September.

The idea was simple: as a user types a question into a search field, the interface would stream back a short AI-generated answer in real time. Not on submit. While they type. The kind of thing that feels fast enough to feel alive.

We had the model call working. The answer quality was solid. The problem was time. From the moment the user paused long enough to trigger the call, to the moment the first token appeared on screen, the gap ran around 900 milliseconds. Our own internal threshold for "this feels like the UI is responding" was 400 milliseconds. At 900ms the cursor blinked, the input field sat empty, and the user's mental model broke: they had typed something and nothing had happened yet.

We shipped a loading spinner. We removed the feature from the roadmap. The model was capable enough. The latency was not.

That was eight months ago. On May 19, Google shipped Gemini 3.5 Flash at Google I/O 2026 with a claim that the model delivers four times the output tokens per second compared to other frontier models. We pulled up the old branch that afternoon.

What "four times faster" actually touches

Raw capability has never been the ceiling on what AI features ship. Latency has been.

Capability is what an AI can do. Latency is what users will wait for it to do.

Every six months the latency floor drops, and three product ideas become possible that were not last quarter.

The 4x claim from Google refers specifically to output throughput: how fast the model generates tokens once it starts responding. The number that matters before that is time-to-first-token (TTFT), which is the delay between submitting a request and seeing the first character of the response. Google published a TTFT benchmark around 150 to 200 milliseconds for 3.5 Flash under normal conditions. Other frontier models in the same tier run between 500 and 900 milliseconds TTFT, depending on load.

Those numbers come from benchmarks. Real-world numbers will vary based on geography, load, and your own network stack. We ran our own tests from a server in Frankfurt against the Google AI Studio endpoint and saw first-token times between 180 and 260 milliseconds on uncached prompts. The September branch at 900ms ran against a different provider and a different model tier. We are not claiming a controlled apples-to-apples comparison. We are claiming the range moved enough to matter.

The feature we could not ship in 2024

The streaming search answer was not a novel idea. Perplexity had shipped something similar. Arc had shipped something similar. The pattern was obvious and the user desire for it was obvious and we had the model quality to pull it off.

What we did not have was the latency headroom.

A threshold exists somewhere around 400 milliseconds where a UI response feels instant enough that the user does not consciously register the wait. Between 400 and 800 milliseconds they notice a pause but tolerate it. Above 800 milliseconds, something breaks: the user interprets the pause as an error, their attention wanders, the context of what they were doing dissolves. The Microsoft research on this ran 20 years ago and the numbers have held up in more recent replication.

Our numbers landed in the third bucket. The feature targeted the first bucket. No amount of UX polish was going to bridge that gap. We tried perceived latency tricks: a shimmer placeholder, an animated typing indicator, text that faded in rather than snapping in. Each one made the wait feel slightly less dead. None of them moved the needle on whether the feature felt like a real-time response or a delayed retrieval.

We shelved it because the quality fell short, and "not good enough" had a specific definition: the interface did not feel alive.

What 4x actually meant when we ran it

We cloned the September branch on May 19, swapped the model string to gemini-3.5-flash, updated the SDK to @google/genai, and ran the same test suite from eight months earlier.

The first-token time in our Frankfurt environment came back at 210 milliseconds on the first run.

We assumed we had a bug and refreshed. 196 milliseconds. 221 milliseconds. 203 milliseconds. The variance across 20 runs sat in a band between 180 and 240 milliseconds, with three outliers above 300 milliseconds.

The feature worked. Not theoretically worked. It worked in the way that mattered: you typed, you paused for a half second, text appeared. The interface felt like it was paying attention.

The timing snippet we added to verify it:

const t0 = performance.now();
const stream = await ai.models.generateContentStream({
  model: "gemini-3.5-flash",
  contents: query,
});

let firstToken = false;
for await (const chunk of stream) {
  if (!firstToken) {
    console.log(`TTFT: ${(performance.now() - t0).toFixed(0)}ms`);
    firstToken = true;
  }
  // pipe chunk.text to the UI
}

Nothing clever in there. One performance mark before the call, one log on the first non-empty chunk. The discipline is just measuring it rather than assuming.

The real budget shift

Here is what turned out to be more useful than the initial test: the latency improvement goes deeper than speed. It changes which features a product team can even bring to the table.

When the average TTFT for a frontier model runs 700 milliseconds, you build AI features that are explicitly asynchronous. The user submits something, a spinner appears, a response comes back. The UX pattern acknowledges the gap and works around it. Every designer and PM on your team has internalized this pattern. When you sketch a new AI feature, the gap is a given constraint.

When the TTFT drops below 300 milliseconds, a new design vocabulary opens. The AI can respond faster than the user can refocus their eyes from the keyboard to the screen. The response is not a result delivered after a wait. The response is the interface updating. The constraint disappears, and with it a whole set of compromises that shaped every decision downstream.

The shift is not that Flash is twice as good. The shift is that crossing the 300 millisecond TTFT threshold unlocks design patterns that did not exist before.

This matters more for some product types than others. If your AI feature runs in the background and returns a summary an hour later, TTFT is not your bottleneck. If your AI feature lives in any surface where a user is actively waiting for feedback, the 200ms gap between what Flash delivers and what the previous generation delivered is the difference between a feature that ships and a feature that gets shelved for the second or third quarter in a row.

The caveats that matter

We are not going to pretend the numbers we saw in Frankfurt are what you will see in Tokyo or Sao Paulo. Gemini 3.5 Flash is a new model on infrastructure that will run under different load conditions than the day we ran our test. Real-world TTFT varies.

The pricing is also a real consideration. Gemini 3.5 Flash costs $1.50 per million input tokens and $9.00 per million output tokens. Gemini 2.5 Flash, the prior generation, runs $0.30 input and $2.50 output. For the streaming search feature we were testing, where prompts are short and answers are short, the 3.5 Flash price premium hurts less than it would on a summarization pipeline chewing through large documents.

Rate limits on the free tier are opaque. We burned through them within an hour of heavy testing. If you are building production traffic around this model, use a paid project from day one.

The thing the announcement buried

The faster model is the headline. The implication that did not make the Google I/O recap coverage is the one that changes product strategy.

For the past two years, the latency floor on frontier models stayed high enough that AI product design defaulted into an async mental model. The user does something. The AI takes a beat. The AI responds. The beat baked into the pattern so deeply that teams stopped treating it as a constraint worth optimizing and started treating it as a defining characteristic of AI interfaces.

Flash below 300ms TTFT breaks that assumption. Not gradually. Abruptly, the way crossing a physical threshold works rather than the way a gradual improvement works.

We think the product implication of that abruptness has not landed yet, and it will land in waves. The first wave is teams like ours who had features stuck in a drawer because latency made them feel wrong. Those features come back out. The second wave is features that nobody proposed because the latency assumption was built into the problem framing, so the product shape never existed in the first place. That second wave takes longer because you have to unlearn the assumption before you can sketch the new design.

The timing benchmark is not the story. The story is what the benchmark unlocks.

The closing

We shipped the streaming search feature on May 21. Two days after the Google I/O announcement. Eight months after we shelved it.

We cannot share usage numbers yet because we gated it to a small beta cohort. What we can say: the first internal tester response was "oh, it actually works." Not "it's pretty good." Not "this is promising." The response captured surprise that the latency had vanished, phrased the way people phrase it when something stops being a problem: they barely mention what the thing does and go straight to commenting on the absence of what annoyed them.

That absence is the product. Not the model, not the benchmark, not the 4x claim. The absence of the wait.

We have a running list of features we have shelved for latency reasons over the past three years. We updated that list the week of May 19. Four of the entries came off the shelf immediately.

The constraint was latency. The constraint moved. The ideas that had stalled against it are moving with it.

If a feature felt impossible six months ago, check whether the latency floor moved.