More Code, Fewer Releases: AI Moved Your Bottleneck to Testing

Published on Oct 12, 2026

#thought-leadership #hybrid-delivery #financial-services #ai #testing #software-delivery

More Code, Fewer Releases

AI Moved Your Bottleneck to Testing

A wide delivery pipeline where a flood of AI-generated code pours in at the development end, piles up into a dense queue at review and testing, and narrows to a thin trickle of changes reaching production.

The pipe got wider at the top. Nobody touched the middle.

Somewhere right now, an engineering leader is showing a slide with some very impressive numbers on it.

Pull requests are up. Commits are up. Story point velocity is up. Lines of code are WAY up.

The slide is called something like “AI Productivity Gains,” and the room nods, because the numbers are real.

Then, someone from release management asks a quiet question that makes everyone in the room reach for their antacid of choice.

How many of these changes are ready for production?

Well, some of the first real data on that question came out earlier this year, and it isn’t very flattering.

CircleCI analyzed more than 28 million CI workflows across thousands of engineering teams for its 2026 State of Software Delivery report. Average throughput, measured as daily workflow runs, jumped 59% year over year. CircleCI called it the biggest increase it has ever recorded. source

Then look at where that activity went. For the median team, throughput on feature branches rose 15%. Throughput on the main branch, where code gets promoted toward production, fell 7%. source

Read that again. I’ll wait.

BOTTOM LINE: teams are writing more code but shipping less of it.

AI has fixed the part of software delivery that was never the real constraint. The real constraint was always everything after the code: review, validation, integration, testing, readiness, and release.

AI hasn’t really touched any of that, at least not at the scale of what it has done to increase speed-to-Pull Request. The net effect here is that AI has merely sent more traffic into the same choke point.

In a modern product shop with continuous delivery, that traffic shows up as a longer review queue.

In a hybrid Financial Services shop, it shows up as a test environment that was already booked back-to-back-to-back until next March.

And people start asking irresponsible questions like:

I know that we said that we need 3 weeks of pre-flight in the SIT-4 environment, but how fast can we really do it?

Ah, there it is, one of my favorite leadership anti-patterns: asking what CAN be done, instead of what SHOULD be done.

The Short Version

If you only remember six things from this article, make them these:

AI made code generation cheap. Review, testing, and release capacity stayed the same price.
The 2026 data shows more code entering pipelines and less of it reaching production for most teams.
AI-generated code is failing more often on the way in and taking longer to recover.
In hybrid delivery environments, the extra volume lands on integrated testing, shared environments, UAT, and release governance, which were the tightest constraints to begin with.
Measuring AI value by code output is how organizations convince themselves they’re faster, even as delivery slows down.
The fix is investing in validation capacity: smaller changes, automated verification, smarter test selection, and evidence captured as work moves.

The Numbers Behind the Slide

$A split visual showing a busy, glowing feature-branch lane with heavy activity on one side, and a thin, dimmer main-branch lane on the other where only a small fraction of the work arrives.$

Feature branches got busy. The main branch got suspiciously quieter. Guess which one your customers use.

The CircleCI numbers get more interesting the closer you look.

The 59% average increase is heavily skewed toward the top. The top 5% of teams nearly doubled their throughput, going from 6.8 to 13.4 daily workflow runs. The median team grew 4%. The bottom quarter saw no measurable increase at all. source

So the headline number describes a handful of teams that were already good at delivery. Everyone else is mostly just generating more branches and merges.

Even the top teams hit the wall. For the top 10%, feature branch activity grew almost 50% while main branch throughput rose only 1%. source

Then, there is this pesky little thing called quality, which, last I checked, was kind of important.

Main branch success rates dropped to 70.8%, the lowest in over five years and well below CircleCI’s recommended benchmark of 90%. Nearly three of every ten attempts to merge toward production are failing. Recovery time for the typical team rose to 72 minutes, up 13%. source

More code. More failures. Slower recovery.

That’s the full picture behind the productivity “increase.”

Thoughtworks, which sponsored the report, put the conclusion bluntly: the delivery pipeline has become the binding constraint on engineering performance, and organizations that miss it will find every dollar spent on AI coding tools producing diminishing returns, or actively degrading their ability to ship reliable software. source

The Review Queue Is Where It Piles Up First

The pattern repeats in data from other sources as well.

Faros AI’s 2026 telemetry across roughly 22,000 developers found task throughput per developer up about 34%, while median time spent in code review rose more than 440% and the ratio of production incidents to pull requests rose about 243%!!! source

Key findings data from the Faros telemetry 2026 report.

Code generation got cheap and fast. Code review and judgment did not.

Code review was built for a world where the author understood every change. The reviewer skimmed the approach, poked at the risky bits, and approved. AI-assisted changes tend to be bigger, arrive faster, and come from an author who may not be able to explain every line. The burden of understanding shifts to the reviewer, who has the same number of hours in a day as last year.

Also, the reviewer is often a more senior (a.k.a. more expensive) resource, with additional demands on their time.

Something has to give. Usually, it’s rigor and/or hygiene.

DORA’s 2025 research on AI-assisted development captured the same tension from the practitioner side. Ninety percent of respondents reported using AI in their work, and more than 80% believed it increased their productivity. Thirty percent reported little or no trust in AI-generated code. DORA also found that AI adoption now improves delivery throughput but still increases delivery instability. source

DORA’s summary line is the one I’d put on the wall: AI is an amplifier. It magnifies the strengths of high-performing organizations and the dysfunctions of struggling ones. source

The tools have improved since the DORA study. Quite a bit, in my opinion.

Here is what no amount of improvement in the tooling can fix: poor delivery hygiene.

If your delivery system was a mess, AI just got you to an even bigger mess, quicker.

Where It Lands in Hybrid Delivery

A Financial Services delivery gauntlet where AI-accelerated development output arrives at a fixed sequence of integrated testing stages, shared test environments, UAT, and release governance, with work stacking up in front of each gate.

Development sped up. The test environment calendar did not get the memo.

Most of the published data comes from CI pipelines in organizations with reasonably modern delivery. That’s the optimistic case.

Now picture a hybrid Financial Services program, in a company that grew through acquisitions, with a less-than-completely modern collection of DevOps tooling that seems to be held together with tribal knowledge and duct tape.

Development teams run in sprints. They’ve adopted AI coding tools, and their velocity looks great. Then, the work leaves the team and enters the part of the lifecycle that was never agile and was never going to be:

system integration testing in shared environments
end-to-end testing across dozens of upstream and downstream systems
regression suites that take days, not minutes
performance and resilience testing
user acceptance testing with business users who also have day jobs
security and control evidence reviews
consolidated release windows
change advisory boards and go/no-go decisions

None of those stages got faster because a developer has a coding assistant.

The SIT environment is still shared by six workstreams. The test data refresh still takes an entire weekend. UAT still depends on the same four business testers. The regression suite still has the same coverage gaps it had last year, while now facing twice the volume of changes.

I’ve written before that hybrid delivery breaks down at the handoff, and governance tends to arrive late, carrying a clipboard. AI makes both problems worse in a very specific way. It increases the volume and speed of work arriving at the exact stages that were already the tightest constraints in the program.

The math isn’t complicated. When work arrives faster than a fixed-capacity stage can process it, the queue grows. Queues that are near capacity don’t grow politely, either. Waiting time climbs much faster than utilization, which is why a testing phase that was merely tight at 80% load becomes a crisis at 95%.

So the program does what programs always do under pressure.

Testing gets compressed. Scope of regression gets “risk-based,” which sometimes means risk-based and sometimes means “we ran what we had time to run.” UAT gets shortened. Defects get deferred. Evidence gets reconstructed at the end of the cycle, perhaps only when needed for an incident. For those of us that have felt that particular pain before, forensic archaeology, under duress, is no fun.

The net result is that the release still goes out, eventually, carrying more change and less validation than anyone would sign off on if they were asked directly.

Let’s come back to that minimal pre-flight before a stream, launch, or release takes over a testing environment. In every case I have seen this, it has not actually helped. Not once. Not ever. Under those conditions, every new testing phase, in every testing environment, starts with one hand tied behind its back.

All of that messiness is what turns “AI made us faster” into incidents, war rooms, lost weekends, and morale falling off a cliff.

The Wrong Metrics Are Increasing the Damage

A satirical executive dashboard glowing with impressive AI productivity numbers on top, while beneath the dashboard surface a long, stalled queue of untested changes waits for validation.

The dashboard measures how fast work enters the system. The customer only derives value from what leaves the system.

Organizations are choosing to measure the wrong things, and that is part of the problem.

When AI coding tools arrived, the easy metrics were all upstream: pull requests opened, commits, code accepted, developer time saved, sentiment surveys. Those are the numbers vendors report and the numbers leaders can show quickly.

They also measure the part of the system that was never the bottleneck.

Reporting AI value by metrics tied to code output is a little like measuring a restaurant’s success by how fast the kitchen chops onions. Impressive knife work. The customers are still waiting forty minutes for a table because there are not enough servers.

The metrics that tell the truth sit further downstream:

main-branch throughput, not feature-branch activity
change lead time from commit to production
change failure rate
time to restore after a failed change
time waiting for review, separate from time in review
time waiting for a test environment
defects found in integrated testing versus in production
escaped defects per release
percentage of planned regression actually executed

Those are less exciting. They also answer the only question that matters to the business: did more value reach customers, safely?

If an organization can’t answer that question, it doesn’t have a clue whether its AI investment is working. It only knows that developers are busy churning out code faster than they ever have.

What Actually Helps

The answer is not to slow down AI-assisted development. That ship has sailed, and frankly the developers are right that the tools help.

The answer is to invest in the other half of the pipeline with the same urgency that went into the first half.

1. Make changes smaller again

AI makes it easy to produce large changes quickly. Large changes are harder to review, harder to test, harder to integrate, and harder to roll back. Put explicit limits on change size, and treat a 1,200-line AI-assisted pull request as a planning failure rather than a productivity win.

2. Automate what a human doesn’t need to judge

Formatting, linting, known vulnerability patterns, dependency checks, contract tests, and baseline security scans should never wait for a person. Every check a machine can run reliably frees up reviewer capacity for the questions only humans answer well: is this the right change, and does it solve the right problem?

That’s the automation continuum applied to validation. Deterministic checks where the rules are knowable. Human judgment where they aren’t.

Lots of companies are using agents for these kind of checks. It’s fast, and much easier than taking the time to know all the rules.

It is also a mix of laziness, fiscal irresponsibility, and cultural red flags.

3. Get smarter about which tests run

Running the entire regression suite for every change doesn’t scale when change volume doubles. Test impact analysis, risk-based selection that is honestly risk-based, and tiered suites let fast feedback happen early while deeper validation runs only when and where it matters. CircleCI’s own analysis credits top-performing teams with faster feedback loops and smarter test selection. source

4. Treat test environments as a constrained asset

In hybrid shops, environment availability is often the single tightest constraint, and it’s frequently managed with a spreadsheet and goodwill. Ephemeral environments where possible, service virtualization for unavailable dependencies, and real scheduling discipline for shared environments can buy back capacity that no AI coding tool can achieve.

Ephemeral environments are a game changer. I know that a lot of Financial Services companies are still trying to get onto the cloud, and even if they have, they might not have ever embraced this concept. SaaS and PaaS vendors also very often require static environments for no other reason than poor architectures, pure greed, or both.

Please, join the rest of us here in the 2020s, will ya?

A couple of additional benefits of ephemeral environments:

they remove the burden of sequencing, which becomes pretty tough when determining which launch might actually be ready to be part of which release seems to change hour-by-hour
they are one of the few changes that can actually shorten the pre-flight for a testing cycle, and do it SAFELY

5. Capture evidence as the work moves

If testing evidence, approvals, and defect disposition are reconstructed at the end, every extra change adds to the end-of-cycle scramble. Capture evidence as work moves through the pipeline, and the release review gets shorter as volume grows, instead of longer.

6. Give AI to the testers, too

Most initial investment in AI virtual assistants went to the people writing code. The bottleneck is now with the people validating it. Test case generation, test data creation, failure triage, defect clustering, and regression analysis are all good candidates for AI investment, as long as the evidence stays traceable and a human owns the sign-off.

The good news? This is already happening, and happening quickly. Test data generation is a huge opportunity in this space, and appears to be getting exponentially better every few months.

7. Measure the system, not the stage

Track flow end to end. If development speeds up and lead time stays flat, the next big gain is sitting in a queue somewhere. Find the queue and target improving it.

It’s what us DevOps/Platform Engineering folks have been doing for a decade or more. Find the biggest bottleneck, work to minimize or eliminate it. Lather, rinse, repeat with the next biggest bottleneck.

Where This Connects

This problem sits right in the middle of everything I keep coming back to.

The handoff between development and testing was already where context went to die. AI adds increased volume and complexity to that handoff.
Governance that arrives late was already expensive. AI makes late governance more expensive per release.
Metrics that report activity instead of readiness were already misleading. AI adds a whole new category of hollow activity metrics that LOOK spectacular.

It’s also why I argue for capturing intent before automating anything, and for capturing, aggregating, and analyzing delivery signals before they become release-week emergencies. When change volume doubles, the program can’t rely on someone remembering which risks matter. The system has to remember automatically, and it has to notice when the queue in front of integrated testing starts growing faster than the queue behind it.

That’s the messy middle, again, like always. It has just gotten even busier.

What Good Looks Like

A balanced delivery pipeline where AI-accelerated development feeds into an equally scaled validation layer with automated checks, smart test selection, ephemeral environments, and continuous evidence capture, producing a steady flow of changes into production.

Good delivery isn’t faster at one stage. It’s balanced across all stages.

A healthy AI-assisted delivery system looks balanced.

Changes stay small.
Automated checks handle everything that doesn’t need human judgment.
Reviewers spend their time on intent and design.
Test selection adapts to what changed.
Environments are scheduled, virtualized, or created on demand.
Evidence accumulates as work moves.
Release reviews get shorter because there are fewer surprises waiting at the end.

And, crucially, leaders measure what reaches production, not what enters the pipeline.

In that system, AI-assisted development really does make the organization faster, because the rest of the pipeline was scaled to match it.

In the other system, AI makes the organization busier, and busy does not decrease time-to-value.

Final Thoughts

AI did exactly what it promised. It made writing code faster.

It never promised to make code reviewed, tested, integrated, accepted, approved, and released faster. Those were always the hard parts, and in hybrid Financial Services delivery, they were the hard parts long before anyone had a coding assistant.

So the bottleneck moved. It moved to the stages with the least flexibility, the most shared dependencies, and the highest consequences when something gets squeezed.

The organizations that win with AI-assisted development won’t be the ones with the most impressive productivity slide. They’ll be the ones that looked downstream, found the overflowing queue, and funded validation like they meant it.

Everyone else will keep writing more code than ever.

They just won’t ship it.

Join the Conversation

Where is AI-generated code piling up in your delivery system?

Code review?

Integration testing?

Shared test environments?

UAT?

Release governance?

Are your AI productivity metrics measuring what enters the pipeline, or what reaches production?

And the honest one: has anyone in your organization compared the AI productivity slide with the release calendar?

I’d love to hear the real-world version. Not the vendor case study. The version from the test environment booking spreadsheet.

About the Author

Joe Mack is a Technology Consulting Senior Principal specializing in technology leadership, enterprise SDLC transformation, release management, deployment governance, and delivery optimization for household name Financial Services companies. Joe is also a lifelong self-learner and builder of systems, and Free Tier Life is one of the ways he is trying to turn those experiences and instincts into something other people can actually use.