Pull any engineering manager aside before their team starts tracking DORA metrics and ask how things are going. Nine times out of ten you get some version of the same answer. Shipping is happening. The team is heads down. Nothing has blown up recently.
None of that is wrong. Not enough to actually work with, though. Velocity tells you how much moved. It says nothing about whether what moved held up, whether the process behind it is improving or quietly degrading, or whether the quiet spell on incidents means the system is healthy or just hasn’t been hit hard recently.
The team looks fine because nothing has gone visibly wrong yet. Whether that is because the system is healthy or because the system has not been stressed recently is genuinely unknowable without better data. That unknowable gap is what DORA metrics close. Not all at once. Not painlessly. Once the numbers exist, the conversations change. That change ends up mattering more than any single metric does.
Also Read: How Runecast Can Help with DORA Compliance?
The Before: Performance as Gut Feel
Before measurement, opinion fills the space where data should be. The senior engineer is convinced the codebase is getting harder to change. The engineering manager is equally convinced the team is moving faster than it ever has. The product lead thinks releases take too long. The tech lead thinks quality is fine given the pace. All of these can be simultaneously true, partially true, or completely mistaken – and without concrete data, there is no way to find out which.
Gut feel has genuine value. Engineers who have worked in a system for years develop real instincts. The problem surfaces when those instincts are the only available input for performance decisions. Instinct does not separate a team shipping confidently from a team shipping nervously. It does not flag when a low incident rate reflects system health versus favorable conditions. It does not see the slow accumulation of technical decisions that will matter in six months.
Performance conversations in this state drift toward activity. Stories closed, features shipped, sprints completed. Real things, but they describe what the team did rather than what the team produced. Closing fifty stories in a sprint and shipping fragility are not mutually exclusive. Without the right metrics, you cannot tell the difference until the fragility becomes an incident.
What Shifts When the Numbers Arrive
First DORA data lands with a specific kind of quiet.
Not alarm, usually. More like the feeling of having been slightly wrong about something for longer than you realized. A few patterns come up so consistently across teams that they almost qualify as universal.
Deployment frequency is lower than the team thought. Not because anyone was being dishonest, but because the mental model of how often deployments happen does not account for the batching, the delays, the releases that got held for one more thing. The cadence felt faster than it was.
Change failure rate is higher than comfortable. Each individual incident was explainable – a bad dependency, an edge case nobody caught, a configuration that looked right. Seeing them as a rate rather than as separate events changes the picture. The incidents were not isolated. They were a pattern.
Failed deployment recovery time varies based on who is available rather than on what the system supports. Some recoveries are fast because the right person happened to be at their desk. Others took hours for the same category of problem. That is not a process. That is luck with a consistent face on it.
Deployment rework rate – the share of deployment activity going toward fixing what was already shipped rather than shipping what is next – turns out to be eating a real fraction of capacity. The team was busy. That part was accurate. What the busyness was composed of was not what anyone assumed.
How the Conversation Changes
Something specific happens to team conversations after three or four months of DORA tracking. The arguments get more useful.
Before, disagreements about performance tended to be irresolvable. Reasonable engineers looking at the same situation could reach completely different conclusions because the evidence was mostly anecdotal. These conversations would end with someone deferring to seniority or letting it drop rather than actually resolving anything.
After, the disagreement has somewhere to go. Not always where people want it to go, but somewhere specific. Deployment frequency is trending down. Change failure rate has been flat for two quarters. Recovery time improved in March and then regressed in May when two people were out. These are not perfect representations of reality. They are specific enough to argue about productively and concrete enough to act on.
The question stops being whether the team is performing well, which has no answer that everyone will accept. It becomes whether particular numbers are moving in particular directions, which does. That is a smaller question in some ways and a more useful one in almost every way.
What Teams Start Doing Differently
Changes from DORA adoption are rarely sudden. They accumulate.
Batch size shrinks. Not because anyone issued guidance about it but because smaller batches show up in the data as lower change failure rates. The team sees this in their own numbers across their own deployments. The incentive does not need to be stated. It becomes visible.
The focus on failed deployments shifts from recovery to prevention. Getting good at recovering is valuable. The data makes it obvious that the same recovery effort, applied upstream, would eliminate the incident rather than just contain it. Teams that internalize this start treating their failure rate as the more important metric and their recovery time as the fallback.
Test coverage starts getting questioned in a different way. A change failure rate that stays elevated alongside a climbing deployment rework rate points at one specific thing: pre-deployment validation is passing against assumptions that no longer match what the system does. Not insufficient tests. Wrong tests – meaning tests that check against a historical version of service behavior rather than the current one. In distributed systems where services deploy independently on their own schedules, the mock files and integration assumptions underlying test suites were written at a point in the past. Every downstream service deployment is a divergence event between what the suite expects and what the system now does. Change failure rate catches the incidents that result. Deployment rework rate captures the remediation effort. Together they form a diagnostic pair that points at validation accuracy rather than validation volume.
Teams that follow this to its conclusion stop adding test cases when failure rates are elevated and start asking whether existing test cases are checking against current behavior or a snapshot from three deployments ago.
The After: Performance as a System
The framing that changes most durably is not about any specific metric. It is about what performance is.
Before DORA metrics, performance is something that happens as a byproduct of people doing work. The team builds things. Things ship. Performance is whatever that process produces.
After, performance is a property of the system the team operates inside. The metrics describe that system – its throughput, its failure characteristics, its maintenance overhead, its resilience. Improving performance means changing system properties rather than asking people to try harder at the same thing.
This does not sound revolutionary but it changes where improvement conversations go. A team trying harder inside a poorly designed system will hit the same ceiling repeatedly. A team redesigning the system will not. Knowing which situation you are in requires the kind of visibility that gut feel cannot provide and DORA metrics can.
Performance discussions start feeling different to the people having them. More grounded. Less about effort and more about structure. Less about who did what and more about what the system is set up to produce. DORA metrics did not cause that shift directly. Visibility did.
What the Data Eventually Reveals
Teams that track DORA metrics for twelve months or more hit a finding they did not expect: the metrics do not converge on a stable good.
The assumption going in is usually that there is a performance level that qualifies as elite, and that reaching it means the hard work is done. The DORA research does establish benchmark ranges – the thresholds that separate high and elite performers from medium and low performers. These feel like targets when teams first encounter them.
What happens in practice is that teams reaching elite ranges on four metrics find the fifth telling a different story. Deployment frequency high. Lead time short. Change failure rate low. Recovery time fast. Deployment rework rate climbing. The first four describe efficient delivery. The fifth describes an increasing share of that efficient delivery going toward fixing what the previous cycle produced.
This is the combination that exposes the gap between delivery efficiency and validation currency. The pipeline is moving fast. The pipeline is also checking things that are becoming less accurate over time as downstream services continue deploying and diverging from what the test suite expects. Rework rate rises not because the team is working poorly but because the validation layer is not keeping pace with the rate of system change.
Getting past this ceiling means shifting investment from test coverage to test currency – from adding more checks to keeping existing checks accurate as the system continues to evolve. That shift does not show up immediately in any single metric. It appears over time as rework rate stabilizes while the other four hold.
DORA metrics do not tell teams how to make that shift. They make it impossible to pretend the shift is not needed.
Also Read: Runecast 6.7 Released with DORA Compliance and Other Features
Final Thoughts
Tracking DORA metrics does not make a team better at building software on its own. What it does is replace the comfortable vagueness of gut feel with something specific enough to act on. That specificity is uncomfortable at the start. Teams that stay with it long enough to move past discomfort and into genuine system redesign tend not to go back. Not because the metrics solved their problems, but because working without them stopped feeling like enough.