tl;dr Software development isn’t engineering, it’s a science! Right?
I created this article, but it has been reviewed and refined with help from AI tools: Claude and Grammarly.
Introduction
The term “software engineering” implies that building software is like building bridges - detailed specs, upfront planning, predictable schedules. But what if software development belongs to a different category of problem?
The Cynefin framework explains why software projects so often resist traditional engineering approaches: why Waterfall methodologies were always doomed to fail, and why Agile embraces the messy, iterative nature of creating software.
Understanding Complexity with the Cynefin Framework
Developed by Dave Snowden, the Cynefin framework categorises problems into four domains (shown in the diagram below): Clear, Complicated, Complex, and Chaotic. Actually, there is a fifth domain, originally called Disorder (these days renamed Confusion). For the purposes of this discussion we’re only interested in the complicated and complex domains:
Complicated
Problems here are solvable with expertise, analysis, and best practices. These problems have knowable cause and effect relationships, and solutions are predictable with the right knowledge. They are best solved through structured processes and planning.
A familiar example is buying a house. If you consult the right experts, evaluate properties against your needs and budget, and follow the established processes for inspections and financing, you can predictably purchase a suitable home. The steps are understood, but they require analysis and expertise to navigate effectively.
Another example of a “complicated” problem is building a bridge. Engineers can calculate the forces, stresses, and materials needed to construct a bridge with a high degree of certainty. They can create detailed plans, blueprints, and schedules to ensure the project’s success.
So a typical engineering project, like building a bridge, falls into the “complicated” domain. While it requires expertise and coordination, the problem itself is knowable and predictable.
Complex
In this domain, cause and effect relationships are unclear until after the fact. Solutions emerge through experimentation, adaptation, and learning. We must Inspect and Adapt in Scrum speak or Probe-Sense-Respond to use Cynefin language. The best way to navigate these problems is through iteration, feedback, and emergent strategies. They are inherently unpredictable and require a different approach than “complicated” problems.
Consider a kids birthday party. We can (hopefully) intuitively understand that it would be futile to create a detailed “project plan”, with milestones to track progress, for a kids party. Instead, all we can do is set some constraints and adapt as the party unfolds, adjusting activities based on the children’s reactions, engagement, and energy levels. A kids party is a complex system.
A typical software project falls into the “complex” domain. In this context, “complexity” doesn’t refer to a tricky algorithm or how hard a problem is to solve. I mean, this could be a contributing factor perhaps. Here, “complexity” refers to the system as a whole. Requirements are often unclear, users’ needs evolve, and technologies shift. In this domain, success doesn’t come from exhaustive planning but from rapid experimentation, user feedback, and pivoting based on new information.
Chaos and Clear
The Cynefin framework also includes “clear” and “chaotic” domains. Clear problems have clear cause and effect relationships, where best practices are sufficient (e.g. following a recipe).
Chaotic problems, on the other hand, require urgent, decisive action to regain control (e.g. responding to a system outage or natural disaster). While software development occasionally dips into chaos (e.g. firefighting production issues on a Friday evening), this isn’t usually where we operate.
With this understanding of Cynefin, let’s now look at why I believe software development should ditch the “engineering” label.
Why Software Development is not “Engineering”
Building a bridge follows a linear path: requirements are fixed (e.g. span a river, support X weight), materials are well understood, and outcomes are predictable with sufficient expertise. Engineers rely on decades of physics, building regulations, standardised materials, and repeatable processes.
Software projects, by contrast, are usually riddled with unknowns:
- Ambiguous Requirements: Stakeholders often don’t know what they want until they see a prototype.
- Evolving Tools: Technologies and frameworks shift rapidly, rendering yesterday’s “best practices” obsolete.
- Human Factors: Software interacts with users, markets, and other systems in unpredictable ways.
Treating software like a bridge (i.e. assuming requirements are fixed and outcomes are plannable) is why Waterfall projects so often fail. Waterfall assumes a “complicated” problem, demanding exhaustive upfront design. But when requirements shift (and they always do), the plan disintegrates.
“But, building a bridge involves lots of people - why isn’t it ‘complex’?”
Large engineering projects do require coordination among hundreds, maybe even thousands, of people, from architects to welders. But the Cynefin framework distinguishes “complicated” from “complex” based on the nature of the problem, not team size.
As we saw above, complicated systems are knowable - engineers can model, calculate, and simulate outcomes before construction begins. Complex systems are different. Software projects often have many unknown unknowns at the outset. For example:
- How will users actually interact with this feature?
- Will this microservice architecture scale under unpredictable load?
- When will the third-party API be available?
These questions can’t be fully answered with upfront analysis. Solutions emerge through trial and error, much like a biologist studying an ecosystem.
Building a bridge is “complicated” (as defined by Cynefin) because the problem itself (e.g. “span this river”) is stable, however many experts it takes to deliver. Software development is “complex” because the system itself is unpredictable, however small the team. Bridges operate in a closed, deterministic world; software exists in an open, adaptive one.
Two teams given the same plans will build the same bridge. Two teams given the same software requirements will produce wildly different products.
“But engineering projects blow out all the time - isn’t that ‘complex’ too?”
Fair. I’ve painted a rosy picture of engineering: fixed requirements, tidy blueprints, predictable outcomes. Reality is messier. The Sydney Opera House was estimated at AU$7 million and scheduled to open in 1963; it opened in 1973 and cost AU$102 million. Megaprojects overrun so reliably that Bent Flyvbjerg built a research career on it and coined the “iron law of megaprojects”: over budget, over time, under benefits, over and over again.
Doesn’t that put engineering in the “complex” domain after all? Mostly, no. Dig into the blowouts and you tend to find optimistic estimates, political pressure to lowball, and scope forced to change from outside - failures of execution and honesty within a problem that was still knowable. The physics of the Opera House never shifted mid-build.
The roof did, though. Utzon’s shell design had no known construction method when the project was approved, and the team spent years probing options before landing on the spherical geometry that made it buildable. The most novel part of the build behaved exactly like a software project: iterative, unpredictable, impossible to estimate upfront. The novel corner of an engineering project can be just as “complex” as any startup.
“We’ve been building software for decades - why is it still ‘complex’?”
Fields like civil engineering have matured over centuries, developing standardised practices, materials, and formulas. But software’s persistent complexity stems from its fundamental nature.
Unlike physical structures, software is made of logic and human intent. Requirements shift as stakeholders discover new needs, markets evolve, or competitors innovate. As I mentioned above, a bridge’s purpose (span a river) is static; software’s purpose (solve a problem, delight users) is dynamic.
A bridge’s load-bearing calculations rely on well-understood physics. Software, however, interacts with layers of dependencies (APIs, libraries, user behaviour, hardware) that create emergent behaviours.
Software is shaped by people (developers, users, stakeholders) whose behaviours, preferences, and misunderstandings introduce variability. Even decades of experience can’t eliminate the unpredictability of human interaction.
“You ship what is in your programmers’ brains” - Andrea Magnorsky
I recently attended YOW! Brisbane and picked this quote up from Andrea Magnorsky’s talk. It speaks to the difficulty of sharing information between team members and the unpredictability of human factors in software development.
Civil engineering’s core principles (e.g. Newtonian physics) haven’t changed in centuries. Software tools, frameworks, and paradigms, however, evolve rapidly. Whilst we often refer to “best practices” in software development, these are frequently fleeting, context dependent, and subject to change.
So then. “Complexity” isn’t a bug of modern software development projects, it’s an inherent feature of it.
“So is no software work ’engineering’ then?”
Some of it is. Cynefin classifies problems, not industries. Plenty of software work sits comfortably in the “complicated” domain: standing up infrastructure you’ve built ten times before, writing yet another CRUD integration against a well-documented API, tuning a query towards a fixed performance target. If you’ve done it before, you can estimate it, plan it, and hand it to an expert. That work genuinely is engineering.
The complex part of software development is discovery: deciding what to build, learning whether anyone wants it, and watching how the system behaves once real users touch it. A single sprint can contain both - a complicated database migration in the morning and a complex “will customers understand this feature?” experiment after lunch.
The trouble is that we brand the whole endeavour “engineering” and then manage all of it, discovery included, as if it were the complicated kind.
Agile and Embracing Complexity
Agile methodologies succeed by acknowledging complexity. Instead of rigid plans, Agile embraces:
- Iterative Development: Deliver small, functional increments to gather user feedback.
- Adaptive Planning: Prioritise flexibility over exhaustive upfront plans and documentation.
- Empirical Process Control: Learn by doing, adjusting course as new information emerges.
This mirrors how scientists work: forming hypotheses, running experiments, and refining theories based on evidence. A startup pivoting after user testing, or a team refactoring code based on new insights, is conducting research, not engineering.
Consider a concrete example. A team builds a new checkout flow. Rather than specifying every interaction upfront, they ship a minimal version to a subset of users. They observe where users hesitate, where they drop off, and what questions support receives. Based on that data, they adjust the flow. This is probe-sense-respond in action, not a failure of planning.
Scientific research thrives on this kind of uncertainty. A biologist doesn’t know what they’ll find when studying a new species; they adapt their methods based on emerging data. Similarly, developers can’t foresee how users will interact with a feature until it’s deployed.
The scientific method is a disciplined way to work through uncertainty. Agile is no free-for-all either; it applies the same discipline to exploring the unknown.
Conclusion
Calling software development “engineering” risks imposing the wrong mindset. Unlike bridges, software isn’t built; it emerges and grows through exploration and adaptation. The Cynefin framework helps us see why: in complex domains, success comes from iterative discovery, not from executing detailed upfront plans.
A bridge stands unchanged for decades; software is never done - it adapts or becomes obsolete.
Perhaps it’s time to retire the term “software engineering” altogether. What would change if we called ourselves “software researchers” instead? Maybe we’d stop promising fixed timelines for unknowable work. Maybe we’d stop treating changing requirements as a failure of discipline and start treating them as new data. Maybe stakeholders would stop expecting blueprints and start expecting hypotheses.
How do you approach software development - as an engineer or a scientist? I’d love to hear your thoughts in the comments below.
Thanks for reading.