Mediations #30: Increasing Efficiency and Effectiveness with Economics & Psychology - Part I
How Socio-Technical Systems, Institutional Economics and Evolutionary Economics Theories can help
Software systems have become incredibly complex over the last decade(s), along with the organizations that build them. As a result, increasing an organization’s efficiency and effectiveness became as challenging as developing a software product.
A problem I see is people spending a lot of energy and time learning the intricacies of their craft (software development, design, marketing, etc.), and almost no time learning the backbone of working in or with an organization: economics and psychology.
Both subjects are often discarded, but they specifically focus on increasing the efficiency and effectiveness of an organization and its people, respectively.
To understand how economics and psychology help navigate an organization, we can use a few different theories as lenses. In this issue, we’ll talk about Economics. Part II will be about Psychology.
Socio-Technical Systems Theory
Maybe the most popular Economics theory these days is the socio-technical systems theory (STS).
STS was born in the 1950s, when coal mines began using machinery. People held the dominant view of workers as “extensions to machines” (does it sound familiar to how we evaluate working with AI?). Researchers sought to understand why they faced a paradox in coal mines: new mechanical technology decreased productivity and worker satisfaction, despite being technically superior.
Over the last seven decades, STS has evolved into an interdisciplinary framework and has been applied to many complex organizations across different fields.
According to STS, optimal organizational performance requires joint optimization of both social and technical subsystems that are divided into six: people, technology (not only software development but also tools used in the organization too), processes, goals, infrastructure (not only software infra but also the infrastructure of an organization) and culture.
Falsely, many leaders focus solely on technology, processes, and goals to increase effectiveness and efficiency, paying less attention to the other three. Moreover, the interconnections among these six and how they affect one another are often not considered. While new industry-altering evolutions like AI are in full swing, this comes at a cost.
Many leaders lack the understanding of how concepts from STS directly contribute to their organization (look at the ‘Great’ link I shared below to learn how). Almost all categories show their results in lagging metrics, making it difficult to connect the dots between a change and its impact. Yet, we can get help from other theories.
Institutional Economics Theory
Institutional economics theory characterizes institutions as extractive or inclusive: extractive ones concentrate power and wealth in small elites, while inclusive ones distribute power through pluralistic mechanisms.
In tech, extractive institutions can concentrate power and limit autonomy through decision bottlenecks (any change requiring multiple approval layers slows down iteration) or by hoarding knowledge and information at the top. These patterns result in long release cycles and teams perpetually blocked by dependencies due to a lack of context or micromanagement.
On the other hand, inclusive institutions distribute authority and enable autonomy through decision rights at the team level (such as teams controlling their roadmap within boundaries), transparency (information access is open to everyone), and flat hierarchies (managers as enablers, not gatekeepers). These patterns result in frequent deployments, teams functioning as “mini-startups” with end-to-end ownership, cross-functional collaboration without permission demands, and knowledge sharing that enables domain ownership.
Creating inclusive institutions within the organization will enable “sustainable wealth” (so to speak) for everyone and advance the organization (using RFCs or ADRs, Netflix’s culture, Spotify Organization Model, or Jeff Bezos’ API Mandate are among the best-known examples of creating inclusive institutions).
The essential mechanism is that the leaders must commit to giving up power. Knowing the results of decades of institutional economics research can help shape culture, infrastructure, and processes directly, while also having an indirect positive impact on goals, technology, and people.
While institutional economics focuses on the institutional structure, evolutionary economics focuses on organizational capabilities, which are more linked to innovation.
Evolutionary Economics
Evolutionary economics favors a variation-selection-retention framework in which organizations generate modifications through experimentation and search processes, select environments in which to run these experiments (e.g., by choosing to compete in a market), and retain successful variants through organizational routines and memory. Innovation is not a discrete event but a continuous process of search under ambiguity and recombining existing knowledge in novel ways.
When we consider adapting AI or developing new AI tools, we need inclusive institutions that allow for innovation, and organizational capabilities and culture that nudge people to use their existing knowledge and experiment with newly available technology to select the right option that increases team and system performance.
For example, the guild or chapter structure in Spotify’s model (guilds are voluntary communities organized by technical domain, such as incident management guild or developer experience guild, that share best practices, coordinate technical debt, and enable expertise development) addresses the trade-off between cross-functional delivery teams and function specialization. Guilds act as meta-routines that share and update other routines. Allowing guilds to form and wield “soft power” to drive changes in organizational routines enables innovation.
At SumUp, I’m part of the incident management guild, and we have put a lot of effort into automation. We improved how each team reports and manages their incidents with small steps, enabling a well-established blameless and effortless incident management culture.
But how these theories play a role in a daily life of a software engineer?
Software Development and Economics
I see economics as a bonus that will help with any software engineering task. A better understanding of the environment helps reduce toil and rework. By learning evolutionary economics, a software engineer can seek small innovations in their work (or join a guild) without expecting divine permission. Using a variation-selection-retention mindset, they can create slight variations in the codebase, select the best, and retain it for the long term.
By understanding institutional economics, engineers can choose the organization they want to work for and assess the company’s long-term value and sustainability. If a team lead has an ambition to collect all the information and power, the engineer in their team might have fewer chances for a promotion. When an engineer faces an extractive organization rather than an inclusive one, they can also plan how to influence it (a crucial skill for any leader) in the absence of transparency and accountability.
We mainly delved into the technical aspects of socio-technical systems, briefly touching on the social elements. When we consider the DORA research, which investigates the capabilities, practices and measures of high-performing technology-driven teams and organizations, leaders have to think in terms of processes (e.g., how to dedicate time to guilds/chapters), people (e.g., how to structure teams), culture (e.g., how to create autonomous teams), technology (which AI tools everyone is encouraged to use), and even infrastructure (if your security team is sitting in the corner office alone, you will have a hard time shifting left on security). They need to create inclusive institutions that innovates using evolutionary methods.
Of course, no leaders will be successful by learning these theories without practicing them at different scales (e.g., first in one team, then in the department) and without combining them with learnings from psychology.
Lastly, all of these learnings are not reserved for the organization’s managers. Anyone can drive a change (it’s just easier when the managers are on the same page) and influence their manager to take tiny steps in the right direction.
We’ll talk about psychology in the next Mediations.
Good to Great
I share max three things I found interesting, sorted by good to great.
Good: I sent this to a few people at work. Also deserves its share here. Building agency is what I wish everyone tries to do. How to be high agency at work by Torsten Walbaum
Better: “…a reminder that self-understanding, like so many facets of life, rarely moves in a straight line.” let me repeat for myself: self-understanding rarely moves in a straight line. From Prison, a bank robber and acclaimed author reflects on time by Psyche.
Great: Adaptive Socio-Technical Systems with Architecture for Flow is a great talk by Susanne Kaiser about the topic we talked this week. She uses Wardley Map, Team Topologies, Domain-Driven Design approaches to structure an organization and their systems. Highly recommended!
Recently, I wrote about
I was looking for the meaning of life for the last few years. I found nothing. Apparently, it was there all the time. All I needed to do was to do a retrospective look (which took me a few weeks to complete). I wrote the end result and the journey of how a problem-solving mindset helped me get better at something I feel is my passion now.
Until next time,
Candost
P.S. My RSS feed didn’t include the whole post before, now it does. Thanks David for reporting!