I was inspired to get out and take some photos in the crisp evening air of Copenhagen in December. Just days away from the winter solstice, it is dark by 3.30pm, and by 10pm when I took to the streets, it is pitch black. Small groups of locals gathered on benches and corners holding tins of beer, the bars closed as COVID cases shoot up across Denmark. The blanket of grey cloud which has covered the city for the past week reflected the lights of the city creating a ghostly halo across the sky, and Christmas lights sparkled from balconies and windows.
In the realm of technology and software development, the phrase “shift left” has gained significant traction. At its core, the “shift left” approach emphasizes the importance of addressing issues and tasks earlier in the development process. This proactive strategy can lead to more efficient, effective, and successful technology implementation projects. Let’s delve deeper into this concept and understand its parallels with the heuristic breadth-first search.
“Shift left” is a philosophy that encourages developers and teams to tackle potential challenges and issues at the earliest stages of a project. Instead of waiting for problems to arise during the testing or deployment phases, the idea is to anticipate and address them during the planning and development stages. This can lead to fewer surprises, reduced costs, and a smoother implementation process.
Analysing the Existing Codebase and Desired Outcomes
Before diving into a new project or adding features to an existing one, it’s crucial to analyze the current codebase. This involves understanding its strengths, weaknesses, and potential areas of improvement. By doing so, teams can identify existing technical debt and areas that might become bottlenecks or pain points in the future.
Once the current landscape is clear, the next step is to define the desired outcomes. What is the end goal? What business value is the project aiming to deliver? By having a clear vision, teams can align their efforts and ensure that every step taken is in the right direction.
Defining Slices of Work
After understanding the starting point and the destination, the journey can be broken down into manageable slices of work. Each slice should:
Deliver Business Value: Every slice should have a tangible benefit, whether it’s a new feature, an optimization, or a bug fix.
Incrementally Increase Solution Maturity: As each slice is completed, the overall solution should evolve and mature, getting closer to the desired outcome.
Reduce Technical Debt: With each slice, any existing technical debt should be addressed, ensuring that the codebase remains clean, efficient, and maintainable.
The Heuristic Breadth First Search Analogy
The process described above can be likened to a heuristic breadth-first search (BFS). In BFS, we explore all the neighbouring nodes at the present depth before moving on to nodes at the next level of depth. Similarly, in the “shift left” approach, instead of diving deep into one aspect of the project, teams tackle a broad range of tasks that deliver immediate value. This ensures that the most pressing issues and valuable features are addressed first, providing quick wins and immediate benefits to the business.
The heuristic aspect comes into play when we prioritize these slices of work based on their potential impact and value. Just as heuristics guide search algorithms to find the most promising paths, our understanding of the business needs and technical challenges guides us in choosing which slices to tackle first.
The “shift left” approach, when combined with the principles of analysing the existing landscape, defining clear outcomes, and breaking the journey into valuable slices, can transform the way projects are executed. By drawing inspiration from heuristic breadth-first search, teams can ensure that they are always moving in the right direction, delivering value at every step, and building solutions that increase the overall maturity of the codebase.
The various scaled agile frameworks make use of some common building blocks which many practitioners will be familiar with. Stories are often the most familiar, as they are the raw material of the SCRUM and Kanban practices which have been applied at the team/squad level for many years, particularly in technical teams. However, in the process of scaling these practices a need for larger aggregations of work was needed, and in response to this need, emerged epics.
Whilst best practice for writing user stories is well documented, the guidelines for scaling this practice to larger groups of activities is not. Having worked with teams using epics for some time, I have seen a range of examples of good epics and even more examples of anti-patterns. Getting epics wrong is probably the number one reason why teams perceive scaled agile to add less value than they expect.
What follows is a summary to serve as guidance for those embarking on backlog creation for a team or squad within a scaled agile organisation.
An epic is a group of stories. Each story is a necessary stepping stone to achieve the objective of the epic.
- An epic can be relatively small or relatively large, but is bigger than a team can deliver in a single sprint. It should generally be a large piece of work, taking between 1-3 months.
- An epic should represent a concrete piece of deliverable value to the end-user. Having a backlog full of epics representing invisible enablers is an anti-pattern.
- Epics should articulated as an outcome. The purpose of the epic is to describe what we want our user to experience, not how to achieve that outcome. Having an epic which is overly prescriptive about how to achieve the outcome is an anti-pattern.
- It should be possible to prioritise all epics against one another. If it isn’t possible to meaningfully prioritise one epic against another, the planning process becomes overly constrained and complex.
- Epics should be mostly deliverable by a single team. Organisations are not perfect, and therefore this criteria is often a tough one. Some dependencies are OK, but if you find yourself in a position where most epics have dependencies to two or more other teams, it may be time to consider reorganising around your business domains and architectural ambitions. My recommended read on this subject is Team Topologies by Matt Skelton.
- An epic should materially improve an underlying KPI or key result (in the OKR framework, for example).
- A team should always learn something from delivery of an epic. Almost all perceived user value is a hypothesis which product teams should be seeking to test at the earliest opportunity.
Benefits of epics
- Enable meaningful prioritisation across teams
- Provide transparency across teams
- Enable efficient evaluation of work against OKRs or KPIs – attempting this story-by-story is very time consuming
- Force product managers to work hard at breaking down long term product aspirations into chunks which deliver user value
- Enable meaningful dependency planning across teams
- Are a mechanism by which to communicate the value of a group of stories
Epics, like stories, must start with the user. When faced with a blank canvas, the use of design thinking methods can lead to a better understanding of the problems the user faces. To better understand who your user might be, the use of personas will allow you to better empathise with the human beings who will ultimately use your product. Personas are also a powerful tool with which to communicate with your team, helping them to truly feel connected with why a given epic is important.
The process for writing an epic within a scaled agile context often begins with a process of slicing up a broader ambition. If you are working with initiatives, each epic is a “slice” of the larger initiative. Slicing is possibly the hardest task a product manager faces on a regular basis. There are several mechanisms by which work can be sliced. Most of these can be linked to an underlying principle of “shifting left”: always try to learn as much as possible as early as possible. By bringing learning forward in the roadmap, you mitigate the risk of committing too much time to endeavours which don’t deliver the expected value.
- Prototyping – can a prototype of a deliverable be developed before committing to a longer roadmap, in a way which meaningfully teaches you something about the underlying hypothesis you may have?
- Architectural slicing – can you deliver value without committing to an expensive engineering challenge, in order to shift learnings left?
- Segment slicing – can you offer products to subsets of users in a way which removes complexity from earlier epics, whilst enabling learning from those users who you do target? Try to think broadly about segmentation, as your existing predefined customer segments may not be the most optimal for this purpose. Experiment with slicing customers based on a broad range of characteristics to find pockets of opportunity to target smaller user groups with similar needs.
- Complexity slicing – can you strip complexity away from the target product in order to deliver value to some users earlier? This is an antidote to the anti-pattern of having too many enabler epics – rather than building a complete solution back to front, build a partial solution through all layers of the technology stack to enable delivery of value early on.
- Stubbing – can you measure interest in a feature by creating a link to the feature before you even begin working on it? Measure the number of clicks on the link to gauge interest before beginning an expensive development journey.
Documenting an epic
An epic, once you have sketched out its place in a backlog, must be documented in such a way as to be meaningfully discussed in a variety of forums, prioritised, and implemented.
Leadership must be able to understand the epic in the context of the overall value proposition for your product. Other product managers must be able to evaluate the priority of the epic against their own in order for you as a team to prioritise effectively. Teams must be able to understand the objective in sufficient detail to be able to get to work implementing and delivering the epic.
A four-step process can be followed to document an epic effectively:
- Write the epic description. The description is a narrative, structured in a similar way to a user story. “As a who, I want to what, so that why“. The who, what, and why must be specific enough for the relationship between the work and the benefit to the user to be clear. Keep in mind that the user needn’t be a customer – it could be an internal team or stakeholder, a system, or a member of your own team.
- “As a user…” is less desirable than “As a controller managing incoming invoices…”
- “…I’d like to manage invoices…” is less desirable than “…I’d like to be able to control, reject, or comment on invoices…”
- “…so that I can do my job quicker” is less desirable than “…so invoices can be quickly and accurately processed to ensure timely payment, increasing creditor perception of our company.”
- Write a summary for the epic. The summary is a short and sharp name for the epic which can be used to unambiguously refer to the epic in prioritisation and status discussions. For example “Invoice controlling workflow”.
- An anti-pattern for epic summaries is to use the “Feature: version 1” formulation. The summary should still communicate as specifically as possible what the outcome of the epic is.
- The summary should not be the full description – avoid using the “As a user…” formulation.
- Write clear acceptance criteria for the epic. The acceptance criteria, as for a story, are a critical tool for the implementing team to understand specifically what is expected from the user’s perspective.
- See acceptance criteria as a checklist of requirements to act as guardrails for the team as they design the solution, and as a way for the team to know when they are done.
- A common mistake is for acceptance criteria to be prescriptive about how to deliver the expected outcome – that is not the purpose, and can fundamentally undermine the autonomy of the team.
- Evaluate the epic. An optional final step can be undertaken, to evaluate the epic using a framework which can be applied consistently across all epics as an aid to prioritisation. Examples of scoring methods which can be adopted include:
Once a backlog has been defined and refined with epics of a high standard, three steps naturally follow. Depending on the cadence of the adopted scaled agile framework, these activities may happen periodically within a product increment (or “sprint of sprints”), or on an ongoing basis more aligned with individual development sprints.
- Prioritisation. Leveraging the evaluation scoring performed in the final stage of epic definition, leadership and the broader product management community prioritise epics against one another to create the overall product backlog.
- Story refinement. The product manager and team(s) working the backlog collaborate to break down the epic in to deliverable stories which ultimately deliver the expected outcome for the user.
- Dependency management. With the deeper knowledge of what is needed to deliver each epic, the team can engage in a forum with other teams to highlight dependencies. The prioritisation across teams from point 1 is used as a reference when deciding whether a team should work on their own stories, or dependencies other teams have on them.
- Estimation & execution. The team implementing the epic estimate the stories within an epic and begin working to deliver the stories in priority order.
Many people today are talking about how uncomfortable they feel about their dependence on devices, and more specifically the content they consume via their devices. Whilst this theme is certainly well established, I have observed very few experimenting with ways of adapting to this trend in a positive manner.
What is interesting is how, in a market where devices are sold on the basis of how they can be depended upon to provide a solution to our needs (and they do), it is often an entirely different kind of dependency which emerges as the dominant factor in our relationship with our devices. Specifically we are using our devices to medicate boredom and anxiety, and later choosing feeds over friends once the dependency has been established. This is not a healthy dependency, in many circumstances probably as undesirable as the long term use of alcohol or recreational drugs for the treatment of these states of mind. Like drugs and alcohol it probably doesn’t feel like a problem until it is too late. But unlike drugs or alcohol, the consequences are less direct, more emergent.
Our devices have quietly inserted themselves into our lines of communication with friends and social circles, into the time and space which we spend with our friends, and thus into the fabric of our relationships. And they are so effective at entertaining, and at incentivising further consumption, that unassuming victims begin to choose, consciously or unconsciously, their devices over their relationships. Think of the husband on the sofa, scrolling through twitter rather than acknowledging his wife’s needs, or the teenage girl, one of many looking at each other only through the lens (literal or otherwise) of what might make a popular post on snapchat or instagram.
As our minds adapt to the growing strength of each hit, so the systems evolve new means to keep us engaged, and in a zero-sum game, our attention on people and relationships slips away, slowly at first, but the snowball is gaining momentum.
For me, the time has come to acknowledge this phenomenon and act. My children deserve my undivided attention, my wife deserves the best of me, and my relationships with wider family and social circles would benefit far more from a phone call than from a rushed WhatsApp message. And so I’m running an experiment. I will switch to a feature phone for a month or so. I’ve selected a device with enough features to ensure the impact on my ability to do my job is not impacted, but with enough compromises in user experience to be an undesirable channel through which to consume, regardless of its theoretical capabilities.
This experiment requires concessions, but I had in mind some capabilities which would help me strike the right balance between convenience and addictive consumption. On my list of considerations were:
- No qwerty keyboard – if I really need to type, I can take the time to pull out an ipad or laptop
- Small screen – if the phone happens to have a twitter app, let’s make sure it’s a crappy experience
- 4G with hotspot capabilities so I can continue to work remotely from my ipad
- Basic maps so I can navigate if I’m in a tough spot
- MP3 player and bluetooth – I can live without Twitter, but not without Bill Evans…
Some concerns I have going in to this experiment are around the ways in which a mobile phone has become so convenient for tasks which aren’t about consumption. For example, I use my phone to manage task lists, to write stubs of blog posts, to take and edit photos, to manage and respond to emails, to pay using Apple Pay, to manage bank accounts and make payments, to hire electric scooters, to study on Coursera or Memrise, to plan routes on public transport, to hire cars… The list of what I am likely to miss is endless, and that is part of the problem.
My hope is that, with a handy partner device (the ipad on which I am writing this post), I will be able to enjoy enough of the convenience of this long list of capabilities without the excessive convenience which so easily leads to a slip in to mindless consumption. Because it leaves me numb, burns my time and feeds my anxiety, and that ain’t good.
Several years ago I was very lucky to become involved in a local initiative in Clapham, London, to establish a community bookshop. The initiative was a response to the closing of the existing local bookshop along with the persistent threat of closure which haunted the local library. Our vision was one of community building, with shared ownership via community shares. We envisioned evening events, a catalyst for a local sharing economy, a safe space for socialising, studying, observing, contemplating. And we built significant momentum, with a committee formed of interested locals working together to identify opportunities to secure premises, grow local awareness, and establish the legal foundation for community ownership.
Unfortunately, the momentum slowed and was ultimately lost altogether when the most likely location for the shop, a new social development in the area, fell through. So, a couple of years on it is time to close down the website, but it will remain archived here at bookshop.danieleverard.com for those interested in reviewing and perhaps learning from the history of the project.
The skate park at Bexhill leisure centre, now earmarked for demolition, has an incredibly passionate and vibrant community of riders and skaters.
The skate park is in desperate need of renovation – several accidents and near misses due to the state of the ramps and concrete occurred during the shooting of these photos. And with demolition on the horizon, the community here deserve a firm commitment from local government to the development of a new location.
Skate parks are positive and sociable community hubs which deserve serious investment to ensure they are safe spaces for up and coming athletes to practice and share their passion with others.
Evolving Specialised Species in Diverse Simulated Ecologies using a Subsumption ArchitectureDownload
Back in 2006, three years in to my degree in “computer systems and software engineering” at the University of York, I embarked on my Masters project. The ambition was quite clear – to use a complementary suite of AI algorithms to simulate speciation. The choice of algorithm for behaviour selection, Rodney Brooks’ subsumption architecture, was dictated by my supervisor, but despite the existence of prior work on the use of the subsumption architecture, the existing implementation was missing key prerequisites for achieving the ambition of the project. Specifically the ability to vary the geographic (and other environmental) conditions such that speciation might occur.
Whilst I learned many things from the arduous process of trying to run successful experiments on this journey, none were so precious as the respect I developed for the complexity of systems. The combination of two core AI algorithms, a broad range of environmental variables and the entropy of unpredictable individual behaviour across a population of hundreds over thousands of generations, led to a huge degree of complex, emergent outcomes. Controlling these outcomes became a huge challenge for me, preventing me from obtaining the evidence I needed to support some conclusions I knew it was possible to draw. The experience was humbling, educational and frustrating in equal parts. The write up I submitted can be viewed at this link. I will eventually edit and post a subsequent revision which includes conclusions drawn after further experimentation past the submission deadline!