Shift Left

All software is delivered into production via some kind of pipeline, these pipelines may vary in length, complexity and effectiveness but they are the route a team relies on to deliver value.

A pipeline may start on the left with a connection to source control and end on the right with a connection to your production environment.

The goal of any introspection of this pipeline should be to shift more and more of the activities it performs further left. Obviously certain tasks, such as deployment, have their position in the pipeline dictated by practicality, but any activity concerned with quality should be as far left as possible.

Early Enforcement and Education

Once sub-optimal code is allowed into a code base a certain amount of technical debt has been incurred, the further right this code goes in the pipeline the higher the cost either in terms of impact to customers if it gets deployed or overhead for developers who must try and fix it.

Automated enforcement of quality as far left as possible in a pipeline is an easy way to try and reduce these impacts, possibly to nothing.

If this enforcement can be at the point of a pull request we can ensure that these sub-optimal changes don't even make it into the mainline trunk.

If it may seem a little heavy handed to place these barriers in the way of developers then they can also be viewed as automated opportunities for education.

If a problem with a change is pointed out early to a developer it gives them an opportunity to learn the why and how of how they can improve the code they are submitting.

If sub-optimal code makes into production these lessons are often more painful to learn and have much bigger consequences.

The Grey Area

Whenever new code is submitted a grey area opens up where we are less sure than we were about the effectiveness of our code in doing what it is designed to do.

If your pipeline does little to enforce quality on the left hand side then this grey area extends all way to your production environment.

The goal of moving quality and testing activities further left is to reduce the size of this grey area to be as small as possible and to get us back to a comfortable feeling as quickly as possible.

Uncertainty in the state of a code base is a bad place to be, not only does it open up the possibility of shipping defects to customers it can also hamper further progress in delivering new features as developers become more tentative in the changes they are willing to make.

Without having a pipeline that is testing, analysing and deploying code on a regular basis this grey area will grow in the build up to you pushing to production creating a fog of fear for everyone involved.

Before You Move On

Developers do and will make mistakes, any strategy built on the idea this can eradicated will fail.

Another goal of shifting left is to highlight when mistakes are made much earlier.

If these mistakes can be pointed out before the developer moves on to their next task the cost of fixing the mistake is likely to be greatly reduced.

Developers tend to get absorbed in their current task and can very quickly forget the details of anything they've done previously.

When mistakes are brought to them when the code has progressed further through the pipeline a large amount of time is likely to be lost as the developer shifts context to re-educate themselves on the change that is causing problems.

If the mistake is highlighted a matter of minutes since they submitted the code they will likely very quickly curse their stupidity or oversight and apply a fix.

Within software engineering we are lucky that the production process for our output can be made extremely fast and easily automated, this gives us a unique opportunity to recover from mistakes and improve quality on a rapid cadence.

Many other engineering disciplines where build time maybe measured in hours, days or months would likely kill for this opportunity.

Embrace this gift that we have and attempt to take it to the nth degree by moving these opportunities ever further left in your pipeline.

Scrum Down

The term scrum was first used in the context of product development to refer to the unified nature in which a team should work, not unlike the way a pack of forwards work together in rugby.

Scrum as an agile framework is now common place in software development however in many instances implementing a perfect incarnation of scrum has become the goal, this leads us away from the reasons scrum was conceived, namely to implement an agile approach to delivery.

This rigid adoption of scrum has lead many to perceive it as something that gets in the way of forward progress and is something that exists for its own sake.

This is the antithesis of what scrum was designed to accomplish so where have we taken a misstep and how do we tackle these misconceptions.

Agile Qualities

In essence scrum is a simple process.

A product owner creates a prioritised list of value, a scrum team decides on a subset of this work they fill they can achieve in a certain time scale, after this time period elapses the team reviews their achievements, looks back on how things could have been done differently, and presents a potentially shippable increment of the product to the product owner.

There are certain key take aways from this process that its important not to lose site of and separates a scrum based approach from its waterfall predecessor.

It promotes flexibility and speed, a scrum team should be nimble and agile such that they can move their focus to different priorities as the product, and the world it exists in, move on. Each new iteration is a chance to tackle to a different problem or react to a new imperative.

It promotes commitment and accountability, the team lays down very clear objectives to be achieved in a specific timescale and hold themselves accountably if they fall short of these goals. This is all done in order to increase output and lay the foundation for more accurate future estimation.

It promotes an iterative and open approach, keeping the product moving forward on a definite cadence with all decisions being made in the open with input from all team members and stake holders.

Scrum Ain't Heavy

The root of many peoples unwillingness to embrace scrum comes from a perceived rigidness in its application.

One source of this frustration can be scrum ceremonies.

Fundamentally scrum only has three ceremonies, a planning session, daily stand-ups and a review/retrospective session.

Because agile promotes communication between team members there will be other occasions where time needs to be allocated to discussion, there is no need for these arrangements to be strictly applied, an organic and informal approach will suffice.

A second point of contention can be what is perceived as a slavish following of the estimation process.

In both cases a dose of reality may be required to understand these things are unavoidable, if we develop a skepticism of the need for communication or we believe that more information about a situation shouldn't trigger a re-evaluation of effort then we are creating the conditions necessary for failure.

If it starts to feel like scrum is getting in the way then its more than likely implementing scrum has become the focus of the team and minds have been diverted from the delivery it is supposed to bring.

Find Your Own Way

Scrum is a framework not an answer and not a cure for all ills.

Scrum provides you with a toolset and some guidance on best practices for getting the best out of your resources using these tools, but this guidance cannot be slavishly followed.

Do not blindly follow how others have decided to implement scrum and always remember the end goal is to adopt an agile mindset. Its possible for scrum to resemble waterfall if this concentration on agile principles isn't observed.

This all means you should find a way that works for your team.

Providing you are following an approximation of the scrum process, and therefore exhibiting the agile qualities it should promote, then you're doing just fine.

It is also highly unlikely that you will be able to properly identify this nirvana immediately, it will be the product of lessons learnt and a willingness to admit the need for improvement via introspection of what is working and what isn't.

If scrum is seen to be something that comes with a rule set and a rigid doctrine then the battle is lost, it holds certain values dear but how you implement them is in your hands.

Dealing with Legacy

Many conversations about the complexity of implementing a new feature or the post mortem of a defect with eventually involve the phrase "the problem is this is legacy code".

Since on this basis it seems inevitable that all code is eventually considered legacy what can we do to ensure that this doesn't cripple our ability to move our product forward?

Well to do this we probably need to start with defining what we mean by legacy code?

Defining Legacy

Whilst the age of a code base is likely to be a contributing factor in whether or not it is considered legacy it is by no means the definitive measure, there are many well established and long serving code bases that are still being actively developed.

Instead we should defined legacy by the effect it has on us, legacy code instills a lack of confidence in those who have to work with it.

This lack of confidence generally comes from either an inability to refactor, an inability to test or potentially the most frightening of all an inability to build the code.

The inability to refactor may come from simply a lack of understanding of how the code works or potentially from the "ball of string effect" that causes refactoring and rewriting to become synonymous, the inability to test usually has similar roots.

The inability to build is somewhat more serious as it immediately rules out any remediation with regard to the other problems the code base may have, its roots generally lie in an overly custom build process that may be heavy reliant on technology that has disappeared from the mainstream.

Without getting into a discussion about how the root causes of these problems could be countered, can we form an escape plan to get ourselves out of this situation?

What Over How

Two of the problems that create a legacy code base can be somewhat addressed by shifting the perspective of the analysis, instead of agonising over "how" this code works start thinking about "what" it should be doing.

Concentrating on "what" code should be doing is all that is required to construct tests for the code, after all "what" is all we have to go on if we are following a TDD approach.

It maybe that we have to accept that these tests are not strictly unit tests and that they aren't following FIRST principles but before embarking on any other change we need to have a crutch to restore confidence that this code is still doing what we need it to do.

Concentrating on "what" is also a decent starting place for refactoring, this view point can act as a microscope to identify what is wrong with the current API, architecture or implementation.

That still doesn't mean that the required refactoring will be easy but it will help shape it, which is a necessary starting point before any technical debt can be dealt with.

Strangling the Problem

Martin Fowler identified an approach to refactoring a legacy system that he termed a Strangler Application.

Using this approach rather than simply starting again and trying to create a new system we gradually build the new system around the edges of the old.

The new code gradually grows taking on more and more of the required functionality until it strangles the legacy code.

To achieve this he identifies two strategies, Event Interception and Asset Capture.

Many systems can be defined by the events that they handle and the data that they capture, when building a strangler system these events and data items are gradually moved to become the responsibility of the strangler system until most of the critical items reside in it.

An important secondary aspect to this approach is that we can make our lives easier in the future by designing a system to be strangled, a modular design where responsibilities and the data model can be moved around and migrated will enable us to replace and re-work in the future.

Whilst it is obviously inevitable that a code base will age it doesn't have to imply that it will eventually becoming a legacy code base with all the negative connotations that may bring.

The ability for a code base to evolve, grow and be maintained is what will prevent it from having the terrifying legacy label applied to it.

For this to be true, no matter how old the code may be, we must have the confidence to refactor it, and this will come from us having the ability to build, test and deploy it.