Scrum Down

The term scrum was first used in the context of product development to refer to the unified nature in which a team should work, not unlike the way a pack of forwards work together in rugby.

Scrum as an agile framework is now common place in software development however in many instances implementing a perfect incarnation of scrum has become the goal, this leads us away from the reasons scrum was conceived, namely to implement an agile approach to delivery.

This rigid adoption of scrum has lead many to perceive it as something that gets in the way of forward progress and is something that exists for its own sake.

This is the antithesis of what scrum was designed to accomplish so where have we taken a misstep and how do we tackle these misconceptions.

Agile Qualities

In essence scrum is a simple process.

A product owner creates a prioritised list of value, a scrum team decides on a subset of this work they fill they can achieve in a certain time scale, after this time period elapses the team reviews their achievements, looks back on how things could have been done differently, and presents a potentially shippable increment of the product to the product owner.

There are certain key take aways from this process that its important not to lose site of and separates a scrum based approach from its waterfall predecessor.

It promotes flexibility and speed, a scrum team should be nimble and agile such that they can move their focus to different priorities as the product, and the world it exists in, move on. Each new iteration is a chance to tackle to a different problem or react to a new imperative.

It promotes commitment and accountability, the team lays down very clear objectives to be achieved in a specific timescale and hold themselves accountably if they fall short of these goals. This is all done in order to increase output and lay the foundation for more accurate future estimation.

It promotes an iterative and open approach, keeping the product moving forward on a definite cadence with all decisions being made in the open with input from all team members and stake holders.

Scrum Ain't Heavy

The root of many peoples unwillingness to embrace scrum comes from a perceived rigidness in its application.

One source of this frustration can be scrum ceremonies.

Fundamentally scrum only has three ceremonies, a planning session, daily stand-ups and a review/retrospective session.

Because agile promotes communication between team members there will be other occasions where time needs to be allocated to discussion, there is no need for these arrangements to be strictly applied, an organic and informal approach will suffice.

A second point of contention can be what is perceived as a slavish following of the estimation process.

In both cases a dose of reality may be required to understand these things are unavoidable, if we develop a skepticism of the need for communication or we believe that more information about a situation shouldn't trigger a re-evaluation of effort then we are creating the conditions necessary for failure.

If it starts to feel like scrum is getting in the way then its more than likely implementing scrum has become the focus of the team and minds have been diverted from the delivery it is supposed to bring.

Find Your Own Way

Scrum is a framework not an answer and not a cure for all ills.

Scrum provides you with a toolset and some guidance on best practices for getting the best out of your resources using these tools, but this guidance cannot be slavishly followed.

Do not blindly follow how others have decided to implement scrum and always remember the end goal is to adopt an agile mindset. Its possible for scrum to resemble waterfall if this concentration on agile principles isn't observed.

This all means you should find a way that works for your team.

Providing you are following an approximation of the scrum process, and therefore exhibiting the agile qualities it should promote, then you're doing just fine.

It is also highly unlikely that you will be able to properly identify this nirvana immediately, it will be the product of lessons learnt and a willingness to admit the need for improvement via introspection of what is working and what isn't.

If scrum is seen to be something that comes with a rule set and a rigid doctrine then the battle is lost, it holds certain values dear but how you implement them is in your hands.

Dealing with Legacy

Many conversations about the complexity of implementing a new feature or the post mortem of a defect with eventually involve the phrase "the problem is this is legacy code".

Since on this basis it seems inevitable that all code is eventually considered legacy what can we do to ensure that this doesn't cripple our ability to move our product forward?

Well to do this we probably need to start with defining what we mean by legacy code?

Defining Legacy

Whilst the age of a code base is likely to be a contributing factor in whether or not it is considered legacy it is by no means the definitive measure, there are many well established and long serving code bases that are still being actively developed.

Instead we should defined legacy by the effect it has on us, legacy code instills a lack of confidence in those who have to work with it.

This lack of confidence generally comes from either an inability to refactor, an inability to test or potentially the most frightening of all an inability to build the code.

The inability to refactor may come from simply a lack of understanding of how the code works or potentially from the "ball of string effect" that causes refactoring and rewriting to become synonymous, the inability to test usually has similar roots.

The inability to build is somewhat more serious as it immediately rules out any remediation with regard to the other problems the code base may have, its roots generally lie in an overly custom build process that may be heavy reliant on technology that has disappeared from the mainstream.

Without getting into a discussion about how the root causes of these problems could be countered, can we form an escape plan to get ourselves out of this situation?

What Over How

Two of the problems that create a legacy code base can be somewhat addressed by shifting the perspective of the analysis, instead of agonising over "how" this code works start thinking about "what" it should be doing.

Concentrating on "what" code should be doing is all that is required to construct tests for the code, after all "what" is all we have to go on if we are following a TDD approach.

It maybe that we have to accept that these tests are not strictly unit tests and that they aren't following FIRST principles but before embarking on any other change we need to have a crutch to restore confidence that this code is still doing what we need it to do.

Concentrating on "what" is also a decent starting place for refactoring, this view point can act as a microscope to identify what is wrong with the current API, architecture or implementation.

That still doesn't mean that the required refactoring will be easy but it will help shape it, which is a necessary starting point before any technical debt can be dealt with.

Strangling the Problem

Martin Fowler identified an approach to refactoring a legacy system that he termed a Strangler Application.

Using this approach rather than simply starting again and trying to create a new system we gradually build the new system around the edges of the old.

The new code gradually grows taking on more and more of the required functionality until it strangles the legacy code.

To achieve this he identifies two strategies, Event Interception and Asset Capture.

Many systems can be defined by the events that they handle and the data that they capture, when building a strangler system these events and data items are gradually moved to become the responsibility of the strangler system until most of the critical items reside in it.

An important secondary aspect to this approach is that we can make our lives easier in the future by designing a system to be strangled, a modular design where responsibilities and the data model can be moved around and migrated will enable us to replace and re-work in the future.

Whilst it is obviously inevitable that a code base will age it doesn't have to imply that it will eventually becoming a legacy code base with all the negative connotations that may bring.

The ability for a code base to evolve, grow and be maintained is what will prevent it from having the terrifying legacy label applied to it.

For this to be true, no matter how old the code may be, we must have the confidence to refactor it, and this will come from us having the ability to build, test and deploy it.

Code Critique

A constant in the life of a developer is the code review, you have completed your assigned task and now you must ask the rest of your team what they think of your efforts.

There aren't many occupations where on a daily basis you openly invite criticism of your work, there are many aspect of development that are more art than science and so asking others for their opinion of your labours is not always done with relish.

However, the purpose of a code review shouldn't be ritual humiliation or driven by a strong desire to find fault, it has many benefits to an effective software development life cycle.

Automate the Mundane

For a code base to have a level of consistency its important that a common coding style be prevalent, one of the items that a code review should cover is conformance to this style but it shouldn't be the responsibility of the other developers reviewing the code to enforce this.

As well as developers the other reviewers should be automated tools who can be relied upon to ensure the agreed upon code style is being adhered to, it is not optimal to have your experienced and talented developers talking about the placement of brackets or the indentation of a code block.

It is also not practical to expect even the best developer to be able to review a piece of code and identify all or any possible bugs that it may introduce, instead another automated reviewer of your code changes should be your unit test runner.

Catching bugs at the earliest possible opportunity is desirable and catching them before they are merged in is about as early as can be.

A Tool For Education

Code reviews aren't just about gatekeeping they are also an opportunity for education both for the person submitting the code and for the people reviewing it.

They represent a good opportunity to demonstrate new thinking on an old problem, smart thinking for a new feature or a post mortem on the cause of a bug.

They can also be a tool for education on architecture, the history of why code is the way it is or a pre-emptive intervention to prevent a possible new bug.

Agile is all about communication, in its purest form this is face to face but in a contemporary development setting code reviews are as good a place as any.

If this communication channel is being used effectively you should be able to see a change in the topics of conversation as the understanding of the code base and the principles behind it grows within the team.

Observation Over Criticism

It can be easy to fall into a critical mindset when reviewing code, to assume that your role here is to find fault and criticise.

The nature of coding means there is nearly always more than one way to solve a problem, some alternative solutions will be flawed but many will simply come down to a question of style or artistic preference.

Don't approach a code review with an aim of criticism, you should have enough faith in every member of your team as to not assume that a code review is required to stop bad code making it into the code base.

Instead see the code review as an opportunity for observation, could some efficiency be gained in this area? Is there the potential for a bug here? Haven't we done something similar to this before?

The opportunity for a code review to be an educational tool should mean that the aim is not just to review the code currently being presented but all to influence the code a developer will submit in the future.

Developers are human like everyone else they will react better to honey than vinegar, don't create an atmosphere where code reviews are a tool for them to beaten with but an opportunity to learn, an opportunity to influence and an opportunity to shape future code reviews.

We Are What We Code

In 1967 Melvin Conway introduced an idea that would eventually become known as Conway's Law.

"organisations which design systems ... are constrained to produce designs which are copies of the communication structures of these organisations"

At the heart of this statement is the realisation that producing software is a collaborative exercise between many areas of an organisation, and as such the effectiveness of that collaboration will be evident in the software that is produced.

To fix your software you need to fix your organisation.

Trash Talk

In order to not produce monolithic software any reasonably complex system is usually broken down into any number of sub-systems.

Once this happens then the nature and structure of how these systems interface with each other must be defined.

In large systems produced by large organisations it will often be the case that different teams will be reasonable for different sub-systems and as such engineers across those different teams must collaborate to define these sub-system interactions.

Ineffective communication between these teams will lead to badly defined or misunderstood requirements that will become evident once these systems go live in the amount of failure recorded or in the amount of inefficiency in achieving outcomes.

To avoid this limit the barriers placed between engineers, let the techies talk to each other whenever the need arises in whatever informal manner they find most effective.

Idle conversations between individuals will very often prevent untold pain in production and also acknowledge the fluidity of implementation meaning not everything can be figured out in advance in a formal setting.

Hierarchy of Needs

All large organisations have a necessary hierarchy to avoid chaos, functions that need to be performed and work that needs to be accomplished is divided among different teams and individuals.

However very often this structure can be manipulated to best suit the needs or wants of individuals, this in turn shapes internal communication and can lead to software being produced that is structured in such a way as to suit the needs of the implementors and not necessarily the users.

Teams should be structured so that they are aligned to the needs and wants of users, in this way what is best for the users will more naturally align with what makes things easiest for those working on the product.

If certain teams are frequent collaborators but have barriers around their interaction because of the way an organisation is structured they will find ways to work more independently, this may not be in the best interests of the user who wants the sub-systems each team is responsible for to interact seamlessly.

Process over Problem Solving

Processes within a large organisation are necessary for many reasons, to enforce consistency, to ensure goals and outcomes are meet and simply to allow for smooth operation.

But a critical mass can be reached where teams and workloads exist solely to service process for its own sake.

The end goal of any process in relation to the purpose of the organisation, and the users it serves, should always be clear and concise.

We don't deploy software for its own sake, we do it because software is a tool to solve problems, all structures and processes within an organisation should be aligned to that goal.

When this isn't the case a stagnation can exist that prevents software delivery, and hence problem solving.

If your organisation isn't structured around slick software delivery it won't deliver software and it won't solve problems.

Its often easy to be critical when software is deemed to not be fit for purpose, its also very easy to see this as purely a technical issue, why aren't our engineers producing better software?

In fact the question to ask is why isn't our organisation producing better software?

Engineers are only one cog in the machine that brought that software to market, much communication will have taken place to lead to that point and ineffective software can nearly always be seen to have its roots in ineffective communication.

Mythical Man Month

In 1975 Fred Brooks wrote what has become a seminal book in software engineering, The Mythical Man Month.

In this book he presents several ideas and observations related to how people develop and engineer software, many of these ideas are still talked about and debated today while we continue to try and learn the lessons that Brook's experiences detail.

I will not attempt to cover all the subjects dealt with in the book but I will present what I feel are some of the reasons why this book is held in such high regard.

Brooks's Law

Central to the book is the presentation of Brooks's Law:

"Adding manpower to a late software project makes it later."

The point here is that attempting to reduce software development planning to the simple mathematics of the amount of work one engineer is capable, of multiplied by the number of engineers, while potentially appealing is ultimately false accounting.

The main reason for this is that it discounts the need for the engineers to act as a team rather than purely as individuals.

Writing software of any significance is a complex business that doesn't lend itself to being sub-divided into well formed tasks that can be worked on in isolation.

Instead engineers will be required to communicate with each other to ensure the many moving parts being developed will be able to be coalesced into a working machine.

For this effective communication to take place a level of knowledge about this end goal and the blueprint for getting there must be inherent in the team, introducing new team members weakens this collective understanding and is a distraction to the team while they bring these new members up to speed.

Irreducible Errors

Brooks makes the observation that in any reasonable complex system there is a tendency towards an irreducible number of errors.

"99 little bugs in the code, 99 little bugs, take one down, patch it around, 127 little bugs in the code."

That is to say that the outcome of fixing one bug maybe to cause bugs in other errors, or that solutions to fix complex bugs in themselves may contain bugs.

This implies that there is a need to manage bugs in a code base rather than to have an unrealistic expectation to eradicate them entirely.

The most important aspect of this being to manage the impact of bugs and to understand those that exist to inform analysis and to formulate plans to fix the most critical without causing equally undesirable consequences to be accidentally implemented.

This also breeds a healthy mindset to expect the code to fail and therefore to build in the automated testing, diagnostics and analytics to enable this failure to be detected and ultimately managed.

Conceptual Integrity

A system should be about more than simply the sum of the various aspects of its implementation, it should in fact implement a concept of functionality that needs to be delivered to end users.

Once this concept has been established it should be guarded and protected, just because a system maybe capable of doing something doesn't mean it should.

Often functionality maybe able to be put to novel or inventive uses but if this violates or does not comfortably sit with the original concept of the system it could potentially degrade users interaction with it by introducing complication or by simply not being discovered by them.

This ultimately leads to a mindset favouring simplicity with relatively complex systems being built over time with users being guided at each stage so they maintain their understanding of what the system is for and how it can be used.

This simplicity and gradual journey towards more complexity will also help the problems of estimation and errors that we've previously discussed as a simpler system is not only easy to use its easier to work on and maintain.

As much as things change in software development they also stay the same, many of the teachings that formed the bedrock of early forays into the art form are as relevant today as they were when they were first written.

Make sure you learn these tried and tested lessons even if the languages and techniques used at the time may not be inline with your current work stream, the fundamental truth of these observations will still hold true.