How to Break Agile

When any school of thought, way of thinking or approach to a problem becomes significantly popular and widely adopted then a counter culture starts to grow to oppose it.

Often it is to the benefit of all of us that perceived wisdom is challenged, however I believe most of the criticism of Agile is actually misdirected, in that it actually relates to flawed implementations of its principles rather then making a killer blow against the entire philosophy.

Agile is not a procedural approach to development it is a way of thinking, a mindset to use to maximise the output of a team.

When Agile is seen to not be working it is usually because of a failure to adopt this mindset.

Fix Time and Scope

When trying to manage a software delivery there are two levers that can be pulled to adjust the outcome, we can make changes to the scope of the release or we can change the date on which software will be delivered.

Changes in scope can either increase or decrease the effort required to complete the delivery, and changes in deadline can either increase or decrease the amount of available effort we can spend prior to releasing.

What we have to realise is a relationship exists between those two things, the fact being that they cannot both be fixed at the same time.

To do so either leads to inefficiency that delays the release of working software or tries to arrange a situation where software is delivered without effort being expended.

Deadlines are a fact of life and it would be foolish to imagine a situation where they don't exist, but as soon as a date is fixed then scope is the only lever left to adjust the teams trajectory towards hitting that target, if that isn't an option then the difficult decision of moving a deadline must be faced.

Failure to Iterate

Although not expressly mentioned, an Agile mindset leads to a certain acceptance that software development is an unsolvable problem.

Your product will never be finished and it will never be perfected.

Once we've learnt that lesson then we can gradually come to the realisation that the next best thing is to try and ship frequent incremental updates to ensure the product is the best it can be right now.

The delivery of this incremental change may be imperceptible to users but allows for a direction of travel to be established.

When the apparent big bang of a new feature is discovered by a user this is actually just one more small delivery, providing the cherry on the cake of several previous unseen steps forward. 

A failure to iterate is often the result of a failure to properly prioritise.

A priority is a singular entity that stands alone, to pluralise prioritises is to admit to a lack of vision for the direction the product should move in, it leads a strategy akin to throwing around features and seeing what sticks.

Speed becomes of the essence as we increasingly fear that users will leave us without the next big thing, when actually users are often happy to have something that works that receives regular attention to ensure that remains the case.  

Breakdown of Trust

Many would find it quite shocking to be asked if they trust their development teams.

But actually it is not uncommon to observe behaviours that indicate that trust has broken down.

First and foremost this is seen during the estimation process, too often there is seen to be a right and wrong answer to providing an estimate.

Development teams shouldn't be asked to estimate as a courtesy, it is because they are experts in there field. 

Repeating estimation without any conversation around scope leads to the madness of asking the same question over and over until the calculation yields a different result.

Trust can also be eroded when a development teams warnings about potential bumps in the road are not heeded.

Too often this is seen as engineers obsessing about technical detail, while on occasion this may be true, this comes from people who understand all the moving parts of the code base and therefore can also envision the areas where the machine may start creaking.

There is more to being Agile then simply adopting its ceremonies and ticking the boxes of its various implementations.

Agile is a philosophical viewpoint on how the unenviable task of software delivery can be approached.

It isn't a formula that can be solved to produce a proof for success.

Many teams start off with good intentions and want to do the right thing but they eventually encounter the pitfalls of allowing their minds to drift to more attractive ways of viewing the problem that promise a solution that will never be delivered.

Learn to recognise the signs of these mindsets creeping in to your team and use these opportunities to re-affirm your commitment to keeping an Agile mindset.

This will inevitably lead to giving up some perceived control over a situation, the first step towards a transition to Agile from more traditional methods is to realise this control is an illusion, the world simply isn't like that.           

Always Deploying

Within most development teams we're becoming used to most things being continuous.

The adoption of Continuous Integration (CI) has given us tooling that can ensure the effort of developers can be combined at the earliest opportunity, and an increasing emphasis on shifting left has made it possible for us to ensure quality is a default position.

But ultimately software has to be deployed to be of any value and despite all the tooling available to smooth this transition into production for many teams this final push is still something thats feared.

If your fearful of deploying code this will become a self fulfilling prophecy and things will go wrong, your fear will grow and you will deploy on an even more infrequent basis.

Break this cycle of fear and move to a situation where deploying to production requires little effort and is something that is a natural consequence of your team delivering.

Not Wrong for Long

The likelihood of an issue occurring after a deployment is proportional to the amount of change being deployed.

If you deploy large change infrequently not only do you need to be confident in your testing regime but its likely that if you do face difficulties you'll find it harder to roll back your changes to your last known good state.

If you deploy small change frequently the chances of an issue arising are reduced but also the fast route into production you've developed means rolling back is simply another deployment accomplished quickly and with little fuss.

This ensures that if the worst happens your not wrong for long.

A long drawn out deployment process is an attempt to make sure nothing ever breaks in production. While that's an admiral intention mistakes will happen and ultimately all this achieves is complicating the recovery from these unavoidable mishaps.

Continuous MVP

A misunderstanding of the concept of a Minimum Viable Product (MVP) is probably the biggest barrier to teams becoming truly Agile in their approach.

Infrequent deployments can exacerbate the tendency to try and squeeze more scope into each release because of the limited opportunities to deliver features.

When features can be deployed at will it becomes much easier to move people to a mindset of shipping a true MVP, they know the next release can be whenever some value has been added to the journey rather than the next scheduled window of opportunity. 

Reducing the time between an idea and a deployment into production also increases the opportunity for user feedback to have a real influence on the direction of a product. 

Feedback is received early meaning large amounts of effort isn't wasted on functionality that users don't find valuable, and once again if need be removing the functionality entirely is just another deployment. 

Don't Break It

Regular deployment of code will also encourage people not to break things.

Large gaps between deployments tends to encourage an acceptance towards parts of the code base being broken, inevitably leading to an increasing feeling of panic when release day does eventually bear down upon us.

When code is to be shipped as soon as its finished not only does code have to be maintained in a working state but this continual green light has to demonstrable, this leads to an increasing effort being put into effective automated testing and good documentation of what the code base working actually means.

Here a distinction can be drawn between Delivery and Deployment, even when operating a Continual Deployment strategy it isn't necessarily the case that every build has to be pushed to production, but it certainly should be the case that it could.

Aside from wanting to deploy new features at the earliest opportunity we can also never predict when an emergency, such as security vulnerability, forces into a deployment we weren't planning on.

In this situations it is definitely advantageous to know everything still works.

Continuous could be described as the mantra for modern development practices, all the activities we value should be happening all the time.

As the last step in the chain that connects us to our users this should also apply to deployment.

As with so many things its good advice to confront your fears, if release day currently comes with feelings of dread then give yourself the impetus to confront these fears and find a solution by having more and more release days.

Delivering into production should never be taken lightly, and quality should always be maintained, but its possible to do this while also trying to give users the value the instant its achieved by your development team.         

Engineering SLAs

As a developer you'll often be asked to explain why you have written certain pieces of code the way you have and some of this feedback may be critical in nature.

When your faced with this situation please don't use the justification "but it works".

There is so much more to software engineering then simply writing working code, if working is the only quality of code worth mentioning then I would wager it won't stay working for very long.

In order to qualify as an engineering discipline we should be aiming our sights a little higher and look for our code, and the systems we use to produce it, to have many more qualities then simply working.

Ability to Deploy Change

Software engineering probably exhibits the fastest rate of change of any engineering discipline.

A code base under active development by a large team may see thousands of changes being made to code daily.  For this reason the ability to accept change is an important property for a code base to have.

This may sound strange, surely any piece of code can be changed? Actually the ability for a code base to be changed, and not re-written, is something that can only be achieved via careful thought.

Many of the solid design principles, such as Single Responsibility, Open Closed and Dependency Inversion exist to promote this acceptance of change without having to throw away large portions of code whenever a new feature is requested.

But the ability to change or modify code is only part of the story, change that isn't deployed is largely a wasted effort, a code base must also demonstrate an ability to deployed easily and without fuss.

This is achieved architecturally via the use of patterns like micro-services but it also requires the confidence given by having well structured automated tests that can be relied upon to identify issues and potential defects.     

Ability to Determine Performance

We deploy code with a purpose in mind and our users expect it to perform that purpose with an acceptable level of performance.

Hearsay and rumour and not effective ways to measure performance, cold hearted metrics that can demonstrate what the code is doing and how well it is doing it must be thought about before deployment and continually measured and reported on.

This should create a feedback loop that when coupled with an effective ability to deploy change will accelerate your product to new heights.

Before the operation of software can be effectively measured it must be clear to everyone what each area of the code base does and how it does it. It must also be clear the reason certain events are generated and there importance to the overall success of the code that achieving its desired outcome. 

Ability to Demonstrate Security

If any single quality of code has been elevated in importance in recent years it is that it should be secure.

Any code deployed almost anywhere can expect to be the subject of inspection and attack, whether the intentions of the attackers are mischievous or criminal it is no longer acceptable for security to not be on the minds of developers at all times.

But security doesn't exist on paper or whiteboards, it exists when a system can demonstrate an ability to detect, repel and recover from an attack.

The ultimate manifestation of a code base that doesn't meet this criteria is one that relies on "Security through Obscurity", there are a lot of smart people out there trying to attack the code they find and you may find your secretes don't stay obscure for long once they turn their attention to your code.

Security must be demonstrated by a willingness to submit your code to deliberate attack.

There must also be an acceptance that sometimes security flaws will be uncovered that must be fixed, sometime they will be found by effective monitoring and once again when coupled with effective deployment of change this can ensure windows of opportunity for attackers are as small as possible and closed as soon as they are found.

The minimum that should be expected from any developer is code that fulfils its original purpose and can be said to be working.

Those that rise to the upper echelons of their profession don't stop when this minimum acceptable state is achieved, they instead realise that code must also demonstrate some other important properties.

In this way they elevate their art beyond the mere production of working code to an engineering discipline with all the rigour and attention to detail that is implied. 

Concerning Seperation

The majority of good engineering and architectural practices in software development can be traced back to the principle of separation of concerns.

A concern may be deemed a reason to change, a functional unit or a set of connected pieces of information but whatever the definition it should be possible to demonstrate clear and distinct separation when it comes to the implementation of particular tasks and actions in a code base.

When this isn't the case we instead see the emergence of spaghetti code, instead of well formed modules of code we are left with interwoven and potentially unintelligible software that is very difficult to pick apart.

It should be possible when explaining the construction of a code base to identify a certain layering along the lines of the concerns that will broadly fulfil common objectives.

Business Logic

Software generally exists to represent a particular business domain, these domains will always have particular rules of operation and practices that define what it is the business does for its users.

This logic when implemented in a code base should be distinct from the implementation of integrations, such as REST APIs, that provide input and also to the mechanism that is used to present the output to the user.

The business logic should be unconcerned with how the data it is using to make decisions is being provided and also to what will be done with the outcomes it is producing, it is purely about logic.

Model

Alongside logic a business domain will also likely have a representation of the data it deals with and how it models its world.

These model classes should be only concerned with representing these entities and be entirely dumb.

It should be very rare to see logical programming constructs in these model classes as the place for logic and model manipulation is in the business logic that represents the rules that govern the management of the model.

A simple example might be that a class representing a bank loan shouldn't contain code to calculate repayment values or fines for late payment, it should simply represent the amount owed by the customer.

Services

No piece of software is self contained, it generally relies on the flow of information in and out of its own sphere of operation.

These flows might be API calls to servers, to functionality within the underlying OS or to other pieces of software deployed alongside it.

All of these interactions should be encapsulated within a service layer, the responsibility of which is to understand the mechanism for the interaction and how to either retrieve or send the relevant information.

A service layer interacts with outside entities so you don't have to, abstracting the detail so other parts of the code base can concentrate on what can be achieved with the information being provided.

These services should also be small enough to make possible the opportunity they can be composed to perform more than the sum of the parts. 

Presentation

Most software at some point must open up a window into its world, both showing the user the current state of its domain whilst also accepting input from the user to change that state.

Any logic in this layer should be solely concerned with choices related to the presentation of data, at no time should it be deciding what data should be displayed only how it should be rendered.

Being the only area of code the user can interact with means it is necessary the presentation layer be the first to be informed of a users input, the code in this area should be the minimum to provide a linkage between this input and the business logic that will decide the outcome, not filtered, second guessed or pre-empted.

Most reasonably complicated software will have more distinct layers than have been presented here.

These might cover hardware abstraction, platform abstraction or any other distinct functional units that can be identified.

Whatever they may be it should be possible to draw a ring around them on an architecture diagram rather than a many sided polygon.

A single concern is a single reason to change and a single reason to cause a bug.

The impact this will have on the maintainability and extensibility of a code base is immeasurable and is pretty much the only way to achieve a clean architecture.

The challenges are firstly to clearly identify these concerns and secondly to have the discipline to resist the temptation to mix them.

As with many good software development practices a healthy dose of pedantry and an obsession with order can go a long way to providing good outcomes.