User Trials and Errors

The lives of a software engineering team would be made a lot easier if they were always feed with exact requirements from the people who will use their software that if delivered would guarantee success.

There are many areas of software development that make it an inexact science, one of these is how to identify and meet users wants and needs. Certain aspects are obvious, no users want software to crash or error, they want software to be secure, reliable and fulfil its main purpose. Beyond this which features users want to see and how they want these feature to work is difficult to predict.

There are certain techniques that we can use to try and smooth this journey towards having happy and content users, but the premise behind adopting these strategies needs to be that we don't have the power to simply predict the right path.

There is no crystal ball, only controlled trial and error.

Analysing Failure

The majority of modern software applications will employ some form of analytics, collecting data about how software is being used for future analysis. Unfortunately in almost equal proportion this task is approached with a certain bias.

Frequently we decide on the data points we will collect with a view, either conscious or subconscious, to prove we are right in our assumptions. We look through a lens focused on how we expect and\or want people to use our software.

This will very often lead to a situation where when the data doesn't prove conclusive we are hampered in further analysis because we aren't collecting the data that would unlock our understanding.

When deciding what data to collect a general rule should be to record everything. Obviously this needs to consider user privacy, anonymity and consent. In this context we are talking about recording how users interacted with the software not their personal details or data.

We also need to set expectations on what our analysis will reveal, analytics will generally raise questions rather than answers. It will point out where we are failing but not the solution. However the value of these observations is that they aren't guesses, they are facts based on real data relating to real users.

Open for Criticism 

Research with users is also common practice with many teams, however this can be equally tainted with the desire to be proven right. We can lean towards wanting to show users our new ideas and gain acceptance for them, however this can be undermined by the natural human reaction to being asked if you want more.

User are often likely to be positive towards the possibility of new features but this doesn't mean the current lack of this feature is their number one gripe with the software.

This kind of research can be much more useful if we are prepared to open ourselves up for a bit of criticism. The insights we can gain from effective analytics will have already highlighted problem areas, engaging with users to ask them exactly why these problems present themselves can yield further insight into what is wrong.

Openly asking people to criticise you is not an easy step to take, and sometime this criticism may not be presented in a useful or constructive manner but if you engage and try and structure the conversation you will learn a lot about the problems that if you can solve will make a big difference to users views on your software.

Trial and Error

A theme that has been running through this post is that you can't necessarily trust what users say or what they do in isolation if it's under controlled circumstances. The only real source of truth is what they do in large numbers when they are using your software in the real world.

This affects not only your analysis of issues but also the effectiveness of your solutions, nothing is certain until you've tried it with real users in real situations. This implies that you have to take a big leap into the unknown whenever you think you have something to improve your users experience but thankfully tools exists to allow a more experimental approach.

By applying A\B testing you can trial your ideas either on a random or targeted proportion of your user base and allow analytics to determine if it has changed user behaviour or has influenced outcomes. This enables you to perform trails on real users in real situations whilst also reducing the impact when you are inevitable on occasion proved wrong.

If we except that perfecting features first time every time is not a realistic ambition, adopting a strategy of trialing data driven experiments in the wild is the next best option.

Much of adopting an agile mindset is in accepting a lack of control over the process of delivering effective software. The techniques it promotes are geared around making the best of the situation given that limitation. Many that want to be agile struggle with letting go of the illusion of control, it isn't always a comfortable situation to be in but that doesn't make any the less true.

Learning to use analytics and techniques like A\B testing will help with the acceptance of this situation and you can even learn to love the excitement of whether this idea will be proven to be the one that makes a big difference, just don't lose heart when it turns out it isn't quite there yet.

Iterate to Accumulate

When assessing your agile maturity it can be easy to fall into the trap of only considering your adherence to the ceremonies and procedures of the particular flavour that you've decided to adopt. Whilst this is obviously important to ensure your working within a framework that provides structure to your teams work, it doesn't provide a measure of how much you've adopted agile values

Describing the values that define agile is a potentially lengthy discussion and I won't attempt to cover them all in a single post, however one value that I think is often overlooked is the need for an iterative approach to all aspects of the delivery lifecycle.

Again the importance of an iterative approach is not born from the process itself but from an understanding of why it represents a better way to solve problems than its waterfall counterpart.

Iterate Understanding

If we define an iteration to be a sequence of requirement gathering followed by execution that is repeated multiple time then we can see that each iteration represents an opportunity to increase our understanding of the problem, possible solutions and the merits of each.

Sometimes an iteration will not make it to end users, we will realise before this that something is wrong with our approach and decide to re-think, or it maybe that we need to go further in what we are trying to deliver. Sometimes it will reach users and based on their feedback or our use of analytics we will realise we can improve. It is also possible that the data derived from users using the software in the wild will indicate a diminished return in iterating further as what we've delivered is achieving its stated aim. 

In all these scenarios we have given ourselves the opportunity to benefit from feedback loops that give us the time and freedom to re-think while the iron is hot. A waterfall approach robs us of that opportunity by increasing the price we pay for mistakes in our understanding, the longer the time period between iterations the less likely you are to be able to quickly act on new information and the more reliant you are on having mastery over all the moves your users could make. If this kind of mastery was possible software development would be a very easy activity and a solved problem.

Iterate Solution

The importance of all the SOLID development principles comes from how they influence the approach to change in a code base. A natural driver to embrace these principles is likely to be fostered within a team that expects to iterate.

An iterative approach will fail if the execution phase always represents large refactoring or is weighed down by technical debt. A reluctance to end up in this situation along with a desire to enable the accumulation of understanding that we've previously discussed will create an opportunity for an architecture to organically grow in the right direction.

A need to ship frequently will also promote the adoption of other methodologies to promote delivery such as increased test automation and effective continuous delivery pipelines. 

Software development teams by their nature are resourceful people who have the skills to be able to shape their world in the way they want to work. Promoting the right values to these teams will nudge them towards developing software in a certain way, an emphasis on being able to effectively and efficiently implement multiple iterations of change will be the mother of invention to bring new ways to shorten the feedback loop.

Iterate Failure

A fundamental lesson in agile development is to learn that you will encounter failure, rather than exhaust energy in trying to avoid the inevitable, failure should be embraced as just another feedback loop.

Failure in this context is not bugs and defects, albeit they are equally a fact of life in software development, failure in this context is misjudging the needs of your users or the effectiveness of your solution in meeting those needs.

In this sense failure isn't something that can be, or indeed should be, hidden in the dark. It should happen in full view worts and all, this is only way that you will get real feedback on what users think of your software, how they use it and what its shortcomings might be. As much as we may think we can head-off this failure by talking to users or asking them what they want and need nothing speaks the truth like putting working software in the hands of large numbers of people to use in their everyday lives.

Techniques to limit the impact of this failure do exist, by using techniques like A/B testing or phased releases we can observe interaction with the software in a targeted way with an ever increasing audience size. 

Only the most trivial problems in software engineering have ever been solved in a one and done manner. Perfection in any feature is probably unobtainable but by paying attention to how your users are interacting with your deliveries you will both identify areas for improvement as well as the time when continued effort is unlikely to bring a proportional increase in user satisfaction.

Agile's promotion of tight and frequent feedback loops provided by continuous releases places this lesson at its heart.  

Pop Goes the Tests

Behaviour Driven Development (BDD) is about so much more than automated tests but the reason these things are so frequently conflated is because an effective implementation of BDD principles presents the opportunity to write tests that proves your systems behaviour.

A frequent mistake when implementing these automated tests is to concentrate too much on the presented User Interface (UI) over the actual behaviour of the software. The purpose of these tests first and foremost should be to validate that things work, that isn't to say that defects in the UI aren't important but they are likely to be less impactful than broken functionality.

Alongside this a frequent criticism you will hear from teams that implemented automated tests is that they are flaky or can't be relied upon, if when tests fail the issue can be with the tests themselves then there becomes little point in running them. The source of this fragility again is often an overemphasis on the UI when implementing the tests.

UI is often an area of frequent change and tests that are too closely tied to a particular implementation of look and feel can become brittle. The Page Object Pattern (POP) is a technique for implementing automated tests that can address this fragility by encapsulating UI details into a single place whilst leaving the rest of the elements that make up the tests to concentrate on function using a business domain language.

Feature Files

Although not expressly part of POP the approach can be undermined when feature files contain too many UI related details. The language used in feature files should reflect the operations users want to complete using concise phrases that both business stakeholders and users would likely use to describe what they are doing.

As an example two different ways of describing a login process might be:

"I'm entering my username into the first text box, I'm entering my password into the second text box, I'm clicking the terms and conditions checkbox, I'm clicking the login button" 

"I'm logging into the site"

The first describes in great detail the interaction with the various UI components on the screen as well as the details of what is happening under the hood, the second is a much more concise explanation of the process the user is trying to complete. The first would need to be changed should the implementation of the login process change, the second would be unlikely to change unless there is a significant change in the flow of the site.

Those that are familiar with the site have an implicit understanding of how the second description is realised and it therefore isn't necessary to go beyond this level of detail.

Page Object Models

Keeping the feature file language simple and concise allows for the code that will underpin the process (usually called step definitions) to be written in such a way as to avoid the need for frequent refactoring when the UI of the experience may change. Obviously since the interactions with the UI need to be automated within the tests this detail does need to be dealt with somewhere, this is where Page Object Models become part of the mix.

Page Object Models provide an encapsulation of the implementation detail of a page and present a business focussed implementation. To return to our previous example the Page Object Model for out login page would define a login method that internally automates the detailed steps listed in the first description, this leaves the test themselves to use the simplified language of the second description.

Should the details of the login process change the the implementation of the Page Object Model would need to be updated but the tests themselves can remain unchanged. This isolation of change promotes stability in the tests allowing them to concentrate on describing and testing function over form.

SOLID Tests

POP and the use of Page Object Models is an example of applying the same engineering discipline to writing tests as to writing the software they are testing. This frequently isn't the case, tests are often the most untidy part of a code base where they are simply churned out rather than thought through and executed with the same care and attention.

Whilst tests may not represent code that is deployed into production they are the mechanism, or at least should be, that gives the confidence to action deployments. Can this confidence be forthcoming if they are sloppily implemented or can't be relied upon to adequately and reliably verify the operation of the system?

Sometimes the reluctance of developers to implement tests is also driven by the fact that the code in this area is difficult to work with and time consuming to change. Putting the implementation of the tests under proper engineering scrutiny can remove this barrier.

If code is worth writing, whether it be implementation of functionality of tests to verify it, then it's worth writing well.

Strangling Tech Debt

Technical Debt will always get you in end, like a circus performer spinning plates you might be able to find ways to keep things intact and maybe even get some more plates spinning, but eventually everything will come crashing down.

You will have reached this point when it is no longer practical or feasible to continue to bolt on extra features or to refactor your way out of trouble. You are left with a code base that whilst it may continue to perform its intended function can no longer evolve in its present form.

This leaves you facing the proposition of a re-write, whilst this may light up the eyes of many developers, this potential new green pasture is not without its downsides. Starting again with a code base also means starting again in terms of credibility in the eyes of the business the system serves. You may see the foundation of sand that the functionality is built upon, but the business sees a system fulfilling its purpose.

Strangler Application

A different approach is that of a Strangle Application, Martin Fowler when defining a Strangler Application draw parallels to the strangler vines of Northern Australia:

"One of the natural wonders of this area are the huge strangler vines. They seed in the upper branches of a fig tree and gradually work their way down the tree until they root in the soil. Over many years they grow into fantastic and beautiful shapes, meanwhile strangling and killing the tree that was their host." 

He uses this as a metaphor for a technique where a new approach to a code base is implemented alongside an existing application and not instead of. The exact mechanism for doing this will depend upon the architecture of the system you are trying to strangle and the technical debt that has stopped it in its tracks.

But imagine as an example instigating a proxy in front of a set of APIs that enables certain calls to gradually be moved over to a new implementation, or an event based system gradually migrating the code that handles certain events over to a new architecture.

This is sometimes referred to as Asset Capture where gradually assets that the system deals with, whether data with in it or interactions from other systems, are migrated to the new way of thinking. This enables new and old code to live alongside each other, giving the new code a chance to prove itself while the old code continues to serve the business in a way they are happy with.

Over time as confidence in the new approach grows and it gains credibility it can assume more responsibility and capture more of the assets of the old system. Ultimately if this proves successful all remnants of the old system will dissolve leaving you with the refactored code base you dreamed of while never having to have faced the blank sheet of paper of a re-write and the risks that entails.

False Dawns

The reduced risk that this approach delivers can, under controlled circumstances, also allow for a degree of experimentation.

If the implementation of the Strangler Application approach allows for responsibility to be passed between new and old in either direction then if a false step is taken with a new implementation approach responsibility can be passed back to the old approach whilst the drawing board is re-visited.

This isn't to suggest that your environments become a playground, but as much as a new idea may look foolproof on paper nothing proves a line of thinking more than it being battle tested in a system fulfilling a real user need.

Any wrong step taken once a re-write has commenced can be extremely costly and much more visible to stakeholders, all of which can damage the prospect of the re-written system gaining traction with the business or enthusiasm for its arrival.

Writing Tomorrows Tech Debt

Whenever we are writing a new piece of code we are sure that this time we are on the right track, this time we have the right design and the right approach. Sometimes this will turn about to be the case, but on a not insignificant number of occasions eventually flaws in our thinking will begin to emerge, exposing cracks in our architecture or implementation approach.

This combined with the benefits we've discussed of a Strangler Application means we need to write our code to at some point in the future be strangled itself, to allow itself to eventually fade away and not to be dragged kicking and screaming.

To a large extent this can achieved by proper adherence to SOLID principles, by properly assigning and segregating responsibilities. It is the loose coupling that this way of thinking promotes that will eventually allow and area of code to be quietly and efficiently replaced with a better idea.

Any time you get the feeling that an area of code would be difficult to refactor out of a system this should serve as warning that you are taking away the opportunity to at some point replace that implementation without having to contemplate a complete re-write.

Whilst starting from scratch on a purely technical level is often quite an attractive option, on a practical level it frequently isn't a realistic prospect. As pragmatic engineers we need to develop strategies that allow us to correct our mistakes without having to take on the risk of asking the business to return to zero lines of working code.

A code base that promotes this approach allows for both the patterns of implementation and the architecture to be evolving and improving as our understanding of the right solution increases and develops.