UI Do Declare

The applicability of software development as an engineering discipline is often debated. Certain areas of coding do exhibit the necessary scientific and structured approach but other areas veer more towards an art form.

User Interface (UI) development definitely falls into the second camp. Not just because it is concerned with the production of attractive and pleasing UI for users to interact with, but as any developer will tell you, UI development often consists of more of a hit and miss or trial and error approach then other more precise areas of coding.

But is this inherent to UI development or have we simply not found the correct tools to be to move this area of coding closer to being true engineering? The attempt to try and find an answer to that question is behind recent trends to try and move to a more declarative approach to UI coding, in this post we will look at what under pins this approach and it's possible advantages.

Declarative vs Imperative

Most software platforms have traditionally chosen an imperative programming model to implement UI. This often involves each screen or page that the user sees being defined some using kind of layout construct formed of a mark-up language.

This means the programmer, before the code executes, fully defines the UI that should be rendered including styling and positioning etc, this is then mutated at run time to include the necessary data and except input from the user.

A declarative approach attempts to defer many of these decisions to runtime rather than having all aspects of the UI being defined ahead of time. Although not exclusively part of the declarative approach, moving to more of a runtime model means that UI can be defined using the same coding primitives and technologies. Rather than having a mark-up language specifically for UI it is defined in the same language as the rest of the application.

Separation of Concerns

Many aspects of good software engineering practices are related to the separation of concerns, as relayed by the Single Responsibility Principle (SRP), a piece of code should do one thing and do it well.

What underlies the adoption of a declarative approach for UI is the desire to separate the two concerns of UI presentation. That is, one area of code should be in charge of what needs to be rendered i.e. we need to except text from the user here and a click here, whilst a separate area of code decides on visual style, positioning and layout.

This allows business logic and data context to influence the structure of the UI without it having to become melded with code concerned with colours, fonts or padding. It also allows a different visual presentation to be applied to similar UI constructs whilst reducing duplication.

Once again isolating and separating different areas of code which are concerned with different aspects of its execution has lead to ongoing benefits when it comes to dealing with future change.

Just Another Area of Code

As already stated, defining UI in code is not an intrinsic requirement of a declarative approach but it does open up the possibility.  Aside from the architectural benefits we've already stated this means the code generating the UI can be crafted using the same tools as the logic that drives it.

This opens up the possibility to write effective unit tests that verify the construction of an applications UI in a way that just isn't possible when defining UI using mark-up, the simple nature of these tests also allows them to part of a shift left strategy ensuring potential visual bugs are blocked before they become a problem.

Even when bugs do make it into an application, having the ability to debug code concerned with UI construction using breakpoints or any other standard mechanism is enormously valuable and will reduce the amount of time needed to find and fix defects.

Software engineering has always been a mixture of different paradigms and approaches, some come and go and are at different times fashionable. A declarative approach to UI won't fit all situations and applications but its potential benefits are undeniable. For this reason it should form part of the tool box of any engineer. Gain an understanding of its principles and practices, experiment with its implementation and recognise opportunities for it to be used in your application.                        

RESTful Maturity

Whenever we need to allow different deployments of software to talk to each other the glue that we fall back on is APIs. Many options exist for their form and structure but REST is now so prevalent that it is the only approach many software engineers know or have been exposed to.

The flexibility offered by a RESTful approach means that whether or not an API is RESTful is more of a spectrum than a binary option. To deal with this Leonard Richardson developed the Richardson Maturity Model, this model attempts to categories APIs based on four levels of maturity.

These levels range from an API being RESTful in name only to the full realisation of what a RESTful approach can offer. If you are a developer or a consumer of REST APIs where do you think you sit on the scale?

Level 0

APIs at this level are categorised by using a single URI and a single HTTP Verb (usually POST). These APIs are simply using HTTP as a transport mechanism for interacting with code remotely.

This level is sometimes referred to as The Swamp of POX (Plain Old XML) since SOAP based Remote Procedure Calls (RPC) using XML occupy Level 0.

There is a lack of structure at this level with APIs simply being a collection of operations.

Level 1

The first step along the road to becoming RESTful is to introduce the concept of Resources to model the business domain. This has the effect of splitting functionality across multiple URIs although generally still using a single HTTP Verb (again usually POST).

If we imagine a system of APIs for interacting with a library a Level 0 API would use the same URI for interacting with authors or books, a Level 1 API would have a URI for interacting with authors and a separate URI for interacting with books.

In this way authors and books become resources in the system, when you receive data about an author you can also retrieve that authors books.

Level 2

Now that we have the concept of Resources Level 2 APIs introduce the concept that they can be manipulated. They do this by using HTTP Verbs to from a CRUD interface for interaction:

Create: POST, Retrieve: GET, Update: PUT, Delete: DELETE

Generally these APIs will also start to make use of HTTP Response codes to provide feedback on the manipulation:

201: Created, 404: Not Found 202: Accepted

At this level interactions with the API become expressive, simply by looking at a clients interaction with the API, even if you weren't previously aware of its structure, inferences about the nature of the interaction can start to be made.

Level 3

Level 3 is often categorised by the term Hypertext As The Engine Of Application State (HATEOAS). Up until this point in order to make full use of the API you had to be aware of all the available APIs and the operations that can be performed.

Level 3 APIs speed up this discovery process by allowing resources to describe the operations that can be performed on them by providing information on the APIs that should be invoked.

To return to library example when a book is queried the data returned would also express that a book can be checked out along with the API that should be called to do so, it would also refer to the API to retrieve more books by that author or possible even more books on that subject.

Developing REST APIs is a journey, as a domain and its resources grow and develop the APIs used to interact with them will move along this maturity model. It is by no means the case that because an API isn't at Level 3 that it is somehow bad, it simply means that its use up until now hasn't matured to a point where the opportunity to move up the levels has presented itself.

The main lesson to draw from this model is that for an API to evolve it needs to become descriptive and expressive. As APIs mature consumers can start making assumptions about how to perform operations because the expressiveness of the APIs allows them to learn more about the resources and operations that make up the domain being modelled.

This helps increase adoption and allows consumers to quickly develop more innovative ways to utilise them. There are various guides and pointers that can be used to ensure APIs are RESTful, these guides will help you move up the maturity model towards the nirvana of Level 3. Don't beat yourself up if you aren't there yet, instead relish the journey and understand the nature of the destination.

Effective Test Automation

As a DevOps mentality has taken hold in the industry teams have increasingly focused on test automation. This is mainly because an effective test automation strategy can be a driver for an increased release cadence by reducing the amount of manual effort involved in declaring code ready for production.

On the face of it this may seem like a straight forward endeavour. Write the tests, run the tests and release the software. However like many aspects of software engineering there is a subtly to its proper application and it is definitely the case that when badly implemented it can lend no value, and at worst can actively degrade quality.

Effective test automation cannot be defined or explained in a single blog post but presented below are a few things to look out for when judging the effectiveness of test automation in your code base.

Red Means Stop

The number one mistake that teams make when implementing automated testing is for there to be no consequences when tests fail. There are often various reasons given for this, there is a bug in that test, it's a timing issue, that wouldn't happen when the application runs for real.

In all those situations if the underlying issue can't be addressed then those tests shouldn't be run because they offer little value. The whole purpose of an automated test suite is to act as a traffic light for the code base, if we're all green we are good to go, if anything is red we need to stop and fix it. Having a situation where software is still released despite tests failing will create a culture of ignoring tests, this will either increase the amount of manual testing you need to do to get comfortable with the codebase or you will knowingly ship with defects.

Much more value can be derived from a smaller set of automated tests that are robust and meaningful than trying to create a larger set of tests that are fragile and where the results require significant interpretation.

Fix Early or Release Broken

Once you have a set of reliable tests you are prepared to put faith in the next most important factor is when you run them. The further right in the development timeline the tests are run the more pressure there will be to ship code regardless of the results.

Depending on the timespan of your project it is also likely the cost of fixing defects found further right in the process will be higher, this will likely be in spite of the fact that any fix may be more like a patch than a well engineered solution.

The further left the tests run, by which we mean the nearer to the time the developer first writes the code, the more time is available to find a fix and the cheaper the fix is likely to be. The closer the tests are run to the proposed release date the more it will become a box ticking exercise due to the significant pressure of continuing regardless of the results.

Variable Scope and Focus

The release of any piece of software is often focussed on particular areas of functionality, this naturally means the potential for bugs or issues is higher in these areas. Whilst there are likely to be core areas of functionality that always need to be verified we can maximise the effectiveness of an automated test suite by allowing it to adapt to the current focus of developers attention.

This shift in focus may be automatic based on analysis of developer commits or it may be via configuration or any such mechanism that allows manual changes in emphasis. Knowing that the available automation resources have been focused on the areas of code most likely to have regressed will go a long way to increasing confidence that these new or adapted features are ready to ship.

The building of an automated test suite is always done with the best of intentions but implementations often end up being sidelined and not given particular relevance in the release process. This usually comes from a view point that simply writing the tests is enough, this isn't the case. Tests that don't relay unequivocal information about the state of the code base, or that are run too late for any information to be effectively acted upon represented wasted effort.

To avoid this decide on your areas of nervousness when releasing and try and develop strategies for these concerns to be addressed via automation. Also treat this automation like any other code, expect it to need refactoring and developing as the code base moves on. Treat it like a living breathing area of code that is your ally in making the important decision over when something is ready to ship. 

Next Level Over Optimisation

Over optimisation is a common pitfall for software engineers. It is frequently categorised by a disproportionate amount of effort being spent on striving for performance levels that have yet to be proven necessary.

However a similar mindset, causing similar diminishing returns, can be seen in other areas of software engineering when too much emphasis is placed on one particular aspect of the code. Software engineers are by nature problem solvers and can often demonstrate compulsive tendencies, this combination can sometimes cause an obsession with providing the ultimate solution to a problem.

Which aspects of code, other than performance, can be over optimised? What are the down sides to the emphasis becoming obsessive.

Code Size

Engineers can full into a trap of trying to devise ways to provide the same solution with an ever decreasing amount of code. Code is not a business asset, our goal isn't to strive to increase the amount of it we have, it is to devise efficient solutions to problems using code as a tool. However measures of efficiency should also include maintainability and extensibility.

To this end there is a tipping point where decreasing the amount of code also decreases the readability. At the time this is first manifested this can seem a minor concern, you still understand how the code works, but maintainability and extensibility and long term goals they can't be derived in the moment.

The over emphasis on reducing code size can be driven from an assumption that it will drive performance and a desire to prove engineering skill by being able to do the same with less. A more pragmatic approach would recognise that the performance benefits, if they do actually exist, are likely to have a negligible impact on users. An engineers skill shouldn't necessarily be judged on any one piece of code, we must also factor in whether that code continues to offer value over a prolonged period of time, both in the form it was originally written and by being adapted and extended over time without being re-written.

Re-Use

You will often see an engineers face light up when they realise they can re-use some previously written code in a new situation. Re-usability is a fine trait for code to have, it increases the speed solutions can be delivered and as previously stated we aren't in the code generation business so opportunities to deliver using less of it should be embraced.

But, sometimes this drives code to be declared re-useable prematurely. When this turns out not to be the case it can lead to the code that is attempting to r-use it being unnaturally bent to fit in with the underlying code, or the interface to the re-useable code degenerating over time as it is mangled to suit slightly different use cases. The fear generated by these potential outcomes can also breed a reluctance the make changes to shared code because of the potential knock on effects.

This is not an argument against code re-use, it is something that should always be strived for. But it's important to develop a sense of when code is truly re-usable and when it would create a rigidness in the code base that will ultimately serve to degrade the intended benefits of re-use. One approach to dealing with this is to not be afraid of refactoring code once the re-usability of code has become apparent, conversely when re-use is starting to cause problems don't be afraid to recognise this fact and allow responsibility for the functionality to be handed back to the consumers.

Configurability

Related to an over emphasis on re-use can be a drive to make every aspect of an area of code configurable. In order to increase the flexibility of code, and therefore open up more opportunities for its re-use, we make every aspect of its functionality configurable.

As with most topics we have discussed, configurability is an admirable quality but if its overdone it can have a negative impact on readability and maintainability. Potential users of the code are presented with a sea of possible options that they may struggle to choose from or appreciate the subtleties of. When it comes to changing the code in question this can be hampered by having to maintain support for a large number of use cases and combinations.

This problem can be addressed by not forcing users to deal with the array of options being presented by offering sensible defaults for particular use cases. If an experienced engineer, who has a good understanding of the problem domain, chooses to dig deeper they can. But consumers who simply want an "out of the box" solution need not dig this deep.

Pragmatism and practicality are important qualities for engineers. Certain aspects of code, most notably the SOLID principles, are important enough to evangelise but good code is not defined by any one measure. Code quality is also not always possible to measure in the moment, decisions made at the time of writing can have unintended consequences further down the line.

It won't be possible to avoid this kind of over optimisation, the approach to a piece of code is based on understanding of the problem at the time of writing. As this knowledge grows mistakes will be surfaced as things go in a different direction to initially anticipated. Don't fear these situations, recognise that they exist, learn to recognise the signs, and develop strategies for finding reverse gear.

Goldilocks Abstraction

Abstraction is one of the more powerful weapons a software engineer has in his or her arsenal. Its proper application is fundamental to ensure the code is engineered and not simply written.

A formal definition of abstraction might be:

"The process of removing physical, spatial, or temporal details in order to more closely attend to other details of interest."

A more practical description would simply be the hiding of implementation detail in order to make a system easier to understand, use and maintain. The moment that we decided to not write code as simply one large procedural block we are engaging in abstraction. Modelling our problem domain and the solution we have engineered into distinct, and hopefully well formed, elements.

However whilst abstraction is fundamental to software engineering it does also demonstrate aspects of a diminishing return that eventually tips into being detrimental to the quality of the code. Abstraction is good but it is possible to have too much of a good thing.

Too Little

When a code base has too little abstraction it can bombard the user with detail. This can make it difficult to separate functionality from implementation making it harder to identify and take advantage of re-use opportunities.

An area of code maybe very close to what you require, but because it is fused with a particular implementation the opportunity for it to be re-used is lost. It is also likely to increase the complexity of maintenance, the impact of required changes in implementation spreads due to the amount of the codebase that is directly exposed to this detail. 

Code with too little abstraction will very often be found to be breaking SOLID principles. A lack of adherence to the Single Responsibility Principle and the Interface Segregation Principle will lead to large unwieldily classes. Breaking the Open-Closed principle will as already stated lead to large scale refactoring for even apparently simple changes.

As referenced in our more formal definition of abstraction, having too little of it exposes a lot of detail, so much so that it is harder to concentrate on the details you actually care about.     

Too Much

So given that rather damming assessment of the impact of having too little abstraction we may assume that we should be trying to implement more of it at every opportunity. In actual fact having too much abstraction can lead to very similar problems.

A codebase with too much abstraction can also hide opportunities for re-use simply because it becomes increasingly difficult to identify what a piece of code is designed to do. This is usually because the abstraction has created a greater distance between the code and the problem space it was originally supposed to work in.

High levels of abstraction can also create problems when debugging code. As you try to step through a problem you seem to plunge ever deeper down a call stack whilst never appearing to get any nearer the code that is doing the actual work or the code that is manifesting the bug.

Too much abstraction can actually increase the amount of detail in a codebase that must be understood to work within it. It may hide implementation detail but it creates large amounts of knowledge related specifically to the layers and layers of code that have been implemented. This will often create a high barrier to entry for new members of the team or even when experienced engineers re-visit once forgotten code. 

Just Right

So we have established that there is a middle ground of just enough abstraction to realise the benefits without creating too much and swinging towards the negatives that can bring. As with many aspects of software engineering it is almost impossible to get this right. Any codebase you work in will on occasions veer either side of the desired middle ground.

When adding an abstraction try to identify clearly what detail it is designed to protect the caller from, how will it make the codebase easier to work within the future and how many other layers of abstraction will it be part of. Having multiple layers of abstraction is not necessarily wrong if the purpose of each can be cleanly and quickly explained and understood.

Also learn to recognise the signs of a codebase having too little or too much abstraction. 

Do relatively simple changes in implementation detail seem to lead to large numbers of points? Are you put off from making changes to dependencies or interaction with lower level systems because of the impact of the change? These types of problems are indicates of too little abstraction.

Do you find it difficult to follow the flow of a piece of functionality from end to end? Do you find it difficult to identify where changes needs to be made when assessing a proposed change in functionality? These can be signs of too much abstraction in a codebase.

As software engineers because of the inherent complexity in coding we often aren't aiming for perfection but simply something slightly better than adequate. An important factor in this is understanding the pro's and con's of all the tools in the toolbox as well as recognising the signs when any particular tool is being over-used. Abstraction is no different, it is a fundamental tool that any engineer must master but this also includes knowing when enough is enough.

User Trials and Errors

The lives of a software engineering team would be made a lot easier if they were always feed with exact requirements from the people who will use their software that if delivered would guarantee success.

There are many areas of software development that make it an inexact science, one of these is how to identify and meet users wants and needs. Certain aspects are obvious, no users want software to crash or error, they want software to be secure, reliable and fulfil its main purpose. Beyond this which features users want to see and how they want these feature to work is difficult to predict.

There are certain techniques that we can use to try and smooth this journey towards having happy and content users, but the premise behind adopting these strategies needs to be that we don't have the power to simply predict the right path.

There is no crystal ball, only controlled trial and error.

Analysing Failure

The majority of modern software applications will employ some form of analytics, collecting data about how software is being used for future analysis. Unfortunately in almost equal proportion this task is approached with a certain bias.

Frequently we decide on the data points we will collect with a view, either conscious or subconscious, to prove we are right in our assumptions. We look through a lens focused on how we expect and\or want people to use our software.

This will very often lead to a situation where when the data doesn't prove conclusive we are hampered in further analysis because we aren't collecting the data that would unlock our understanding.

When deciding what data to collect a general rule should be to record everything. Obviously this needs to consider user privacy, anonymity and consent. In this context we are talking about recording how users interacted with the software not their personal details or data.

We also need to set expectations on what our analysis will reveal, analytics will generally raise questions rather than answers. It will point out where we are failing but not the solution. However the value of these observations is that they aren't guesses, they are facts based on real data relating to real users.

Open for Criticism 

Research with users is also common practice with many teams, however this can be equally tainted with the desire to be proven right. We can lean towards wanting to show users our new ideas and gain acceptance for them, however this can be undermined by the natural human reaction to being asked if you want more.

User are often likely to be positive towards the possibility of new features but this doesn't mean the current lack of this feature is their number one gripe with the software.

This kind of research can be much more useful if we are prepared to open ourselves up for a bit of criticism. The insights we can gain from effective analytics will have already highlighted problem areas, engaging with users to ask them exactly why these problems present themselves can yield further insight into what is wrong.

Openly asking people to criticise you is not an easy step to take, and sometime this criticism may not be presented in a useful or constructive manner but if you engage and try and structure the conversation you will learn a lot about the problems that if you can solve will make a big difference to users views on your software.

Trial and Error

A theme that has been running through this post is that you can't necessarily trust what users say or what they do in isolation if it's under controlled circumstances. The only real source of truth is what they do in large numbers when they are using your software in the real world.

This affects not only your analysis of issues but also the effectiveness of your solutions, nothing is certain until you've tried it with real users in real situations. This implies that you have to take a big leap into the unknown whenever you think you have something to improve your users experience but thankfully tools exists to allow a more experimental approach.

By applying A\B testing you can trial your ideas either on a random or targeted proportion of your user base and allow analytics to determine if it has changed user behaviour or has influenced outcomes. This enables you to perform trails on real users in real situations whilst also reducing the impact when you are inevitable on occasion proved wrong.

If we except that perfecting features first time every time is not a realistic ambition, adopting a strategy of trialing data driven experiments in the wild is the next best option.

Much of adopting an agile mindset is in accepting a lack of control over the process of delivering effective software. The techniques it promotes are geared around making the best of the situation given that limitation. Many that want to be agile struggle with letting go of the illusion of control, it isn't always a comfortable situation to be in but that doesn't make any the less true.

Learning to use analytics and techniques like A\B testing will help with the acceptance of this situation and you can even learn to love the excitement of whether this idea will be proven to be the one that makes a big difference, just don't lose heart when it turns out it isn't quite there yet.

Iterate to Accumulate

When assessing your agile maturity it can be easy to fall into the trap of only considering your adherence to the ceremonies and procedures of the particular flavour that you've decided to adopt. Whilst this is obviously important to ensure your working within a framework that provides structure to your teams work, it doesn't provide a measure of how much you've adopted agile values

Describing the values that define agile is a potentially lengthy discussion and I won't attempt to cover them all in a single post, however one value that I think is often overlooked is the need for an iterative approach to all aspects of the delivery lifecycle.

Again the importance of an iterative approach is not born from the process itself but from an understanding of why it represents a better way to solve problems than its waterfall counterpart.

Iterate Understanding

If we define an iteration to be a sequence of requirement gathering followed by execution that is repeated multiple time then we can see that each iteration represents an opportunity to increase our understanding of the problem, possible solutions and the merits of each.

Sometimes an iteration will not make it to end users, we will realise before this that something is wrong with our approach and decide to re-think, or it maybe that we need to go further in what we are trying to deliver. Sometimes it will reach users and based on their feedback or our use of analytics we will realise we can improve. It is also possible that the data derived from users using the software in the wild will indicate a diminished return in iterating further as what we've delivered is achieving its stated aim. 

In all these scenarios we have given ourselves the opportunity to benefit from feedback loops that give us the time and freedom to re-think while the iron is hot. A waterfall approach robs us of that opportunity by increasing the price we pay for mistakes in our understanding, the longer the time period between iterations the less likely you are to be able to quickly act on new information and the more reliant you are on having mastery over all the moves your users could make. If this kind of mastery was possible software development would be a very easy activity and a solved problem.

Iterate Solution

The importance of all the SOLID development principles comes from how they influence the approach to change in a code base. A natural driver to embrace these principles is likely to be fostered within a team that expects to iterate.

An iterative approach will fail if the execution phase always represents large refactoring or is weighed down by technical debt. A reluctance to end up in this situation along with a desire to enable the accumulation of understanding that we've previously discussed will create an opportunity for an architecture to organically grow in the right direction.

A need to ship frequently will also promote the adoption of other methodologies to promote delivery such as increased test automation and effective continuous delivery pipelines. 

Software development teams by their nature are resourceful people who have the skills to be able to shape their world in the way they want to work. Promoting the right values to these teams will nudge them towards developing software in a certain way, an emphasis on being able to effectively and efficiently implement multiple iterations of change will be the mother of invention to bring new ways to shorten the feedback loop.

Iterate Failure

A fundamental lesson in agile development is to learn that you will encounter failure, rather than exhaust energy in trying to avoid the inevitable, failure should be embraced as just another feedback loop.

Failure in this context is not bugs and defects, albeit they are equally a fact of life in software development, failure in this context is misjudging the needs of your users or the effectiveness of your solution in meeting those needs.

In this sense failure isn't something that can be, or indeed should be, hidden in the dark. It should happen in full view worts and all, this is only way that you will get real feedback on what users think of your software, how they use it and what its shortcomings might be. As much as we may think we can head-off this failure by talking to users or asking them what they want and need nothing speaks the truth like putting working software in the hands of large numbers of people to use in their everyday lives.

Techniques to limit the impact of this failure do exist, by using techniques like A/B testing or phased releases we can observe interaction with the software in a targeted way with an ever increasing audience size. 

Only the most trivial problems in software engineering have ever been solved in a one and done manner. Perfection in any feature is probably unobtainable but by paying attention to how your users are interacting with your deliveries you will both identify areas for improvement as well as the time when continued effort is unlikely to bring a proportional increase in user satisfaction.

Agile's promotion of tight and frequent feedback loops provided by continuous releases places this lesson at its heart.  

Pop Goes the Tests

Behaviour Driven Development (BDD) is about so much more than automated tests but the reason these things are so frequently conflated is because an effective implementation of BDD principles presents the opportunity to write tests that proves your systems behaviour.

A frequent mistake when implementing these automated tests is to concentrate too much on the presented User Interface (UI) over the actual behaviour of the software. The purpose of these tests first and foremost should be to validate that things work, that isn't to say that defects in the UI aren't important but they are likely to be less impactful than broken functionality.

Alongside this a frequent criticism you will hear from teams that implemented automated tests is that they are flaky or can't be relied upon, if when tests fail the issue can be with the tests themselves then there becomes little point in running them. The source of this fragility again is often an overemphasis on the UI when implementing the tests.

UI is often an area of frequent change and tests that are too closely tied to a particular implementation of look and feel can become brittle. The Page Object Pattern (POP) is a technique for implementing automated tests that can address this fragility by encapsulating UI details into a single place whilst leaving the rest of the elements that make up the tests to concentrate on function using a business domain language.

Feature Files

Although not expressly part of POP the approach can be undermined when feature files contain too many UI related details. The language used in feature files should reflect the operations users want to complete using concise phrases that both business stakeholders and users would likely use to describe what they are doing.

As an example two different ways of describing a login process might be:

"I'm entering my username into the first text box, I'm entering my password into the second text box, I'm clicking the terms and conditions checkbox, I'm clicking the login button" 

"I'm logging into the site"

The first describes in great detail the interaction with the various UI components on the screen as well as the details of what is happening under the hood, the second is a much more concise explanation of the process the user is trying to complete. The first would need to be changed should the implementation of the login process change, the second would be unlikely to change unless there is a significant change in the flow of the site.

Those that are familiar with the site have an implicit understanding of how the second description is realised and it therefore isn't necessary to go beyond this level of detail.

Page Object Models

Keeping the feature file language simple and concise allows for the code that will underpin the process (usually called step definitions) to be written in such a way as to avoid the need for frequent refactoring when the UI of the experience may change. Obviously since the interactions with the UI need to be automated within the tests this detail does need to be dealt with somewhere, this is where Page Object Models become part of the mix.

Page Object Models provide an encapsulation of the implementation detail of a page and present a business focussed implementation. To return to our previous example the Page Object Model for out login page would define a login method that internally automates the detailed steps listed in the first description, this leaves the test themselves to use the simplified language of the second description.

Should the details of the login process change the the implementation of the Page Object Model would need to be updated but the tests themselves can remain unchanged. This isolation of change promotes stability in the tests allowing them to concentrate on describing and testing function over form.

SOLID Tests

POP and the use of Page Object Models is an example of applying the same engineering discipline to writing tests as to writing the software they are testing. This frequently isn't the case, tests are often the most untidy part of a code base where they are simply churned out rather than thought through and executed with the same care and attention.

Whilst tests may not represent code that is deployed into production they are the mechanism, or at least should be, that gives the confidence to action deployments. Can this confidence be forthcoming if they are sloppily implemented or can't be relied upon to adequately and reliably verify the operation of the system?

Sometimes the reluctance of developers to implement tests is also driven by the fact that the code in this area is difficult to work with and time consuming to change. Putting the implementation of the tests under proper engineering scrutiny can remove this barrier.

If code is worth writing, whether it be implementation of functionality of tests to verify it, then it's worth writing well.

Strangling Tech Debt

Technical Debt will always get you in end, like a circus performer spinning plates you might be able to find ways to keep things intact and maybe even get some more plates spinning, but eventually everything will come crashing down.

You will have reached this point when it is no longer practical or feasible to continue to bolt on extra features or to refactor your way out of trouble. You are left with a code base that whilst it may continue to perform its intended function can no longer evolve in its present form.

This leaves you facing the proposition of a re-write, whilst this may light up the eyes of many developers, this potential new green pasture is not without its downsides. Starting again with a code base also means starting again in terms of credibility in the eyes of the business the system serves. You may see the foundation of sand that the functionality is built upon, but the business sees a system fulfilling its purpose.

Strangler Application

A different approach is that of a Strangle Application, Martin Fowler when defining a Strangler Application draw parallels to the strangler vines of Northern Australia:

"One of the natural wonders of this area are the huge strangler vines. They seed in the upper branches of a fig tree and gradually work their way down the tree until they root in the soil. Over many years they grow into fantastic and beautiful shapes, meanwhile strangling and killing the tree that was their host." 

He uses this as a metaphor for a technique where a new approach to a code base is implemented alongside an existing application and not instead of. The exact mechanism for doing this will depend upon the architecture of the system you are trying to strangle and the technical debt that has stopped it in its tracks.

But imagine as an example instigating a proxy in front of a set of APIs that enables certain calls to gradually be moved over to a new implementation, or an event based system gradually migrating the code that handles certain events over to a new architecture.

This is sometimes referred to as Asset Capture where gradually assets that the system deals with, whether data with in it or interactions from other systems, are migrated to the new way of thinking. This enables new and old code to live alongside each other, giving the new code a chance to prove itself while the old code continues to serve the business in a way they are happy with.

Over time as confidence in the new approach grows and it gains credibility it can assume more responsibility and capture more of the assets of the old system. Ultimately if this proves successful all remnants of the old system will dissolve leaving you with the refactored code base you dreamed of while never having to have faced the blank sheet of paper of a re-write and the risks that entails.

False Dawns

The reduced risk that this approach delivers can, under controlled circumstances, also allow for a degree of experimentation.

If the implementation of the Strangler Application approach allows for responsibility to be passed between new and old in either direction then if a false step is taken with a new implementation approach responsibility can be passed back to the old approach whilst the drawing board is re-visited.

This isn't to suggest that your environments become a playground, but as much as a new idea may look foolproof on paper nothing proves a line of thinking more than it being battle tested in a system fulfilling a real user need.

Any wrong step taken once a re-write has commenced can be extremely costly and much more visible to stakeholders, all of which can damage the prospect of the re-written system gaining traction with the business or enthusiasm for its arrival.

Writing Tomorrows Tech Debt

Whenever we are writing a new piece of code we are sure that this time we are on the right track, this time we have the right design and the right approach. Sometimes this will turn about to be the case, but on a not insignificant number of occasions eventually flaws in our thinking will begin to emerge, exposing cracks in our architecture or implementation approach.

This combined with the benefits we've discussed of a Strangler Application means we need to write our code to at some point in the future be strangled itself, to allow itself to eventually fade away and not to be dragged kicking and screaming.

To a large extent this can achieved by proper adherence to SOLID principles, by properly assigning and segregating responsibilities. It is the loose coupling that this way of thinking promotes that will eventually allow and area of code to be quietly and efficiently replaced with a better idea.

Any time you get the feeling that an area of code would be difficult to refactor out of a system this should serve as warning that you are taking away the opportunity to at some point replace that implementation without having to contemplate a complete re-write.

Whilst starting from scratch on a purely technical level is often quite an attractive option, on a practical level it frequently isn't a realistic prospect. As pragmatic engineers we need to develop strategies that allow us to correct our mistakes without having to take on the risk of asking the business to return to zero lines of working code.

A code base that promotes this approach allows for both the patterns of implementation and the architecture to be evolving and improving as our understanding of the right solution increases and develops.              

The Art of Imperfection

If you've spent any amount of time with a development team you will recognise the obsessive, compulsive and perfectionist behaviour that whilst possibly a stereotype also has a bucket full of truth.

This strive for perfection can be a teams downfall, leading to stagnation caused by a reluctance to commit and release. Clearly this is a difficult juggling act, swing too far the other way and pragmatism gives way to simply bad software.

The ability to apply well reasoned pragmatism to everyday development situations is a core skill that will generally come with experience as idealism is complemented by the experience of shipping, or indeed not shipping, software.

This isn't an exact science and this post is by no means a definitive guide on how to achieve it. You'll have to find your own way to achieve this in your world but hopefully these pointers will help you.

Imperfections in Architecture

If any architect tells you they are 100% happy with their architecture I would question how much they understand about how it has been implemented. We all strive to follow a ratchet mechanism where every change we implement improves our technical capability and moves our architecture forward. However, software engineering is a far too complicated discipline to never take a wrong step.

You will at various points define an architecture that you think will provide a structure for never ending success and be proven wrong when the implementation goes in a direction you didn't anticipate and is less than ideal.

Whilst a good architect will hopefully keep these situations to a minimum they won't be able to eliminate them entirely. Instead they can be judged on how quickly they recognise these situations, how they contain their impact and how they formulate a plan to reverse and try a different direction.

Adherence to SOLID design principles will help you in these situations, they will naturally ensure that loose coupling avoids the transmission of technical debt across a code base. They will also allow for a cookie cutter approach to fixing the issue where the problem area can be cleanly removed and improved upon.

Imperfections in Code

All developers will recognise the situation where you can't let go of the feeling that a piece of code can be improved. Either adding functionality or trying to achieve the same functionality with less code are common goals that lead to a reluctance to declare something as done.

Both of these goals are to be applauded but they must be approached with due to regard to the YAGNI principle (Your Aren't Going to Need It). Coding for a situation that will never arise is a fairly good definition of over-engineering, along with coding for a level of performance that you will never need to obtain.

The major benefit of following SOLID principles is how they allow for effective future refactoring. They allow for simple extensions to functionality along with the ability to change the implementation of existing functionality.

The majority of code must be accepted as a continual work in progress, only the most trivial areas of a code base are ever "finished". It's fine for code to be good enough for now if this is coupled with a clear plan of how it would be changed in the future should requirements change.

Imperfections in Features

We write software to implement features and our willingness to declare a features as ready is equally as problematic as our willingness to declare code as done. What we must realise is that it is the vacuum caused by a lack of user feedback that leads to this gun shy nature.

Obviously the initial inception of an idea requires reasoned thought as to the best way to address a user need, however before long this idea will have been implemented well enough that the overhead of continued refinement will not be matched by addtional user positivity.

Any time we place ourselves in the shoes of our users we are in danger of over thinking based on incorrect assumptions. Any refinement cycle that isn't driven by observations of users using released software has reduced odds of hitting the mark.

Getting ourselves to be comfortable with releasing what might be deemed good enough is an important mindset for a team to attain. Create something functional, prove that users will engage it with and look for where this engagement could be streamlined, increased or otherwise improved.

Perfection in software development is all about the journey and not the destination for the simple fact that perfection will never be achieved. Those that excel are able to optimise in any given moment whilst also managing the areas that need improvement in the longer term.

Whilst shipping code that doesn't work is never going to benefit anybody this aside failing to ship is likely to do you more damage in the long term than shipping something that might not be 100% perfect.