Unit of Work

The number of individual work items required to achieve a software engineering outcome can be staggering. If the execution of these tasks is to result in success they need to be organised into a structure to avoid chaos.

Within agile teams we have developed a structure containing many different types of work items, features, themes, epics, stories, tasks and possibly many others are an every day part of teams lexicon and operation.

Whilst their definition can sometimes seem arbitrary or open to interpretation they shouldn't be seen as a collection of things to do. They must be written and grouped to achieve a purpose and have a defined outcome. Whilst this may seem obvious it can be surprising how often we can lose sight of this.

Defined Value

The purpose of properly segregating what work needs to be completed isn't simply to ensure a team is kept occupied. It must also serve to help planning and act as documentation for what is going to be built, what has been built and why.

This means each item must explain its value to the business. Providing functionality for users is clearly valuable to any business but constraining work items to only being described in these terms is limiting and doesn't acknowledge that there are other reasons to write code that benefits the business.

This needs to be equally true when we group together work items. As an example it can be tempting to treat items such as epics simply as large collections of user stories, but such a loose definition can lead to the collection being superficial. No matter what the scale completing an item of work should bring a self contained benefit, it shouldn't imply a benefit that isn't actually achievable until more work is done.

Do No Harm

If a team is to follow a DevOps mentality, to ship continuously on an automated basis, then each work item must result in shippable code. This doesn't mean it needs to implement an entire feature, it maybe that nothing changes the user can see or interact with whilst still achieving a business benefit of moving towards something bigger.

However it shouldn't result in the code base being unstable or incomplete unless further items are completed. If this is the case then it indicates that the original division of work was wrong or that a different implementation approach was required.

Sometimes a dose of realism is required to accept that the required functionality requires a large amount of effort, dividing the work up into too smaller chunks to try and change that situation won't result in achieving the goal any quicker.

A Plan Not An Idea

Work items shouldn't document ideas they should document plans to achieve a statable and desirable outcome.

Generally if you review an untidy backlog many of the work items that have lingered for sometime will be ideas that haven't made the journey from initial inception to work the team can and should do. There are many ways to document ideas but they must be refined and pass through a filtering process before they should be considered for execution by the team.

This shouldn't be seen as stifling creativity, teams absolutely should be free to spit ball ideas, but when it comes to spending some of the teams valuable effort and expertise on delivering one of them then it needs to be undertaken with a clear plan of what they will do and why it is worthwhile them doing it.

Nothing is hard and fast with agile, whilst we may have taken a decision to push back against large upfront planning that doesn't mean we have adopted chaos. Agile does involve planning but with a dose of realism on what can be controlled and what can't. It isn't possible to do this planning without understanding the work that will get you there. The effort of your team is a currency that you must plan to spend wisely and not to squander on a series of unrelated tasks with no clear purpose.

Drowning Under The Waterfall

Whilst the software engineering fraternity has widely embraced agile methodologies it has also struggled to let go of its waterfall roots. The behaviours that waterfall encourages have not been entirely let go and continue to creep into the ways that teams approach delivering code.

It is understandable why this is a difficult break-up, the benefits that waterfall claims to offer are intoxicating. It claims to offer certainty and the ability to plan an exact strategy for success, but ultimately these are hollow promises that it cannot deliver.

It cannot deliver because the world it is intended to operate in doesn't exist.

Crystal Ball

A vital piece of equipment required to implement an effective waterfall strategy is a crystal ball. It requires stakeholders to have certainty over their requirements with no gaps or uncertainties, it also requires developers to understand all the problems that will arise when trying to implement that vision.

The reason for this is that waterfall creates elongated feedback loops. It requires initial up front design, development and finally test. If at any point stakeholders are found to have missed or misunderstood a requirement, or developers to have not anticipated an implementation problem, the price paid in elapsed time is heightened.

Software development is fair too complicated to implement a strategy based on certainty. This complication exists both in its implementation and in defining the required outputs. Effective development strategies shorten feedback loops and encourage collaboration in an attempt to combat rethinking and lessen its impact.

Excluding the User

The big bang release approach encouraged by waterfall has the unintended consequence of sidelining end users. Stakeholders act as a proxy for the user, as they do in most development methodologies, but the issue caused by waterfall is it takes away the possibility for them to be wrong.

A plan based on perfectly predicating users wants and needs is high risk and likely to fail. A more realistic approach to accept the possibility of being wrong and to structure releases to reduce the size and impact of any possible mis-step.

Every release should be a learning experience whilst equally allowing time and resource to implement those learnings. If each release is a wager on your understanding of your users how much of your development teams effort and hard work will you risk?

Linear World

One of the most alluring aspects of waterfall is its ability to predict outcomes over long periods of time allowing a teams planning horizon to be far into the future. The issue with this is it implies a linear and predictable understanding of all the factors that can influence the outcome of a project.

Very little about human behaviour is linear, this applies equally to people charged with delivering a project as well as the intended users once the project delivers.

The interesting aspect of this flaw is that the illusion of certainty in predications is so attractive that we fail to be to able to let it go despite contradictory and frequent evidence to the contrary. It has almost become a cliché for an IT project to be late and over budget, this is despite the fact that all these projects would have started with a carefully crafted plan that was supposed to mean this time it would be different.

No-one wants to hear the answer "we don't know" or "we're not sure" but if uncertainty exists this answer invites the possibility of a contingency to be planned, "we got it wrong" offers no such hope.

No-body can claim that any methodology has all the answers, but an advantage of agile is its willingness to embrace the things we don't know and structure our approach accordingly. As attractive as it may be to be seduced by a methodology that claims to hold all the cards its important to remember that we develop code in the real world and we are therefore subject to its foibles and unpredictability, learn to live with that and you'll start making better decisions.  

The Art of Software Engineering

Software development is rightly categorised as an engineering discipline, there are many well established principles and practices that deliver more robust and scalable code. But compared to its contemporaries such as structural or mechanical engineering it is still a relative young practice. 

This combined with the fact that coding can be a blank canvas with many possible ways to achieve an outcome mean it can also exhibit traits more akin with an art form.

Non-Scientific Measures

Developers will often describe code as being good or bad, this is very rarely a judgement based on measurement and opinion on the quality of code may not even always be universal. These opinions can drive new proposals for solutions to common scenarios, providing new takes on old problems.

Whilst I'm sure this can happen in other engineering disciplines it feels like software is unique in the way it can revisit old problems that already have solutions. To put this in context, structural engineers probably no longer debate how to build a wall, a correct solution has been found and we've moved onto other areas.

Correctness in software engineering can be more opaque, it is weighted against the opinions and beliefs of the engineer making the assessment. 

This isn't necessarily a negative, a willingness to re-evaluate and even criticise can be a strength if it's used as a driver to ensure that all possible avenues have been explored and no ideas discounted.

Valued Characteristics

One of the things that can drive a change of view of the correctness of code can be a change in the characteristics that we most value. Most languages and frameworks have many options for tooling that can be used to quantify these measures and different philosophies may choose to put more emphasis on one or the other.

This might be testability, reusability, speed or efficiency. Whilst when questioned engineers would realise the benefits of all these things they may choose to sacrifice one for the other because they feel it's a better measure of correctness.

To continue our analogy, the valued characteristics of a wall are unlikely to change. It needs to be a strong and durable, dissenting voices amongst engineers are likely to be few and far between and unlikely to gain traction.

An advantage of this can be that it develops strategies and practices to optimise certain aspects of our code. Even if you decide that this optimisation isn't the sole focus of your engineering effort their existence is still an advantage when you feel that particular aspect of your software could be improved.

No Raw Material

Where software engineering's uniqueness is harder to question is in its lack of a raw material. Most other engineering disciplines take raw materials, whether it be bricks and mortar or steel and plastic, and turn them into the desired structures and assemblies.

The raw material of software engineering is the creativity, ingenuity and effort of the developers that practice it. The advantage of this is the freedom it gives to evolve quickly and achieve impressive things on minimal budgets, the disadvantage comes when mindsets assume closer alignment with more traditional engineering.

Scaling software development is a difficult, and possibly still unsolved problem. If you need to build a wall then clearly having more people laying bricks will speed up the process or enable you to build a bigger wall. Taking the same approach to painting a portrait is likely to lead to an incoherent and unsatisfying outcome.

Realistically software engineering exists somewhere between those two extremes. In the early days of a project adding more engineers can increase output whilst maintaining quality, but this relationship is not linear and will very quickly breakdown if the differences to traditional engineering aren't realised.

It maybe that these aspects of software engineering are related to its relatively young age, after another hundred years of software development maybe we will see less flux. Or it could be the case that software engineering has qualities that make it inherently unique in its approach. Rather than something to be feared we should embrace the freedom our discipline gives us to try new things, evolve our thinking and change the shape of the code we build. 

Code Recycling

If you observe a group of software engineers discussing an area of code for a long enough period you will hear the phrase "reuse". We rightly hold the benefits of code reuse in high esteem, it can reduce development time, provide increased levels of reliability and increase the adoption of desired practices.

But is this the whole story? Are there disadvantages to reuse? What different forms can reuse take?

This is potentially a large area of discussion with almost philosophical overtones but one potential approach is to analyse what is being reused and identify the pro's and con's of the approach.

Reuse of Effort

The simplest form of reuse that is often considered sinful is copy and paste, this is where code is simply duplicated, and potentially modified, but results in the same code existing in multiple places in a code base.

This allows the developer doing the copying to take advantage of the thought and effort expended by another to solve a shared problem. The downside comes from the duplication of code. Code is not an asset to an organisation because it needs to be managed and maintained, duplicating an area of code creates the potential to duplicate bugs and the effort of fixing them.

So code should never be copied?

Well the answer to that question is almost always going to be yes but as with many things it doesn't hurt to apply some pragmatism on occasions. Copying a small number of lines of code when the effort of defining an abstraction and creating a dependency for other areas of code may not always be the wrong thing to do.

The analysis of whether or not this is the wrong thing to do needs to take into the account the number of times the code has been copied. Copying it once when defining an abstraction might be problematic might be able to be justified, copying it any more times than that is indicating that the opportunity for reuse, and its associated benefits, is presenting itself.

Reuse of Code

When we've made the assessment that copying the code would be wrong then we look to reuse the code itself. This can take many forms whether it be creating a library or having shared source code.

This is where the benefits of reuse are realised. Reusing code decreases development time by providing ready made solutions to common problems, reusing the code in multiple places allows it to be battle hardened and honed overtime.

So code reuse is always a good thing?

Well the answer to this is usually going to be yes but it isn't always an absolute. Reusing code creates dependencies and this doesn't come without its own potential headaches. A large web of dependencies can stifle the ability for a code base to be refactored, maintained and grow. It is no surprise that many of the SOLID principles relate to reducing the scope of individual areas of code this will therefore reduce the scope of the dependency on it.

This shouldn't be taken as an argument against reuse but as a word of caution to think it through. Reuse code when the abstraction will hold and the opportunity for reuse is clear and unambiguous.

Reuse of Function

Ultimately we aren't actually trying to reuse code but functionality and we have developed ways to achieve this that can go beyond code reuse.

Software as a Service (SaaS) is common place among cloud providers but the same approach can be taken within your own code base. Divide and conquer has long been an effective strategy for engineering well defined code bases and techniques such as microservices have this at their heart.

A library doesn't have to be the only way to reuse functionality, using REST APIs can also allow functionality to be reused whilst also providing a really clear interface with the potential to evolve over time as requirements change.

This won't always be the correct approach and will come down to the scale of functionality that needs to be reused. Having a microservice to perform operations on strings would rightly be deemed ridiculous, but caching or complex data manipulation maybe more likely candidates.

Identifying and effectively implementing code reuse is a core aspect of software engineering. We have many tools available to us to achieve this and whilst it will nearly always be the right thing to do the potential pitfalls must also be considered and factored into the strategy.