A Test Of Strength

Fortunately most developers now realise the importance of unit testing, so if were all writing tests how do we make sure were all writing good ones.

On the face of it it may seem that the addition of any unit tests must be a good thing right? Well actually no, badly written unit tests can be counter productive and do more harm then good. Badly written tests are unlikely to be maintained and the results are often not trusted, you should never hear anyone say "Yeah that one always fails".

So how do we tell good tests from bad?

Triple AAA Standard

A good unit test should consist of three clear sections.

First we Arrange, we create the conditions that represent the situation were trying to test. This may involve setting up mock objects to return certain values or setting certain data on the class under test (CUT).

If this arrangement phase is difficult to achieve it should be treated as a smell that the CUT is too tightly coupled to its dependencies, has a lot of internal state or that its APIs have side-effects.

Next we Act, we exercise the CUT by using the API were trying to test.

This section of the test should be small, a unit test should be focused on testing one and only one aspect of the CUT, if this cannot be done without multiple API calls again this is a smell of internal state and side-effects being at play.

Lastly we Assert, we verify that what we wanted to happen actually happened.

Again this section should be relativity small if our test is limited to testing only aspect of the class.

By following this triple AAA pattern we can

• Maintain The Tests: In each test it can clearly be seen what conditions are being tested, which API of the CUT is being tested and what the expected outcome is, this makes it much easier to fix a broken test if something changes in the API or the CUT's implementation.
• Read The Tests: It should be easy to read tests written using the AAA pattern making it easier to identify missing coverage, this can also help make the tests act as documentation for the CUT. 
• Identify Smells In The Design: As we've already said an inability to be able to follow the AAA pattern is indicative of a smell, for example if you see Assert steps in the Act phase ask yourself why the design of the CUT makes that necessary (incidentally this is a lot less likely to happen if your employing TDD).
• Identify Bad Tests: Tests that shouldn't qualify as unit tests will jump out at you with large Act or Assert sections demonstrating that there doing too much.

Flying FIRST Class

We should never have any fear about running unit tests we should be happy to run them on every commit. For this reason they should be Fast, if the tests take an excessively long time to run we might decide to not run them on every build. If we do that how much bad code or bugs are going to make it into the build before we spot that theirs a problem?

Each test should only ever fail for one reason. Each test should be Isolated, both from external factors relating to the CUT's dependencies but also from each other. The purpose of unit tests is to test the functionality of a single class not a group of classes.

Unit tests should be Repeatable you should get the same results every time they run with no intermittent failures. If this isn't true it means either the CUT cannot always be trusted to do its job, its behaviour is non-deterministic or that the tests do not adequately control the environment in which the tests are being conducted.

Tests should be Self-Verifying, the results of the tests need to be trust-worthy. If all tests are green we should be happy to ship if any test is red we should be prepared to hold back until the problem is investigated and fixed. The instant we don't trust green tests or we ignore red tests were deciding to throw the dice on every build, unit testing should not be seen as a chore they should be how we verify the health of the code base.

Test should be written in a Timley manner they should be written before the CUT so that the tests drive development (TDD) we should be writing code to meet the tests not writing tests to meet the code as this proves very little about whether the code is doing what it is supposed to do now or in the future.

What this adds up to is that unit testing shouldn't be an afterthought in your development it should be front and centre.

You should take as much pride in how you right your tests as how you write your code, what those tests tell you about your code base should be clear and unambiguous and you should trust that message. 

As your code base develops the tests should be run early and often, they should go forth and multiply and no broken test should be left behind. 

The Three Ages Of A Developer

Our growth as developers goes through many stages as we learn more skills and gain experience but an important aspect of this development is how we view the world and our role in it.

We basically always think we know it all, we just change out minds about what it is, and why its important we know all about it.

The Science Bit

As fresh faced graduates we've been through years of hard work to learn the science of computation and the essence of technology. Then we discover the world doesn't welcome us with open arms.

The problem with many technology courses at universities is there not vocational, science has very little involvement in the day to day life of a developer. Should we be teaching the Shannon Sampling Theorem or Source Control, Diffie–Hellman Key Exchange or Debugging, Tree Models or Testing.

Obviously to produce a well rounded graduate education should be a mixture of all of these things but this lack of a real world view point is what produces the shock most graduates feel when they make there first steps into world of work.

Engineering The World

As we take on-board the skills a successful developer needs we make the transition to being a software engineer.

We understand the value of SOLID design principles, we know the importance of testing our code and we value the code quality in the systems we deliver into the world.

The next bump in the road comes when we realise business's and users don't actually care about any of this stuff.

Users obviously want products to be well engineered because they want them to work but they don't have or want to have any appreciation for what's going on under the hood. You want your car to be well engineered but you don't want to have discussion with the mechanic about the inner workings of the engine.

Business's are all about the development of product not the development of code. Their view point is similar to that of the user, they want what they want, the details are your problem.

This lack of appreciation for engineering is sometimes a difficult pill to swallow, the important thing to realise is that both business's and users do care they just don't know they do.

Badly engineered code will impact both at some point either via things breaking or via the production line grinding to a halt because of a fragile unmaintainable code base.

The Realist

The differing viewpoints of engineers and business's often leads to mistrust, engineers doubt the business's competency in knowing what they want and business's don't understand developers reluctance to build what needs building.

This mistrust is a shame because everyone ultimately wants the same thing.

As developers we must take on-board defining the right product is difficult and as such there will be many changes in requirements along the way. Its because of this that good software should deal with change and product should be delivered too users early and often.

Business's must realise that developers want to feel proud of the code they work on and as a result can be protective about anything that they think will affect this quality, they also don't want to store up trouble for themselves further down the line by making bad choices.

Ultimately as developers we must be realistic about the purpose of what it is we do, our job is to use the high degree of skill we have to help the business to produce products that users want to use. We are in the business of product development and we use engineering to get their.

The Tip Of The Iceberg

Whenever your writing software you should be thinking about the user and don't forget the user can be another developer. Every class you write is eventually going be used by someone else, either another developer at your company or someone externally if your writing an SDK or library.

With this in mind you should be designing the class to best suit the needs of those users. One of the things to keep in mind to achieve this should be to limit what those users need to understand, this might be limiting the numbers of API calls they need to make or keeping the concept count to a minimum.

I look at this as reducing the surface area of code the user is exposed to, let them only have to deal with the tip of the iceberg. So what are the benefits of keeping this surface area small to the user.

• The bigger the surface area of code I'm exposed to the greater the chance that any change you make is going to affect me.
• Every concept I have to learn to use your code is a barrier to entry. This means it might take me longer to figure out how to use your code and increases the likelihood I'll make a mistake or misuse your API.
• The smaller the footprint of your code the easier it is for me to mock it out when I come to testing my code, your helping me maintain the quality of my code. 

So if we agree keeping the surface area small is a good thing how do we achieve it.

I've Seen This One Before

As you develop your code try and solve similar problems in the same way, if you use the observer pattern to monitor one aspect of your system then use it in all these situations.

When you use multiple techniques to achieve the same goal its forcing the user to adapt, this takes time and pushes the inconsistency into their code.

Consistency gives the user confidence they understand how there supposed to use your code and their speed in using it will increase.

Looks Great....What Is It?

As much as possible try and use standard solutions to common problems. As we become experienced developers we develop a keen eye for well known design patterns, when you use these patterns your presenting the user with a fast lane to being able to integrate with your code.

When you get creative in these situations your creating extra and unfamiliar concepts for the user to learn.

A user of your code should never be asking the questions "How does this code work?", "How have they implemented this?". Deriving answers to these questions that shouldn't exist is wasting time before your code and mine is integrated.

It Didn't Look Like That Before

Once you have defined how your code works don't make a change unless it clearly benefits the user, public APIs are not for changing.

A user will soon get fearful and angry if you pull the rug out from under them with every release, force them to learn new concepts, integrate with new APIs or fix broken integrations with existing APIs.

When a user integrates with your code they should only see enough of the iceberg to enable them to take advantage of what your offering. They shouldn't have to understand the whole machine just learn how to drive it, every time you make a change to this aspect think about the brain of your user and try not to exercise it too much.

Abstract Thinking

Abstraction is one of the so called four pillars of object-oriented programming, at its heart its about hiding the details of implementation and concentrating on the functionality offered to the user.

"Abstractions should NOT depend on details. Details should depend on abstractions."

As someone creating an abstraction I should construct it to present the functionality in the most convenient and logical way from the point of view of the user, then figure out the details of how I achieve that interface.

As the user of the abstraction I don't want to be exposed to the detail of the implementation because it was easier for you to code that way.

So if thats how we approach abstractions, why should we want to use them?

Sorry, I've Had To Make A Change

Fundamentally good software design is about dealing with change. Change is a fact of life for a variety of different reasons, software that can't adapt will become fragile and untrustworthy.

One of the things most likely to change are details,You want to change the details of a server API? You want to change the schema of a database? You want to use a different 3rd party library to handle analytics?

This is where abstractions will save the day. By depending on an abstraction only concerned with functionality you are completely isolated from changes in the details of implementation, and at some point these detail will change.

Let Me Worry About That

Good use of abstraction will help bring encapsulation, if I don't know the details of an implementation I can't interfere with them.

By not exposing details you gain protection from someone relying on something you didn't intend them to use, from misunderstanding the purpose or consequences of a method or by just generally not understanding your thinking.

Abstractions allow details to be your domain, you handle that and I'll use the functionality.

Test The Code, Not Your Patience

By not having any dependency on detail it becomes trivial to provide mock implementations when testing your code.

Many people reading this may be saying "Ah well my testing framework can handle this even when I use concrete classes".

My answer would be I'm sure it can, I'm also sure that your tests take longer to write, are more fragile, and that eventually you'll hit something you can't test.

Higher Level Thinking

When we put the effort in to define what abstractions our system is going to need we are forced to think about the bigger picture of what it is we are trying to build and how we want it to be used.

All implementation is detail, its only importance should be that it works and is well engineered. Getting bogged down in the details and only thinking at a low level, especially when first designing the system, will ultimately lead to a system without a clear vision or goal.

In my view there are very few reasons to not put all functionality behind abstractions.

• Its very difficult to predict change, why would you not want to be protected against it?

• Why would you not want your code to be testable?

• Why would you not want to protect a class from outside interference?

• Why would you not want to think about the big picture?

When designing good software abstract thinking isn't weird it should be the norm.

KISS Me Quick

There's quite often a difference between what people are impressed by when your trying to sell them on a product, concept or idea and what they actually want come delivery time.

Bells and whistles will draw gasps initially but when they actually come to use your code they just want it to work, is being brilliant 80% of the time better than being fit for purpose 100% of the time?

Its unfortunately quite common place for a simple idea to be turned into a complicated implementation, what takes real skill is to engineer beautiful simplicity.

Worse Is Better

In 1989 Richard P. Gabriel described a principle that was summarised as "Worse is Better", this principle pitched the idea that quality does not necessarily grow with functionality.

"In software making (and perhaps in other arenas as well) it is better to start with a minimal creation and grow it as needed." 

Using this model, four principles are applied to a design in descending order of importance.

First and foremost a design should be Simple both to implement and to use, simplicity should never be sacrificed to meet any other criteria.

Secondly a design should be Correct in all observable aspects, but it is better to be simple than correct.

Next a design should be Consistent but in driving for consistency we should not end up with an overly complicated implementation, especially when the inconsistent parts of the design would be on the periphery of the system.

Lastly a design should be Complete, but no other quality of the design should be sacrificed in the pursuit of completeness.

You Aren't Going To Need It

At the the heart of the application of these design principles is the concept of YAGNI.

Too often a simple design is contorted into a confusing mess in an effort to cover every possible eventuality. While ideally a design would be full-proof we should prefer a simple well understood design that will be fit for purpose in all but the most extreme situations.

It will be much easier to think through how such a solution can be adapted to meet changing future requirements than to produce a rigid unmaintainable but complete design up front.

Complexity Costs

Unnecessary complication ultimately costs,

• It costs the developers time to implement.
• It costs the testers time to test.
• It costs the customers time to understand the system and report the bugs that will undoubtedly exist.
• It costs more of the developers and testers time to fix the bugs and introduce new functionality.

Paying this cost to implement a feature or cover an edge case that isn't going to be needed makes little sense.

Following these principles isn't an excuse to not think things through, complexity should be measured relative to the problem thats trying to be solved.

A system designed to put a satellite into space will by most peoples standards be complicated but the same rules should apply, someone who understands the problem should smile at the simplicity of the solution.

Customers can be very unforgiving, they are unlikely to be impressed by the brain powered required to produce a monolithic all encompassing design if they can't figure out how to use it or it continually fails under the weight of its own complexity. What they will ultimately be satisfied with is something that does what it says on the tin, day in, day out.    

Can You Smell That?

As developers over time we learn a broad range of skills, we learn to become technicians and actually produce the code, we learn to become architects and structure the solution and we also develop a sense of smell.

This sense of smell is the invaluable intangible quality of an experienced developer, the ability to know when something just doesn't look right.

Trust Your Instincts

Quite often you won't be able to articulate exactly what is making you feel uncomfortable about a particular piece of code or design but trust yourself, you know what good and bad code looks like.

Its important to bottom out what you think the smell is, its much better to take an hour now then to ignore what your experience is telling you and store up trouble further down the line.

Using Some Air Freshener

Talking it over with someone else in your team will force you to try and put into words what you think the problem is.

Try to work out if your issue with the code is related to the design or the implementation, is it related to performance or maintainability, gradually home in whats bugging you.

Is this API just wrong or do I just not like this switch statement? 

Am I worried about how this scales or do I just think we should split this function up a bit?

A lot of problems in development have been encountered before, some research online might well turn up other peoples thoughts on similar issues and how they went about solving them.

If you think its related to the structure of the code a quick review of standard design patterns might bring the light bulb moment, thats the problem we've got!

I've Smelt This Before

The complete list of all code smells is probably endless but there are certain standard smells that you should get used to the stench of, things like,

• This code would be really difficult to change in the future.
• This code is really difficult to test.
• Its difficult to explain to others how this works.
• This part of the code is always breaking. 
• We seem to copy and paste this a lot.

Its important to distinguish these smells from personal preference, just because you wouldn't have coded that way doesn't mean its wrong, but if your nose starts twitching just take the time to work out where that smell is coming from.

A good sense of smell is what distinguishes good developers from mediocre ones, always trust your instincts there what makes you one of the good ones, don't just settle for something that works take a deep breath and smell the roses.  

Bricks and Cement

There are two properties of code that can give a pretty good indication as to the quality and the adherence to SOLID design principles, coupling and cohesion.

Coupling, at the class level, is an indication of the interdependence of one or more classes. If I make a change in class A how much impact does that have on class B? When I'm coding class B how much do I have to care about the implementation of class A? Coupling between classes should always be loose, the impact of change between classes should be minimal if not zero, classes should never be concerned about the implementation of others. 

Cohesion, within a class, is an indication of how much the functionality belongs together, how closely related are the functions the class presents. Cohesion within a class should be high, the class should be focused on doing what it says on the tin.

These two properties usually have an inverse relationship, classes with loose coupling normally have high cohesion and vice-versa, but how do we make sure we achieve this?

Do One Thing and Only One Thing

Classes that do more than one thing have multiple reasons to change, there generally exposed to more areas of the code base both in terms of how many dependencies they have and how many classes are dependent on them. This web of interdependency leads to tight coupling and fragile code.

Almost by definition a class that does more than one thing has low cohesion. A tell tale sign of this is when a classes functions and member variables can easily be segregated, this is telling us there are multiple classes just waiting to be set free from the monster we have created.

Abstraction NOT Implementation

Coupling doesn't get much tighter than when classes are forced to be aware of and concerned with each others implementation.

Implementation is detail, classes should only be aware of the contract its dependencies have agreed to abide by, making it happen is your problem!

By inverting the control of dependencies and using an interface driven design we create loose coupling between dependents, if we also ensure that interfaces are well focused on supplying a single piece of functionality we will promote high cohesion within the implementation.

We Might Have Gone Too Far

In reality it is possible to take loose coupling and high cohesion too far.

Many systems, especially on mobile, have a variant of the event aggregator pattern, that is where classes can communicate by sending events out into the ether and some mediator will ensure that they make it to the appropriate recipient (for example NSNotification on iOS and Intent on Android).

On the face of it this seems as loose as coupling can get and in many situations is the ideal solution, but.....this shouldn't be used as an excuse not to have to think about the architecture of a system and how the bricks should fit together, we don't want a random collection of classes we want an engineered system. This type of pattern can also act as a smelly "get out of jail free card", I've coded myself into a corner but its ok I can just fire of this event and someone will save me!

A class could have perfect cohesion if it only presented a single, atomic operation, but the likely complexity of such classes and the resulting large number of dependencies that would be required to achieve an overall goal would lead to a complex potentially unmanageable design.

Loose coupling and high cohesion are a happy consequence of following SOLID design principles, we can use these principles to build a wall made of cohesive bricks and loose cement that can be adapted over time as we change the shape of our house, remember software development isn't a game of Jenga.

Continuous Product

As developers were not used to being part of a production line but what if when we merged in our latest and greatest code a shiny new product came rolling off the line.

By applying continuous integration we can ensure a never-ending pipeline of product. 

Continuous Everything

Having our code being built frequently and automatically leads to several benefits, continuous integration first got its name because it was able to ease the nightmare of integration hell when prior to a release developers who had been working in isolation all of a sudden had to make their code play nicely together.

By continually testing, measuring and running our code we can apply a process of continuous quality that is feeding back metrics of the quality and health of our product, this gives us a warm feeling inside but also enables us to jump on issues as they happen rather than having a nasty surprise when customers get their hands on it.

By having our code base be the result of small pieces of effort being frequently added to the whole we can have continuous product where we are always in a position to be able to ship the latest and greatest to the customer. A key trait of agile is to deliver to your customers early and often, you can't get earlier or more often then automatically.

Building the Production Line

A continuous integration strategy should have certain key traits in order to reap the benefits automation can bring.

• Build Everything.
• Test Everything.
• Report Everything.
• Fix Everything.

Every commit into the main branch of a product should be built automatically; this build should be fast and easily configurable. Not every merge is going to go flawlessly but the sooner the problem is known the better.

Every build should be tested, you may have unit tests, functional tests and integration tests, some of which will take longer than others but at the very least every build should come with a stamp of approval that it does what it says in the tin.

The results of every build should be visible to everyone; this doesn't just mean the developers but everyone who has a vested interest in seeing a successful product.

Whenever a build produces any problems, whether they are integration problems, broken tests or quality issues they should be fixed at the earliest possible opportunity. The instant a team becomes used to a build being broken all the benefits of continuous integration are lost, if it is a problem you genuinely don't care about then it shouldn't be part of the build. The health of the build and therefore the health of the product should be the top priority on everyone’s list.

Rather than a working build of your software being a future goal somewhere down the road make it a continual output of the work your doing and keep your customers and stakeholders happy on a continual basis automatically.  

Slicing Up The Cake

As good SOLID developers when we're designing a system we're trying to ensure that the classes within it,

• Depend on abstractions not implementations.
• Are loosely coupled to the classes around them. 
• Have a well defined singular role within the overall design.

A good way of achieving this is to produce a design that uses layers to define how a class fits into the grand plan.

Each layer within the design represents a slice of the functionality required to deliver the overall system, all classes within each layer share a common relationship in the functionality they offer. 

The design has a clear structure, lines of interaction can be easily traced and any piece of the jigsaw can be changed with a predictable and limited impact.

Each layer of the design knows only about the functionality offered by the layer below via the abstractions it presents. 

Passing it Down The Line

Each class has a singular responsibility that involves it knowing how to use its dependencies to deliver the contract defined by the abstractions it presents for the layer above.

The effect of making a change to any implementation of an interface is nil. The implementation of every abstraction can be kept very simple. 

Combining this layered approach with an interface driven design and applying the dependency inversion principle makes these classes very testable. Mocks are passed in, expectations are set and high coverage is achieved.

Up and Down Not Left and Right

Notice how in the diagram the lines showing interactions only go up and down. If a class had lines going left or right to another class within the same layer this would represent a invisible side-effect to the layer above. 

Classes in layer 2 can see every class in layer 3 and should be able to use any of those classes independently without a care for how the layer is implemented.

Invisible interactions between classes will lead to classes above needing to have implied knowledge about the implementation of the abstractions below to account for the side effects, we will have broken the encapsulation that an interface driven design is supposed to bring.

The Surface Area for Change

A second reason to prefer not to have classes on the same layer interacting is in limiting the impact of change.

In the diagram the effect of changing any individual abstraction is only felt by the class that use that interface in the layer above.

If classes within each layer were also interacting the surface area of code that is affected by a change increases. It can then be very easy for a small change to one abstraction to rapidly propagate through that layer, and to layers above, resulting in the entire design being invalidated.

So when your slicing up how your design is going to be implemented make sure your knife is razor sharp and every cut is clean with no rough edges that you could get snagged on in the future. 

Don't Worry, Its Magic

"For my next trick I will code myself out of this corner"

Software development is an engineering discipline and as such large parts of the day to day aspects involve turning the handle and watching code roll off the production line.

While this can be fun in its own right their is obviously a creative element to development which is more rewarding. However these creative juices need to be channeled in the right direction, they need to be used to solve problems that need solving, not problems that shouldn't have existed in the first place.

A Sense of Deja-Vu

The same problems and challenges crop up again and again in development, and because of this as a collective engineers have gradually refined solutions to these problems, we refer to these solutions as design patterns.

I would never judge a developer by whether or not the names of design patterns roll off their tongue, the important thing is to recognise when one of these standard problem is encountered and knowing that the best solution is probably the one you used last time.

Being creative in these situations should be setting off alarm bells that we might have taken a miss step further up the line. There is a fine line between a creative solution to a standard problem and a hack.

Will the next person understand your magic? Will they have to be even more creative when they want to modify it?

Create to Innovate

Your creative energies should be used to innovate, use them to recognise a problem a client needs solving and envision a system or product that would help them. Use them to create a vision of the how to build this awesome product thats going to change your users lives.

Thats not to say that development needs to be boring, a painter doesn't paint because he likes the mechanics of using the brush, he does so because he wants to paint a picture.    

Be creative in what your software does, strive for simplicity in how it does it. Be an artist when deciding on the purpose and design of the software your building, be a technician when making it happen.   

We Do Scrum, But...

The majority of companies now either use scrum or they think they probably should be, but do they know why? Is it because they understand the benefits it can bring or is it because they feel they have to describe themselves as agile because we'll everyone is right?

Iteration with Purpose

Even when applying a waterfall methodology you would attempt to split up the building of a system into chunks, the question to ask is why iterate rather than gradually build?

The goal of a sprint (or a group of 2 or 3 sprints that might form a release) should be to continually and rapidly deliver working software to a client. You don't know your building the right thing until your clients tell you that you are, getting this feedback early and often is they key to validating your product or service.

Sprints aren't about splitting up the work required to build a large system, they are providing a way to start with a small working variant of your product (an MVP) and then continually iterate on that product to deliver more and more value to a client.

The delivery on a weekly basis of working software to the client should be the measure of success.

Can't We All Just Get Along 

Another key aspect to agile is how the work gets done, its all about the team.

A team of talented, well motivated people is best placed to know how problems are best solved, they should be trusted to do so. Let the business define the problems, let the team decide on the solutions.

The team should be a tight-knit bunch who are continually communicating, not in meetings but face to face whenever they need each other to bounce ideas around.

The teams focus should be on delivering quality code that does just enough to give the client the next iteration.

Times they are a Changing

Agile is about taking a dose of reality, the world and the people in it (who we usually refer to as users) are complicated. Its difficult to predict what they want and its always subject to change.

Painstakingly trying to figure them out and spending a long time to build what you think they're dreaming of is a gamble that very often isn't going to pay off and will consume a lot of resource in doing so.

Instead embrace the change, give them what they want by constantly asking them to validate your ideas.

Continually give them something new to play with, don't give them promises, aspirations or dreams give them working software over and over again.

And We Know This How?

"It's all about the analytics, we use big data to drive insights"

You'll hear phrases like this from a multitude of different companies in very different market places, but do they really understand what it is they are saying?

We Need Something for the Website

Far too often analytics is used purely as a sales tool, its perfectly valid to quote figures and statistics about your product or service that show it in a good light to convince people of its merit. But, if you only ever use analytics to validate answers you've already decided upon to questions you've already defined you're missing the opportunity that data can give you to better understand your company, the market you're in and your customers.

Analytics should be taken for what it is no matter what it shows, whether its inconclusive or even negative. Don't assume that because you don't get the answer you want that you must just be doing the analysis wrong.

Who Knows What This All Means

Big data is a very misunderstood concept, its about so much more than just scale, its about,

• Volume
• Variety
• Velocity
• Variability
• Veracity

Big data is a large amount of data, covering a wide range of areas, from a lot of different sources, of varying quality that can change rapidly and is unpredictable.

Its complex because your customers, and all human beings, are a complicated bunch who'll exhibit unpredictable behaviour, only by having an open-mind about how they are interacting with your product or service will you gain true insight into how to improve.

Trying to forecast what big data is going to tell you is a fools errand and misses the point entirely, if you think you understand big data, you don't understand big data.

The analysis of big data should be used to provide answers to questions you didn't know you were supposed to be asking, to show trends you weren't expecting and to identify patterns you don't fully understand.

In the long term its much more beneficial to expect the data to show you what you’re doing wrong then to assume it will confirm you've been right all along.

Be Careful What You’re Analysing

As a demonstration that you have to understand the context and circumstances in which your data was collected and what it is actually showing you, a famous study in America attempted to show if peoples twitter feed could be used to show the state of the economy especially unemployment.

So they searched for certain keywords in tweets, one of which was "jobs", they saw some unusual patterns in the data, unfortunately the data they had was collected at the time of the death of Steve Jobs, patterns don't always mean what you may think they mean.

So approach analytics with a cautionary attitude, have a predisposition to negativity and an enthusiasm for improvement, if you do your best to prove you’re getting it wrong and fail then you’re truly going to be getter it right.