Sunday 30 September 2018

Adolescent Behaviour


The start-up holds an almost mythological place in the mindset of the technology industry, often perceived as a zen like state with all elements in perfect balancing producing perfect software.

This article is not meant to be an attack on the notion of a start-up, many large organisations can benefit from the principles and practices they observe.

Instead it presents some views of behaviours often observed in a start-up culture, not necessarily good or bad but simply a consequence of being new on the block with limited resources and a strong desire to be successful quickly.

Bend the Truth

Start-ups are often in a constant state of seeking funding, whether directly via venture capitalists or investors, or indirectly via trying to attract customers to provide an income stream. This means they are very often pitching, both formally and informally, they are trying to get their message across and advertise their capabilities and ambitions.

This may seem a controversial statement but sometimes they lie, and actually this is ok.

Especially in the very early days if a start-up was honest about their current capabilities they would have very little to pitch so it is natural that this gets embellished to present future goals as already being in the here and now.

The critical factor is that everyone needs to recognise that the truth is being bent.

Those being pitched to need to understand the nature of a start-up in this position and judge them more on whether they think they can get to the destination they are pitching and not whether they believe they have already arrived.

In turn the start-up should not present whats required to its developers as if this was always supposed to be the case and acknowledge hard work and savvy will be required to meet expectations.

It is possible to sell a dream as reality and hit deadlines but all parties must acknowledge the nature of the pitch and agree to work together to make the lie true.

Building Front to Back

Another consequence of the constant need to be pitching is the hunt for the wow factor. No matter the level of engineering genius it is difficult to get people excited about an effective and scalable infrastructure design or testable and adaptable software architecture.

This leads to a natural concentration on the frontend over the backend, to developing an awe inspiring UI\UX that is teasing at the functionality and possible ingenuity underneath.

In time if the venture is to be a success an equally effective backend will be joined with this snazzy and flashy front end but initially there will be growing pains as the first iteration creaks under the weight of the frontend ambitions.

Again the important factor here is an acknowledgement of reality, such that we don't over estimate what is achievable in this initial phase of the organisations development, along with the fact that effort will need to be expended on building the capability that was hinted at.

Make Noise

The goal of many start-ups is not to scale their original idea into a Goliath, starting from zero and scaling out to infinity is an extremely difficult undertaking. Thats why often many are looking for a shortcut that often involves becoming part of a larger organisation.

A tactic for achieving this is to make noise about the possibilities your technology points at, to wet the appetite of other organisations and tempt them into taking a chance that this could be the next big thing.

This tends to lead to effort being put into the core technological aspects of a proposition and not necessarily on scaling it out. Scale is put off to another day when the resources available to the organisation may have grown significantly and quickly, both financial and in terms of experience of taking things to the next level.

As with the previous points this approach isn't a problem providing all involved are in on it and honest about the objective of the initial short and medium term goals.

Not all the points made in this article are relevant or accurate for all start-ups. So called unicorns who start life or soon achieve vast resources are often not constrained by the issues mentioned here.

But this is a very different situation to be in compared to a small band of people trying to build success from humble beginnings, this presents a unique challenge without a guaranteed blue print for success. This leads to certain unique behaviours that while at first may seem questionable is understandable given the circumstances and can lead to success despite the initial seemingly overwhelming odds.

Sunday 16 September 2018

Advancing the Art


Software Engineers have an intimate relationship with programming languages. Vigorous Arguments and spirited discussions will ensue whenever the advantages of one over the other is debated.

Sometimes these arguments while enjoyable revolve around technical nuance. Whilst different paradigms exist, functional, procedural or object oriented being examples, languages within each are often exhibit similar features and functionality.

Most languages continue to evolve, introducing new ideas to try and stay relevant to developers and the industry. Sometimes these new features have a fundamental impact on the approaches that are taken to writing code. In this article I'll present what I feel to be some such features, they aren't necessarily recent but there impact cannot be overestimated.

Managed Runtimes

The differences between managed an unmanaged code lies less in the constructs of the language itself and more in the target they are compiled to.

Unmanaged code compiles to the instruction set of a physical machine, whereas managed code compiles to be run on an idealised virtual machine. Many examples of this exist but two of the most common are the Java Virtual Machine (JVM) and the .NET Common Language Runtime (CLR).

The original intention of this approach was to make code more portable, the so called "Compile once run anywhere" methodology, but it has also brought many additional benefits.

These benefits have been related to both performance and security but chief among them is the advent of garbage collection. Because of this a whole generation of developers now don't  know the pain of managing memory, trying to match-up allocation and deallocations, pointer arithmetic and heap exhaustion. Whilst it is still clearly possible to generate memory leaks many tools now exist to help with this.

The benefits of a managed runtime are continuing to be felt, emerging technologies such as Web Assembly are attempting to bring more languages choices to web development, the implications of these approaches and the impact they will have on software are probably still yet to be realised.

Asynchronous Programming

Phil Karlton is often credited with the quote: 

"There are only two hard things in Computer Science: cache invalidation and naming things"

Many have suggested adding to that list, one that would get my vote would be concurrency. As code evolves beyond the trivial there almost inevitably comes a point where we need to be able to do multiple things at once, equally inevitable are the problems we get ourselves into when we reach this point.

There was a time when developers were directly exposed to thread scheduling and expected to manage this complex and error prone area. Thankfully modern languages have provided abstractions to protect us from ourselves and allow concurrency to be approached in a safe manner.

These abstractions are usually hiding a large amount of complexity and they do not mean that concurrency is always completely safe and bug free, but as with managed memory they do allow developers to not expose themselves directly to this complexity and thus give them a fighting chance of getting things right.

Lambda Expressions

In the past all functionality had to be bound to an identifier, this made it difficult for functionality to be passed between entities. Its true that languages like C have function pointers but even in this scenario the functionality itself is not unbound and can be a difficult concept to use effectively.

The emergence of lambda expressions has enabled variables relating to functionality to be defined that are truly first class citizens, this can lead to very expressive code that provides extremely readable solutions to what were once difficult problems.

A good example of such a technique is the .NET Language Integrated Query (LINQ) feature. Using this feature operations on large datasets that would have traditionally required potentially large blocks of complicated code can now be very simply expressed.

The ability to pass functionality around as easily as we do data opens up new techniques and approaches that previously wouldn't have been impossible or at least would have made for inelegant and buggy code.

Reflection

It used to be that code was mostly geared around operating on various forms of data structures, with the invention of reflection it was possible for code to become meta, code could be written to operate on other pieces of code.

This can be a double edged sword, on occasion reflection can be used to get developers out of corners that they are responsible for painting themselves into. Reflection is a powerful technique that is sometimes used as a nuclear option when more efficient solutions could have been found.

Despite this the addition of reflection to the toolbox was a big step forward for the art of software engineering and opened up whole new ways of thinking about problems and there solutions.

Many people reading this will have other language features that they feel should be on this list. My views on this are likely to be influenced by my own experience in development but this speaks to the diversity that exists within software engineering.

We should consider ourselves lucky to work in an industry where the raw materials available to us are constantly evolving and growing. These advances allow previous scars to heal, whether these be related to memory leaks, deadlocking or pointer confusion, the pain of dealing with these problems inspires us to solve them once and for all so that future generations of engineers don't have to experience the same frustrations.                                    

Sunday 9 September 2018

Rears on Seats


An adage that could be applied to many software projects would be something along the lines of from small beginnings large inefficient teams grow.

In the early days of a project a well focused, dedicated and small team can appear to achieve miracles in terms of the scale of what can be built and the time it takes to build it. When faced with such tremendous success it is a common and natural reaction to assume that if we grow this well performing team then we will be able to achieve even more in even less time.

Unfortunately this thinking is flawed but despite this its a lesson the software industry has failed to learn almost since its inception.

Software as a Commodity

Much of this flawed thinking comes from viewing software as a commodity, more engineers means more software and more software means more functionality? The problem with this approach is it views software as a raw material with intrinsic value, this in turn equates software development with a production line.

Engineers aren't simply producing chunks of software to package and ship to production, they are in fact trying to work as a team to craft and refine a single piece of software.

To continue the production line analogy, software engineers aren't all building individual cars, they are a race team trying to perfect a single race car. When viewed like this it should be clear that more craftsmen won't help complete the job any quicker or to any greater degree of quality.

Software has no intrinsic value, we aren't employing engineers to produce more of it, we are employing them to produce just enough for us to extract value, and to try and increase the amount of value we can squeeze out of any giving quantity of code.

Scaling Capability Not Code

As an organisation engineering software its important to understand within the context of scaling what is it your are trying to scale?

If this is simply the amount of code you can produce then increasing the number of engineers in your organisation will achieve this, if however you want to scale your capability to deliver reliable and robust functionality then the solution maybe more subtle.

Software engineering resource is more akin to a sponge than a brick. It can be counter productive to try and line them all up to build a wall, instead you need to concentrate on squeezing more out of the resource you already have.

This doesn't mean working them harder, it means having faith in their abilities and asking them how they think they could be more efficient or more productive. At a certain point the answer will be that more engineers are required, reaching this point prematurely will ultimately have a much bigger negative impact.

Your engineers are experts in the development of software both in general and with regard to the software you currently have, they will also be experts on how your current processes can be improved and refined.

People Bring Overhead

Software engineers do not exist or work in isolation, software is developed by teams. As both the number of people in a team and the number of teams themselves grow this comes with an increase in certain overheads.

Effective communication between engineers will decline, more formal processes will grow and dynamism will take a back seat.

Of course this isn't inevitable, many organisations have large and successful teams. But effort is required to think about how to structure these teams to ensure that barriers don't grow between them.

Ineffective communication between teams is quite possibly the number one reason why large numbers of engineers can fail to be more than or even equal to the sum of their parts.

There are no hard and fast rules as to how to accomplish this but trying to resist the temptation to throw resource at a problem will at least to delay these headaches until the point where there is more certainty that a team needs to grow.

As with many things an important step when trying to scale software development is acceptance that you'll probably get it wrong. Recognising the signs that will highlight this will ensure that it isn't your wrongness that scales. The next step is in embracing the fact that imposing a scaling strategy is likely to be ineffective, instead, placing it in the hands of those that are being asked to scale has much more potential to produce the right answer.