Twelve Factor App - Build, Release and Run

The concept of the Twelve Factor app was developed by engineers at Heroku to describe the core principles and qualities that they believe are key to the adoption of the Software as a Service (SaaS) methodology.

First published in 2011 the Heroku platform is unashamedly opinionated in the enforcement of these principles, the relevance of them to effective software development has only intensified as the adoption of cloud computing has increased the number of us deploying and managing server side software.

The fourth principle relates to the strict separation of the build, release and run phases of application development:

"The delivery pipeline should strictly consist of build, release, run."

Development to Deployment

A codebase is transformed from source code to a working application in three stages.

The Build stage converts source code into an executable binary. The resultant binary is a combination of source code produced by the development team, 3rd party dependencies and the underlying framework being used to produce the application.

The Release stage takes the binary and combines this with configuration suitable for the environment that it will be deployed into. 

Finally the Run stage starts the application in the environment, bringing it to life and making the functionality it provides available to the rest of the estate.

The exact nature of all of these stages will vary greatly depending on the technologies being used during development but the essence of the three distinct stages remains. Build your code, combine it with configuration and deploy, then finally run your code in the environment.

An app following the twelve factor principles strives to maintain this strict separation.

Isolation of Change

The first important aspect of maintaining this separation is to isolate where functional changes can be made. As an example functional changes to the application should not be made as part of the run phase. Any such changes cannot be pushed upstream to the build phase and therefore create inconsistency during development and testing.

It maybe that configuration changes made during the release phase can also affect functionality but these changes can be easily applied during development in a consistent and controlled manner.

By properly isolating change we increase the repeatability of the entire process to produce consistent results, not only in a production environment but for local and shared development environments as well.

Always Releasing

Release and run and phases should be as simple and quick as possible. Not only is this a desirable quality for any piece of engineering but in a modern cloud based architecture deployment is a frequently occurring operation.

As an environment scales out with more copies of your application being required a reliable and fast deployment mechanism is essential in being able to respond to demand.

A slick and fast deployment process also reduces the amount of time it takes from developer commit to running in production. The compound effect of these marginal gains enables consistent gains and improvements in your applications performance and effectiveness.

Finally an effective release process also enables strong rollback strategies when a change into production needs to be reversed. The more complicated the release process the more any rollback is a leap into the unknown and the less likely that the environment can be returned to a working state quickly.

The need to keep these three phases separate may on the face of it seem obvious but inefficiencies and sub-optimal changes can easily cause the line between them to be blurred over time. Recognising the impact this has and being on the look out for anything that goes against this goal is key to maintaining a healthy development environment. As with most aspects of software engineering this is more of a goal than an absolute but being driven by the correct principles will never normally steer you wrong.    

Twelve Factor App - Backing Services

The concept of the Twelve Factor app was developed by engineers at Heroku to describe the core principles and qualities that they believe are key to the adoption of the Software as a Service (SaaS) methodology.

First published in 2011 the Heroku platform is unashamedly opinionated in the enforcement of these principles, the relevance of them to effective software development has only intensified as the adoption of cloud computing has increased the number of us deploying and managing server side software.

The third principle relates to the management of backing services:

"All backing services are treated as attached resources and attached and detached by the execution environment."

Backing Services

Applications will routinely need to consume resources that are not part of the code being deployed. These might be databases, queuing system or communication services such as the sending of email or push messages.

The management and the deployment of these services will likely be a mixture of those being supplied by the same team that are deploying the application and those that our sourced from third parties.

These services are often also drivers of change, either because of a change in supplier or because the team wants to move to a new technology. Generally these services also need to be highly available with outages not being tolerable.

Attached Resources

If an app is written to follow twelve factor principles then it will treat all backing services as attached resources accessible via a URL defined by the configuration within the environment.

The goal of this approach is to the limit the impact of needing to change the source of the functionality. If we have written custom code to interact with a resource or we are using a proprietary SDK then any change in the technology being used will mean code changes and a new deployment of the application is required.

By accessing the resource via a URL, provided by the environment, then in theory we can change the location and technology of these services without the need for any code changes.

Minimal Changes

In practice this will often be difficult to achieve, if for example we change to a different database technology this might require at least tweaks to our code if the feature sets have some variation. 

However by making efforts to push the functionality provided by the services to the edge of our application we have done everything we can to avoid unnecessary change.

It is often a marker for good software architecture for there to be identifiable seams in the code that enable these kinds of changes to be achieved with minimal overall change in the structure of the code. I think this a more realistic aim than hoping to never need to make code changes when a service changes.

An application is hardly ever able to contain all the functionality that's required to achieve the overall aim, it will always be the case that the code will need to interact with external services. Ensuring the nature of these services is not hard baked into the code needs to be a top priority not just to allow for future changes but also to ensure a clean overall architecture.

Twelve Factor App - Configuration

The concept of the Twelve Factor app was developed by engineers at Heroku to describe the core principles and qualities that they believe are key to the adoption of the Software as a Service (SaaS) methodology.

First published in 2011 the Heroku platform is unashamedly opinionated in the enforcement of these principles, the relevance of them to effective software development has only intensified as the adoption of cloud computing has increased the number of us deploying and managing server side software.

The third principle relates to the management of configuration within an environment:

"Configuration that varies between deployments should be stored in the environment."

Separate Configuration From Code

All applications will have configuration to control its operation. This will range from database connection strings to credentials for accessing a backing service and smaller items that might influence the look and feel of the UI.

Sometimes it can be tempting to simply define these values in the source code as constants. Whilst this maybe the easiest approach it comes with several disadvantages, some configuration items are sensitive in nature but are exposed to anyone that has access to the code base, plus, when these items need to vary with the environment being deployed into then we now have to deploy different source code in order to change the configuration. 

Some of these issues can be addressed by separating configuration into separate files within the repository. While this may mean that we no longer need to modify source to change the configuration of an environment we do still have to maintain a different deployment process per environment based on files in the repository, and secrets are still available to anyone with access.

Using Environment Variables

The next iteration of separating configuration from code is to make configuration part of the environment the code is being deployed into, completely removing it from the files associated with the deployment.

This can be achieved by using environment variables,  the code once deployed reaching out to the environment to get its required configuration values when they are needed.  

This approach creates a very clean separation between the code and the configuration and isolates the code from any changes between environments. We deploy exactly the same code into each environment from a developer debugging the application locally to our final deployment into our production environment.

Also, because we can separate the process of setting the environment variables from the deployment of the code we can have tighter controls on who can see and set secret configuration values.

Moving Away From Bundling

The use of environment variables has an addtional benefit in that it gives us the chance to move away from the bundling or batching of configuration per environment

While this approach works in most situations it can present a scaling problem if you need many environment or you need to be able to create new environment quickly. With the traditional file based configuration approach an appropriately named file is required per environment.

A more scalable approach is to group configuration values based on the role they perform in the environment, these more granularly sets of values can be combined in the environment to produce an overall configuration.

All developers will have encountered problems and bugs caused by a configuration mishap. These can be unavoidable due to the often complex nature of a reasonably large application. Even by moving configuration into the environment you are unlikely to completely remove the possibility for a mistake to creep in. However, simplifying the approach and completely removing it from the applications source code is certainly a step in the right direction. 

Twelve Factor App - Dependencies

The concept of the Twelve Factor app was developed by engineers at Heroku to describe the core principles and qualities that they believe are key to the adoption of the Software as a Service (SaaS) methodology.

First published in 2011 the Heroku platform is unashamedly opinionated in the enforcement of these principles, the relevance of them to effective software development has only intensified as the adoption of cloud computing has increased the number of us deploying and managing server side software.

The second principle relates to the declaration and management of dependencies:

"All dependencies should be declared, with no implicit reliance on system tools or libraries." 

Code Dependencies

Virtually all software has dependencies, this will range from your particular language and technology platform of choice to the libraries you consume for functionality that is not practical or not expedient for you to write yourselves.

For this reason most programming environments provide some sort of dependency management tooling. .NET as NuGet, Ruby as Gems and Swift as CocoaPods to name a few. Some of these tools provide the option to install packages locally for use within the codebase or more widely as system packages.

The declaration and installation of dependencies should always be explicit and clearly identified, there should never be an implicit reliance on the system to provide a dependency that could cause a runtime issue if for whatever reason it is found to be missing.

Aside from the clear declaration of any dependency, its use within the codebase should be as equally well signposted and clear for developers to see. This can be achieved by following normal good coding practices such as ensuring class dependencies are declared in constructors and using dependency injection to avoid a class creating its own dependencies internally.

System Dependencies

It can be challenging to completely avoid having a reliance on the underlying system on which your software will run. As with the code dependencies we've previously discussed it is important that these dependencies are clearly stated and managed. It should not be the case that the existence of a system dependency is assumed. A common scenario for this issue to make itself apparent is if the software in question invokes the shell to run commands, for example running Curl commands or manipulating the file system.

These kinds of dependencies should either be fulfilled by using the appropriate areas of your technology stack, by vendoring the necessary packages into your application or by using technologies such as Docker to ensure that the necessary dependencies are installed on the system as part of the deployment of your software.

The more your software has a dependency on the system it is running on the less portable it becomes and the more challenges you will have if you need to change or adapt your deployment strategy.

Self Explanatory Application

The aim of this principle is to ensure your software is self explanatory. By this we mean that a developer can gain all the knowledge they need to run and work on your application simply by looking at its repository.

An application that is self explanatory is also agile in nature. This agility manifests itself both in the effectiveness of the teams working on the software and in the options that become available for hosting and deployment.

Dependencies are virtually impossible to avoid, but all developers will have war stories of a vexing issue they've had to deal with that has boiled down to an unclear or complicated interaction with a dependency. 

No strategy is likely to avoid these kinds of issues permanently but recognising that dependencies are a potential source of pain and doing all we can to mitigate them is a sensible approach to take.

Twelve Factor App - Source Code

The concept of the Twelve Factor app was developed by engineers at Heroku to describe the core principles and qualities that they believe are key to the adoption of the Software as a Service (SaaS) methodology.

First published in 2011 the Heroku platform is unashamedly opinionated in the enforcement of these principles, the relevance of them to effective software development has only intensified as the adoption of cloud computing has increased the number of us deploying and managing server side software.

The first principle relates to the management of the applications source code:

"There should be exactly one codebase for a deployed service with the codebase being used for many deployments"

Source Code Management

Very few if any teams would now debate the need for source control. Even if you are coding as a lone range the ability to properly organise and manage updates to source code is fundamental to engineering software as opposed to simply coding for fun.

A twelve factor app is contained in a single repository and deployed multiple times.

Whilst there is now very little, if any, debate on the need for source control there are still practices that should be followed to maximise the benefits. These include but aren't limited to:

• An effective branching strategy to properly separate day to day development from release activity.
• Using source control features such as git's commit squashing to ensure a clean and readable history of key code base changes.
• The ability to revert and re-deploy changes when their introduction cause unintended and undesirable consequences.

Self Contained Repository

The repository for your application should contain everything that is required to build and run your application. If your application is actually multiple applications masquerading as a single entity then each distinct application should itself be written as a twelve factor app. 

The first of the twelve factor principles actually states that applications sharing code, at a source code level, is undesirable and instead should be based on libraries using a dependency management solution. This is in contrast to the monolithic repository approach that has gained popularity over recent years.

My personal feeling is that the essence of the principle can still be achieved whilst also benefiting from a monolithic repository. It is still the case that cloning a single repo gives you everything you need to build, run and deploy an application. Being able to view the source of all your internally written dependencies provides a great deal of insight into how your code base works, and still allows a single code base to be deployed multiple times.

Many Deployments

There are many reasons why you may need to deploy your application multiple times. It could be that the application provides functionality needed by multiple aspects of your system, or in a cloud computing world your application may be deployed multiple times as your application scales up or out.

For this process to be efficient all deployments should be using the same code base and deployment mechanism. That is to say that the code base shouldn't require modification based on the environment or context it is being deployed into.

If this isn't the case it makes the predicability of the impact of changes harder to gauge and will lead to instability if the differences between deployments is not properly understood by all members of the team.

The first of the twelve factor principles may now seem so fundamental to good software development as to be an obvious mandate. It's fundamental nature means it provides the required under-pinning to build a solid foundation for the principles to come. It is also never a bad thing to be reminded of fundamentals that may seem like common sense but can easily be forgotten.

Source code makes up the building blocks of any application and so it isn't surprising that an effective strategy for the development of good software would start with the principles that should govern the management of this essential commodity.    

As a Service Principles

Software as a Service (Saas) is a delivery model where consumers pay to access functionality provided by a system when they need it without having to manage, purchase or be responsible for the software providing the functionality.

The large scale adoption of SaaS as a delivery model by providers has been fuelled by the adoption of cloud hosting. The emergence of the cloud made it possible for providers to easily stand-up and scale compute resource in order to provide functionality to large numbers of consumers. This in turn meant consumers no longer had to own their own infrastructure and install software in order to take advantage of the service.

As the use of SaaS has grown the importance of the underlying architecture being suitable for cloud hosting has become ever more important, this has lead to the concept of the Twelve Factor App. This manifesto defines twelve factors that should be adhered to in order to build software with a scalable architecture. I'm not going to go through all twelve factors in this post, instead I'm going to concentrate on the principles that underpin the definition of these factors and which they are designed to promote.

Declarative Infrastructure

In the past knowledge about the architecture and make-up of a systems infrastructure was either held inside engineers heads or within cumbersome documentation that had a strong propensity to become outdated or inaccurate. This meant whenever changes were required or disaster struck the number of people who could effectively deal with the situation was limited, it also makes reversing changes more challenging and provides a barrier to entry for new engineers joining the team.

To deal with this situation we can use declarative formats to define our underlying infrastructure. Sometimes also referred to as Infrastructure as Code this approach uses technologies like Terraform, Chef or Puppet to drive the generation and maintenance of infrastructure in a machine and human readable format that shares many of the properties of code.

This approach allows these files to be managed under source control providing more robustness around the history of changes in the infrastructure and the ability to rollback. Since these files are driving the shape of the infrastructure they also cannot become outdated or out of sync with reality. The repeatability of applying these files also means scaling or recreating your infrastructure, in both development and production environments, becomes an error free process.

Finally new engineers joining the team, providing they are familiar with the technology being used, can easily and quickly get to grips with the nature of the infrastructure and become effective.

Clean Contract

One of the benefits of the cloud is portability, the ability too quickly move between hosting environments, operating systems or hardware configurations. The ability to do this can be easily undermined if your software does not have a clean and well defined contract between itself and it's dependencies.

This can be achieved by proper adherence to dependency declaration and dependency isolation. Software should never rely on a dependency implicitly being part of the environment it's operating in. All dependencies are explicitly declared via some form of manifest that forms part of the applications source code. Many technologies exist for achieving this depending upon the stack you are using to develop your software. Gemfiles, NuGet, Chocolatey and many others all provide a way for software to declare that it is reliant on certain other packages being available. This not only reduces the risk of unfilled dependencies once software is deployed but also makes it easier for developers to simply clone the code and build when getting started in the team.

This applies equally to the availability of system tools that maybe dependent on the underlying operating system, every effort should be made to isolate these dependencies and ensure that the functionality is provided to the application in an explicit and declarative way.

Minimal Divergence

Many issues and problems that present themselves in production will be meet with incredulity by developers since everything worked fine on their machine. These situations are born from the divergence of development and production environments. Many of the principles and techniques we've already discussed work towards reducing this possibility and ensuring as far as possible all environment are the same.

This is achieved by the proper application of continuous integration and deployment. This has the effect of reducing the amount of time between code being developed and being available in the environment and ensures that the process of code being developed and deployed aren't separate responsibilities.

An important aspect to achieving this is also to ensure that technology stacks don't vary between environments. The same database, web server and operating systems are in use everywhere that the code runs. Clearly the resource available in these environments is likely to be different but the fundamental fabric of the technology is kept in line.

The adoption of a cloud based SaaS delivery model is about more than simply where your code is hosted. Rather it is about divorcing your code from having any meaningful relationship with it's hosting environment, the freedom this gives allows for agility, efficiency and productivity.

As with most things whether or not you are embracing a SaaS model is not a binary operation that is on or off. It's about placing yourself on a scale but trying to adhere to the principles described here will help you reach the tipping point where the benefits of the approach will start to be realised. 

Responsibility Segregation

A consistent property of bad code is a lack of segregation between responsibilities. Relatively large classes will implement multiple facets of functionality and therefore be responsible for more than one aspect of a system.

This will lead to code that is difficult to follow, difficult to maintain and difficult to extend. Those large classes will frequently be modified because they are responsible for many things, if some of these changes are sub-optimal then technical debt gradually accumulates and grows. To finish the vicious circle this can compound the original problem leading to more technical debt and the downward spiral continues.

Command Query Responsibility Segregation (CQRS) is a design pattern focused on addressing this situation by defining clear responsibility boundaries and encouraging proponents to ensure these boundaries aren't breached.

Commands and Queries

Within the CQRS pattern functionality is either a command or a query.

A query is a piece of functionality that given a context will interrogate a data source to return the requested data to the caller. Critically a query should be idempotent and not change the state of the underlying data in any way.

A command is more task driven, it is a piece of functionality that given a context will change the state of an underlying data source or system. Because of its inherent side effects it should not be used to return data to the user as this is the role of a query, instead just returning the result of the downstream operation. This requirement around what a command should return can in practice be difficult to achieve, more often than not some data is required to come back from the command but the important aspect is that callers are aware that commands perform operations on data and therefore have side effects.

Although not explicitly part of the pattern an effective CQRS implementation will also not chain queries or command together. The layers of abstraction this builds can make the code difficult to follow and understand, this can lead to unintended consequences when a caller doesn't realise the chain of events that will unravel. Instead queries and commands should be composed by callers making individual and separate calls to each element in turn, potentially passing data between them and building an aggregated response to return upstream.

Database Origins

Although in the last section the pattern is presented in abstract terms, and CQRS can be applied too many different areas, the origins of the approach comes from the application of CRUD when dealing with databases.

Within this world there can be many advantages to treating reads and writes differently. Firstly there can be advantages to using different models depending on whether data is being queried or modified, also the load presented by reads and writes is often not symmetrical so being able to separate the workloads can bring performance and efficiency advantages.

Having strong abstractions over the top of data access also enables more flexibility in the approach to underlying storage with users being protected from the nuances this may involve via the interface presented by commands and queries.

Advantages

First and foremost the advantage of CQRS is the re-use that can be achieved by the promotion of separating concerns. When code does one thing and does it well the opportunity for re-use is increased. Quite often when classes are bigger and do more the functionality they offer will always be almost what you need but not quite. This either leads to a new class being created with a slightly modified interface, leading to duplication, or the existing class being tweaked leading to its integrity being further degraded.

Effectively segregated code is also likely to be easier to test since the interface to the code will be simpler and it is likely to have fewer dependencies.

Finally the code base as a whole will be understandable with a clearer structure. New members of your team will quickly be able to asses what the code base is capable of by looking at the queries and commands that can be executed.

No one pattern can be a solution for all problems but certain qualities of well constructed code should be promoted above all others. Segregation of responsibility is one of those qualities, it is almost the very definition of good architecture to promote this quality and ensure its adoption and adherence.

As a code bases grows and evolves you will likely have to introduce additional concepts alongside that of commands and queries, providing these new elements have a strong identity and clearly defined role within your system then this will enable your code to grow whilst still maintaining the well defined structure that is a recipe for success.

Striding To Be Secure

Whenever software is deployed it is a virtual certainty that at some point it will come under some form of attack. This might be via a bot evaluating your infrastructure for the possibility of exploiting known vulnerabilities, or a concerted effort from hackers to make your code expose data and functionality it shouldn't.

Resources like the OWASP Top 10 can help you recognise common security mistakes but each piece of software, and the use cases it implements, will present varying and sometimes specific security flaws. This means it can be a valuable exercise to take a step back and try and analyse your software from the point of view of an attacker.

STRIDE is a mnemonic that can help with this kind of threat analysis by identifying the six categories of attack that hackers may try and perpetrate against your code.

Spoofing

An authenticated system will rely on some mechanism for a user to identify themselves. Spoofing is when an attack is successfully able to identify themselves as another user. This doesn't necessarily mean breaking passwords but on attacking the mechanism your system users to prove on subsequent requests that authentication has taken place.

As a rather trivial example if your system relied on users including an HTTP header indicating their user ID as a means of authentication this could easily be spoofed by an attacker by simply including the ID of the user they are trying to spoof.

Spoofing can actually occur before users even get to your code via attacks such as DNS or TCP/IP spoofing where attacks imitate your site into to lure unsuspecting users into entering their information.

These attacks will generally be countered by careful analysis of authentication systems to ensure that identity cannot be falsified.

Tampering

Tampering occurs when an attack is successfully able to modify data in transit or at rest for a malicious purpose.

The various forms of injection attack represent the classic examples of tampering. This may be SQL injection, cross site scripting or any attack that allows an attacker to inject their own code into the application.

Tampering attacks can be addressed by taking a healthy distrust in all input from the outside world and sanitising it before it gets anyway near forming part of the execution path.

Repudiation

Repudiation is the act of being able to deny that an act or operation took place. This will generally occur if your system does not have sufficient logging to be able to track all user operations, or by allowing attacks to change or destroy logs in order to cover their tracks.

The defence against repudiation is the robust implementation of audit logging. This should cover all user interactions but also the behaviour of your infrastructure and any other data source that can be used to forensically analyse whats was happening in your system at any given point of time.

Information Disclosure

Information disclosure is perhaps the worst nightmare of any business if its systems come under attack, it occurs any time an attacker is able to view data that they shouldn't be allowed to see.

This can be caused by improper application of authorisation, insecure transport mechanism, a lack of encryption, or a lack of segregation between elements of a system allowing hackers to jump from a non-critical element to a more critical part of the system.

Your systems production data needs to be treated with the utmost care and attention, access controls and authorisation must be robustly implemented to ensure that only entitled users are ever allowed to view or export data.

Denial of Service

Denial of Service (DoS) attacks are unique in the sense that they are not necessarily aimed at extracting data from your system or causing to execute specific functionality for an attacker, instead they are simply designed to stop your software being able to offer its intended functionality to your user base.

They can take many different forms but generally involve presenting your code and your infrastructure with more work than it is capable of handling, this means your site become unavailable to legitimate users or to become so slow as to be useless to them.

The exact method of protection against these kinds of attacks will vary depending on your functionality and infrastructure but will usually depend on being able to effectively measure and categorise the traffic entering your system alongside the ability to deny and block suspicious traffic at the edge of your network.

Elevation of Privilege

Privilege elevation occurs whenever a user is able to perform operations that they shouldn't be able to perform based on their role within the system, they are generally higher level functions usually reserved for administrators.

These attacks will generally rely on an insecure authorisation mechanism, as an example if a users role is controlled via a query string element then an attacker will be able to elevate their system privilege by simply inserting this element into their requests.

We deploy our software into a dangerous world, at some point it will come under attack. There is no silver bullet that means security can be deemed as finished. You are involved in a constant battle with attackers but you can often gain great insight into your system and identify areas for improvement by trying to think the way they think.

Getting to Production

If you're a server side developer the fruition of your efforts is achieved once your code gets to production. For a team to be effective and efficient it's important that this final stage in releasing is not a scary or frightening proposition. Delivering value into production is the whole reason for your team to exist so it should be a natural and unhindered consequence of your efforts.

Achieving this productive release chain requires a deployment strategy that inspires confidence by being slick, repeatable and with a get out of jail solution for when things don't work out quite as expected.

There is no one correct strategy, this will depend on the makeup of your team, the code you are writing and the nature of your production environment. Each possibility has it's advantages and disadvantages and it will be up to you to decide what works best for your situation.

Redeploy

Perhaps the simplest strategy involves simply redeploying code to your existing servers. This will usually involve momentarily blocking incoming traffic whilst the new code is deployed and verified.

The advantage of this strategy is it doesn't involve standing up any additional infrastructure to support the release and is a straight forward and simple process to follow. However it does have disadvantages.

Firstly it involves service downtime, you are not able to service requests whilst the release is in process. Additionally since the release must be tested after deployment, whilst traffic is being blocked, it can mean this testing is conducted under pressure that is not necessarily conducive to thoroughness.

The other major disadvantage is the inability to roll back the release should a problem develop. Either you must fix forward by deploying updated code or by redeploying the previous release. Both of these mean repeating the original release process by blocking traffic, deploying and testing.

Blue Green

Using this strategy you maintain two identical production stacks, one blue and one green. At any point in time one is your active stack serving production traffic and one is inactive and only serving internal traffic, or stood down entirely. The release process involves deploying to the inactive stack, conducting appropriate testing and then switching production traffic to be served from the newly deployed code.

The advantage of this strategy is it involves virtually no down time and allows for thorough testing to be completed, on the infrastructure that will serve the traffic, prior to the release without causing impact to users. It also has the substantial benefit that should the worst happen and you need to roll back the release this is simply achieved by switching traffic back to the previously active stack, which has not been modified since it was last serving production traffic.

The main disadvantage of this approach is in the cost of maintaining two production stacks. However taking advantage of modern cloud techniques means your inactive stack only needs to be up and running in the build up to a release reducing the addtional cost.

A second disadvantage can be seen if your application is required to maintain a large amount of state, the transition of this data between the two stacks needs to be managed in order to avoid disruption to users.

Canary

Canary deployments can be viewed as a variant of the Blue Green approach. Whereas a Blue Green deployment moves all production traffic onto the new code all at once a Canary deployment does this in a more gradual phased manner.

The exact nature of the switch will depend on your technology stack. When deploying directly to servers it will likely take the form of applying a gradually increasing traffic waiting between the old and new stacks. As traffic is moved onto the new code error rates and diagnostics can be monitored for issues and the weighting reversed if necessary.

This approach is particularly well suited to a containerised deployment where the population of containers can gradually be migrated to the new code via the normal mechanism of adding and removing containers from the environment.

The advantage of this approach is that it makes it far easier to trial new approaches and implementations without having to commit to routing all production traffic to the new code. Sophisticated techniques can be applied to the change of traffic weighting to route particular use cases or users to the new code whilst allowing the majority of traffic to be served from the tried and tested deployment.

However there can be disadvantages, the deployment of new code, and any subsequent roll back, are naturally slower although this can depend on your technology stack. Depending on the nature of your codebase and functionality having multiple active versions in production can also come with its own problems in terms of backwards compatibility and state management.

As with most decisions related to software engineering whether or not any particular solution is right or wrong can be a grey area. The nature of your environment and code will influence the choice of approach, the important thing to consider is the properties that any solution should offer. These are things like the ability to quickly roll back to a working state, the ability to effectively monitor the impact of a release along with factors such as speed and repeatability.

Keep these factors in mind when choosing your solution and you will enjoy many happy years of deploying to production.

What They Don't Teach You

Although it's possible to enter the world of software engineering from many different backgrounds it is still the case that many engineers will have studied some related technological discipline.

Armed with this hard earned knowledge they enter the world of professional development assured that they can hit ground running. Whilst this maybe true on a purely technical level there are many aspects of being a professional developer that unfortunately isn't taught at universities or colleges.

I should at this juncture admit that these views are mainly based on my experience of university compared to the world of work, which is admittedly now some time ago. However I think it is still the case that sometimes the curriculum can focus on the theoretical over the practical which isn't always to the benefit of the industry.

Source Control

Aside from their IDE of choice and the ticket management system their team chooses to employ the other item developers will interact with on a daily basis is source control. In this regard the proper use of source control is crucial to the effectiveness of a team.

Understanding the different approaches to branching strategies, having an appreciation of more advanced features, along with the basic etiquette of source control are all skills that will help new team members integrate into a team quickly and smoothly.

Although some courses may attempt to cover certain aspects of source control and may explain some of the available tooling options, a lack of practical experience in working on a code base within a team can hinder students in gaining an appreciation for why source control is so important. 

Continuous Integration

In a similar vein a lack of experience in working on a code base with a group of collaborators can also hinder an appreciation for the importance of Continuous Integration (CI).

The reason for the existence of CI is to solve the problem of integration hell that arises when code from multiple developers needs to be combined into a single build for release. CI has been around for long enough for even many experienced developers to not remember the troublesome days before "the build box". But what still remains prevalent is the need for stability in main line code and the shift left mentality designed to protect it.

A lack of understanding of the importance of CI can also be caused by a lack of exposure to the need to release. Obviously students are well aware of the importance of deadlines but nothing quite compares to the pressure to release combined with the scrutiny of a real user base to sharpen the mind and develop strategies to avoid mistakes.     

Legacy Code

The opportunities for developers to work on truly greenfield projects are very often few and far between. Usually in the majority of codebases or systems there are areas of legacy code that may be sub-optimal in certain aspects but that are so crucial to the correct operation of the software that everyone must tread careful when making changes.

This is not to say that legacy code should never be dealt with but the strategies employed for doing that are often very different to the rip it up and start again philosophy that may be employed in other areas of code.

Learning to work effectively with code you didn't write is a fact of life for all developers. Interpreting code written by others along with writing your own code to be understandable by some future reader is something all developers have to learn.

The points made here shouldn't be interpreted as criticism for those that have a lack of experience. The majority of valuable lessons all developers need to learn are born from the experience of making mistakes, the scar tissue that these mistakes develop are almost a rite of passage for many engineers. You're nobody until you've broken the build or deployed a bug to production.

In this sense it may seem unfair to blame academic institutions for not being able to produce experienced engineers. Whilst its true you can't teach experience you can foster an appreciation for the wider world that awaits. Few people study technological subjects purely as an academic exercise, the majority are doing so with an eye to becoming a professional in the industry, in this regard including some of the more vocational skills would benefit all concerned.