There are certain topics that all software engineers regardless of the technology stack they work in or the programming language they use will bump into. One of those topics is asynchronous programming.
Asynchronous programming is where a unit of work is completed independently from the code that invoked it. A large number of languages have support for writing asynchronous code, mostly this is in a concurrent style where multiple units of work are allowed to progress simultaneously without actually executing in parallel.
Unfortunately just because an engineering topic is common place doesn't mean it is well understood, this is true for asynchronous programming. This is less to do with an understanding of its implementation and more to do with an appreciation for why and when to use asynchronous programming in your code base.
Asynchronous Programming vs Multi-threading
The most common misconception about asynchronous programming is that it makes you code faster or more performant, introducing asynchronous techniques will not inherently improve the performance of you code. Another common misconception is that asynchronous programming is all about introducing multiple threads of execution.
Both of these misconceptions are understandable given that well implemented asynchronous code can appear to be running faster and on occasion can employ multiple threads but neither of these things are intrinsic to the purpose of asynchronous programming.
Asynchronous programming is actually about making the most of available resources by increasing throughput. It attempts to ensure that the maximum number of work items are allowed to progress alongside each other. Each individual work item might not complete any quicker from start to finish but the total amount of work completed in a given time frame is increased.
For a real world example imagine a takeaway kitchen on a busy Saturday night. If each member of staff takes an order, prepares it, waits for it to cook and then delivers it then the takeaway is operating in a synchronous manner, the number of orders it can cope with at any one time is dictated purely by the number of workers.
An asynchronous approach looks to reduce the amount of idle time each worker has while dealing with an individual order. While a worker is waiting for an order to cook they start preparing the next, this makes better use of the available resource and enables the takeaway to deal with more orders simultaneously without introducing anymore resource.
To complete the analogy multi-threading would be the equivalent of introducing more workers into the kitchen. To begin with this may increase the number of orders leaving the kitchen but eventually workers will start to get in each others way and overall efficiency will fall.
Bounding
In general there are two main reasons for the completion of a task to be held up. A task is CPU bound if it would be completed faster by more compute resource, a task is IO bound if its completion is reliant on the transfer of data via IO operations.
When dealing with CPU bound tasks the previously explained misconceptions about asynchronous programming and multi-threading can cause these techniques to be applied in an inappropriate manner. We have a thread that is blocked by a CPU bound task, we therefore create a new thread to handle the CPU bound task allowing the first thread to be unblocked.
However there are two issues with this approach, firstly the creation of threads is not free it involves the allocation of memory and other resources with only a finite number of them being available. Secondly, we have still ended up in a position where a thread is being blocked by the CPU bound task. The now unblocked thread is presumably still waiting for the result of the CPU bound task so this is the anthesis of the asynchronous approach by using an increased level of resource in an inefficient way.
Asynchronous programming is tailor made for dealing with IO bound tasks by allowing valuable resource to not be held waiting for a relatively long task, compared to CPU execution speed, to complete. Again creating more threads to deal with this won't get away from the fact that the task itself is bound by constraints.
There is however an exception to this rule, in GUI applications that have the concept of a UI thread moving any bounded task onto another thread is a practical step to take. In this situation the UI thread is a special resource that must be protected since it is the only resource that can update the user interface.
Deadlocks
In the opening paragraph of this post we said that the majority of software engineers will have encountered asynchronous programming, alongside this I am also willing to bet that most of these engineers will have accidentally implemented a deadlock when they were learning these techniques.
A deadlock occurs when multiple processes are all waiting on each other to complete in a circular manner meaning no task can progress and all are permanently blocked. For a deadlock to occur a few conditions are necessary.
There must be at least one resource that operates mutual exclusion meaning only one process can interact act with it and it is non-shareable. A process is currently holding the lock on this resource whilst waiting for another process to complete when the second process needs the locked resource, this circular wait is the essence of a deadlock.
There are many problems in software engineering where the first line of defence against them is to be aware that they exist. The second defence lies in using our experience of encountering them to develop a smell for when they might be about to present themselves. While it may be possible to come up with large numbers of code examples for these kinds of problems ultimately it is these slightly more intangible defences that will protect us.
Similarly there are certain intrinsic programming concepts, such as asynchronous programming, that are difficult to master. The issue isn't that engineers fail to become experts its that they fail to appreciate the danger they can represent. A good engineer is very often a humble one who understands the limits of his or her knowledge whilst having an appreciation for what lies beyond.
These engineers will tread carefully around thorny topics and take care to look for the potential problems they know can exist. Treating the complex with distain is usually the first step towards making the most common of mistakes.
