Load Balancing is Impossible

13th July 2016 in London at CodeNode

There are 85 other SkillsCasts available from FullStack 2016 - the conference on JavaScript, Node & Internet of Things

Please log in to watch this conference skillscast.

581502725 640

Load balancing is something most of you assume is a solved problem. But the idea that load balancing is "solved" could not be further from the truth. If you use multiple load balancers, the problem is even worse. Most of you use "random" or "round-robin" techniques, which have certain advantages but are highly inefficient. Others use more complex algorithms like "least-conns," which can be more efficient but have horrific edge cases. "Consistent hashing" is a very useful technique, but only applies to certain problems. T

here are several factors that exist both in theory and practice that make efficient load balancing an exceptionally hard problem. For instance:

  • Poisson request arrival times;

  • Exponentially distributed response latency;

  • Oscillations when sharing data between multiple load balancers .

You will explore techniques and algorithms such as Randomized Least-conns, Join-Idle-Queue, and Load Interpretation. Load balancing perfectly may be impossible in the real world, but you can do better than “random,” “round-robin,” and naive “least-conns”.

You will walk away with an understanding of why traditional load balancing techniques are sub-optimal, practical techniques for improving it, as well as longer term theoretical approaches. You will also learn what makes load balancing such a difficult problem and why it's impossible to solve precisely.


Load Balancing is Impossible

Tyler McMullen

Tyler McMullen is CTO at Fastly, where he’s responsible for the system architecture and leads the company’s technology vision. As part of the founding team, Tyler built the first versions of Fastly’s Instant Purging system, API, and Real-time Analytics. Before Fastly, Tyler worked on text analysis and recommendations at Scribd. A self-described technology curmudgeon, he has experience in everything from web design to kernel development, and loathes all of it. Especially distributed systems.