Please log in to watch this conference skillscast.
Eventually consistent data is common in modern web applications. We all know about eventually consistent data stores, popularised by Dynamo and Cassandra, but there are other sources of eventually consistent data. Mobile applications can go offline, but that doesn't stop people from using them, requiring synchronising data when they come back online. For robustness and to reduce network latency, applications may be hosted in multiple data centres connected by relatively high latency connections. It's infeasible to maintain strong consistency between these connections, again requiring synchronisation at regular intervals. How do we reconcile eventually consistent data?
One way is to use data types that we are guaranteed to merge without conflict. Then we easily reconcile all the different replicas in our system with a simple merge operation that we know cannot fail. These data types are known as conflict-free (sometimes commutative or convergent) replicated data types (CRDTs).
It seems straightforward to create, say, CRDT counters, but can we create CRDTs for complex data types such as sets? It turns out we can. There are wide variety of CRDTs available.
YOU MAY ALSO LIKE:
- The Elements of Tagless Final Style (SkillsCast recorded in May 2019)
- Akka Streams for Scala | ScalaCon Workshop (Online Course on 10th - 12th May 2021)
- ScalaCon 2021 (Online Conference on 18th - 22nd May 2021)
- An Introduction to Cats Effect with Adam Rosien (Online Meetup on 22nd April 2021)
- Abstract Data Types In The Region Of Abysmal Pain, And How To Navigate Them (SkillsCast recorded in September 2019)
- The Last Frontier and Beyond (SkillsCast recorded in August 2019)
Reconciling Eventually-Consistent Data with CRDTs
Noel has been interested in computers for a long time, particularly the leverage that computers give to people. He followed this interest to a PhD in machine learning, focusing on Bayesian nonparametrics and reinforcement learning. He still finds machine learning very interesting, but right now is more involved with programming and programming languages. A large part of his work is helping people become more effective with functional programming.