Please log in to watch this conference skillscast.
Eventually consistent data is common in modern web applications. We all know about eventually consistent data stores, popularised by Dynamo and Cassandra, but there are other sources of eventually consistent data. Mobile applications can go offline, but that doesn't stop people from using them, requiring synchronising data when they come back online. For robustness and to reduce network latency, applications may be hosted in multiple data centres connected by relatively high latency connections. It's infeasible to maintain strong consistency between these connections, again requiring synchronisation at regular intervals. How do we reconcile eventually consistent data?
One way is to use data types that we are guaranteed to merge without conflict. Then we easily reconcile all the different replicas in our system with a simple merge operation that we know cannot fail. These data types are known as conflict-free (sometimes commutative or convergent) replicated data types (CRDTs).
It seems straightforward to create, say, CRDT counters, but can we create CRDTs for complex data types such as sets? It turns out we can. There are wide variety of CRDTs available.
YOU MAY ALSO LIKE:
- The Elements of Tagless Final Style (SkillsCast recorded in May 2019)
- ScalaCon 2022: Opening Keynote Networking Party (in London on 4th October 2022)
- ScalaCon 2022 (Online Conference on 4th - 8th October 2022)
- LJC: I Started Testing In Production... Then I Went On Holiday (Online Meetup on 15th August 2022)
- Real-time Stream Processing in Spring Made Easy (in London on 25th August 2022)
- A History of Enterprise Monads (SkillsCast recorded in May 2021)
- Connecting the dots - building and structuring a functional application in Scala (SkillsCast recorded in May 2021)
Reconciling Eventually-Consistent Data with CRDTs
Noel has been interested in computers for a long time, particularly the leverage that computers give to people. He followed this interest to a PhD in machine learning, focusing on Bayesian nonparametrics and reinforcement learning. He still finds machine learning very interesting, but right now is more involved with programming and programming languages. A large part of his work is helping people become more effective with functional programming.