Please log in to watch this conference skillscast.
Would you like to learn how to reconcile eventually consistent data? Join Noel's talk and learn how to do this through conflict-free (sometimes commutative or convergent) replicated data types (CRDTs). In this talk Noel will describe the foundations of CRDTs, give some examples of known data types in Scala, and discuss issues that arise in practice. You will learn how to create straightforward CRDT counters, as well as how to create CRDTs for complex data types such as sets.
Eventually consistent data is common in modern web applications. We all know about eventually consistent data stores, popularised by Dynamo and Cassandra, but there are other sources of eventually consistent data. Mobile applications can go offline, but that doesn't stop people from using them, requiring synchronising data when they come back online. For robustness and to reduce network latency, applications may be hosted in multiple data centres connected by relatively high latency connections. It's infeasible to maintain strong consistency between these connections, again requiring synchronisation at regular intervals. How do we reconcile eventually consistent data?
One way is to use data types that we are guaranteed to merge without conflict. Then we easily reconcile all the different replicas in our system with a simple merge operation that we know cannot fail. These data types are known as conflict-free (sometimes commutative or convergent) replicated data types (CRDTs).
It seems straightforward to create, say, CRDT counters, but can we create CRDTs for complex data types such as sets? It turns out we can. There are wide variety of CRDTs available.
YOU MAY ALSO LIKE:
Reconciling Eventually-Consistent Data with CRDTs
Noel is a partner at Underscore, a global Scala consultancy, where he uses his expertise to develop simple and scalable systems for clients. His main professional interests are programming languages and machine learning.