Reconciling Eventually-Consistent Data with CRDTs
Please log in to watch this conference skillscast.
Eventually consistent data is common in modern web applications. We all know about eventually consistent data stores, popularised by Dynamo and Cassandra, but there are other sources of eventually consistent data. Mobile applications can go offline, but that doesn't stop people from using them, requiring synchronising data when they come back online. For robustness and to reduce network latency, applications may be hosted in multiple data centres connected by relatively high latency connections. It's infeasible to maintain strong consistency between these connections, again requiring synchronisation at regular intervals. How do we reconcile eventually consistent data?
One way is to use data types that we are guaranteed to merge without conflict. Then we easily reconcile all the different replicas in our system with a simple merge operation that we know cannot fail. These data types are known as conflict-free (sometimes commutative or convergent) replicated data types (CRDTs).
It seems straightforward to create, say, CRDT counters, but can we create CRDTs for complex data types such as sets? It turns out we can. There are wide variety of CRDTs available.
YOU MAY ALSO LIKE:
Thank you to our sponsors and partners
Platinum
Gold
Silver
Bronze
Reconciling Eventually-Consistent Data with CRDTs
Noel Welsh
Noel has been interested in computers for a long time, particularly the leverage that computers give to people. He followed this interest to a PhD in machine learning, focusing on Bayesian nonparametrics and reinforcement learning. He still finds machine learning very interesting, but right now is more involved with programming and programming languages. A large part of his work is helping people become more effective with functional programming.