Please log in to watch this conference skillscast.
Would you like to learn how to reconcile eventually consistent data? Join Noel's talk and learn how to do this through conflict-free (sometimes commutative or convergent) replicated data types (CRDTs). In this talk Noel will describe the foundations of CRDTs, give some examples of known data types in Scala, and discuss issues that arise in practice. You will learn how to create straightforward CRDT counters, as well as how to create CRDTs for complex data types such as sets.
Eventually consistent data is common in modern web applications. We all know about eventually consistent data stores, popularised by Dynamo and Cassandra, but there are other sources of eventually consistent data. Mobile applications can go offline, but that doesn't stop people from using them, requiring synchronising data when they come back online. For robustness and to reduce network latency, applications may be hosted in multiple data centres connected by relatively high latency connections. It's infeasible to maintain strong consistency between these connections, again requiring synchronisation at regular intervals. How do we reconcile eventually consistent data?
One way is to use data types that we are guaranteed to merge without conflict. Then we easily reconcile all the different replicas in our system with a simple merge operation that we know cannot fail. These data types are known as conflict-free (sometimes commutative or convergent) replicated data types (CRDTs).
It seems straightforward to create, say, CRDT counters, but can we create CRDTs for complex data types such as sets? It turns out we can. There are wide variety of CRDTs available.
YOU MAY ALSO LIKE:
- FullStack Bytes with Jolyon Russ! (in London on 25th October 2017)
- F# eXchange 2018 (in London on 5th - 6th April 2018)
Reconciling Eventually-Consistent Data with CRDTs
Noel has over fifteen years experience in software architecture and development, and over a decade in machine learning and data mining. Examples of the projects he's been involved with include one of the first commercial products to apply machine learning to the Internet (eventually acquired by Omniture), a BAFTA award winning website, and a custom CMS used daily by thousands of students. Noel is an active writer, presenter, and open source contributor. Noel has a PhD in machine learning from the University of Birmingham.