Shopping around with CRDTs at Whisk

8th December 2014 in London at Business Design Centre

There are 54 other SkillsCasts available from Scala eXchange 2014

Please log in to watch this conference skillscast.

499544174 640

In this talk, David will describe how they use Commutative Replicated Data Types (CRDTs) at Whisk to provide a seamless, cross-device shopping list. CRDTs are general-purpose patterns for achieving eventually-consistent synchronisation. They achieve this by communicating in operations - “diffs” between one state and the next - which have magical properties of being replayable (idempotence) in any order (commutativity). Operations can be recorded in an operations log and used to reconstruct current state on a client, whilst respecting that client’s own local state. One of the concerns with using CRDTs is that this log of operations can grow indefinitely.

Whisk is a shopping list app that connects online recipes to online and offline supermarkets. Their users build their shopping lists from online recipes using embedded widgets, browser extensions, our webapp, and our mobile apps. Their lists must stay in sync, or they won’t eat. To complicate matters, users carry their lists with them to supermarkets - the best naturally-occurring examples of Faraday cages. Whisk's apps must work seamlessly with very patchy networks, but also in fully offline mode. To achieve this, their apps use local storage, and sync mechanisms use CRDTs.

Whisk have implemented two specific CRDTs to support shopping lists at Whisk, provided to their clients through an API built with Scala and underpinned by Cassandra. David will describe their approach and architecture, and discuss some practical measures they've taken to address the problem of an ever-growing operations log.

This talk should appeal to anyone interested in cross-device data synchronisation, and practical applications of Scala.


Thanks to our sponsors

Shopping around with CRDTs at Whisk

David Brooks

David is Chief Technology Officer at Whisk, a startup that understands online recipes and makes it easy to buy their ingredients.