A data layer in Clojure - Intermediate

Please log in to watch this conference skillscast.

The three main lessons you will learn are:

1) the goto data layer infrastructure build around Kafka, Onyx and aggressive usage of materialized views, with emphasis on how to build a system that requires relatively little effort upfront but can grow with one's needs.

2) The problem of managing data and a case for declarative and queryable data descriptions. How these can be used as the basis for automatic materialized view inference where specialized views and data crossings are inferred from raw incoming data or other views based on a combination of heuristics, statistical analysis (seasonality, outlier removal, ...) and predefined ontologies. Doing so is a practical way to maintain a large number of views, increasing their availability and abstracting the complexity into declarative rules, rather than having an ETL pipeline with dozens or even hundreds of hand crafted tasks.

3) how and why Clojure is a natural choice for tasks that involve a lot of data manipulation, touching both on functional programming and lisp-specifics such as code-is-data.

YOU MAY ALSO LIKE:

• The Subtle Art of Recommendation (Algorithms) (SkillsCast recorded in July 2019)
• Apache Pulsar and the Streaming Ecosystem (SkillsCast recorded in May 2021)
• Islands in the Stream - What country music can teach us about event driven systems (SkillsCast recorded in May 2021)