Alex Dean will talk about building Snowplow, an open source event analytics platform, on top of Scala and key libraries and frameworks including Scalding, Scalaz and Spray. He will highlight some of the data processing tricks and techniques picked up along the way, particularly: schema-first development; monadic ETL; datatable-based testing; data transformation maps. He will also introduce some of the Scala libraries the Snowplow team have open sourced along the way (such as scala-forex, referer-parser, scala-maxmind-geoip).
YOU MAY ALSO LIKE:
- Building robust data pipelines in Scala (SkillsCast recorded in December 2014)
- Lightbend Scala Language - Professional (in London on 9th - 10th September 2019)
- Lightbend Scala Language - Expert (in London on 11th - 13th September 2019)
- Scala eXchange London 2019 (in London on 12th - 13th December 2019)
- Keynote by Dick Wall on Why API Design Matters, and Why Yours Sucks! (and mine sucks too!) (in London on 24th June 2019)
- London Java Community June (in London on 25th June 2019)
- Lighting Talk: The Side Effects of Microservices: Making the Industry Better (SkillsCast recorded in May 2019)
- Lightning Talk: Multiple Hosting Targets for Microservices: What Could Be Easier? (SkillsCast recorded in May 2019)
Building data processing applications in Scala: the Snowplow experience
I'm the co-founder and tech lead at Snowplow Analytics, the open source web and event analytics platform (https://github.com/snowplow/snowplow). Snowplow is almost exclusively written in Scala, using a range of technologies including Scalaz, Scalding and Spray. I spend a lot of time working with distributed systems (historically Hadoop, increasingly Kinesis, Kafka et al) to deliver really scalable event stream processing. I'm also the author of Unified Log Processing from Manning Publications (http://manning.com/dean/).