Staging reactive data pipelines using Kafka as the backbone

Please log in to watch this conference skillscast.

Kafka has become the de facto platform for reliable and scalable distribution of high-volumes of data. However, as a developer, it can be challenging to figure out the best architecture and consumption patterns for interacting with Kafka while delivering quality of service such as high availability and delivery guarantees. It can also be difficult to understand the various streaming patterns and messaging topologies available in Kafka.

In this talk, Jaakko will share with you the patterns the team has successfully employed in production and provide the tools and guidelines for other developers to choose the most appropriate fit for given data processing problem. The key points for the presentation are: patterns for building reactive data pipelines, high availability and message delivery guarantees, clustering of application consumers, topic partition topology, offset commit patterns, performance benchmarks, and custom reactive, asynchronous, non-blocking Kafka driver.

YOU MAY ALSO LIKE:

• Functional Domain Models in Practice with Marco Heimeshoff (Online Workshop on 17th - 21st October 2022)
• Domain Driven Design & Legacy Systems with Marijn Huizendveld (In-Person) (in London on 22nd - 24th November 2022)
• ScalaCon 2022: Opening Keynote Networking Party (in London on 4th October 2022)
• ScalaCon 2022 (Online Conference on 4th - 8th October 2022)
• Open Source MLOps - Model Deployment (Online Meetup on 29th September 2022)
• Team Topology - Enablement mode (Online Meetup on 6th October 2022)
• Building a Culture to Take Control of Your Software (SkillsCast recorded in July 2022)
• Gravitee APIM Behind Istio Service Mesh (SkillsCast recorded in May 2022)