Staging reactive data pipelines using Kafka as the backbone

Please log in to watch this conference skillscast.

Kafka has become the de facto platform for reliable and scalable distribution of high-volumes of data. However, as a developer, it can be challenging to figure out the best architecture and consumption patterns for interacting with Kafka while delivering quality of service such as high availability and delivery guarantees. It can also be difficult to understand the various streaming patterns and messaging topologies available in Kafka.

In this talk, Jaakko will share with you the patterns the team has successfully employed in production and provide the tools and guidelines for other developers to choose the most appropriate fit for given data processing problem. The key points for the presentation are: patterns for building reactive data pipelines, high availability and message delivery guarantees, clustering of application consumers, topic partition topology, offset commit patterns, performance benchmarks, and custom reactive, asynchronous, non-blocking Kafka driver.

YOU MAY ALSO LIKE:

• Domain Models in Practice with Marco Heimeshoff (Online Course on 9th - 13th August 2021)
• Sociotechnical Domain-Driven Design with Kacper Gunia (Online Course on 23rd - 24th August 2021)
• Software Architecture Gathering Digital 2021 (Online Conference on 13th - 14th October 2021)
• ScalaCon 2021: November Edition (Online Conference on 2nd - 5th November 2021)
• LDN *Virtual* Talks Jun 2021 — Concordium Takeover (Online Meetup on 28th June 2021)
• Tooling Up! - Contract testing with Pact (Online Meetup on 7th July 2021)
• Patterns for a Successful Cloud Native Transformation (SkillsCast recorded in June 2021)
• Managing Commercial Platforms: The Architect’s Emerging Skill (SkillsCast recorded in June 2021)