Continuous applications face many challenges as they grow to production. Often, due to the rapid increase in the number of devices or end-users or other data sources, the size of their data set grows exponentially. This results in a backlog of data to be processed. The data will no longer be processed in near-real-time. Redis Streams enables you to collect both binary and text data in the time series format. The consumer groups of Redis Stream help you match the data processing rate of your continuous application with the rate of data arrival from various sources. Apache Spark’s Structured Streaming API enables real-time decision making for Continuous Applications. In this session, Roshan will perform a live demonstration of how to integrate open source Redis Streams with Apache Spark’s Structured Streaming API using Spark-Redis library. He will also walk through the code and demo a live continuous application.

Talk Description: Continuous applications have 3 things in common:

1) They collect data from sources (ex: IoT devices)

2) process them in real-time (example: ETL)

3) deliver them to machine learning serving layer for decision making

YOU MAY ALSO LIKE:

• Fast Track to DevOps Fundamentals (in London on 23rd - 24th September 2019)
• CloudNative London 2019 (in London on 25th - 27th September 2019)
• Running Stateful Applications in Kubernetes (in London on 5th September 2019)
• Real-time Systems with Spark Streaming and Kafka (in London on 30th September - 1st October 2019)
• Leonardo De Marchi's Deep Learning Fundamentals (in London on 22nd - 23rd October 2019)
• Haskell eXchange 2019 (in London on 10th - 11th October 2019)
• You got served: How Deliveroo improved the ranking of restaurants (in London on 27th August 2019)
• AlphaGo Zero: Mastering the Game of Go without Human Knowledge (in London on 27th August 2019)
• Genetic Programming for RL on Minecraft (SkillsCast recorded in August 2019)
• Designing trust and transparency using AI (SkillsCast recorded in August 2019)