Please log in to watch this conference skillscast.
The talk will be about a project Oliver and the Datatonic team did with TfL, where they used Apache Beam and Tensorflow to predict congestion. The talk will focus in detail on how and why these technologies were used in the use case at hand. n-episode-ii-predicting.html
redicting congestion is an important part of any traffic management system. With accurate forecasting traffic can be effectively regulated, ensuring safe and fast journeys on the roads. Join Oliver as he shares a deep learning model which accurately forecasts congestion based on road sensor data from Transport for London (TfL).
The IoT (internet of things) nature and scale of the raw sensor data (5 TB, 120 billion rows) demands extensive preprocessing as a first step towards a predictive traffic model. Oliver and his team used Apache Beam for this task as it let them create efficient data pipelines which can be executed in distributed frameworks such as Apache Spark or Apache Flink. Beam natively handles streaming workloads which makes it an ideal candidate for large scale preprocessing of real-time data such as streams from road sensors.
The preprocessed data was then used to train a neural network to predict the congestion ahead of time. A recurrent neural network (RNN) was chosen to model the traffic time-series for each of the sensors. This deep learning architecture was implemented in Tensorflow and it allowed us to accurately model the time-series including the correlations between the sensors on the road network. With this end-to-end example Oliver will demonstrate how Beam and Tensorflow can be used to build predictive models for time series data.
YOU MAY ALSO LIKE:
- Deep Learning for Recommender Systems (SkillsCast recorded in July 2018)
- Fast Track to Machine Learning with Louis Dorard (in London on 15th - 17th July 2019)
- Leonardo De Marchi's Deep Learning Fundamentals (in London on 22nd - 23rd October 2019)
- Infiniteconf 2019 - A one-day community celebration of Big Data, Machine Learning and AI (in London on 4th July 2019)
- IWDS 23: Intro to Natural Language Processing (NLP) - 1/2 (in London on 24th June 2019)
- Keynote by Emily Robinson on Creating a Strong Data Science Portfolio (in London on 24th June 2019)
- Achieve 100% Analytics Adoption with a New Class of Enterprise Intelligence (SkillsCast recorded in June 2019)
- Data Engineering in Badoo: handling 20 billion events per day (SkillsCast recorded in June 2019)