Predicting congestion on London’s roads with Beam and Tensorflow - Intermediate

Please log in to watch this conference skillscast.

redicting congestion is an important part of any traffic management system. With accurate forecasting traffic can be effectively regulated, ensuring safe and fast journeys on the roads. Join Oliver as he shares a deep learning model which accurately forecasts congestion based on road sensor data from Transport for London (TfL).

The IoT (internet of things) nature and scale of the raw sensor data (5 TB, 120 billion rows) demands extensive preprocessing as a first step towards a predictive traffic model. Oliver and his team used Apache Beam for this task as it let them create efficient data pipelines which can be executed in distributed frameworks such as Apache Spark or Apache Flink. Beam natively handles streaming workloads which makes it an ideal candidate for large scale preprocessing of real-time data such as streams from road sensors.

The preprocessed data was then used to train a neural network to predict the congestion ahead of time. A recurrent neural network (RNN) was chosen to model the traffic time-series for each of the sensors. This deep learning architecture was implemented in Tensorflow and it allowed us to accurately model the time-series including the correlations between the sensors on the road network. With this end-to-end example Oliver will demonstrate how Beam and Tensorflow can be used to build predictive models for time series data.

More information on the project (including some results and images) can be found here and here.

YOU MAY ALSO LIKE:

• Deep Learning for Recommender Systems (SkillsCast recorded in July 2018)
• Deep Learning with F#: An Experience Report (SkillsCast recorded in October 2021)
• Ends in Data (SkillsCast recorded in December 2020)