1) Desire to reuse scale-out and on-demand-scaling technologies developed for big-data analysis

2) Very high data volume output from the simulations which is costly to reproduce, meaning that the outputs are saved and mined subsequently for multiple scenarios and analyses.

3) <>, where rapid analysis of initial results is used to adjust the pricing parameters and the space of market conditions which is simulated

A similar convergence pattern can be seen in a number of other fields in science and engineering, e.g., climate modelling simulation, which will make this talk of interest to a wide audience.

In this talk, Sebastien assume you know about overfitting and regularization, and will dissect insidious ways to overfit, as well as the no free lunch theorem. He will explore points of contact between big data and machine learning from an engineering perspective. Finally, Sebastien will present more advanced ML topics which are worth knowing about for data scientists, Bayesian optimization and Auto-ML.

Recommender system technology is the core of Netflix and Amazon's business model and has lead to a tremendous increase in sales and customer satisfaction. Other retailers have seen sales increases of 5-15%, and now recommender systems are making their way to other industries to help customers find products faster, help salespeople find collateral and configure solutions, and help companies accelerate their product development by finding the right components to make products that meet market needs.

Real-time recommender systems are one of the sweetspot use cases for native graph databases. Key goals for a good recommender system include relevance, novelty, serendipity and recommendation differentiation. In this talk, Pieter will demonstrate how you can have full and accurate control of the recommender system with Neo4j, interactive response at scale, and "on the fly" tuning for a fast time to market.

Join Brian and discover a variety of open source tools for extracting the content, identifying elements and structure and analyzing the text can be used in distributed, microservice-friendly ways.

During this talk, you will discover the new requirements that streaming imposes on systems and how they are met by combinations of popular tools like Kafka, Spark, Flink, and Akka. Dean will argue that streaming and microservice architectures are actually converging.

The company uses reinforcement learning and bayesian optimisation to analyse customer feedback data, which is provided via a Facebook Messenger bot. This data is then used to tell IntelligentX’s master brewer what to brew next. This ensures each new version is more finely tuned to customers' tastes. Join IntelligentX co-founder Rob McInerney as he shares how they came up with the idea, how it works and what the future holds for ML-based consumer products.

Find out more on IntelligentX here.

Neural Networks were first created in an attempt to mimic the biological neural networks in the human Brain [3]. Such types of machine learning models are now being extensively used in many prediction problems due to their multi-layer-structures and their ability to generalize based on a plethora of different parameters accounting for many of their initial weaknesses (and their tendency to overfit or underfit). The advances in computing power and specifically the usage of GPUs has allowed such machine learning models to be run at greater speeds [4] shaping the form of today’s deep learning.

Join Marios and learn about the StackNet model – a computational, scalable and analytical framework implemented in Java that resembles a feedforward neural network and uses Wolpert’s stacked generalization in multiple levels to improve accuracy in classification problems. In contrast to feedforward neural networks, rather than being trained through back propagation, the network is built iteratively one layer at a time using Wolpert’s stacked generalization. StackNet’s ability to improve accuracy is demonstrated via creating different instances of StackNet models with multiple levels and architectures which are then used to rank best the likelihood of a certain song being created before or after 2002 using a set of 90 numerical attributes out of 515,345 songs that come from a subset of the Million Song Dataset[1].

[1] Bertin-Mahieux, T., Ellis, D. P., Whitman, B., & Lamere, P. (2011, October). The Million Song Dataset. In ISMIR (Vol. 2, No. 9, p. 10).

[2] Koren, Y. (2009). The bellkor solution to the netflix grand prize. Netflix prize documentation, 81, 1-10.

[3] Rosenblatt, F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65(6), 386.

[4] Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85-117.

[5] Wolpert, D. H. (1992). Stacked generalization. Neural networks, 5(2), 241-259.

He will also briefly cover Social Networking applications, recommendation engines, graph visualisations and Java Unmanaged Extensions in neo4j.

He will point out the potential pitfalls that lie in wait and how to avoid them. What’s a good architecture and modelling pattern to work towards? Matt and his team have been using neo4j full time for 18 months and will share what has worked for them as well as sharing some of their glorious and spectacular failures.

Warning: Contains hairballs, Java, Python, React.js and Kiwis.

Join Daniele and Eleanor in this talk and learn how to build an AI for Pac-Man leveraging Monte Carlo Tree Search. With the help of D3.js you can verify the correctness of the algorithm and enter in the mind of the machine.

redicting congestion is an important part of any traffic management system. With accurate forecasting traffic can be effectively regulated, ensuring safe and fast journeys on the roads. Join Oliver as he shares a deep learning model which accurately forecasts congestion based on road sensor data from Transport for London (TfL).

The IoT (internet of things) nature and scale of the raw sensor data (5 TB, 120 billion rows) demands extensive preprocessing as a first step towards a predictive traffic model. Oliver and his team used Apache Beam for this task as it let them create efficient data pipelines which can be executed in distributed frameworks such as Apache Spark or Apache Flink. Beam natively handles streaming workloads which makes it an ideal candidate for large scale preprocessing of real-time data such as streams from road sensors.

The preprocessed data was then used to train a neural network to predict the congestion ahead of time. A recurrent neural network (RNN) was chosen to model the traffic time-series for each of the sensors. This deep learning architecture was implemented in Tensorflow and it allowed us to accurately model the time-series including the correlations between the sensors on the road network. With this end-to-end example Oliver will demonstrate how Beam and Tensorflow can be used to build predictive models for time series data.

More information on the project (including some results and images) can be found here and here.

Seen the flexibility and the advantages of GP models, it is not surprising that recent research has been focused on extending GP models to BigData problems.

Join Roberta as she shares an overview of GPs in a machine learning context and how they can be used for BigData problems. Explore the basic definition of a Gaussian distribution and GPs, with particular attention to covariance functions. Roberta will then explain how GPs are used for inference problems and present relevant application areas, their advantages and their limitations.

Join Mark for a practical session with examples of R code. So if the veracity, variety and value of data are more important aspects than volume and velocity then this talk is for you.

This software application is composed by state-of-the art cross-domain data collection, integration and visualisation capabilities, advanced functionalities such as notification about changes in the projects a user has subscribed to, dashboards, as well as filtering mechanisms by geographical opportunity areas, funding, project costs, among others. Over 12,000 pieces of city data (open, private and commercial) relevant to infrastructure planning and delivery are currently included in the tool, ranging from years 2015 to 2050. Data sets include projected growth (population and jobs), planning frameworks, development proposals, infrastructure projects, infrastructure capacity, environmental threats (such as flooding information), and skills requirements.

Ultimately, this solution allows infrastructure providers to identify opportunities to deliver infrastructure jointly, reducing the potential for disruption and driving down the cost of construction.

At Onfido, they apply computer vision and machine learning techniques to the detection of physical and digital fraud in identity documents.

When it comes to fraud detection, they’re often dealing with a problem for which they have hugely unbalanced datasets. Moreover, they only understand what they've seen before; if they notice something different, how do they know if it’s really fraudulent, and how confident can they be with that decision?

Join Jacques for this lightning talk as he explores the problem of fraud detection, highlight some of the biggest challenges that they face in this area and cover some of Onfido's efforts to circumvent these challenges, touching on topics of anomaly detection and data generation.

The three main lessons you will learn are:

1) the goto data layer infrastructure build around Kafka, Onyx and aggressive usage of materialized views, with emphasis on how to build a system that requires relatively little effort upfront but can grow with one's needs.

2) The problem of managing data and a case for declarative and queryable data descriptions. How these can be used as the basis for automatic materialized view inference where specialized views and data crossings are inferred from raw incoming data or other views based on a combination of heuristics, statistical analysis (seasonality, outlier removal, ...) and predefined ontologies. Doing so is a practical way to maintain a large number of views, increasing their availability and abstracting the complexity into declarative rules, rather than having an ETL pipeline with dozens or even hundreds of hand crafted tasks.

3) how and why Clojure is a natural choice for tasks that involve a lot of data manipulation, touching both on functional programming and lisp-specifics such as code-is-data.

Join Robert as he demonstrates Dataflow’s capabilities through a real-time demo with practical insights on how to manage and visualize streams of data.

TensorFlow is an increasingly popular open source Machine Intelligence library that is especially well-suited for deep learning. The Google Cloud Machine Learning Engine (Cloud ML Engine) lets you do distributed training and serving of your TensorFlow models at scale.

Join Yufeng as he kicks off with an introduction to TensorFlow concepts and then walks through how to use Cloud ML Engine to do distributed training and scalable serving of your trained models. You will explore the design decisions available to you in scaling up your machine learning, and talk about their tradeoffs.

Streaming real time datasets from sensors in connected cars, wearables & IoT devices into Power BI dashboards utilising Cortana Intelligence Suite features such as IoT Hubs, Events Hub, HD Insights, Streaming Analytics and Machine Learning to ingest, clean, store & analyse and enhance your datasets for visualisation in Power BI is an affordable Cloud solution for end users to spin up consume and then pause as and when required without large IT infrastructure overheads costs.

David Moss, Microsoft MVP Data Platform & an Azure Cortana Intelligence & Power BI end to end Solutions Architect, will teach you how to fire up this stack in 45 minutes. This is an essential skill that all IoT, Data Scientists and AI learners require within their toolset.

Lindsey and Phil will share with you how the Guardian has used a range of technologies including Apache Spark and PrestoDB on AWS to support simple ingestion and fast querying of a wide range of datasets. Learn why it’s important to decouple storage from compute and raw data sources from optimised query formats and why there’s still no single perfect solution.

Join Alan as he illustrates a new type of ensemble method which makes use of information about how the deep learning algorithm is structured, in order to improve training times, diversity generation, and ultimately accuracy.

Alan will also give examples of existing methods from his research at Birkbeck, and provide experimental results.

IOT and Event Based systems can process huge volumes of data. Which typically needs to be stored and read in near real time for event processing, in addition to being read in bulk to feed data hungry learning systems. Apache Cassandra provides a high performance, scalable, and fault tolerant database platform with excellent support for time series data models typically seen in IOT systems. It's millisecond (or better) latency can support systems that react to events in real time, while scalable bulk reads via batch processing systems such as Apache Hadoop and Apache Spark can support learning applications. These features, and more, make Cassandra an ideal persistence platform for modern data intensive, event driven, systems.

In this talk Aaron Morton, CEO at The Last Pickle, and Jon Haddad, Principal Consultant at The Last Pickle, will discuss lessons learned using Cassandra for IOT systems. They will explain how Cassandra fits into the modern technology landscape and dive into data modelling for common IOT use cases, capacity planning for huge data loads, tuning for high performance, and integration with other data driven systems. Whether starting a new project, or deep into the weeds on an existing system, attendees will leave will leave with an understanding of how Apache Cassandra can help build robust infrastructure for IOT systems.

Join Kaz and discover how you can combine Cloud Machine Learning and BigQuery to realize this vision. By sharing a demo, you'll see how BigQuery's power of "democratizing enterprise data" can be enhanced with a deep neural network model trained with Cloud Machine Learning.

This presentation aims to be a return of experience implementing these pipelines on a real-world big data project.

At Millesime.ai, they're using Spark to predict fine wine market price and trading strategy for the next 6 months and more. Through this use case, you'll explore the core concepts and principles behind machine learning pipelines, what algorithms Spark MLlib covers and how to implement them using Transformers, Estimators and Model Selection through Hyperparameter Tuning.

In the end we'll try to have an objective summary of the lessons learnt developing this project: the good, the bad and the ugly, with some common pitfalls we've run into and how to avoid them.

Duncan will start by defining linked data, and the value it holds for a data scientist. You'll explore use cases that show how analysing connections using specific data mining and visualisation techniques can help you gain deeper understanding of a topic.

Most of this talk will describe Duncan's own experience building an interactive tool for visualising the connections between different music genres.

Join Duncan and see how he used SPARQL to mine and query data from DBPedia, and then JavaScript code libraries, including d3, to visualise the connections. He’ll include code snippets, and publish some source code on GitHub for you to try. He’ll share what he learned together with best practices you should follow when designing your own visualisation.

• Data Science (Analytics to discover and prove valuable insights)

• Data Engineering (Implementing robust operational data flows and applications)

• Governance for Data & Architecture (Alignment of multiple projects and the business)

Asking Data Scientists to be responsible for both Analytics and Engineering is to combine roles as diverse as those of a benign hacker and a DBA. At best, one person will be an amateur in one of those fields but to charge one person with the conflicting responsibilities of discovery and operations is to underrate one of them to the disadvantage of the organisation.

Join Kenneth and get an overview of Big Data Issues & solutions in respect of

• Resourcing and Managing concurrent Analytics Projects

o Sharing research on the separation of responsibilities for Analytics and Engineering

o Data Acquisition and Managing the “Data Lake”

• Planning for the future

o Operational Algorithms become MISSION CRITICAL

o Ontology

o Translation of Data Models from Conceptual to schemas for Object Relational,

Document, RDF, Graph, etc.

o Python v Scala v R, etc.

o Movement of data

o Architecture for timely, robust and sustainable operations

• Governance work and forums.

In this talk Marco will describe the intuitions behind this family of algorithms, you'll explore some of the Python tools that allow us to implement modern NLP applications and we'll conclude with some practical considerations.