Detecting antisocial comments: an adventure in machine learning at theguardian.com - Intermediate

8th December 2016 in London at Business Design Centre

There are 57 other SkillsCasts available from Scala eXchange 2016

Please log in to watch this conference skillscast.

606909121 640

How do you detect antisocial behaviour in online communities? This is a very real problem for the Guardian as we have tens-of-thousands of comments a day. During this talk, you will learn how, as a small team of non-data-scientists, Nicolas and team used Apache Spark running on Amazon’s Elastic Map Reduce (‘EMR’) platform to detect abusive comments. The results? A better community and significantly reduced workload for our moderation team.

You will explore Nicolas and Thomas's real-life example to introduce machine learning concepts and Apache Spark. The particular focus will be on patterns of behaviour in online communities, but the talk should also be interesting to you if you happen to be relatively new to machine learning, or interested in classification problems more generally - for example, spam detection. You will also discover problems the team encountered along the way and how to avoid them yourself!

The Call for Papers is now open for Scala eXchange 2017! Submit your talk for the chance to join a stellar line-up of experts on stage. Find out more.

Get your tickets for Scala eXchange 2017!


Detecting antisocial comments: an adventure in machine learning at theguardian.com - Intermediate

Thomas Kaliakos

Thomas Kaliakos is a Software Engineer at Ovo Energy. He is a machine learning enthusiast. He's also a professional day-dreamer, amateur philosopher and lover of asking "why?"

Nicolas Long

'Nic Long is Tech Lead for the Discussion (user commenting) team at the Guardian, where he has been working for the last couple of years. He loves building scalable APIs in functional languages - in particular Scala and Clojure. He is passionate about developing new ways of journalism on the web and other technology platforms.