Sketch Algorithms
Please log in to watch this conference skillscast.
In this talk, you will look at how to efficiently (in both space and time) summarize large, potentially unbounded, streams of data by approximating the underlying distribution using so-called sketch algorithms. The main approach you are going to be looking at is summarization via histograms. Histograms have a number of desirable properties: they work well in an online setting, are embarrassingly parallel, and are space-bound. Not to mention they capture the entire (empirical) distribution which is something that often gets lost when doing descriptive statistics.
Building from that you will delve into related problems of sampling in a stream setting, and updating in a batch setting, and highlight some cool tricks such as capturing time-dynamics via data snapshotting. To finish off we will touch upon algorithms to summarize categorical data, most notably count-min sketch. The talk is motivated by my work at Metabase -- an opensoucre analytics tool -- where you heavily utilize histograms in building our "data scientist in a box".
YOU MAY ALSO LIKE:
- The Subtle Art of Recommendation (Algorithms) (SkillsCast recorded in July 2019)
- Fast Track to DevOps Fundamentals (in London on 23rd - 24th September 2019)
- CloudNative London 2019 (in London on 25th - 27th September 2019)
- Keynote by Ryn Daniels on Designing Sustainable Ops Cultures (in London on 21st August 2019)
- Functional Concurrency in .NET with C# and F# (in London on 9th - 10th September 2019)
- ProgNET London 2019 (in London on 11th - 13th September 2019)
- Haskell eXchange 2019 (in London on 10th - 11th October 2019)
- London Business Analytics Group (in London on 21st October 2019)
- How do we use technology to augment human judgement in projects? (SkillsCast recorded in May 2019)
- Project Data Analytics Lightening Talks (SkillsCast recorded in April 2019)
Thanks to our sponsors
Sketch Algorithms
Simon Belak
Simon built his first computer out of Lego bricks and learned to program soon after. Emergence, networks, modes of thought, limits of language and expression are what makes him smile (and stay up at night). The combination of lisp and machine learning put him on the path of always striving to make himself redundant if not outright obsolete.
Please log in to watch this conference skillscast.
In this talk, you will look at how to efficiently (in both space and time) summarize large, potentially unbounded, streams of data by approximating the underlying distribution using so-called sketch algorithms. The main approach you are going to be looking at is summarization via histograms. Histograms have a number of desirable properties: they work well in an online setting, are embarrassingly parallel, and are space-bound. Not to mention they capture the entire (empirical) distribution which is something that often gets lost when doing descriptive statistics.
Building from that you will delve into related problems of sampling in a stream setting, and updating in a batch setting, and highlight some cool tricks such as capturing time-dynamics via data snapshotting. To finish off we will touch upon algorithms to summarize categorical data, most notably count-min sketch. The talk is motivated by my work at Metabase -- an opensoucre analytics tool -- where you heavily utilize histograms in building our "data scientist in a box".
YOU MAY ALSO LIKE:
- The Subtle Art of Recommendation (Algorithms) (SkillsCast recorded in July 2019)
- Fast Track to DevOps Fundamentals (in London on 23rd - 24th September 2019)
- CloudNative London 2019 (in London on 25th - 27th September 2019)
- Keynote by Ryn Daniels on Designing Sustainable Ops Cultures (in London on 21st August 2019)
- Functional Concurrency in .NET with C# and F# (in London on 9th - 10th September 2019)
- ProgNET London 2019 (in London on 11th - 13th September 2019)
- Haskell eXchange 2019 (in London on 10th - 11th October 2019)
- London Business Analytics Group (in London on 21st October 2019)
- How do we use technology to augment human judgement in projects? (SkillsCast recorded in May 2019)
- Project Data Analytics Lightening Talks (SkillsCast recorded in April 2019)
Thanks to our sponsors
Sketch Algorithms
Simon Belak
Simon built his first computer out of Lego bricks and learned to program soon after. Emergence, networks, modes of thought, limits of language and expression are what makes him smile (and stay up at night). The combination of lisp and machine learning put him on the path of always striving to make himself redundant if not outright obsolete.