Awesome CI/CD Data Pipelines for Distributed Data-Sources
Please log in to watch this conference skillscast.
The foundational principle of microservices is to segregate functionality into abstracted units. What if you want to empower your business with self-service access to almost every data source?
Bringing data together after painstakingly designing separation is counter-intuitive. Would you even have any of the benefits of a micro-service architecture anymore if you have one big fat database with all your data in it?
Chris will explain how, at Landbay, we maintain totally different data stores and built an awesome downstream data-warehouse reporting solution with a robust data pipeline. He will cover the Dos & Don’t as he talks through the evolution of the data pipeline architecture, touching on many technologies as he goes, from Redshift, Glue & Apache Spark to Stitch, dbt, Great Expectations & binlogs on the cloud.
YOU MAY ALSO LIKE:
- Lightning Talk: CDC testing is dead – Long Live Swagger! (SkillsCast recorded in November 2017)
- Fast Track to DevOps Fundamentals (in London on 23rd - 24th September 2019)
- CloudNative London 2019 (in London on 25th - 27th September 2019)
- Keynote by Ryn Daniels on Designing Sustainable Ops Cultures (in London on 21st August 2019)
- Software Architecture for Developers Workshop (in London on 9th - 10th September 2019)
- Functional Concurrency in .NET with C# and F# (in London on 9th - 10th September 2019)
- ProgNET London 2019 (in London on 11th - 13th September 2019)
- You got served: How Deliveroo improved the ranking of restaurants (in London on 27th August 2019)
- XP as we would do it today (in London on 27th August 2019)
- Genetic Programming for RL on Minecraft (SkillsCast recorded in August 2019)
- You know Selenium? So you think you know Appium? (SkillsCast recorded in August 2019)
Thanks to our sponsors
Awesome CI/CD Data Pipelines for Distributed Data-Sources
Chris Burrell
Chris joined Landbay, a peer-to-peer fin-tech start up as the lead software engineer and then Head of Technology. With 10+ years’ of commercial experience, he designed, developed and continues to deliver the microservice architecture that was built from the ground-up to 50+ microservices. Prior to working with Landbay, his most recent roles involved helping architect and develop some of the largest UK customer-facing platforms in the UK, such as the EE online shop, Three, The AA, Vodafone, etc. Outside of his work life, Chris develops some open-source Bible study software.
Please log in to watch this conference skillscast.
The foundational principle of microservices is to segregate functionality into abstracted units. What if you want to empower your business with self-service access to almost every data source?
Bringing data together after painstakingly designing separation is counter-intuitive. Would you even have any of the benefits of a micro-service architecture anymore if you have one big fat database with all your data in it?
Chris will explain how, at Landbay, we maintain totally different data stores and built an awesome downstream data-warehouse reporting solution with a robust data pipeline. He will cover the Dos & Don’t as he talks through the evolution of the data pipeline architecture, touching on many technologies as he goes, from Redshift, Glue & Apache Spark to Stitch, dbt, Great Expectations & binlogs on the cloud.
YOU MAY ALSO LIKE:
- Lightning Talk: CDC testing is dead – Long Live Swagger! (SkillsCast recorded in November 2017)
- Fast Track to DevOps Fundamentals (in London on 23rd - 24th September 2019)
- CloudNative London 2019 (in London on 25th - 27th September 2019)
- Keynote by Ryn Daniels on Designing Sustainable Ops Cultures (in London on 21st August 2019)
- Software Architecture for Developers Workshop (in London on 9th - 10th September 2019)
- Functional Concurrency in .NET with C# and F# (in London on 9th - 10th September 2019)
- ProgNET London 2019 (in London on 11th - 13th September 2019)
- You got served: How Deliveroo improved the ranking of restaurants (in London on 27th August 2019)
- XP as we would do it today (in London on 27th August 2019)
- Genetic Programming for RL on Minecraft (SkillsCast recorded in August 2019)
- You know Selenium? So you think you know Appium? (SkillsCast recorded in August 2019)
Thanks to our sponsors
Awesome CI/CD Data Pipelines for Distributed Data-Sources
Chris Burrell
Chris joined Landbay, a peer-to-peer fin-tech start up as the lead software engineer and then Head of Technology. With 10+ years’ of commercial experience, he designed, developed and continues to deliver the microservice architecture that was built from the ground-up to 50+ microservices. Prior to working with Landbay, his most recent roles involved helping architect and develop some of the largest UK customer-facing platforms in the UK, such as the EE online shop, Three, The AA, Vodafone, etc. Outside of his work life, Chris develops some open-source Bible study software.