The biggest initial hurdle to success with Big Data isn’t technical - it’s management. Your data engineering project’s initial success is predicated on your management team correctly staffing and resourcing it. This runs opposite to how most data engineering teams are started and run. If you just choose the best technologies, things will just fall into place. They don’t and that’s a common pattern for failure.
But how do you correctly do something that’s so new? This could be your team’s first data engineering project. What should the team look like? What skills should the team have? What should you look for in Data Engineer (because you’ll probably have to hire a Software Engineer and train them)? What are some of the management pitfalls?
In this talk, we will cover the most common reasons why data engineering teams fail and how to correct them. This will include ways to get your management to understand that data engineering is really complex and time consuming. It is not data warehousing with new names. Management needs to understand that you can’t compare a data engineering team to the web development team, for example.
Jesse will share the stories of teams who haven’t set up their data engineering culture correctly and what happened. Then, Jesse will talk about the teams who’ve turned around their culture and how they did it. FInally, Jesse will share the skills that every data engineering team needs.
YOU MAY ALSO LIKE:
- YOW! Data 2022 (Online Conference on 1st - 2nd June 2022)
- YOW! London: In-Person (in London on 22nd - 23rd September 2022)
- YOW! London: Online (Online Conference on 22nd - 23rd September 2022)
- Foundations of Data Teams (SkillsCast recorded in May 2021)
- Introduction to Knowledge Graphs (In-Person Workshop) with Howard Knowles (in London on 13th June 2022)
- Introduction to Knowledge Graphs (Online Workshop) with Howard Knowles (Online Workshop on 21st July 2022)
- Shift to Data as a Product, Leverage In-Business Expertise to Scale Analytics (Online Meetup on 22nd May 2022)
- Getting Geospatial Data on The Web (SkillsCast recorded in February 2022)
- Deep Learning with F#: An Experience Report (SkillsCast recorded in October 2021)