Top 5 things we hear from data leaders
TL;DR: After talking to 100+ data leaders in 2023, five key challenges emerged: teams stuck in reactive mode instead of driving strategy, low data literacy across organizations, siloed knowledge as teams grow, persistent data quality issues, and the struggle to find signal in the AI noise. The good news? AI might be the key to solving many of these challenges.
Curious to know what’s on the mind of other data leaders? Or curious to know how they’re thinking about AI? We’ve spoken with over 100 data leaders in 2023, so we thought we’d share the top 5 themes we’ve heard. The data leaders that we’ve spoken to range from solo-data teams at 20 person series A startups, all the way to Fortune 100 companies.
1. They want to elevate the role of the data team
Most data leaders feel like their team spends a disproportionate amount of their time “resolving tickets” (whether or not there is a ticketing system), or reactively responding to requests and putting out fires. These range from “Can you quickly pull the sales numbers for California in the past 30 days?” to “The Revenue By Customer dashboard doesn’t seem correct, can you please take a look?” The more the data team handles these requests, the less time they spend on exploratory data analysis that can actually move the business forward, the lower the perceived value of the data team, the lower the budget etc. This quickly turns into a vicious cycle where the team truly does no more than handle tickets, and lack the bandwidth for deeper analysis that actually moves the business forward.
2. They want to raise the level of data literacy across the organization
“If only our marketing team understood what our data actually looks like, they would be able to articulate their asks more clearly.” or “My product team would probably be able to uncover some really valuable insights for the business, but they self-censor and don’t ask for the data because they don’t want to make the ask”: if either of these sound familiar, you’re not on your own. This theme is particularly true in smaller organizations where the data is manageable, but the biggest barrier to entry is the SQL-level of individuals across the team. There’s an extremely strong desire for the data team to not be the gate-keepers, but rather, let anyone within the organization explore data on their own (within certain bounds) and let creativity flourish in every parts of the organization.
3. Knowledge is siloed
The larger the organization the bigger the issue. There comes a point when the data and data infrastructure has grown so much, there has been enough turnover on the team over the years and the team has grown to a certain size, where no one individual understands every part of the data. This is when teams tend to decentralize with embedded analysts in smaller teams and you start seeing dedicated Slack channels to handle specific requests. Knowledge breaks down, communication overhead increases, team efficiency decreases.
4. Recurring data quality issues get in the way of reporting
Within this theme, we hear two sub-themes: 1. The data is just wrong and cannot be used 2. The data is good, but the internal data team uses downstream data from the product team, and this data will constantly shift under their feet. The latter is particularly true in SaaS organizations where data teams typically build reports off of data generated by and powering the product. This issue is typically mitigated using dbt or other software-development best-practices, but can be hard to avoid. The former tends to crop up when a new leader typically steps in and realizes that the data that has been used up to date has been wrong all along or post-acquisitions when all the pieces have not yet been joined.
5. AI is everywhere, it’s hard to separate the signal from the noise
Data leaders are in exciting, but daunting times. AI is rushing ahead, offering many solutions, but also moving so fast that it’s hard to keep up. Existing solutions are trying to retain their foothold by wedging AI in their current solutions, while other solutions have seen their whole value proposition disappear overnight - thus need to reinvent themselves - while still others are rethinking data and analytics from the ground-up in an AI-first world.
At Fabi.ai we feel like there isn’t a more exciting time to be a data leader. We believe that AI is going to completely transform the way data analysts and scientist operate and how analysis is done within the enterprise. AI is going to empower the data experts to move faster on critical analyses that move the business forward while opening the door to less technically-literate, but just as capable, individuals across the organization. In other words, we believe the data literacy will increase across the entire organization and data will be behind every single decision when the data exists. We also believe that AI is going to break down knowledge silos. AI is perfect at cutting through mountains of information and boiling it down to its essence. Solutions that tap into these knowledge bases (wikis, logs, code bases etc.) will quickly 10X the productivity of the data team.
If you’d like to see the future of data analytics, we’d love to show it to you.
