Adapt or fall behind: a review of the Databricks executive survey on AI and its use

Andras Nagy
Andras Nagy
20 Nov 2023 · 4 min read

The race is now on, and enterprises are gearing up to prepare their organizations to be ready for AI. MIT Technology Review Insight recently teamed up with Databricks to survey 600 C-suite executives to learn how tech leaders think about AI. 

The result? “Laying the foundation for data- and AI-led growth” is a dense read packed with insights, possibly leaving you wondering how you can AI-proof your organization (jump to the bottom to see the full report). This article shares some of the findings and how DATAPAO can be a partner on the road to adoption.

Get ready for AI adoption with data as a sidekick

Worldwide IT spending is forecast to more than double from 4.3% to 8.8% in the 2024 Q1, and most of the investments will go into AI and data. While the latter will see most of the money spent on modernization, the former, as expected, will be about adopting the new technology.

With most sectors rating their industry adoption speed as “fast” or “very fast”, two-thirds of tech execs believe that the race is not only on but not joining will catch up to you at a similar speed. 

This probably confirms your suspicion: the time is now. But you may wonder: how am I to do that? Rushing to adopt is expected, but there are a few action points you need to take care of.

Building a solid data platform

“Data infrastructure that worked five years ago doesn’t work now. For example, when smaller, we could serve data from an operational database. But when we reached this kind of scale, we had to build a performance-oriented architecture to be able to store data more efficiently in a cost-effective manner while also serving customer needs.” That’s Murali Brahmadesam, Chief Technology Officer and Head of Engineering at Razorpay.

The skinny? Without a reliable data infrastructure, you might find yourself in a pickle. Adopting AI will pose new challenges to your organization: mammoth data processing demands and governance matters require you to have a robust system. 

Modernizing legacy software

If you work at an enterprise, your data infrastructure is probably fleshed out. For quite a long time. That’s why adopting AI is tricky: many services require real-time data processing, something legacy tools can’t tackle – this means your first job is to modernize your data architectures.

The next step is threefold. You need to simplify your existing architecture, consolidate disparate data, and unify data governance models (as you might have multiple ones co-existing). But let the tech leaders tell you the same with more illustration:

Modernizing your data infrastructure based on priorities

Enterprises use legacy software – we know that. But there’s more. The problem is that they use many legacy software. Just take a glimpse:

The larger your organization, the more the systems

If your company eclipses the 1-billion threshold, it’s pretty certain that you at least use 10 different data, AI, and ML systems. If that’s the case, you probably also have some housekeeping to do.

ADOPTING AI DOESN’T HAVE TO BE A HASSLE

FROM DESIGN TO IMPLEMENTATION, OUR EXPERTS AT DATAPAO HELP YOU BUILD A SYSTEM THAT WITHSTANDS EVEN THE HEAVIEST WORKLOAD.

Harvesting your data

In case you already have a trusty data infrastructure in place (and you don’t need to revamp a legacy system either), there’s still much you can do to become a data-accelerated organization. By exploring your data use cases, we can identify data sources that are not yet utilized – this helps us find the low-hanging fruits and create value with data. 

To keep it simple, this can be done in three ways: Data-for-Automation (DfA) to reduce friction, Data-for-Decision (DfD) to improve decision-making efficiency, and Data-for-Product (DfP) to enhance your product’s functionality. 

Investing in talent

39% of survey respondents think the best data-related investment they can make is training and upskilling their staff. This can be a struggle without proper guidance – this is how you can grow up for the task:

  • Augment organizational resources. If you lack the in-house talent or simply don’t have the resources to recruit, external consultants can join any project to help you out with the execution.
  • Enablement. Have a team that needs to get up to speed regarding data governance? Want your leaders to be more data-aware? With trainings delivered by external parties, your talent will grow up to the task fast.

Investing in talent is money well spent. It needs some time to see the positive effects, but it will pay dividends and lay the groundwork for a solid base before adopting AI solutions.

Keeping your focus on data governance and quality

AI is becoming ubiquitous. This automatically means that your data governance framework needs to be foolproof. Yes, you need to get data accuracy and integrity right, but your governance must also take care of data privacy and security. 

No wonder 60% of tech leaders think that it’s “very important” to have a single governance model for data and AI – if investing in talent is the groundwork for adopting AI, setting up a proper governance framework is the engine that ensures everything works fluently.

Towards a brighter, AI and data-lead future

AI isn’t just a buzzword that keeps tech leaders busy for the next few quarters. It has a serious upside, potentially transforming entire industries by reaching new heights of productivity. Similarly, the downside of not getting AI right in your organization can have a vast impact across functions. 

Just remember: a functioning AI integration always starts with a top-notch data platform.
And to read the entire report, check out the Databricks article here.