Overall, two-thirds of enterprises cite security as their top data management challenge. This is according to a new 451 Research survey, which also discovered that many companies are concerned about data compliance.
DataOps is on the rise
451 Research surveyed 150 representatives from US organisations with over 1,000 employees and a minimum of 2 petabytes of managed data. Respondents had a thorough understanding of their company's data management strategy.
Approximately a fifth of respondents said that they had already integrated DataOps into their culture. Meanwhile, 37% were in the process of adopting DataOps and 34% were still defining their strategy.
Last year, 73% of companies planned to invest in DataOps according to a survey commissioned by Nexla. Nevertheless, half of the survey respondents lacked a sufficient number of backend data engineers on their data teams.
Company concerns regarding data
In the 451 Research survey, the majority of respondents said that security was their primary data management issue. Although, 40% said that compliance requirements were also a cause for concern.
"Data security is a perennial concern, of course, while addressing data quality is essential if organisations are to ensure they are making decision based on accurate data," the report stated. Respondents also cited the governance of distributed data as one of their problems.
From on-premises and multiple clouds, data can exist in various locations and on multiple platforms. However, the distribution of data across multiple platforms and locations complicates a company's ability to understand what data it currently possesses.
Inability to handle data
As a result, many enterprises are unable to harness the true power of their data. This is particularly problematic considering that almost a third of organisations said that their data grows at a rate of 100 gigabytes to 500 gigabytes per day.
In addition to this, 19% said that their data grew between 500 gigabytes and 1 terabyte per day. 12% of the respondents said their growth rate was between 1 terabyte and 2 terabytes, while 13% said it was two terabytes per day.
As a consequence, it is becoming increasingly more expensive for companies to store and analyse data on-premise. Despite this, more companies are beginning to adopt DataOps in order to efficiently manage their data.
According to the report, companies are reporting improved agility and efficiency since integrating DataOps. However, respondents also stated that the security and compliance associated with DataOps was their primary driver for investment.
What toolkits do data scientists require to be efficient? Listen to our latest podcast with Charles Givre, advisory board member and Vice President, Lead Data Scientist and CSO/COO of Deutsche Bank
