Big Data and Analytics: Benefits, Challenges and Pitfalls


In today’s complex business environment, big data is growing in acceptance and importance.

It is a critical decision-making resource for executives, especially those managing large organisations, who are using big data and analytics to track consumer sentiment, build customer loyalty, gain competitive advantages, and make more effective business decisions.

Governments depend on big data and analytics to support infrastructural developments, improve healthcare services, efficiently manage available natural resources, and mitigate risks and disasters.

Other sectors such as oil and gas, tourism, and pharma are also following suit and joining forces with big data investments. Be it the UAE’s Ministry of Energy’s Oil & Gas Big Data Project or pharma companies leveraging technology to boost insulin production, data has become indispensable to businesses and organisations in many ways.

Key benefits of analytics for enterprises

Businesses can use data to create a customised shopping experience for customers, prompting a website to suggest products that customers are likely to purchase.

Businesses can analyse their competitors in real-time, set prices, make better offers than their competitors, and even analyse a competitor’s negative reviews to determine how they can out-perform them.

Data can also help businesses to optimise their workflow and improve employee performance. By monitoring employee performance, human resources departments can better understand how employees spend much of their day, identifying areas in the workflow where efficiency is lost.

For financial businesses or businesses working with online transactions, careful data monitoring can improve security. Credit card companies are one example of how data can help eliminate fraud threats; by monitoring consumer behaviour, these companies can detect suspicious transactions, flag accounts, and catch (or even prevent) fraud early on.

Also Read: Top 3 Big Data Challenges For Enterprises

Challenges CIOs and IT teams face when deploying analytics

Firms often lack the right data and analytics organisational structure. They do not have the right resources, skills or technology to deal with the changes that a future-oriented data and analytics strategy that is focused on delivering business value brings.

Technologies such as AI can take over specific routine tasks, so employees can work on more-complex problems. However, instead of automating processes and freeing up capacity for future-oriented digital projects, data and analytics leaders find themselves spending time and money on mandatory activities like compliance.

In some cases, central IT departments get detached from the numerous business units they support that include marketing, HR and sales who have their own analytics initiatives. Although more and more business units are embracing analytics, nevertheless they should still involve IT in all data and analytics- related projects, as they bring the expertise and can ensure that all security and compliance policies are being followed.

Also Read: Are You Keeping A Tab On The Cloud? 

Common pitfalls to avoid when deploying analytics

Hyper-focusing on building expensive backend systems: Data analytics should be all about end-users. Companies typically place too much emphasis and focus on building an expensive backend data infrastructure without realising the importance of usability and access.

Existing infrastructure must go: Organisations like to ditch their existing analytical architectures once they commit to a new product, as they believe the older systems cannot be leveraged. However, new analytical solutions can and should be implemented with complementary technical architectures that integrate with existing tools and solutions to enable new use cases.

Complex planning needed: Many companies usually find it challenging to implement data analytics projects because of the assumption that it would involve very complex planning. This, however, can be avoided if they keep it simple. Companies should have a clear objective and be prepared to manage change properly. They can also look for other power users who are multipliers and champions for the benefit of the new system.

The future of analytics market in the Middle East

The analytics market in the Middle East is poised for growth due to the growing adoption and application of big data within multiple industries. The Middle East and Africa retail analytics market is projected to grow at a CAGR of 22 per cent to reach $484 million by 2023, due to the growth of the retail industry and the increasing urbanisation and commercialisation across the region.

The healthcare analytics market in the same region is estimated to reach $1.49 billion by the end of 2025, at a growth rate of almost 19 per cent. The increased adoption of electronic health records over the past few years has further fuelled the growth of analytics in the healthcare sector.

In the GCC, for instance, the governments have already established bold plans to position themselves at the forefront of the digital revolution. But many countries in the Middle East still remain in the strategising phase and are working on developing more concrete applications.

Stephen Gill is the Academic Head of the School of Mathematical and Computer Sciences, Heriot-Watt University Dubai.