Data Management Is Becoming More Important And Challenging

Data-Management-Is-Becoming-More-Important-And-Challenging

“More organisations are turning to data disciplines that are maturing around and directly related to AI and ML to help manage, optimise, and analyse increasing data flows,” says, Bill Waid, General Manager, Decision Management, FICO.

In this interview, Waid talks about why organisations need to bring in data, map it, and extract value out of it for successful digital transformation and how alternative data enhances the accuracy of consumer credit profiles

Excerpts from the interview:

Is data management still a fundamental challenge? Why?

Data management is becoming more important and challenging. Ninety per cent of the world’s data was created in the last two years, with no signs of slowing down. As more and more organisations embark on their digital transformation journeys, they will need to have the ability to bring in data and effectively move it, map it, operationalise it, and extract value out of it.

To do this, organisations must provide a holistic view of internal and external data and end-to-end data integration, transformation, and management tools. In addition to streamlining business processes and reducing waste, AI solutions will create more data and inform customised digital experiences, relevant offers, and features/products that customers want.

We’re seeing more organisations turn to data disciplines that are maturing around and directly related to AI and ML to help manage, optimise, and analyse increasing data flows.

How does alternative data enhance the accuracy of consumer credit profiles?

The field has expanded over the last few years regarding using alternative data in credit risk assessments. The use of alternative data is being driven by competitive forces and a growing imperative for a more inclusive economy. Effective alternative data can give a B2C organisation an essential edge over its peers. Assessing this value and comparing it to the cost of data acquisition is a maturing discipline in many organisations, but it’s becoming an increasingly important trade-off to factor into customer-centric data strategies.

An estimated 3 billion adults worldwide don’t have credit and credit records. Responsibly engaging that market in financial services is a priority for many lenders.

However, there are now a variety of sources of alternative data that lenders can leverage to inform credit decisions of potential customers, including

  • Transaction Data (sourced internally)
  • Telecom / Utility / Rental Data
  • Social Profile Data
  • Clickstream Data
  • Audio and Text Data
  • Social Network Analysis
  • Survey / Questionnaire Data

These data sources can add predictive value on margin to credit risk models based on traditional data and demonstrate a consumer’s ability to manage their finances and credit repayment trends.

What are the current technological trends and challenges in the analytics sector?

Incorporating analytics into smart applications and business processes is exploding. It’s enabling an unprecedented level of automation in the back, mid, and front office. This, of course, includes AI, ML, and all the techniques and innovations arising around a massive level of global investment, but we’re still on the frontier. R&D is advancing faster than practitioners can keep up, and the nature of the innovation remains out of mainstream reach, particularly in intensive regulatory environments.

There aren’t a lot of organisations that have integrated a sufficiently diverse architecture for analytic development and execution into end-to-end digital intelligence. For a business that runs from data management and ingestion through data transformation and feature engineering into AI, ML and traditional analytic models and then into the decision with integrated orchestration and queue management for human or virtual agent intervention along the way.

Few have addressed the business KPI monitoring, and ML optimised instrumentation can wrap these workloads into composable, semi-autonomous business services. That’s a challenging problem to solve, and while some are on various stages of the journey, we have a lot of work to do.

The major challenge organisations face is transparency to full lineage, explainability, and the intricate dependency tree. The level of interactive complexity that needs to be managed in the design-time environments is many times more dynamic than in the operational environment. As a case in point, I spoke to a Chief Risk Officer who knows he needs to get his pricing models into compliance to avoid a hefty fine. Still, the dependency impacts inside his operating environment are so opaque that he has decided to pay the fine to buy his organisation the time it needs to unwind and map the dependencies that he needs to have confidence in approving a pricing change that won’t cost his business more than the fine. In all our enthusiasm over advances in AI and ML, we sometimes lose sight of the real challenges we’re facing in the market every day to make it all come together.

FICO is focused on regulatory compliance in applying AI and ML. The ethical, safety, and fairness concerns associated with a lack of caution create enormous legal vulnerabilities, business risk, and societal consequences. Senior leadership and boards must understand and enforce immutable AI model governance to drive the responsible use of AI in organisations. They need to establish governance frameworks to monitor AI models to ensure the decisions they produce are accountable, fair, transparent, and responsible.

Business scenario simulation can help us with these types of complex problems. They give the power to safely hypothesise and test “what if” scenarios and use the resulting insights to experiment with pain points in the business and then pivot strategies to achieve higher performance. You can see past the day-to-day minutiae and visualise the long-term impacts of important changes – both upstream and downstream.

An applied intelligence approach enables organisations not just to gain analytic insights but operationalise these insights in operational decisions to create business outcomes. More organisations adopt an applied intelligence platform, enabling business users to lead innovation and collaborate more effectively with data science. IT to optimise customer journeys with AI-powered intelligence. Doing so allows organisations to gain and operationalise a richly contextualised view of the customer, apply the appropriate AI and advanced analytic techniques to gain competitive insights, use these findings to make better business decisions, and then put these decisions into action.

How are enterprises using a data mesh approach? Is the trend gaining traction?

Data has always been a highly contextual asset. As the data landscape expands and as features continue to become critical resources for our businesses, the trend to deeper contextualisation is unavoidable. To the extent that data mesh architectures help us draw lineage between knowledge experts and the deepening context of data, it is a natural fit and will continue to gain relevance. The broader trend is how organisations are flexing hard to become more data-driven and data-centric. As that happens, we need to virtualise data across the silos in organisations, and data mesh concepts help that a great deal, but there are many ways of looking at it, which are equally useful. We need to stay focused on solving the problem and be careful of over-rotate on a particular technique.

As data proliferate at ever-increasing velocity, more organisations will turn to strategies like data mesh and platform approaches to improve data management and governance holistically across the business. Additionally, the self-service approach data mesh offers will further empower business users to capture, analyse, and manage data in a tailored fashion.

How do you leverage AI, ML, automation, and other emerging technologies in your operations?

AI has been used extensively within FICO for more than 30 years. Our fraud modelling group pioneered much of our work in the AI space, obtaining some of the earliest patents on neural nets to detect fraudulent credit card transactions. This happened because the fraud use case is very well suited to AI, which drives highly predictive models with low false-positive rates even in a significant “class imbalance” between the fraud/non-fraud outcomes being predicted.

FICO’s core business is in helping organisations first assess risk and then make risk/reward-based decisions informed by advanced analytics. FICO has been developing and operationalising AI and ML for decades, developing many of the tools and methods that make AI effective in consumer and small business credit decisioning and fraud detection, from both transaction-oriented (payments) and customer/account-oriented (identity theft, account takeover) perspectives.

What data science and analytics podcasts do you listen to?

I stay up to speed through reading more than listening. I subscribe to The Sequence, to some relevant subscriptions from Medium, and I find McKinsey’s emails useful to stay connected with trends in business. They provide a lot of feedback on industrial applications of AI and ML. I also occasionally find the room to plug into industry seminars about Data and Model Ops, where I find that iron sharpens iron.

If you liked reading this, you might like our other stories

Datatechvibe Explains: Data Mesh
When The Chips Are Down