Assessing the business impact of artificial intelligence

AI is in a period of transformation, both as a technology and in how it is used. More and more companies are taking AI pilot projects out of test labs and deploying them at scale, and some are seeing huge benefits. Despite the uncertainties surrounding AI, ignoring its potential can lead to companies doing business in old ways and risk going bankrupt.

However, for many companies, deriving value from AI can be difficult to predict. Their models may not be tuned; the training data set may not be large enough; customers may have doubts, concerns about bias, ethics and transparency, and more. Bringing an AI strategy forward before it is ready to move the AI ​​program into production, or before its results have been properly reviewed, could cost the company money, or worse, be detrimental to the business.

So how do you know whether an AI project will change or destroy your company? Without a direct ROI number, companies have to find creative ways to determine this. Here’s how IT leaders and industry insiders measure the value of AI.

Mature vs. Breakthrough Technologies

Measuring the business value of any initiative or technology is not always a linear calculation, and AI is certainly no exception, especially given maturity and business potential. Empirically proven and predicted variables—such as data mining, cost savings and training, and the ability to invest in and promote new uses—influence decisions about acceptable ROI, but give a certain level of trust to the technology, whether emerging or For mature technologies, they are all crucial.

Chris Mattmann, CTO and Innovation Officer of NASA’s Jet Propulsion Laboratory, said that some AI case applications are already very mature. Take automating business processes as an example.

He said: "Every company has boring stuff, and we do too. So we automate a lot of things, like processing tickets, searching, data mining, using AI to look at contracts and subcontracts."

JPL does this by using commercialized technologies, including DataRobot and Google Cloud, Mattmann said. To determine whether a particular technology is worth investing in, companies consider whether it will save cost, time, and resources. "It's already mature, so it should be able to demonstrate that."

For technologies that are at a medium level of maturity, JPL looks at whether the technology has the ability to implement new capabilities and at what cost. He said: "We are going to Mars, for example, and there is a thin pipe for deep space telecommunications. And today, there is enough bandwidth to send about 200 pictures a day from Mars to Earth.

The wonderful people we sent The Mars rovers have pea-sized brains, and they're running iPhone 1 processors. We only leave things in space that have been radiation-hardened, and we trust them to withstand the deep space environment. We know that the chips that perform well are the ones that are older chip, so we're not doing advanced AI or ML on the rover.

But the Ingenuity helicopter was originally intended as a technology demonstration and not the core of the mission, and it's powered by a Qualcomm Snapdragon processor, the AI ​​chip. We proved that it's possible to have newer chips and do more AI."

Here, AI will enable new use cases that are currently unachievable. For example, instead of sending back 200 images a day, the rover could use AI to analyze the images and send a million text descriptions to Earth; for example, there is a dry lake bed in a specific direction, and we can get better results through text than Stage more visibility.

Finally, for cutting-edge experimental AI technologies, success will be measured by whether they allow new science to be done, and new papers to be written and published.

Mattmann said: "There is a cost to train and build models. Companies like Google and Microsoft have ready access to massive training data, but the JPL data set is difficult to obtain and requires PhD-level experts to analyze and Labeling. At NASA, our cost to train new AI models is 10 to 20 times higher than in the commercial industry.”

Here, the emergence of new technologies allows NASA to create AI models with fewer human labels.

AI Measurement and Scope of Impact

When there is no direct way to measure the business impact of an AI project, companies look at related key performance indicators (KPIs) Mining data. These proxy variables are often related to business goals, including customer satisfaction, time to market, or employee retention.

Atlantic Health System is a typical example. Sunil Dadlani, the company's senior vice president and CIO, believes that patients at his company are at the center of every decision. So in many ways, the return on investment in AI is measured by observing improvements in patient care. Those patient-centered metrics include shorter hospital stays, faster treatment times, faster insurance eligibility verification and faster prior insurance authorization, he said.

Another project involves using AI to support radiologists in reviewing scans. One of the KPIs is the frequency with which radiologists are alerted to potentially abnormal findings. “As of April 2022, 99% of our radiologists reported using AI to analyze more than 12,000 study reports,” Dadlani said, adding that this has triggered nearly 600 alerts so doctors can address potentially serious issues as quickly as possible. ”

Richard Davis, a partner in the management consulting, business and technology transformation team of RSM, the fifth largest accounting firm in the United States, believes that at RSM, AI investment follows two closely connected paths: one is productivity and analysis Tools can help employees work better; second, customers use the same or similar tools.

For example, when working with a client, RSM may be asked to pull data from multiple systems (accounting, sales and marketing, HR, logistics) and consolidate everything into a single pane. AI can help speed up the process, Davis said, and AI can then be used to identify how work flows through these systems and where potential challenges and roadblocks may be.

So how does a company know if its AI is heading in the right direction?

Davis declined to provide details on RSM's investment in AI plans or ROI, but said: "One, we can measure the usage of the tool very clearly. Over time, we hope to see What you get is more efficient delivery of engagement."

He also said that increased engagement should lead to increased productivity. So if it used to take a week to get something done, now the goal might be to get it down to a day.

Focus on business benefits

Measuring the success of AI can also be subjective. Evaluating an AI project is as much an art as developing AI itself, says Eugenio Zuccarelli, an AI research scientist at MIT and a data scientist in the retail industry.

Still, it’s important to be able to explain the impact of AI on the business, Zuccarelli said. He said: "KPIs should not be set around the model itself, but should be set around the business and people, which should be the ultimate goal of the project." Otherwise, choose one that seems successful but does not actually translate into an effective impact on the company. Technical indicators are too easy.

Zuccarelli, who has also held data science roles at companies including BMW and Tesla, also warned against measuring progress in isolation. For example, if an AI project aims to improve something that is already improving for other reasons, a control group is needed to determine how much of the improvement is actually caused by the AI.

Vladislav Shapiro, who has many years of experience in the financial services industry, said that other valuable KPIs for AI projects may be to reduce false alarms or automatically remove excessive privileges. Shapiro is the founder of Costidity, a consulting group specializing in IT security and identity governance management.

In a recent AI-driven security deployment, false positive rates were reduced threefold, and many previously manual processes were automated, he said.

“When you show these numbers to a company’s senior management, they understand that all of the above measures reduce the risk of a breach and increase accountability and governance,” he said.

Progressive measurement of success

Sanjay Srivastava, chief digital strategist at global professional services company Genpact, said that cost reductions caused by automation are the simplest and clearest way to demonstrate the economic benefits of AI. The way. But AI can also facilitate new revenue streams and even revolutionize a company’s business model.

For example, with the help of AI, an aircraft engine manufacturer found that it could better predict failures and improve logistics, so it could start providing engine services. He said: "For the end consumer, buying flight miles is better than buying the engine itself. This is a new business model that changes the way the company operates because it is supported by AI technology. At the same time, it is good for the business The impact is also clear."

So, in order to justify the investment in AI over that time, this particular manufacturer needs this long-term goal, but translates it into something that can be measured in other ways. short-term projects.

He also said, "Instead of saying that in ten years we will change the industry, in the first year we will start to look at what parts we need to stockpile. You haven't changed the flight mileage of the industry, you just said , we need the right parts in the right quantities. This is a year-long project to optimize warehouse systems and reduce investment in inventory."

In addition to supply chain optimization, other short-term progress measures include customer satisfaction.

He said: "If the plane is stuck in Mumbai for five days waiting for a part, the customer will have a bad feeling."

Alignment with strategic vision

There is also the reality that some AI projects may impact losses in the short term, but remain important and transformative in the long term. For example, companies using robots to provide customer service can solve some monotonous tasks. Gartner analyst Whit Andrews said: "But chatbots can also have drawbacks. Because some people are good at upselling and want to interact with people, companies may not want to use chatbots."

This comes back to you What kind of company do you want to be? "At some point you have to ask yourself if your company is one where, for example, if a delivery is messed up, the customer can call and ask what exactly went wrong, and then you interact with them directly to try to fix the problem after the delivery. Market to them every month."

If a company is committed to AI-driven transformation, backed by measurable ROI, and has a customer-centric vision, it may be overlooking the direct affect revenue metrics and instead focus on other potentially more meaningful metrics.

Andrews said: "A more automated company may be more successful because it is increasing market share. But you can develop your data so that you can reach them at a time that is more relevant to them. . If there's something you can point to and say, logical thinking tells us, it will make our customers happier and our employees more successful, then let's make it happen."

Source: www. cio.com​

The above is the detailed content of Assessing the business impact of artificial intelligence. For more information, please follow other related articles on the PHP Chinese website!