Have you ever made these fatal mistakes in AI projects?

​Translator|Bugatti

Reviewer|Sun Shujuan

Since data is the core of artificial intelligence (AI), AI and machine learning (ML) It’s no surprise that systems need enough good data to learn. Large amounts of high-quality data are generally required, especially for supervised learning methods, to properly train an AI or ML system. How much data is required depends on the model of AI being implemented, the algorithms used, and other factors such as internal data and third-party data. For example, neural networks require large amounts of data to train, while decision trees or Bayesian classifiers do not require as much data to obtain high-quality results.

So, you may think that the more data, the better, right? Please think again. Organizations with large amounts of data (even exabytes of data) realize that having more data does not solve the problem as expected. Indeed, with more data comes more questions. The more data you have, the more data you need to clean and prepare, the more data you need to label and manage, the more data you need to secure, protect, reduce bias and other measures. When you start increasing the amount of data, small projects can quickly turn into large projects. In fact, large amounts of data often kill projects.

Clearly the missing step between identifying a business problem and organizing data to solve that problem is determining what data is needed and how much of it is actually needed. You need enough data, but don’t have too much: no more, no less, just right. Unfortunately, organizations often jump into AI projects without understanding the data. Organizations need to answer many questions, including figuring out where the data is, how much data it already has, what state it is in, which characteristics of the data are most important, internal and external uses of the data, data access challenges, requirements to enhance existing data, and other key factors and questions. Without answering these questions, AI projects may fail or even drown in a sea of ​​data.

1. Better understand the data

In order to understand how much data you need, you must first understand where the data is in the structure of the AI ​​project s position. One visual way to help us understand the increasing value we get from data is the "DIKUW Pyramid" (sometimes called the "DIKW Pyramid"), which shows how the data foundation can be transformed through information, knowledge, understanding and wisdom. Help get greater value.

With a solid data foundation, you can gain deeper insights at the next layer of information, which can help you answer fundamental questions about that data. Once you've made basic connections between data to gain information insights, you can find patterns in that information and understand how the pieces of information connect together to gain deeper insights. Organizations can gain more value by building on the knowledge layer and understanding why these patterns occur, helping to understand the underlying patterns. Finally, you can get the most value from information at the intelligence level by deeply understanding the cause and effect of information decisions.

This recent wave of AI focuses most on the knowledge layer, as machine learning provides insights to identify patterns on top of the information layer. Unfortunately, machine learning hits a bottleneck at the understanding layer, because finding patterns is not enough to make inferences. We have machine learning, but we don’t have machine reasoning to understand why patterns occur. You see this limitation every time you interact with a chatbot. While machine learning-based natural language processing (NLP) is very good at understanding human speech and inferring intent, it encounters limitations when trying to understand and reason. For example, if you ask your voice assistant if you want to wear a raincoat tomorrow, it doesn't understand that you're asking about the weather. It's up to humans to provide this insight to machines because the voice assistant has no idea what rain actually is.

2. Stay data aware to avoid failure

Big data has taught us how to handle large amounts of data. Not just how the data is stored, but how all that data is processed, manipulated and analyzed. Machine learning adds even more value by processing the different types of unstructured, semi-structured or structured data that organizations collect. Indeed, this recent wave of AI is actually a wave of big data-driven analytics.

But it’s for this very reason that some organizations are taking a big hit when it comes to AI. Rather than running AI projects from a data-centric perspective, they focus on the functional aspects. To navigate AI projects and avoid fatal mistakes, organizations must better understand not only AI and machine learning, but also the several “Vs” of big data. It’s not just about how much data there is, but also about the nature of the data. Some of the V’s of big data include:

• Quantity: The absolute amount of big data owned.
• Speed: The speed at which big data changes. Successfully using AI means applying AI to high-speed data.
• Diversity: Data can come in many different formats, including structured data like databases, semi-structured data like invoices, and unstructured data like emails, images, and video files. Successful AI systems can handle this diversity.
• Authenticity: This refers to the quality and accuracy of the data and how much you trust that data. Garbage in, garbage out, especially in data-driven AI systems. Therefore, successful AI systems need to be able to handle widely varying data quality.

With decades of experience managing big data projects, organizations that are successful in AI have primarily been successful in big data. Organizations that have seen AI projects fail often approach AI problems with an application development mindset.

3. Too much wrong data and insufficient correct data are killing AI projects

Although the AI ​​project started correctly, the lack of necessary data, lack of understanding, and lack of Solving real problems is killing AI projects. Organizations continue to move forward without a true understanding of the data and data quality required, which creates real challenges.

One of the reasons organizations make this data mistake is that they don’t have any real approach to AI projects other than using agile or application development methodologies. Yet successful organizations have realized that using a data-centric approach includes data understanding as the first stage of a project approach. The CRISP-DM approach, which has been around for more than 20 years, specifies data understanding as the next step after business needs are identified. Based on CRISP-DM and combined with agile methods, the Cognitive Project Management with AI (CPMAI) approach requires data understanding in the second phase. Other successful approaches also require understanding the data early in the project, because AI projects are, after all, data projects. How do you build a successful program on data if you approach it without understanding the data? This is definitely a fatal mistake you want to avoid.

Original link: https://www.forbes.com/sites/cognitiveworld/2022/08/20/are-you-making-these-deadly-mistakes-with-your -ai-projects/?sh=352955946b54

The above is the detailed content of Have you ever made these fatal mistakes in AI projects?. For more information, please follow other related articles on the PHP Chinese website!