What is big data? What are the characteristics of big data?

Big data refers to a collection of data that cannot be captured, managed and processed with conventional software tools within a certain time range. It requires new processing models to have stronger decision-making power, insight discovery and process optimization capabilities. Massive, high-growth and diversified information assets. Characteristics of big data: 1. Huge amount of data; 2. Diverse data forms and wide range of data sources determine the diversity of big data forms; 3. High speed, that is, rapid data growth and fast processing; 4. Low value density ; 5. High commercial value.

The operating environment of this tutorial: Windows 7 system, Dell G3 computer.

What is big data

Big data (big data), an IT industry term, refers to the inability to use conventional software within a certain time range The collection of data captured, managed and processed by tools is a massive, high-growth and diverse information asset that requires new processing models to have stronger decision-making power, insight discovery and process optimization capabilities.

In the "Big Data Era" written by Victor Meyer-Schonberg and Kenneth Cukier, big data refers to the use of all data instead of shortcuts such as random analysis (sampling survey). Analysis and processing. The 5V characteristics of big data (proposed by IBM): Volume (capacity), Velocity (high speed), Variety (diversity), Value (low value density), and Veracity (authenticity).

Features

• ##Capacity (Volume): The size of the data determines the value and potential information of the data considered;

• Variety: the diversity of data types;

• Velocity: refers to the speed of obtaining data;

• Variability (Variability): hinders the process of processing and effectively managing data.

• Veracity: The quality of data.

• Complexity: The amount of data is huge and comes from multiple channels.

• Value (value): Rational use of big data to create high value at low cost.

What are the characteristics of big data

1. The volume of data is huge

With the Internet With the development of the industry, a lot of data on user network behaviors are generated and accumulated in daily operations. For example, social e-commerce platforms generate orders every day, posts, comments and short videos published by various short videos, forums and communities, emails sent every day, and pictures, videos and music uploaded, etc., the scale of data generated by countless individuals It is very huge, and the data volume has already reached the PB level. If such large-scale data wants to be processed, analyzed, and counted, it needs to have a large enough capacity. Therefore, one of the characteristics of big data is its huge volume.

2. Diverse data forms

The wide range of data sources determines the diversity of big data forms. Any form of data can be useful. Currently, the most widely used is the recommendation system, such as Taobao, NetEase Cloud Music, Toutiao, etc. These platforms will analyze users' log data to further recommend things that users like. Log data is clearly structured data, and there are also some data that are not clearly structured, such as pictures, audios, videos, etc. These data have weak causal relationships and require manual annotation.

3. High speed

The high speed of big data refers to the rapid growth of data and rapid processing. Every day, data from all walks of life is growing exponentially. In many scenarios, data is time-sensitive. For example, search engines need to present the data users need within a few seconds. When enterprises or systems face rapidly growing amounts of data, they must process it at high speed and respond quickly.

4. Low value density

The low value density of big data means that among massive data sources, there are very few truly valuable data, and much of the data may be wrong. It is incomplete and cannot be used. Generally speaking, the density of valuable data in the total data is very low, and refining data is like surfing the sand.

5. High commercial value

Compared with traditional small data, the greatest value of big data is to mine out future trends and trends from a large amount of irrelevant data of various types. Model prediction analyzes valuable data, and through in-depth analysis using machine learning methods, artificial intelligence methods, or data mining methods, new rules and new knowledge are discovered, and applied to various fields such as agriculture, finance, and medical care, so as to ultimately improve social governance, Improve production efficiency, promote the effectiveness of scientific research, and realize its commercial value.

Recommended: "

Programming Video"

The above is the detailed content of What is big data? What are the characteristics of big data?. For more information, please follow other related articles on the PHP Chinese website!