What are the three sources of big data?

What are the three sources of big data?

1. Transaction data, including POS machine data, credit card swipe data, etc.;

2. Human data, including emails, documents, pictures, and through WeChat, blogs, Twitter, etc. The generated data stream;

3, machine and sensor data, such as data from sensors, meters and other facilities.

What is big data?

Gartner, a research organization for “big data”, gives this definition. "Big data" requires new processing models to have stronger decision-making power, insight discovery and process optimization capabilities to adapt to the massive, high growth rate and diversified information assets.

The definition given by McKinsey Global Institute is: a data collection that is so large that its acquisition, storage, management, and analysis greatly exceed the capabilities of traditional database software tools. It has massive data scale, rapid It has four major characteristics: data flow, diverse data types and low value density.

The strategic significance of big data technology lies not in mastering huge data information, but in professional processing of these meaningful data. In other words, if big data is compared to an industry, then the key to making this industry profitable is to improve the "processing capabilities" of data and achieve the "value-added" of data through "processing".

Technically, the relationship between big data and cloud computing is as inseparable as the two sides of the same coin. Big data cannot be processed by a single computer and must use a distributed architecture. Its characteristic lies in distributed data mining of massive data. But it must rely on distributed processing, distributed database and cloud storage, and virtualization technology of cloud computing.

With the advent of the cloud era, big data (Big data) has also attracted more and more attention. The analyst team believes that big data is generally used to describe the large amounts of unstructured and semi-structured data created by a company, which would take too much time and money to download to a relational database for analysis. Big data analytics is often associated with cloud computing because real-time analysis of large data sets requires frameworks like MapReduce to distribute work to tens, hundreds, or even thousands of computers.

Big data requires special techniques to efficiently handle large amounts of data over a tolerable amount of time. Technologies applicable to big data include massively parallel processing (MPP) databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

Recommended tutorial: "PHP"

The above is the detailed content of What are the three sources of big data?. For more information, please follow other related articles on the PHP Chinese website!