What does big data in the book 'Big Data Era' mean?

Big data in the book "Big Data Era" refers to "all data or all data", also known as "huge data", which refers to the amount of data involved that is so huge that it cannot be passed through the current mainstream Software tools can capture, manage, process, and organize information within a reasonable time to help enterprises make more positive business decisions.

The operating environment of this tutorial: Windows 7 system, Dell G3 computer.

Big data in the book "Big Data Era" refers to "all data or all data".

Big data (big data), or huge amount of data, refers to the amount of data involved that is so large that it cannot be captured, managed, and managed within a reasonable time by current mainstream software tools. Process and organize information to help companies make more positive business decisions.

In "The Age of Big Data" written by Victor Meyer-Schoenberg and Kenneth Cukier, big data refers to the use of all data without shortcuts such as random analysis (sampling survey). method) 4V characteristics of big data: Volume, Velocity, Variety, and Value.

History of the development of the concept of big data:

The earliest reference to the term "big data" can be traced back to the open source project Nutch of apache org. At the time, big data was used to describe large data sets that needed to be batch processed or analyzed simultaneously to update web search indexes. With the release of Google MapReduce and Google File System (GFS), big data is no longer just used to describe large amounts of data, but also covers the speed of processing data.

As early as 1980, the famous futurist Alvin Toffler enthusiastically praised big data as "the cadenza of the third wave" in his book "The Third Wave". .

However, starting around 2009, “163 big data” became a popular vocabulary in the Internet information technology industry. The U.S. Internet Data Center pointed out that the data on the Internet will grow by 50% every year and double every two years. At present, more than 90% of the data in the world was generated in recent years. In addition, data does not simply refer to the information people publish on the Internet. There are countless digital sensors on industrial equipment, cars, and electricity meters around the world, measuring and transmitting information about position, movement, vibration, temperature, humidity, and even chemistry in the air at any time. Changes in matter also generate massive amounts of data information.

Conceptual structure of big data:

Big data is just a manifestation or characteristic of the development of the Internet to the present stage. There is no need to myth it or maintain awe of it. , with the backdrop of technological innovation represented by cloud computing, these data that were originally difficult to collect and use have begun to be easily utilized. Through continuous innovation in all walks of life, big data will gradually create more for human beings. value.

Secondly, if you want to systematically recognize big data, you must decompose it comprehensively and carefully. I will start from three levels:

The first level is theory, and theory is recognition. It is the only way to know, and it is also the baseline that is widely recognized and disseminated. I will understand the industry's overall description and characterization of big data from the definition of the characteristics of big data; deeply analyze the preciousness of big data from the discussion of the value of big data; gain insight into the development trend of big data; and start from the special and important issue of big data privacy. Examine the long-term game between people and data from a perspective.

The second level is technology. Technology is the means to embody the value of big data and the cornerstone of progress. I will explain the entire process of big data from collection, processing, storage to result formation from the development of cloud computing, distributed processing technology, storage technology and perception technology respectively.

The third level is practice, and practice is the ultimate value manifestation of big data. I will describe the beautiful scene that big data has shown and the blueprint for its upcoming realization from four aspects: Internet big data, government big data, enterprise big data and personal big data.

Characteristics of the big data concept:

Compared with traditional data warehouse applications, big data analysis has the characteristics of large data volume and complex query and analysis. The article "Architecting Big Data: Challenges, Current Situation and Prospects" published in "Journal of Computer Science" lists several important features that a big data analysis platform needs to have, and analyzes the current mainstream implementation platforms - parallel databases, MapReduce and hybrids based on the two. The architecture is analyzed and summarized, and their respective advantages and disadvantages are pointed out. At the same time, the current research status of each direction and the author's efforts in big data analysis are introduced, and future research is prospected.

The four "Vs" or characteristics of big data have four levels: First, the volume of data is huge. From the TB level to the PB level; second, there are many data types. The web logs, videos, pictures, geographical location information, etc. mentioned above. Third, the processing speed is fast and the 1-second rule can quickly obtain high-value information from various types of data. This is also fundamentally different from traditional data mining technology. Fourth, as long as the data is properly utilized and analyzed correctly and accurately, it will bring high value returns. The industry summarizes it into four "V" - Volume, Variety, Velocity, and Value.

To some extent, big data is the cutting-edge technology of data analysis. In short, the ability to quickly obtain valuable information from various types of data is big data technology. Understanding this is critical, and it’s what drives this technology’s potential to reach so many businesses.

Use of big data concept:

Big data can be divided into fields such as big data technology, big data engineering, big data science and big data application. What people are talking about the most now is big data technology and big data applications. Engineering and scientific issues have not yet been taken seriously. Big data engineering refers to the systematic engineering of planning, construction, operation and management of big data; big data science focuses on discovering and verifying the laws of big data and its relationship with natural and social activities during the development and operation of big data networks.

The Internet of Things, cloud computing, mobile Internet, Internet of Vehicles, mobile phones, tablets, PCs, and various sensors spread across every corner of the earth are all data sources or carrying methods.

Some examples include weblogs, RFID, sensor networks, social networks, social data (thanks to the data revolution in society), Internet text and files; Internet search indexes; call detail logging, astronomy, atmospheric science, genomics , biogeochemical, biological, and other complex and/or interdisciplinary scientific research, military reconnaissance, medical records; photographic archives, video archives; and large-scale electronic commerce.

The role of big data

For general enterprises, the role of big data is mainly reflected in two aspects, namely the analysis and use of data and secondary processing development projects. By analyzing the big data of Xijin Information, we can not only dig out hidden data, but also use these hidden messages to improve our customer base through physical sales. As for the secondary development of data, it is often used in network service projects. By summarizing and analyzing this information, we can develop personalized plans that meet customer needs and create a new advertising and marketing method. What you need to understand here is that combining products and services through big data analysis is not an accident. Those who realize this are often leaders in the data era.

To sum up, the application of big data not only marks the progress of the times, but also inspires people to conduct deeper exploration. In addition, for the research on big data, in addition to the above content, it is also necessary to understand the three characteristics of big data, namely large scale, fast operation speed and data diversity. By studying these three aspects, it is not only easier to observe the nature of the data, but also conducive to the effective operation of the software processing platform.

For more related knowledge, please visit the FAQ column!

The above is the detailed content of What does big data in the book 'Big Data Era' mean?. For more information, please follow other related articles on the PHP Chinese website!