Table of Contents
1. Preliminary use of NumPy
2. Use of Matplotlib package – graphical data processing
3. Deep learning theoretical method – similarity calculation (can be skipped)
1. Similarity calculation based on Euclidean distance
2. Similarity calculation based on cosine angle
4. Visual display of data statistics (taking precipitation in Bozhou City as an example)
Quartiles of data
Home Backend Development Python Tutorial In-depth understanding of Python data processing and visualization

In-depth understanding of Python data processing and visualization

Mar 21, 2022 pm 05:43 PM
python

This article brings you relevant knowledge about python, which mainly introduces related issues about data processing and visualization, including the preliminary use of NumPy, the use of Matplotlib package and data statistics Visual display, etc. I hope it will be helpful to everyone.

In-depth understanding of Python data processing and visualization

Recommended learning: python tutorial

1. Preliminary use of NumPy

Tables are general representations of data form, but it is incomprehensible to the machine, that is, it is unrecognizable data, so we need to adjust the form of the table.
The commonly used machine learning representation is a data matrix.
In-depth understanding of Python data processing and visualization
We observed this table and found that there are two types of attributes in the matrix, one is numeric type and the other is Boolean type. So we will now build a model to describe this table:

#  数据的矩阵化import numpy as np
data = np.mat([[1,200,105,3,False],[2,165,80,2,False],[3,184.5,120,2,False],
              [4,116,70.8,1,False],[5,270,150,4,True]])row = 0for line in data:
    row += 1print( row )print(data.size)print(data)
Copy after login

The first line of code here means introducing NumPy and renaming it to np. In the second line, we use the mat() method in NumPy to create a data matrix, and row is the variable introduced to calculate the number of rows.
The size here means a table of 5*5. You can see the data by printing the data directly:
In-depth understanding of Python data processing and visualization

2. Use of Matplotlib package – graphical data processing

Let’s still look at the top table. The second column is the difference in housing prices. It is not easy to see the difference intuitively (because there are only numbers), so we hope to draw it (Research The method for numerical differences and anomalies is to draw the distribution of data ):

import numpy as npimport scipy.stats as statsimport pylab
data =  np.mat([[1,200,105,3,False],[2,165,80,2,False],[3,184.5,120,2,False],
              [4,116,70.8,1,False],[5,270,150,4,True]])coll = []for row in data:
    coll.append(row[0,1])stats.probplot(coll,plot=pylab)pylab.show()
Copy after login

The result of this code is to generate a graph:
In-depth understanding of Python data processing and visualization
So that we can clearly see it There is a difference.

The requirement for a coordinate chart is to show the specific values ​​of data through different rows and columns.
Of course, we can also display the coordinate diagram:
In-depth understanding of Python data processing and visualizationIn-depth understanding of Python data processing and visualization

3. Deep learning theoretical method – similarity calculation (can be skipped)

Similarity There are many calculation methods, and we choose the two most commonly used ones, namely Euclidean similarity and cosine similarity calculation.

1. Similarity calculation based on Euclidean distance

Euclidean distance is used to represent the true distance between two points in three-dimensional space. We all know the formula, but we rarely hear the name:
In-depth understanding of Python data processing and visualization
So let’s take a look at its practical application:
This table shows the ratings of items by three users:
In-depth understanding of Python data processing and visualization
d12 represents the similarity between user 1 and user 2, then there is:
In-depth understanding of Python data processing and visualization
Similarly, d13:
In-depth understanding of Python data processing and visualization
It can be seen that user 2 is more similar to User 1 (the smaller the distance, the greater the similarity).

2. Similarity calculation based on cosine angle

The starting point for the calculation of cosine angle is the difference in the included angle.
In-depth understanding of Python data processing and visualization
In-depth understanding of Python data processing and visualization
It can be seen that compared to user 3, user 2 is more similar to user 1 (the more similar the two targets are, the smaller the angle formed by their line segments)

4. Visual display of data statistics (taking precipitation in Bozhou City as an example)

Quartiles of data

Quartiles are the statistical median scores A kind of digit, that is, the data is arranged from small to large, and then divided into four equal parts. The data at the three dividing points is the quartile.
First quartile (Q1), also called lower quartile;
Second quartile (Q1), also called median;
Third quartile (Q1), also called lower quartile;

The gap between the third quartile and the first quartile is also called the four-point gap (IQR).

若n为项数,则:
Q1的位置 = (n+1)*0.25
Q2的位置 = (n+1)*0.50
Q3的位置 = (n+1)*0.75

四分位示例:
关于这个rain.csv,有需要的可以私我要文件,我使用的是亳州市2010-2019年的月份降水情况。

from pylab import *import pandas as pdimport matplotlib.pyplot as plot
filepath = ("C:\\Users\\AWAITXM\\Desktop\\rain.csv")# "C:\Users\AWAITXM\Desktop\rain.csv"dataFile = pd.read_csv(filepath)summary = dataFile.describe()print(summary)array = dataFile.iloc[:,:].values
boxplot(array)plot.xlabel("year")plot.ylabel("rain")show()
Copy after login

以下是plot运行结果:
In-depth understanding of Python data processing and visualization
这个是pandas的运行
In-depth understanding of Python data processing and visualization
这里就可以很清晰的看出来数据的波动范围。
可以看出,不同月份的降水量有很大差距,8月最多,1-4月和10-12月最少。

那么每月的降水增减程度如何比较?

from pylab import *import pandas as pdimport matplotlib.pyplot as plot
filepath = ("C:\\Users\\AWAITXM\\Desktop\\rain.csv")# "C:\Users\AWAITXM\Desktop\rain.csv"dataFile = pd.read_csv(filepath)summary = dataFile.describe()minRings = -1maxRings = 99nrows = 11for i in range(nrows):
    dataRow = dataFile.iloc[i,1:13]
    labelColor = ( (dataFile.iloc[i,12] - minRings ) / (maxRings - minRings) )
    dataRow.plot(color = plot.cm.RdYlBu(labelColor),alpha = 0.5)plot.xlabel("Attribute")plot.ylabel(("Score"))show()
Copy after login

结果如图:
In-depth understanding of Python data processing and visualization
可以看出来降水月份并不规律的上涨或下跌。

那么每月降水是否相关?

from pylab import *import pandas as pdimport matplotlib.pyplot as plot
filepath = ("C:\\Users\\AWAITXM\\Desktop\\rain.csv")# "C:\Users\AWAITXM\Desktop\rain.csv"dataFile = pd.read_csv(filepath)summary = dataFile.describe()corMat = pd.DataFrame(dataFile.iloc[1:20,1:20].corr())plot.pcolor(corMat)plot.show()
Copy after login

结果如图:
In-depth understanding of Python data processing and visualization
可以看出,颜色分布十分均匀,表示没有多大的相关性,因此可以认为每月的降水是独立行为。

今天就记录到这里了,我们下次再见!希望本文章对你也有所帮助。

推荐学习:python学习教程

The above is the detailed content of In-depth understanding of Python data processing and visualization. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

How to solve the permissions problem encountered when viewing Python version in Linux terminal? How to solve the permissions problem encountered when viewing Python version in Linux terminal? Apr 01, 2025 pm 05:09 PM

Solution to permission issues when viewing Python version in Linux terminal When you try to view Python version in Linux terminal, enter python...

How to efficiently copy the entire column of one DataFrame into another DataFrame with different structures in Python? How to efficiently copy the entire column of one DataFrame into another DataFrame with different structures in Python? Apr 01, 2025 pm 11:15 PM

When using Python's pandas library, how to copy whole columns between two DataFrames with different structures is a common problem. Suppose we have two Dats...

Can Python parameter annotations use strings? Can Python parameter annotations use strings? Apr 01, 2025 pm 08:39 PM

Alternative usage of Python parameter annotations In Python programming, parameter annotations are a very useful function that can help developers better understand and use functions...

Python hourglass graph drawing: How to avoid variable undefined errors? Python hourglass graph drawing: How to avoid variable undefined errors? Apr 01, 2025 pm 06:27 PM

Getting started with Python: Hourglass Graphic Drawing and Input Verification This article will solve the variable definition problem encountered by a Python novice in the hourglass Graphic Drawing Program. Code...

How to use Python and OCR technology to try to crack complex verification codes? How to use Python and OCR technology to try to crack complex verification codes? Apr 01, 2025 pm 10:18 PM

Exploration of cracking verification codes using Python In daily network interactions, verification codes are a common security mechanism to prevent malicious manipulation of automated programs...

How do Python scripts clear output to cursor position at a specific location? How do Python scripts clear output to cursor position at a specific location? Apr 01, 2025 pm 11:30 PM

How do Python scripts clear output to cursor position at a specific location? When writing Python scripts, it is common to clear the previous output to the cursor position...

Python Cross-platform Desktop Application Development: Which GUI Library is the best for you? Python Cross-platform Desktop Application Development: Which GUI Library is the best for you? Apr 01, 2025 pm 05:24 PM

Choice of Python Cross-platform desktop application development library Many Python developers want to develop desktop applications that can run on both Windows and Linux systems...

How to dynamically create an object through a string and call its methods in Python? How to dynamically create an object through a string and call its methods in Python? Apr 01, 2025 pm 11:18 PM

In Python, how to dynamically create an object through a string and call its methods? This is a common programming requirement, especially if it needs to be configured or run...

See all articles