How to use the Requests library in Python-Python Tutorial-php.cn

Table of Contents

Installation of Requests library

http basic request

Basic GET request

Basic POST request

Timeout configuration

Session Object

SSL Certificate Verification

Proxy

Home

Backend Development

Python Tutorial

How to use the Requests library in Python

巴扎黑

Mar 18, 2017 am 09:48 AM

python

I talked about the use and methods of Python's urllib library, the basic use of Python network data collection Urllib library, and the advanced usage of Python's urllib.

Today we will learn how to use the Requests library in Python.

Installation of Requests library

Use pip to install. If you have installed the pip package (a Python package management tool, I don’t know if you can use Baidu), or an integrated environment, such as Python (x, y) or anaconda, you can directly use pip to install the Python library.

$ pip install requests

Copy after login

After the installation is completed, let’s take a look at the basic method:

    #get请求方法
    >>> r = requests.get(&#39;https://api.github.com/user&#39;, auth=(&#39;user&#39;, &#39;pass&#39;))
#打印get请求的状态码
    >>> r.status_code
200
#查看请求的数据类型，可以看到是json格式，utf-8编码
    >>> r.headers[&#39;content-type&#39;]
&#39;application/json; charset=utf8&#39;
    >>> r.encoding
&#39;utf-8&#39;
#打印请求到的内容
    >>> r.text
u&#39;{"type":"User"...&#39;
#输出json格式数据
    >>> r.json()
    {u&#39;private_gists&#39;: 419, u&#39;total_private_repos&#39;: 77, ...}

Copy after login

Let’s take a look at a small chestnut:

#小例子
import requests

r = requests.get(&#39;http://www.baidu.com&#39;)
print type(r)
print r.status_code
print r.encoding
print r.text
print r.cookies
&#39;&#39;&#39;请求了百度的网址，然后打印出了返回结果的类型，状态码，编码方式，Cookies等内容
输出：&#39;&#39;&#39;
<class &#39;requests.models.Response&#39;>
200
UTF-8
<RequestsCookieJar[]>

Copy after login

http basic request

requests library Provides all basic request methods of http. For example:

r = requests.post("http://httpbin.org/post")
r = requests.put("http://httpbin.org/put")
r = requests.delete("http://httpbin.org/delete")
r = requests.head("http://httpbin.org/get")
r = requests.options("http://httpbin.org/get")

Copy after login

Basic GET request

r = requests.get("http://httpbin.org/get")
#如果想要加参数，可以利用 params 参数：
import requests
payload = {&#39;key1&#39;: &#39;value1&#39;, &#39;key2&#39;: &#39;value2&#39;}
r = requests.get("http://httpbin.org/get", params=payload)
print r.url

#输出：http://httpbin.org/get?key2=value2&key1=value1

Copy after login

If you want to request a JSON file, you can use the json() method to parse it. For example, write a JSON file yourself and name it a.json with the following content:

["foo", "bar", {
"foo": "bar"
}]
#利用如下程序请求并解析：
import requests
r = requests.get("a.json")
print r.text
print r.json()
&#39;&#39;&#39;运行结果如下，其中一个是直接输出内容，另外一个方法是利用 json() 方法
解析，感受下它们的不同:&#39;&#39;&#39;
["foo", "bar", {
"foo": "bar"
}]
[u&#39;foo&#39;, u&#39;bar&#39;, {u&#39;foo&#39;: u&#39;bar&#39;}]

Copy after login

If you want to get the raw socket response from the server, you can get r.raw. However, stream=True needs to be set in the initial request.

r = requests.get(&#39;https://github.com/timeline.json&#39;, stream=True)
r.raw
#输出
<requests.packages.urllib3.response.HTTPResponse object at 0x101194810>
r.raw.read(10)
&#39;\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03&#39;

Copy after login

In this way, the original socket content of the web page is obtained.

If you want to add headers, you can pass the headers parameter:

import requests

payload = {&#39;key1&#39;: &#39;value1&#39;, &#39;key2&#39;: &#39;value2&#39;}
headers = {&#39;content-type&#39;: &#39;application/json&#39;}
r = requests.get("http://httpbin.org/get", params=payload, headers=headers)
print r.url
#通过headers参数可以增加请求头中的headers信息

Copy after login

Basic POST request

For POST requests, we generally need to add some parameters to it. Then the most basic parameter passing method can use the data parameter.

import requests

payload = {&#39;key1&#39;: &#39;value1&#39;, &#39;key2&#39;: &#39;value2&#39;}
r = requests.post("http://httpbin.org/post", data=payload)
print r.text
#运行结果如下：
{
"args": {}, 
"data": "", 
"files": {}, 
"form": {
"key1": "value1", 
"key2": "value2"
}, 
"headers": {
"Accept": "*/*", 
"Accept-Encoding": "gzip, deflate", 
"Content-Length": "23", 
"Content-Type": "application/x-www-form-urlencoded", 
"Host": "http://httpbin.org", 
"User-Agent": "python-requests/2.9.1"
}, 
"json": null, 
"url": "http://httpbin.org/post"
}

Copy after login

You can see that the parameters were passed successfully, and then the server returned the data we passed.

Sometimes the information we need to send is not in the form of a form. We need to send data in JSON format, so we can use the json.dumps() method to serialize the form data.

import json
import requests

url = &#39;http://httpbin.org/post&#39;
payload = {&#39;some&#39;: &#39;data&#39;}
r = requests.post(url, data=json.dumps(payload))
print r.text

#运行结果：
{
"args": {}, 
"data": "{\"some\": \"data\"}", 
"files": {}, 
"form": {}, 
"headers": {
"Accept": "*/*", 
"Accept-Encoding": "gzip, deflate", 
"Content-Length": "16", 
"Host": "http://httpbin.org", 
"User-Agent": "python-requests/2.9.1"
}, 
"json": {
"some": "data"
}, 
"url": "http://httpbin.org/post"
}

Copy after login

Through the above method, we can POST data in JSON format

If you want to upload a file, just use the file parameter directly:

#新建一个 test.txt 的文件，内容写上 Hello World!
import requests

url = &#39;http://httpbin.org/post&#39;
files = {&#39;file&#39;: open(&#39;test.txt&#39;, &#39;rb&#39;)}
r = requests.post(url, files=files)
print r.text

{
"args": {}, 
"data": "", 
"files": {
"file": "Hello World!"
}, 
"form": {}, 
"headers": {
"Accept": "*/*", 
"Accept-Encoding": "gzip, deflate", 
"Content-Length": "156", 
"Content-Type": "multipart/form-data; boundary=7d8eb5ff99a04c11bb3e862ce78d7000", 
"Host": "http://httpbin.org", 
"User-Agent": "python-requests/2.9.1"
}, 
"json": null, 
"url": "http://httpbin.org/post"
}

Copy after login

In this way we will complete it successfully uploaded a file.

requests supports streaming uploads, which allows you to send large data streams or files without reading them into memory first. To use streaming upload, you only need to provide a file-like object for your request body, which is very convenient:

with open(&#39;massive-body&#39;) as f:
requests.post(&#39;http://some.url/streamed&#39;, data=f)

Copy after login

Cookies

If a response contains cookies, then we can use Cookies variable to get:

import requests

url = &#39;Example Domain&#39;
r = requests.get(url)
print r.cookies
print r.cookies[&#39;example_cookie_name&#39;]

Copy after login

The above program is just a sample, you can use the cookies variable to get the cookies of the site

In addition, you can use the cookies variable to send cookie information to the server:

import requests

url = &#39;http://httpbin.org/cookies&#39;
cookies = dict(cookies_are=&#39;working&#39;)
r = requests.get(url, cookies=cookies)
print r.text
#输出：
&#39;{"cookies": {"cookies_are": "working"}}&#39;

Copy after login

Timeout configuration

You can use the timeout variable to configure the maximum request time

requests.get(‘Build software better, together’, timeout=0.001)

Copy after login

Note: timeout is only valid for the connection process and has nothing to do with the download of the response body.

In other words, this time only limits the requested time. Even if the returned response contains a large amount of content, it will take some time to download.

Session Object

In the above requests, each request is actually equivalent to initiating a new request. This is equivalent to the effect of using a different browser to open each request separately. That is, it does not refer to a session, even if the same URL is requested. For example:

import requests

requests.get(&#39;http://httpbin.org/cookies/set/sessioncookie/123456789&#39;)
r = requests.get("http://httpbin.org/cookies")
print(r.text)
#结果是：
{
"cookies": {}
}

Copy after login

Obviously, this is not in a session and cookies cannot be obtained. So what should we do if we need to maintain a persistent session on some sites? Just like using a browser to browse Taobao, jumping between different tabs actually creates a long-term session.

The solution is as follows:

import requests

s = requests.Session()
s.get(&#39;http://httpbin.org/cookies/set/sessioncookie/123456789&#39;)
r = s.get("http://httpbin.org/cookies")
print(r.text)
#在这里我们请求了两次，一次是设置 cookies，一次是获得 cookies
{
"cookies": {
"sessioncookie": "123456789"
}
}

Copy after login

It is found that cookies can be successfully obtained. This is to establish a session.

So since the session is a global variable, we can definitely use it for global configuration.

import requests

s = requests.Session()
s.headers.update({&#39;x-test&#39;: &#39;true&#39;})
r = s.get(&#39;http://httpbin.org/headers&#39;, headers={&#39;x-test2&#39;: &#39;true&#39;})
print r.text
&#39;&#39;&#39;通过 s.headers.update 方法设置了 headers 的变量。然后我们又在请求中
设置了一个 headers，那么会出现什么结果？很简单，两个变量都传送过去了。
运行结果:&#39;&#39;&#39;
{
"headers": {
"Accept": "*/*", 
"Accept-Encoding": "gzip, deflate", 
"Host": "http://httpbin.org", 
"User-Agent": "python-requests/2.9.1", 
"X-Test": "true", 
"X-Test2": "true"
}
}

Copy after login

What if the headers passed by the get method are also x-test?

r = s.get(&#39;http://httpbin.org/headers&#39;, headers={&#39;x-test&#39;: &#39;true&#39;})

#它会覆盖掉全局的配置：
{
"headers": {
"Accept": "*/*", 
"Accept-Encoding": "gzip, deflate", 
"Host": "http://httpbin.org", 
"User-Agent": "python-requests/2.9.1", 
"X-Test": "true"
}
}

Copy after login

What if you don’t want a variable in the global configuration? It's easy, just set it to None.

r = s.get(&#39;http://httpbin.org/headers&#39;, headers={&#39;x-test&#39;: None})
{
"headers": {
"Accept": "*/*", 
"Accept-Encoding": "gzip, deflate", 
"Host": "http://httpbin.org", 
"User-Agent": "python-requests/2.9.1"
}
}

Copy after login

The above is the basic usage of session session.

SSL Certificate Verification

Now that you can see websites starting with https everywhere, Requests can verify SSL certificates for HTTPS requests, just like a web browser. To check the SSL certificate of a certain host, you can use the verify parameter, because the 12306 certificate was not invalid some time ago. Let’s test it:

import requests

r = requests.get(&#39;https://kyfw.12306.cn/otn/&#39;, verify=True)
print r.text
#结果：
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

Copy after login

Let’s try github’s:

import requests

r = requests.get(&#39;Build software better, together&#39;, verify=True)
print r.text

Copy after login

Well, normal request, because there is too much content, I won’t paste the output.

If we want to skip the certificate verification of 12306 just now, set verify to False:

import requests

r = requests.get(&#39;https://kyfw.12306.cn/otn/&#39;, verify=False)
print r.text

Copy after login

Once found, the request can be made normally. By default verify is True, so you need to set this variable manually if necessary.

Proxy

If you need to use a proxy, you can configure individual requests by providing the proxies parameter to any request method.

import requests

proxies = {
"https": "http://41.118.132.69:4433"
}
r = requests.post("http://httpbin.org/post", proxies=proxies)
print r.text
#也可以通过环境变量 HTTP_PROXY 和 HTTPS_PROXY 来配置代理
export HTTP_PROXY="http://10.10.1.10:3128"
export HTTPS_PROXY="http://10.10.1.10:1080"

Copy after login

The above is the detailed content of How to use the Requests library in Python. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress images for free

Clothoff.io

AI clothes remover

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

How to fix KB5055612 fails to install in Windows 10?

3 weeks ago By DDD

Roblox: Grow A Garden - Complete Mutation Guide

3 weeks ago By DDD

Roblox: Bubble Gum Simulator Infinity - How To Get And Use Royal Keys

3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Mandragora: Whispers Of The Witch Tree - How To Unlock The Grappling Hook

3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Nordhold: Fusion System, Explained

3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial

1668

CakePHP Tutorial

1427

Laravel Tutorial

1329

PHP Tutorial

1273

C# Tutorial

1256

Related knowledge

PHP and Python: Different Paradigms Explained Apr 18, 2025 am 12:26 AM

PHP is mainly procedural programming, but also supports object-oriented programming (OOP); Python supports a variety of paradigms, including OOP, functional and procedural programming. PHP is suitable for web development, and Python is suitable for a variety of applications such as data analysis and machine learning.

Choosing Between PHP and Python: A Guide Apr 18, 2025 am 12:24 AM

PHP is suitable for web development and rapid prototyping, and Python is suitable for data science and machine learning. 1.PHP is used for dynamic web development, with simple syntax and suitable for rapid development. 2. Python has concise syntax, is suitable for multiple fields, and has a strong library ecosystem.

How to run sublime code python Apr 16, 2025 am 08:48 AM

To run Python code in Sublime Text, you need to install the Python plug-in first, then create a .py file and write the code, and finally press Ctrl B to run the code, and the output will be displayed in the console.

PHP and Python: A Deep Dive into Their History Apr 18, 2025 am 12:25 AM

PHP originated in 1994 and was developed by RasmusLerdorf. It was originally used to track website visitors and gradually evolved into a server-side scripting language and was widely used in web development. Python was developed by Guidovan Rossum in the late 1980s and was first released in 1991. It emphasizes code readability and simplicity, and is suitable for scientific computing, data analysis and other fields.

Python vs. JavaScript: The Learning Curve and Ease of Use Apr 16, 2025 am 12:12 AM

Python is more suitable for beginners, with a smooth learning curve and concise syntax; JavaScript is suitable for front-end development, with a steep learning curve and flexible syntax. 1. Python syntax is intuitive and suitable for data science and back-end development. 2. JavaScript is flexible and widely used in front-end and server-side programming.

Golang vs. Python: Performance and Scalability Apr 19, 2025 am 12:18 AM

Golang is better than Python in terms of performance and scalability. 1) Golang's compilation-type characteristics and efficient concurrency model make it perform well in high concurrency scenarios. 2) Python, as an interpreted language, executes slowly, but can optimize performance through tools such as Cython.

Where to write code in vscode Apr 15, 2025 pm 09:54 PM

Writing code in Visual Studio Code (VSCode) is simple and easy to use. Just install VSCode, create a project, select a language, create a file, write code, save and run it. The advantages of VSCode include cross-platform, free and open source, powerful features, rich extensions, and lightweight and fast.

How to run python with notepad Apr 16, 2025 pm 07:33 PM

Running Python code in Notepad requires the Python executable and NppExec plug-in to be installed. After installing Python and adding PATH to it, configure the command "python" and the parameter "{CURRENT_DIRECTORY}{FILE_NAME}" in the NppExec plug-in to run Python code in Notepad through the shortcut key "F6".

See all articles