How to use the urllib.request module to send HTTP requests in Python 3.x

How to use the urllib.request module to send HTTP requests in Python 3.x

In the actual development process, we often need to send HTTP requests to interact with the server. Python provides the urllib.request module, which is one of the modules in the Python standard library for handling URL requests. In this article, we will learn how to send HTTP requests using the urllib.request module.

urllib.request module overview

The urllib.request module is Python3’s built-in HTTP request module, which provides a series of methods to send and process HTTP requests. It can implement common HTTP request methods such as GET request and POST request, and also supports setting request headers, form data, cookies and other functions.

To use the urllib.request module, we first need to import it:

import urllib.request

We can then use the methods in the urllib.request module to send HTTP requests.

Send a GET request

To send a GET request and get the content of the server response, we can use the urlopen() method in the urllib.request module. The example is as follows:

import urllib.request

# 发送 GET 请求
response = urllib.request.urlopen('http://www.example.com')

# 获取服务器响应的内容
content = response.read()

# 打印服务器响应的内容
print(content)

In this example, we first use the urlopen() method to send a GET request, and the requested URL is http://www.example.com. Then, we called the response.read() method to obtain the content of the server response. Finally, use the print() method to print out the content.

Send a POST request

To send a POST request and upload form data, we can construct a urllib.request.Request object and use the urlopen() method to send the request. An example is as follows:

import urllib.request
import urllib.parse

# 构造表单数据
data = urllib.parse.urlencode({'key1': 'value1', 'key2': 'value2'}).encode()

# 构造请求对象
request = urllib.request.Request('http://www.example.com', data)

# 发送 POST 请求
response = urllib.request.urlopen(request)

# 获取服务器响应的内容
content = response.read()

# 打印服务器响应的内容
print(content)

In this example, we first construct a form data using the urllib.parse.urlencode() method. Then, use the encode() method to convert it to a byte stream. Next, we construct a urllib.request.Request object and pass it the URL and form data as parameters. Finally, use the urlopen() method to send the request and get the content of the server response.

Set request header

If you need to set request headers, such as User-Agent, Referer and other information, you can use the add_header() method of the urllib.request.Request object. An example is as follows:

import urllib.request

# 构造请求对象
request = urllib.request.Request('http://www.example.com')

# 设置请求头
request.add_header('User-Agent', 'Mozilla/5.0')

# 发送请求
response = urllib.request.urlopen(request)

In this example, we first construct a urllib.request.Request object and pass the URL to it as a parameter. Then, use the add_header() method to set a User-Agent request header. Finally, the request is sent using the urlopen() method.

Processing the request results

After sending the request, we can get the server's response by calling the response related method. Commonly used methods include:

• response.read(): Get the content of the server response and return data in the form of byte stream.
• response.getheaders(): Get the header information of the server response and return a list.
• response.getheader(name): Get the response header information of the specified name.

The example is as follows:

import urllib.request

# 发送 GET 请求
response = urllib.request.urlopen('http://www.example.com')

# 获取服务器响应的内容
content = response.read()

# 获取服务器响应的头信息
headers = response.getheaders()

# 获取指定名称的响应头信息
content_type = response.getheader('Content-Type')

# 打印结果
print(content)
print(headers)
print(content_type)

In this example, we first send a GET request and get the response from the server. Then, we called the response.read(), response.getheaders() and response.getheader(name) methods respectively to obtain the content, header information and response header information of the specified name from the server response. Finally, print out the results.

Summary

In summary, we learned how to use the urllib.request module to send HTTP requests. Through the urllib.request module, we can easily send GET requests, POST requests, upload form data, set request headers, etc. This is very helpful for us to interact with the server, obtain data, etc.

I hope this article will help everyone understand and use the urllib.request module. thanks for reading!

The above is the detailed content of How to use the urllib.request module to send HTTP requests in Python 3.x. For more information, please follow other related articles on the PHP Chinese website!