Getting started with python crawlers (3)--Using requests to build Zhihu API

This article mainly introduces you to the relevant information about python Getting started with crawlers and using requests to build ZhihuAPI. The article introduces it in detail through sample code, which is of great help to everyone. It has certain reference and learning value. Friends who need it can take a look below.

Preface

The use of requests is introduced in the crawler series article Elegant HTTP Library Requests. This time we use requests to build a knowledge base. The functions of the Hu API include: sending private messages, liking articles, following users, etc. Because any function involving user operations requires logging in, it is recommended to understand Python to simulate Zhihu login before reading this article. Now assume that you already know how to use requests to simulate Zhihu login.

Idea Analysis

The process of sending a private message is that the browser sends an HTTP request to the server, and the request message includes the request URL and request headerHeader, and the request body. As long as these information are clear, it is easy to use requests to simulate the browser sending private messages.

Open the Chrome browser, find a user, click to send a private message, and track the network request process of the private message.

Look at the request header information first

The request header header contains cookies login information, in addition There is also an authorization field, which is used for user authentication. At the same time, this field is also stored in cookies (in order to prevent cookie information from being leaked, I made a mosaic). This information must be carried when requests are requested.

Let’s take a look at the request URL and request body

The request URL is www.zhihu.com/api/v4/messages , the request method is POST, the request body

{"type":"common","content":"你好，我是pythoner","receiver_hash":"1da75b85900e00adb072e91c56fd9149"}

The request body is a json string, the type and content are easy to understand, but I don’t know what receiver_hash is and need to be further determined, but you should guess that this is A field similar to user id.

Now the question is, how to find the user's id through the URL of the user's homepage? In order to completely simulate the entire process of private messaging, I specially registered a Zhihu account.

If you don’t have an extra mobile phone number, you can use Google to search for “receive sms online”. There are many mobile phone numbers on the Internet that provide free online reception of text messages. The homepage of my registered account: https://www.zhihu .com/people/xiaoxiaodouzi

First try to follow the trumpet account, and then find the trumpet account in the list I follow. When I move the mouse to the trumpet account's avatar, I find that there is an HTTP network request.

The request URL is www.zhihu.com/api/v4/members/xiaoxiaodouzi. The back part of this URL "xiaoxiaodouzi" corresponds to the back part of the Xiaoxiao homepage URL. This Part of it we call url_token.

The data returned by interface is the user’s personal public information.

{ 
 ...
 "id":"1da75b85900e00adb072e91c56fd9149",
 "favorite_count":0,
 "voteup_count":0,
 "commercial_question_count":0,
 "url_token":"xiaoxiaodouzi",
 "type":"people",
 "avatar_url":"https://pic1.zhimg.com/v2-ca13758626bd7367febde704c66249ec_is.jpg",
 "is_active":1492224390,
 "name":"\u6211\u662f\u5c0f\u53f7",
 "url":"http://www.zhihu.com/api/v4/people/1da75b85900e00adb072e91c56fd9149",
 "gender":-1
 ...
}

We can clearly see that there is an id field. As we guessed before, the receiver_hash field in the private message is the user's id.

Code implementation

Now we have clarified the idea of ​​the private message function, and code implementation is a matter of course.

User information

#In order to get the receiver_hash dictionary required for the private message interface, we first need to obtain the user information, which contains the id for value.

@need_login
def user(self, url_token):
 """
 获取用户信息,
 :param url_token:
 url_token 是用户主页url中后面部分
 例如: https://www.zhihu.com/people/xiaoxiaodouzi
 url_token 是 xiaoxiaodouzi
 :return:dict
 """
 response = self._session.get(URL.profile(url_token))
 return response.json()

Send a private message

@need_login
def send_message(self, user_id, content):
 """
 给指定的用户发私信
 :param user_id: 用户ID
 :param content: 私信内容
 """
 data = {"type": "common", "content": content, "receiver_hash": user_id}
 response = self._session.post(URL.message(), json=data)
 data = response.json()
 if data.get("error"):
 self.logger.info("私信发送失败, %s" % data.get("error").get("message"))
 else:
 self.logger.info("发送成功")
 return data

The above two methods are placed in a class called Zhihu. I only listed the key code. The @need_login involved is a user authentication decorator, indicating that this method requires login before it can operate. If you are careful, you may notice that I did not explicitly specify the Header field in each request because I initialized it in the init.py method.

def init(self):
 self._session = requests.session()
 self._session.verify = False
 self._session.headers = {"Host": "www.zhihu.com",
    "Referer": "https://www.zhihu.com/",
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36'
      ' (KHTML, like Gecko) Chrome/56.0.2924.87',
    }
 self._session.cookies = cookiejar.LWPCookieJar(filename=cookie_filename)
 try:
 self._session.cookies.load(ignore_discard=True)
 except:
 pass

Call execution

from zhihu import Zhihu

if name == 'main':
 zhihu = Zhihu()
 profile = zhihu.user("xiaoxiaodouzi")
 _id = profile.get("id")
 zhihu.send_message(_id, "你好,这是来自Python之禅的问候")

After the execution was completed, the trumpet successfully received the private message I sent.

Finally, we can follow similar ideas to implement functions such as following users and liking them.

[Related recommendations]

1. Getting started with python crawlers (5)--regular expression example tutorial

2. Getting started with python crawlers (4)--Detailed explanation of HTML text parsing library BeautifulSoup

3. Getting started with python crawlers (2)--HTTP library requests

4. Getting started with python crawlers (1) - Quickly understand the HTTP protocol

5. Share an example of using a Python crawler to simulate Zhihu login

The above is the detailed content of Getting started with python crawlers (3)--Using requests to build Zhihu API. For more information, please follow other related articles on the PHP Chinese website!