Cookie introduction and simulated login demonstration

CookieIt can be said that it is closely related to our lives. When you shop on Taobao, those recommended products are the masterpieces of Cookie.

In the previous article, we explained the basic operations of Selenium PhantomJs. Today we bring you an introduction to Cookie and a simulated login demonstration.

What are cookies?

In computer terminology, it refers to a method that allows the website server to store a small amount of data on the client's hard drive or memory, or read data from the client's hard drive. a technology.

Open Chrome's developer tools (F12), click on Network, click on a request URL, you can see a cookie information in the request header (Request Headers). Of course, we can see a Set-Cookie information from the above response header, which is some information about the server setting cookies to the browser, such as cookie scope, time, etc.

Purpose of Cookie

Storage user login information to determine whether the user is logged in; save user browsing footprints;

Cookie is stored by the server on the client information. This information is generated by the server and interpreted by the server.

When making a request, the client needs to send untimed cookies to the server. The server needs to parse cookies to determine user information.

Our browser will automatically store cookies every time you browse the web. For example, when you open the clear browser browsing history, there will be cookie information.

Cookies bring us a lot of convenience. They can also record our browsing footprints and the time we stay on the page. For example, when you are crazy about Taobao, Taobao's recommendations for products you like are generated based on your cookies to obtain which products you have browsed.

Simulated login demonstration

Let’s take Zhihu as an example. https://www.zhihu.com/people/yu-kun-73/answers

This URL is my Zhihu information page. Of course, you can use your own Zhihu page as an example. If we visit him directly, we will see the login button above, but I am already logged in and accessing. The login button will not be displayed, but some links to personal edit profiles.

So, what if we use a crawler to access.

import requests
 
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36'
}
url = 'https://www.zhihu.com/people/yu-kun-73/answers'
resp = requests.get(url, headers=headers).text
print(resp)

Through the printed results, we can see that there is a login button in the html code. This proves that we are not logged in to this page. So, if we crawl some websites that require a login account to crawl data, we must solve the login problem. So how to implement login.

The answer is our Cookie. We said in a previous article that the requests library is very simple when handling cookies. We just need to add request cookies like we added 'User-Agent'.

import requests
 
headers = {
    'Cookie': # 你的登录过后的浏览器cookies,
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36'
}
url = 'https://www.zhihu.com/people/yu-kun-73/answers'
resp = requests.get(url, headers=headers).text
print(resp)

When we look at the print information again, we can see a link to edit personal information. This means we have successfully logged in. Isn't it very simple? Don't worry, the hard part will come later.

at last

So, if we want to log in to some complex websites, such as Sina Weibo, which requires logging in to obtain information, using cookies to log in will definitely not work. Because these websites will frequently update the algorithms of some websites, our cookies will expire after a while, so we need to post our login information to the login page of these websites.

These login information parameters are generally stored in the Form Data under the request header of the login page. As long as we post this information, we can log in smoothly. But if you encounter a verification code, it will be very troublesome. The problem of verification code has always been a threshold for testing crawlers. Let’s study this by ourselves.

The above is the detailed content of Cookie introduction and simulated login demonstration. For more information, please follow other related articles on the PHP Chinese website!