简体中文(ZH-CN) English(EN) 繁体中文(ZH-TW) 日本語(JA) 한국어(KO) Melayu(MS) Français(FR) Deutsch(DE)
Home > Backend Development > PHP Tutorial > body text

How to implement distributed crawler using client IP

WBOY
Release： 2016-08-08 09:06:43
Original
1279 people have browsed it

If you use a server-side crawler, you will encounter various problems. How can you use the visitor's IP to access the crawled website when opening the web page, and then upload the data? Can this be achieved by a distributed crawler? Ajax gets the crawled data and then sends it to your own server?

Are there any similar examples or open source projects?

Reply content:

If you use a server-side crawler, you will encounter various problems. How can you use the visitor's IP to access the crawled website when opening the web page, and then upload the data? Can this be achieved by a distributed crawler? Ajax gets the crawled data and then sends it to your own server?

Are there any similar examples or open source projects?

You are stealing user privacy, it won’t work~

The basic principle is to create a hidden iframe and then request the target website. After the request is successful, use ajax to save it to the local server. . Because many websites have implemented anti-crawling strategies, server-side crawlers often fail. In this case, it is very useful to use client-side crawlers.

However, the user experience is not very good. . . .

Related labels：
php distributed web crawler
source：php.cn
Previous article：I created a folder in the yii2 framework. There is no problem running it locally, but when I pull up the server, a server 500 error occurs. Next article：Why can YIi2 use external objects without Include using namespaces?
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Latest Articles by Author
Latest Issues
PHP arrays obtained from URL parameters do not behave as expected I have a URL parameter that contains the category ID and I want to treat it as an array li...
From 2024-04-06 22:09:02
0
1
1428
Where should I place CustomLog directive in apache I'm using php:7.2-apachedocker. I need to disable health check url login access log. Based...
From 2024-04-06 22:03:59
0
1
990
What is the format of the variables in the return value? I am a new learner of php. I found a piece of code: if($x<time()){return[false,'error']...
From 2024-04-06 21:55:20
0
1
778
Problems encountered when using opentbs to generate odt files: values ​​of the same key are displayed in the same row instead of separate columns. I'm using a library called OpenTbs to create odt using PHP, I'm using it because columns a...
From 2024-04-06 20:18:18
0
1
483
Group MySQL results by ID for looping over I have a table with flight data in mysql. I'm writing a php code that will group and displ...
From 2024-04-06 17:27:56
0
1
406
Related Topics
More>
Popular Recommendations
Popular Tutorials
More>
Related Tutorials
Popular Recommendations
Latest courses
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template
About us Disclaimer Sitemap
php.cn：Public welfare online PHP training，Help PHP learners grow quickly！