Table of Contents
PHP curl 抓取AJAX异步内容示例,curlajax
php curl抓取ajax数据一段时间后就无响应
php使用curl抓取一个网站的内容被拒绝
Home php教程 php手册 PHP curl 抓取AJAX异步内容示例,curlajax

PHP curl 抓取AJAX异步内容示例,curlajax

Jun 13, 2016 am 09:25 AM
ajax curl asynchronous

PHP curl 抓取AJAX异步内容示例,curlajax

其实抓ajax异步内容的页面和抓普通的页面区别不大。ajax只不过是做了一次异步的http请求,只要使用firebug类似的工具,找到请求的后端服务url和传值的参数,然后对该url传递参数进行抓取即可。

利用Firebug的网络工具

如果抓去的是页面,则内容中没有显示的数据,是一堆JS代码。

Code

$cookie_file=tempnam('./temp','cookie');
$ch = curl_init();
$url1 = "http://www.cdut.edu.cn/default.html";
curl_setopt($ch,CURLOPT_URL,$url1);
curl_setopt($ch,CURLOPT_HTTP_VERSION,CURL_HTTP_VERSION_1_1);
curl_setopt($ch,CURLOPT_HEADER,0);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_FOLLOWLOCATION,1);
curl_setopt($ch, CURLOPT_ENCODING ,'gzip'); //加入gzip解析
//设置连接结束后保存cookie信息的文件
curl_setopt($ch,CURLOPT_COOKIEJAR,$cookie_file);
$content=curl_exec($ch);

curl_close($ch);

$ch3 = curl_init();
$url3 = "http://www.cdut.edu.cn/xww/dwr/call/plaincall/portalAjax.getNewsXml.dwr";
$curlPost = "callCount=1&page=/xww/type/1000020118.html&httpSessionId=12A9B726E6A2D4D3B09DE7952B2F282C&scriptSessionId=295315B4B4141B09DA888D3A3ADB8FAA658&c0-scriptName=portalAjax&c0-methodName=getNewsXml&c0-id=0&c0-param0=string:10000201&c0-param1=string:1000020118&c0-param2=string:news_&c0-param3=number:5969&c0-param4=number:1&c0-param5=null:null&c0-param6=null:null&batchId=0";
curl_setopt($ch3,CURLOPT_URL,$url3);
curl_setopt($ch3,CURLOPT_POST,1);
curl_setopt($ch3,CURLOPT_POSTFIELDS,$curlPost);

//设置连接结束后保存cookie信息的文件
curl_setopt($ch3,CURLOPT_COOKIEFILE,$cookie_file); 
$content1=curl_exec($ch3);
curl_close($ch3);
Copy after login

php curl抓取ajax数据一段时间后就无响应

尝试伪造 头信息 :Host 、Referer、User-Agent 等
 

php使用curl抓取一个网站的内容被拒绝

刚写的。希望有用
>
 

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Tutorial on updating curl version under Linux! Tutorial on updating curl version under Linux! Mar 07, 2024 am 08:30 AM

To update the curl version under Linux, you can follow the steps below: Check the current curl version: First, you need to determine the curl version installed in the current system. Open a terminal and execute the following command: curl --version This command will display the current curl version information. Confirm available curl version: Before updating curl, you need to confirm the latest version available. You can visit curl's official website (curl.haxx.se) or related software sources to find the latest version of curl. Download the curl source code: Using curl or a browser, download the source code file for the curl version of your choice (usually .tar.gz or .tar.bz2

How to solve the 403 error encountered by jQuery AJAX request How to solve the 403 error encountered by jQuery AJAX request Feb 20, 2024 am 10:07 AM

Title: Methods and code examples to resolve 403 errors in jQuery AJAX requests. The 403 error refers to a request that the server prohibits access to a resource. This error usually occurs because the request lacks permissions or is rejected by the server. When making jQueryAJAX requests, you sometimes encounter this situation. This article will introduce how to solve this problem and provide code examples. Solution: Check permissions: First ensure that the requested URL address is correct and verify that you have sufficient permissions to access the resource.

How to solve jQuery AJAX request 403 error How to solve jQuery AJAX request 403 error Feb 19, 2024 pm 05:55 PM

jQuery is a popular JavaScript library used to simplify client-side development. AJAX is a technology that sends asynchronous requests and interacts with the server without reloading the entire web page. However, when using jQuery to make AJAX requests, you sometimes encounter 403 errors. 403 errors are usually server-denied access errors, possibly due to security policy or permission issues. In this article, we will discuss how to resolve jQueryAJAX request encountering 403 error

How to handle 301 redirection of web pages in PHP Curl? How to handle 301 redirection of web pages in PHP Curl? Mar 08, 2024 am 11:36 AM

How to handle 301 redirection of web pages in PHPCurl? When using PHPCurl to send network requests, you will often encounter a 301 status code returned by the web page, indicating that the page has been permanently redirected. In order to handle this situation correctly, we need to add some specific options and processing logic to the Curl request. The following will introduce in detail how to handle 301 redirection of web pages in PHPCurl, and provide specific code examples. 301 redirect processing principle 301 redirect means that the server returns a 30

How to get variables from PHP method using Ajax? How to get variables from PHP method using Ajax? Mar 09, 2024 pm 05:36 PM

Using Ajax to obtain variables from PHP methods is a common scenario in web development. Through Ajax, the page can be dynamically obtained without refreshing the data. In this article, we will introduce how to use Ajax to get variables from PHP methods, and provide specific code examples. First, we need to write a PHP file to handle the Ajax request and return the required variables. Here is sample code for a simple PHP file getData.php:

How to solve the problem of jQuery AJAX error 403? How to solve the problem of jQuery AJAX error 403? Feb 23, 2024 pm 04:27 PM

How to solve the problem of jQueryAJAX error 403? When developing web applications, jQuery is often used to send asynchronous requests. However, sometimes you may encounter error code 403 when using jQueryAJAX, indicating that access is forbidden by the server. This is usually caused by server-side security settings, but there are ways to work around it. This article will introduce how to solve the problem of jQueryAJAX error 403 and provide specific code examples. 1. to make

PHP and Ajax: Building an autocomplete suggestion engine PHP and Ajax: Building an autocomplete suggestion engine Jun 02, 2024 pm 08:39 PM

Build an autocomplete suggestion engine using PHP and Ajax: Server-side script: handles Ajax requests and returns suggestions (autocomplete.php). Client script: Send Ajax request and display suggestions (autocomplete.js). Practical case: Include script in HTML page and specify search-input element identifier.

Advanced Guide to Python asyncio: From Beginner to Expert Advanced Guide to Python asyncio: From Beginner to Expert Mar 04, 2024 am 09:43 AM

Concurrent and Asynchronous Programming Concurrent programming deals with multiple tasks executing simultaneously, asynchronous programming is a type of concurrent programming in which tasks do not block threads. asyncio is a library for asynchronous programming in python, which allows programs to perform I/O operations without blocking the main thread. Event loop The core of asyncio is the event loop, which monitors I/O events and schedules corresponding tasks. When a coroutine is ready, the event loop executes it until it waits for I/O operations. It then pauses the coroutine and continues executing other coroutines. Coroutines Coroutines are functions that can pause and resume execution. The asyncdef keyword is used to create coroutines. The coroutine uses the await keyword to wait for the I/O operation to complete. The following basics of asyncio

See all articles