php curl批处理-可控并发异步
php curl批处理--可控并发异步
在php的手册里面有一段代码:
$mrc = curl_multi_init(); //发出请求 ....... $active = null; do { $mrc = curl_multi_exec($mh, $active); } while ($mrc == CURLM_CALL_MULTI_PERFORM); while ($active && $mrc == CURLM_OK) { if (curl_multi_select($mh) != -1) { do { $mrc = curl_multi_exec($mh, $active); } while ($mrc == CURLM_CALL_MULTI_PERFORM); } } //下面是处理请求返回的结果
这段do...while是用来控制并发的,效果是将$mrc所有的请求完毕,再执行response结果
但如果我有1000个请求,那么curl批处理将并发1000个请求,显然是不合理,所以应该要控制一个并发数,并且将剩余的连接添加到请求队列里:
参考:http://www.onlineaspect.com/2009/01/26/how-to-use-curl_multi-without-blocking/
$mh = curl_multi_init(); $ch = array(); $chunck = 10; //并发控制数 $all = count($urls);//所有的请求url数组 $chunck = $all > $chunck ? $chunck : $all; $options = array( CURLOPT_HEADER=>FALSE, CURLOPT_RETURNTRANSFER=>TRUE, CURLOPT_FOLLOWLOCATION=>TRUE, CURLOPT_MAXREDIRS=>5, CURLOPT_USERAGENT=>'Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20100101 Firefox/6.0' ); for($i = 0 ; $i <br><br>使用下来效果很好,没有副作用,并发数可控,应用之处多多,自己发挥想象吧 <div class="clear"> </div>

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics



Both curl and Pythonrequests are powerful tools for sending HTTP requests. While curl is a command-line tool that allows you to send requests directly from the terminal, Python's requests library provides a more programmatic way to send requests from Python code. The basic syntax for converting curl to Pythonrequestscurl command is as follows: curl[OPTIONS]URL When converting curl command to Python request, we need to convert the options and URL into Python code. Here is an example curlPOST command: curl-XPOST https://example.com/api

We know that in C language, 'while' keyword is used to define a loop that works based on the condition passed to the loop. Now, since the condition can have two values, true or false, the code inside the while block will be executed repeatedly if the condition is true and will not be executed if the condition is false. Now, by passing parameters to the while loop, we can differentiate between while(1) and while(0) because while(1) is a loop where the condition is always considered true and hence the code inside the block will start executing repeatedly. Furthermore, we can state that it is not 1 that is passed to the loop that makes the condition true, but if any non-zero integer is passed to the while loop, then it will be considered as the true condition, so

To update the curl version under Linux, you can follow the steps below: Check the current curl version: First, you need to determine the curl version installed in the current system. Open a terminal and execute the following command: curl --version This command will display the current curl version information. Confirm available curl version: Before updating curl, you need to confirm the latest version available. You can visit curl's official website (curl.haxx.se) or related software sources to find the latest version of curl. Download the curl source code: Using curl or a browser, download the source code file for the curl version of your choice (usually .tar.gz or .tar.bz2

PHP8.1 released: Introducing curl for concurrent processing of multiple requests. Recently, PHP officially released the latest version of PHP8.1, which introduced an important feature: curl for concurrent processing of multiple requests. This new feature provides developers with a more efficient and flexible way to handle multiple HTTP requests, greatly improving performance and user experience. In previous versions, handling multiple requests often required creating multiple curl resources and using loops to send and receive data respectively. Although this method can achieve the purpose

From start to finish: How to use php extension cURL for HTTP requests Introduction: In web development, it is often necessary to communicate with third-party APIs or other remote servers. Using cURL to make HTTP requests is a common and powerful way. This article will introduce how to use PHP to extend cURL to perform HTTP requests, and provide some practical code examples. 1. Preparation First, make sure that php has the cURL extension installed. You can execute php-m|grepcurl on the command line to check

How to handle 301 redirection of web pages in PHPCurl? When using PHPCurl to send network requests, you will often encounter a 301 status code returned by the web page, indicating that the page has been permanently redirected. In order to handle this situation correctly, we need to add some specific options and processing logic to the Curl request. The following will introduce in detail how to handle 301 redirection of web pages in PHPCurl, and provide specific code examples. 301 redirect processing principle 301 redirect means that the server returns a 30

In Linux, curl is a very practical tool for transferring data to and from the server. It is a file transfer tool that uses URL rules to work under the command line; it supports file upload and download, and is a comprehensive transfer tool. . Curl provides a lot of very useful functions, including proxy access, user authentication, ftp upload and download, HTTP POST, SSL connection, cookie support, breakpoint resume and so on.

How to set cookies in php curl: 1. Create a PHP sample file; 2. Set cURL transmission options through the "curl_setopt" function; 3. Pass the cookie in CURL.
