Home Backend Development PHP Tutorial Discuss the analysis of file_get_contents and curl efficiency and stability_PHP tutorial

Discuss the analysis of file_get_contents and curl efficiency and stability_PHP tutorial

Jul 21, 2016 pm 03:08 PM
curl file get and content analyze crawl Discuss efficiency Stablize website

I have worked on many products that capture content from other websites, and I am used to using the convenient and fast file_get_contents function, but I always encounter the problem of failure to obtain it. Even though I set the timeout according to the examples in the manual, it does not work most of the time. :

Copy code The code is as follows:

$config['context'] = stream_context_create(array('http' => ; array('method' => "GET",
'timeout' => 5//This timeout is unstable and often does not work
)
));

At this time, if you look at the server’s connection pool, you will find a bunch of similar errors, which will give you a headache:
file_get_contents(http://***): failed to open stream…
As a last resort, I installed the curl library and wrote a function replacement:
Copy the code The code is as follows:

function curl_file_get_contents($durl){
   $ch = curl_init();
   curl_setopt($ch, CURLOPT_URL, $durl);
   curl_setopt($ch, CURLOPT_TIMEOUT, 5);
   curl_setopt($ch, CURLOPT_USERAGENT, _USERAGENT_);
   curl_setopt($ch, CURLOPT_REFERER,_REFERER_);
   curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
   $r = curl_exec($ch);
   curl_close($ch);
   return $r;
 }

So no more problems other than real network issues.
This is a test done by others about curl and file_get_contents:
The number of seconds it takes for file_get_contents to crawl google.com:
2.31319094
2.30374217
2.21512604
3.30553889
2.30124092
Time used by curl:
0.68719101
0.64675593
0.64326
0.81983113
0.63 956594
Is there a big gap? Haha, from my experience, these two tools are not only different in speed, but also in stability. It is recommended that friends who have high requirements for the stability of network data capture use the curl_file_get_contents function above. Not only is it stable and fast, it can also fake the browser and spoof the target address!

www.bkjia.comtruehttp: //www.bkjia.com/PHPjc/327449.htmlTechArticleI have made many products that capture the content of other websites. I am used to using the convenient and fast file_get_contents function, but I always get it. When encountering the problem of failure to obtain, despite following the examples in the manual...
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
WWE 2K25: How To Unlock Everything In MyRise
1 months ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

How to set up Cheat Engine in Chinese? Cheat Engine setting Chinese method How to set up Cheat Engine in Chinese? Cheat Engine setting Chinese method Mar 13, 2024 pm 04:49 PM

CheatEngine is a game editor that can edit and modify the game's memory. However, its default language is non-Chinese, which is inconvenient for many friends. So how to set Chinese in CheatEngine? Today, the editor will give you a detailed introduction to how to set up Chinese in CheatEngine. I hope it can help you. Setting method one: 1. Double-click to open the software and click "edit" in the upper left corner. 2. Then click “settings” in the option list below. 3. In the opened window interface, click "languages" in the left column

Private deployment of Stable Diffusion to play with AI drawing Private deployment of Stable Diffusion to play with AI drawing Mar 12, 2024 pm 05:49 PM

StableDiffusion is an open source deep learning model. Its main function is to generate high-quality images through text descriptions, and supports functions such as graph generation, model merging, and model training. The operating interface of the model can be seen in the figure below. How to generate a picture. The following is an introduction to the process of creating a picture of a deer drinking water. When generating a picture, it is divided into prompt words and negative prompt words. When entering the prompt words, you must describe it clearly and try to describe the scene, object, style and color you want in detail. . For example, instead of just saying "the deer drinks water", it says "a creek, next to dense trees, and there are deer drinking water next to the creek". The negative prompt words are in the opposite direction. For example: no buildings, no people , no bridges, no fences, and too vague description may lead to inaccurate results.

Analysis of the reasons why the secondary directory of DreamWeaver CMS cannot be opened Analysis of the reasons why the secondary directory of DreamWeaver CMS cannot be opened Mar 13, 2024 pm 06:24 PM

Title: Analysis of the reasons and solutions for why the secondary directory of DreamWeaver CMS cannot be opened. Dreamweaver CMS (DedeCMS) is a powerful open source content management system that is widely used in the construction of various websites. However, sometimes during the process of building a website, you may encounter a situation where the secondary directory cannot be opened, which brings trouble to the normal operation of the website. In this article, we will analyze the possible reasons why the secondary directory cannot be opened and provide specific code examples to solve this problem. 1. Possible cause analysis: Pseudo-static rule configuration problem: during use

Example of Curl Get command Example of Curl Get command Mar 20, 2024 pm 06:56 PM

In Linux, URL or Curl client is a popular command line utility that allows you to transfer data over the network using various protocols such as HTTPS, HTTP, FTP, etc. It allows you to send and receive data using its get, post and request methods. Among them, you need to use the "get" method frequently. Therefore, it becomes crucial to learn various methods and various options that you can use to increase your productivity. "Performing a curl operation is as simple as entering a few simple commands. Although it seems simple, many users do not fully realize its potential. Therefore, this short guide provides some information on how to perform curl operations on Linux systems. Example using the "curlget" command." Curl

Analyze whether Tencent's main programming language is Go Analyze whether Tencent's main programming language is Go Mar 27, 2024 pm 04:21 PM

Title: Is Tencent’s main programming language Go: An in-depth analysis. As China’s leading technology company, Tencent has always attracted much attention in its choice of programming languages. In recent years, some people believe that Tencent mainly adopts Go as its main programming language. This article will conduct an in-depth analysis of whether Tencent's main programming language is Go, and give specific code examples to support this view. 1. Application of Go language in Tencent Go is an open source programming language developed by Google. Its efficiency, concurrency and simplicity are loved by many developers.

After multiple transformations and cooperation with AI giant Nvidia, why did Vanar Chain surge 4.6 times in 30 days? After multiple transformations and cooperation with AI giant Nvidia, why did Vanar Chain surge 4.6 times in 30 days? Mar 15, 2024 am 10:46 AM

Recently, Layer1 blockchain VanarChain has attracted market attention due to its high growth rate and cooperation with AI giant NVIDIA. Behind VanarChain's popularity, in addition to undergoing multiple brand transformations, popular concepts such as main games, metaverse and AI have also earned the project plenty of popularity and topics. Prior to its transformation, Vanar was the predecessor of TerraVirtua, a company founded in 2018. Initially, it was a platform that provided paid subscription-based virtual reality (VR) and augmented reality (AR) content and supported cryptocurrency payments. The company was founded by co-founders Gary Bracey and Jawad Ashraf, of whom Gary Brac

Usability analysis of Hongmeng system from actual operation Usability analysis of Hongmeng system from actual operation Mar 23, 2024 am 08:21 AM

As an operating system that has attracted much attention in recent years, Hongmeng system has attracted much attention. Hongmeng OS is hailed as an alternative system launched by Huawei in response to the ban on the Android system. It has the advantages of independent controllability and cross-platform. However, the quality of a system not only depends on its technical strength, but more importantly, the user experience, that is, ease of use. This article will analyze the ease of use of Hongmeng system from the perspective of actual operation. 1. Installation and settings The installation and settings of Hongmeng system are relatively simple and not complicated for ordinary users. during installation

Analyzing Solana's DEX layout: Is Jupiter the future of ecology? Analyzing Solana's DEX layout: Is Jupiter the future of ecology? Mar 26, 2024 pm 02:10 PM

Source: Shenchao TechFlow As a high-profile emerging project in the Solana ecosystem, Jupiter has quickly emerged in the DeFi field despite its short launch. However, even in such a rapidly developing environment, the improvement of economic models and the stability of token prices are still crucial. Without these supports, a project can easily fall into a vicious cycle that may ultimately lead to its decline or even its inability to sustain itself. Therefore, Jupiter needs to continuously optimize its economic design and ensure token price stability to ensure the long-term development and success of the project. The Solana chain has performed strongly in the past week, with its token SOL rising rapidly in the secondary market, and Jupiter’s token $JUP also rising in the past two weeks.

See all articles