Home > Backend Development > PHP Tutorial > How Can I Efficiently Check for 404 Errors in PHP Before Web Scraping?

How Can I Efficiently Check for 404 Errors in PHP Before Web Scraping?

Linda Hamilton
Release: 2024-12-19 17:44:10
Original
703 people have browsed it

How Can I Efficiently Check for 404 Errors in PHP Before Web Scraping?

Easy Way to Test a URL for 404 in PHP

Finding that your code is running into issues due to URLs returning 404 is a common pain point in web scraping. To resolve this efficiently, implementing a test at the code's start to verify if a URL has a 404 response is essential.

While suggestions like using @fsockopen() may not account for redirects, a more suitable approach is utilizing curl's curl_getinfo() function. Here's how:

// Initialize a cURL handle with the given URL
$handle = curl_init($url);
// Enable return of transfer as a string
curl_setopt($handle, CURLOPT_RETURNTRANSFER, TRUE);
// Get the response (HTML or data linked to the URL)
$response = curl_exec($handle);
// Check for 404 (file not found) response
$httpCode = curl_getinfo($handle, CURLINFO_HTTP_CODE);
// Handle 404
if ($httpCode == 404) {

    /* Perform your custom 404 handling here. */
}
// Close the curl session
curl_close($handle);
// Continue processing with the retrieved $response
Copy after login

By incorporating this code, you can effectively check for 404 responses, allowing your code to skip the problematic URLs and proceed with the available ones.

The above is the detailed content of How Can I Efficiently Check for 404 Errors in PHP Before Web Scraping?. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Latest Articles by Author
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template