Tutorial on using PHP to crawl Douban movie reviews

With the continuous expansion and development of the film market, people's demand for films is also getting higher and higher. As for movie evaluation, Douban Film Critics has always been a more authoritative and popular choice. Sometimes, we also need to perform certain analysis and processing on Douban film reviews, which requires using crawler technology to obtain information about Douban film reviews. This article will introduce a tutorial on how to use PHP to crawl Douban movie reviews for your reference.

1. Get the page address of Douban movie

Before crawling Douban movie reviews, you need to get the page address of Douban movie. The page address of the movie can be obtained through a search engine or directly accessing the movie list interface of Douban Movies. For example, if we want to get the page address of the movie "The Shawshank Redemption", we can search on the movie list interface of Douban Movies, or we can directly enter the page address of the movie in the browser: https://movie.douban.com/subject/ 1292052/.

2. Use PHP to access the Douban movie page and obtain the page content

PHP provides the cURL library for web page access and data acquisition. The following is a PHP code example to access the Douban Movies page and obtain the page content:

$url = 'https://movie.douban.com/subject/1292052/';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$html = curl_exec($ch);
curl_close($ch);
echo $html;

In the code, the page address of Douban Movies is first defined, and then the curl_init() function is used to initialize the cURL object. Then use the curl_setopt() function to set the parameters of the curl object, where CURLOPT_URL sets the accessed URL address and CURLOPT_RETURNTRANSFER sets the return result as a string. Then use the curl_exec() function to perform the cURL crawling operation and assign the result to the variable $html. Finally, use the curl_close() function to close the cURL session and release related resources, and then output the value of the variable $html.

Execute the above code to output the HTML content of the Douban movie page.

3. Parse the HTML content of the Douban movie review page

After obtaining the HTML content of the page, you need to use PHP to parse the HTML content and extract the Douban movie review information.

In PHP, you can use the SimpleXML extension to parse XML or HTML data. The following is a PHP code example, which is used to parse the HTML content of the Douban movie review page and extract the rating and review content of the movie review:

$url = 'https://movie.douban.com/subject/1292052/';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$html = curl_exec($ch);
curl_close($ch);

// 使用 SimpleXML 解析 HTML 内容
$xml = simplexml_load_string($html);
$ratings = array();
$comments = array();

// 获取影评评分和评论内容
foreach ($xml->xpath('//div[@class="comment-item"]') as $comment) {
    $rating = $comment->div[2]->span[2]->text();
    $comment_content = $comment->div[2]->p->text();
    $ratings[] = (string)$rating;
    $comments[] = (string)$comment_content;
}

print_r($ratings);
print_r($comments);

In the code, the page address of the Douban movie is first defined, and then curl is used. Page access and operations to obtain HTML content.

Next use the simplexml_load_string() function to parse the HTML content into a SimpleXML object. Then use the xpath() function and XPath query statement to obtain all movie review information on the page, use the foreach statement to traverse each movie review, and obtain the ratings and comment content. Finally, store the ratings and comments into the $ratings and $comments arrays respectively, and output the contents of these two arrays.

Execute the above code to parse out all the film review information in the Douban film review page.

4. Output movie review information

Next, output the obtained movie review information. You can use HTML and CSS to beautify the output content. The following is a PHP code example for beautifying and outputting the movie review information on the Douban movie review page:

$url = 'https://movie.douban.com/subject/1292052/';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$html = curl_exec($ch);
curl_close($ch);

// 使用 SimpleXML 解析 HTML 内容
$xml = simplexml_load_string($html);
$ratings = array();
$comments = array();

// 获取影评评分和评论内容
foreach ($xml->xpath('//div[@class="comment-item"]') as $comment) {
    $rating = $comment->div[2]->span[2]->text();
    $comment_content = $comment->div[2]->p->text();
    $ratings[] = (string)$rating;
    $comments[] = (string)$comment_content;
}

// 输出美化的影评信息
echo '<style>table, th, td {border: 1px solid black; border-collapse: collapse;} th, td {padding: 5px;}</style>';
echo '<table><tr><th>评分</th><th>评论内容</th></tr>';
for ($i = 0; $i < count($ratings); $i++) {
    echo '<tr><td>' . $ratings[$i] . '</td><td>' . $comments[$i] . '</td></tr>';
}
echo '</table>';

In the code, first use curl to obtain the HTML content of the Douban movie page, and then use SimpleXML to parse the HTML content and obtain Ratings and review content for movie reviews.

Next, use HTML and CSS to beautify the output content, including adding table styles, setting table headers and table content, and using a for loop to traverse each movie review and output the ratings and comments.

Execute the above code to output the film review information of the beautified Douban film review page.

To sum up, this article introduces a tutorial on how to use PHP to crawl Douban movie reviews. Through learning, readers can learn how to use PHP and cURL technologies to access web pages and capture data, as well as how to use SimpleXML extensions to parse HTML content and beautify output.

The above is the detailed content of Tutorial on using PHP to crawl Douban movie reviews. For more information, please follow other related articles on the PHP Chinese website!