phpSpider Advanced Guide: How to use regular expressions to extract web content?-PHP Tutorial-php.cn

phpSpider Advanced Guide: How to use regular expressions to extract web content?

WBOY

Release： 2023-07-24 20:30:01

Original

1437 people have browsed it

phpSpider Advanced Guide: How to use regular expressions to extract web content?

Foreword:
When developing web crawlers, we often need to extract specific content from web pages. Regular expressions are a powerful tool that can help us perform pattern matching in web pages and extract the required content quickly and accurately. This article will give you an in-depth understanding of how to use regular expressions to extract web content in PHP, and comes with example code.

1. Basic syntax of regular expressions
Regular expression is a way to describe character patterns. Use regular expressions to flexibly match, find, and replace strings. The following is some basic syntax of regular expressions:

Character matching:
.: Matches any character
[]: Matches any character within brackets
w: Matches any letter, number or underscore
d: Matches any number
s: Matches any blank character
: Matches Word boundaries
Repeat match:
- : Match 0 or more repetitions of the previous character
- : Matches 1 or more repetitions of the previous character
?: Matches 0 or 1 repetition of the previous character
{n} : Matches exactly n repetitions of the previous character
{n,} : Matches at least n repetitions of the previous character
{n,m} : Matches at least n times of the previous character , repeat
up to m times Escape characters:
: Escape special characters, for example. Indicates matching dot

2. Use the preg_match function for regular matching
PHP provides a series of functions for processing regular expressions, the most commonly used of which is the preg_match function. This function is used to perform regular string matching. The following is the basic usage of the preg_match function:

$pattern = '/正则表达式/';
$string = '要匹配的字符串';
$result = preg_match($pattern, $string, $matches);

Copy after login

Among them, $pattern is the regular expression to be matched, $string is the string to be matched, $result is the Boolean value of the matching result, and $matches is to store the matches. Array of results.

3. Example Demonstration
Let us use an example to illustrate how to use regular expressions to extract web page content.

Suppose we want to extract all links from the following target web page:

<html>
<body>
<a href="https://www.example.com/link1">Link 1</a>
<a href="https://www.example.com/link2">Link 2</a>
<a href="https://www.example.com/link3">Link 3</a>
</body>
</html>

Copy after login

We can use the following regular expression to match all links:

$pattern = '/<as+href=["'](.*?)["'].*>(.*?)</a>/';

Copy after login

Then, we You can use the preg_match_all function to store all matching results in a two-dimensional array:

$pattern = '/<as+href=["'](.*?)["'].*>(.*?)</a>/';
$string = '
            
              Link 1
              Link 2
              Link 3
            
          ';
preg_match_all($pattern, $string, $matches);

var_dump($matches[1]);  // 输出所有链接

Copy after login

After executing this code, we will get the following output:

array(3) {
  [0]=>
  string(23) "https://www.example.com/link1"
  [1]=>
  string(23) "https://www.example.com/link2"
  [2]=>
  string(23) "https://www.example.com/link3"
}

Copy after login

In this way, we succeeded All links are extracted from the web page.

4. Notes
It is worth noting that when using regular expressions for crawler development, you should pay attention to the following points:

Greedy and non-greedy
By default, regular expression repeat matching is greedy, that is, it matches as many times as possible. We can use ? to change greedy matching to non-greedy matching.

For example, the following regular expression will greedily match the entire string "abcdef":

$pattern = '/a.*b/';
$string = 'abcdef';
preg_match($pattern, $string, $matches);
var_dump($matches[0]);  // 输出'abcdef'

Copy after login

If we change greedy matching to non-greedy matching, only The shortest substring:

$pattern = '/a.*?b/';
$string = 'abcdef';
preg_match($pattern, $string, $matches);
var_dump($matches[0]);  // 输出'ab'

Copy after login

Line breaks in HTML tags
When extracting web content, you often encounter line breaks contained in HTML tags. In order to match content containing newlines, we can add the s modifier to the regular expression pattern:

$pattern = '/<p>(.*)</p>/s';
$string = '<p>This is a paragraph.</p>
           <p>This is another paragraph.</p>';
preg_match_all($pattern, $string, $matches);
var_dump($matches[1]);  // 输出两个段落的内容

Copy after login

Summary:
Through the introduction of this article, you already understand how to use regular expressions Expression method to extract web page content in PHP. Regular expressions are a very powerful tool for efficiently extracting the information you need. I hope this content can help you better develop web crawlers.

The above is the detailed content of phpSpider Advanced Guide: How to use regular expressions to extract web content?. For more information, please follow other related articles on the PHP Chinese website!