Use Crawler component to analyze HTML in laravel-PHP Tutorial-php.cn

Use Crawler component to analyze HTML in laravel

巴扎黑

Release： 2023-03-14 18:40:02

Original

5537 people have browsed it

This article mainly introduces the use of Symfony's Crawler component to analyze HTML in laravel. Friends in need can refer to it

The full name of Crawler is DomCrawler, which is a component of the Symfony framework. What is outrageous is that DomCrawler does not have Chinese documentation, and Symfony has not translated this part, so development using DomCrawler can only be explored bit by bit. Now I will summarize the experience in the use process.

The first thing is to install

1 2	`composer` `require` `symfony/dom-crawler` `composer` `require` `symfony/css-selector`

Copy after login

css-seelctor is a css selector. Some functions will be used when selecting nodes with css

The example used in the manual is

use Symfony\Component\DomCrawler\Crawler;
$html = <<<‘HTML‘
Hello World!
Hello Crawler!
HTML;
$crawler = new Crawler($html);
foreach ($crawler as $domElement)
{
var_dump($domElement->nodeName);
}

Copy after login

The printed result is

1	`string ‘html‘ (length=4)`

Copy after login

because of this paragraph The nodeName of the html code is html. My English is not good. When I started using it, I thought the program was wrong. . .

In the actual use process, if new Crawler ($html) will have garbled characters, it should be related to the page encoding, so you can use the following method, first initialize the crawler, and then add node

1 2	`$crawler` `=` `new` `Crawler();` `$crawler->addHtmlContent($html);`

Copy after login

The second parameter of addHtmlContent is charset, and the default is utf-8.

For other examples, please refer to the official documentation, http://symfony.com/doc/current/components/dom_crawler.html

Record the usages that you have tried little by little at work

filterXPath(string $xpath) method, according to the manual, the parameter of this method is $xpath, and p, p and other blocks are often used.

1 2	`echo` `$crawler->filterXPath(‘//body/p‘)->text();` `echo` `$crawler->filterXPath(‘//body/p‘)->last()->text();`

Copy after login

The output is the text of the first and next p tag block

1	`var_dump($crawler->filterXPath(‘//body‘)->html());`

Copy after login

Output the html within the body

foreach ($crawler->filterXPath(‘//body/p‘) as $i => $node) {
$c = new Crawler($node);
echo $c->filter(‘p‘)->text();
}

Copy after login

filterXPath obtains an array of DOMElement blocks. Each DOMElement block can use a new crawler object to continue parsing

$nodeValues =
$crawler->filterXPath(‘//body/p‘)->each(function (Crawler $node, $i) {
return $node->text();
});

Copy after login

crawler provides each loop and uses closure functions to simplify the code. However, please note that this way of writing $nodeValues results in an array, which requires further processing.

Other usage

1	`echo` `$crawler->filterXPath(‘//body/p‘)->attr(‘class‘);`

Copy after login

You can get the value "message" of the class attribute corresponding to the first p tag ”

1 2	`$crawler->filterXPath(‘//p[@class="样式"]‘)->filter(‘a‘)->attr(‘href‘);` `$crawler->filterXPath(‘//p[@class="样式"]‘)->filter(‘a>img‘)->extract(array(‘alt‘, ‘href‘))`

Copy after login

The above are some methods of obtaining tag attributes

Filter and filterXPath are different. The manual says css selector. I don’t quite understand. I It is understood that the elements contained in XPath nodes such as p need to be tried in actual development.

Generally speaking, I feel that DomCrawler is easier to use than simple html dom. Maybe it is because I use it more easily.

The above are just the basic functions of Crawler. For more usage, please refer to the functions in the Crawler part of the symfony manual

http://api.symfony.com/3.2/Symfony/Component/DomCrawler/Crawler .html

The main problem with Crawler is that there are too few examples. There are no usage examples in the function manual, so you can only explore it in actual use. . . .

symfony's documentation about DomCrawler, which contains a few examples

http://symfony.com/doc/current/components/dom_crawler.html

The above is the detailed content of Use Crawler component to analyze HTML in laravel. For more information, please follow other related articles on the PHP Chinese website!