Home > Backend Development > PHP Tutorial > How does Goutte get the url in the a tag, or a useful PHP crawler library, thank you

How does Goutte get the url in the a tag, or a useful PHP crawler library, thank you

WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWB
Release: 2016-08-08 09:06:35
Original
1364 people have browsed it

How does Goutte get the url in the a tag? Or a useful PHP crawler library, thank you

<code><?php
require('./Vendor/autoload.php');
use Goutte\Client;

/**
* 
*/
class Spider 
{
    private $_client;
    private $_crawler;
    public  $_news = [
        'title'   => [],
        'link'    => [],
        'content' => [],
        'source'  => [],
        'date'    => [],
    ];

    public function __construct()
    {
        try {
            $this->_client  = new Client();
            $this->_crawler = $this->_client->request('GET', 'http://www.ningshan.gov.cn/Category_90/Index.aspx');
            // $client->getClient()->setDefaultOption('config/curl/'.CURLOPT_TIMEOUT, 10);
        } catch (Exception $e) {
            throw new \Exception($e->getMessage(), 1);
        }
    }

    public function getDate()
    {
        $this->_crawler->filter('div#list>ul>li>span')->each(function ($node) {
           $this->_news['date'][] = $node->text();
        });
    }

    public function getTitle()
    {
        $link = $this->_crawler->selectLink('宁陕县召开政协八届二十二次次常委会')->link();
        var_dump($link->getUri);die;
        $this->_crawler->filter('div#list>ul>li>a')->each(function ($node) {
           if ($node->text() !== '宁陕要闻') {
                $this->_news['title'][]  = $node->text();
                $this->_news['link'][]   = $node->link();
                $this->_news['source'][] = '宁陕要闻';
           }
        });
    }
}

//-----------------------------------
try {
    $spider = new Spider();
    $spider->getDate();
    $spider->getTitle();

    echo json_encode($spider->_news, JSON_UNESCAPED_UNICODE);
} catch (Exception $e) {
    echo $e->getMessage();
}

</code>
Copy after login
Copy after login

Reply content:

How does Goutte get the url in the a tag? Or a useful PHP crawler library, thank you

<code><?php
require('./Vendor/autoload.php');
use Goutte\Client;

/**
* 
*/
class Spider 
{
    private $_client;
    private $_crawler;
    public  $_news = [
        'title'   => [],
        'link'    => [],
        'content' => [],
        'source'  => [],
        'date'    => [],
    ];

    public function __construct()
    {
        try {
            $this->_client  = new Client();
            $this->_crawler = $this->_client->request('GET', 'http://www.ningshan.gov.cn/Category_90/Index.aspx');
            // $client->getClient()->setDefaultOption('config/curl/'.CURLOPT_TIMEOUT, 10);
        } catch (Exception $e) {
            throw new \Exception($e->getMessage(), 1);
        }
    }

    public function getDate()
    {
        $this->_crawler->filter('div#list>ul>li>span')->each(function ($node) {
           $this->_news['date'][] = $node->text();
        });
    }

    public function getTitle()
    {
        $link = $this->_crawler->selectLink('宁陕县召开政协八届二十二次次常委会')->link();
        var_dump($link->getUri);die;
        $this->_crawler->filter('div#list>ul>li>a')->each(function ($node) {
           if ($node->text() !== '宁陕要闻') {
                $this->_news['title'][]  = $node->text();
                $this->_news['link'][]   = $node->link();
                $this->_news['source'][] = '宁陕要闻';
           }
        });
    }
}

//-----------------------------------
try {
    $spider = new Spider();
    $spider->getDate();
    $spider->getTitle();

    echo json_encode($spider->_news, JSON_UNESCAPED_UNICODE);
} catch (Exception $e) {
    echo $e->getMessage();
}

</code>
Copy after login
Copy after login

Looking for it now

<code>$crawler = $client->request('GET', 'http://www.symfony.com/blog/');
$link = $crawler->selectLink('Security Advisories')->link();
print_r($link->getUri());</code>
Copy after login

Manual: http://symfony.com/doc/curren...
GIT: https://github.com/FriendsOfP...

Collection reference: http://flc.ren/2016/06/528.html

Related labels:
php
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Latest Issues
php data acquisition?
From 1970-01-01 08:00:00
0
0
0
PHP extension intl
From 1970-01-01 08:00:00
0
0
0
How to learn php well
From 1970-01-01 08:00:00
0
0
0
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template