Web crawler - python crawler, the data to be crawled is not in the source code of the web page, how to deal with it?

Question

I want to capture the scores of live broadcast events: live broadcast links, but I can only capture the date, home and away teams, etc. using Scrapy. My question is: are the scores loaded via Script? Is it in bf4.js in this script? How to accurately obtain the score as shown below in this Request Method: GET situation?
The source code of the web page viewed through the developer tools is as follows:
<li label="Football, China League One, Wuhan Zall" id="saishi93287" data-time="2017-06- 24 15:00">15:00 <b>China League One 15th Round</b> Yunnan Lijiang<img src="//duihui.qiumibao.com/zuqiu/yunnanlijiang.png" > < span> - </span> <img src="//duihui.qiumibao.com/zuqiu/wuhanzhuoer.png" > Wuhan Zall<a href="/zhibo/zuqiu/2017/0624yunnanlijiangvswuhanzhuoer.htm" target="_blank">Wuhan Culture and Sports Guangzhou Competition PPTV</a> <a href="http://wenzi.zhibo8.cc/zhibo/zuqiu/2017/0624yunnanlijiangvswuhanzhuoer.htm" target="_blank">Text </a> <a href="//www.zhibo8.cc/shouji.htm" target="_blank">Watch live broadcast on mobile phone</a> <a href="http://www. 188bifen.com/" target="_blank">score</a> <a href="http://ogzq2.wanjiashe.com/game.php?sid=51&zhibo" target="_blank">UEFA Champions League New football uniform</a> </li>

That is, the score is not displayed in <span> - </span>. How can we capture the rendered score webpage?

Answer 1

数据在https://bifen4pc.qiumibao.com/json/list.htm?74520

Answer 2

Then use Selenium to get it,

Link description