Jadi saya mempunyai kod yang mengalih keluar nama+harga mineral daripada 14 halaman (setakat ini) dan menyimpannya ke fail .txt. Saya mula-mula cuba menggunakan Halaman1 sahaja, kemudian saya ingin menambah lebih banyak halaman untuk mendapatkan lebih banyak data. Tetapi kemudian kod itu mengambil sesuatu yang tidak sepatutnya - nama/rentetan rawak. Saya tidak menjangka ia akan merebut yang itu, tetapi ia berjaya dan memberikan harga yang salah kepada yang ini! Ia berlaku selepas mineral mempunyai "nama yang tidak dijangka" ini dan kemudian keseluruhan senarai yang lain mempunyai harga yang salah. Lihat di bawah:
Jadi oleh kerana rentetan ini berbeza daripada rentetan lain kod selanjutnya tidak boleh memisahkannya dan memberikan ralat:
cutted2 = split2.pop(1)
^^^^^^^^^^^^^
IndexError: pop index out of range
Saya cuba mengabaikan ralat ini dan menggunakan salah satu kaedah yang digunakan dalam halaman Stackoverflow yang berbeza:
try: cutted2 = split2.pop(1) except IndexError: continue
Ia sebenarnya berkesan, tiada ralat berlaku...tetapi kemudian ia memberikan harga yang salah kepada mineral yang salah (seperti yang saya perhatikan)! Bagaimanakah saya boleh menukar kod untuk mengabaikan nama "pelik" ini dan meneruskan senarai? Di bawah ialah kod penuh, saya ingat ia berhenti di URL5 dan memberikan ralat indeks pop timbul ini:
import requests
from bs4 import BeautifulSoup
import re
def collecter(URL):
headers = {"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"}
soup = BeautifulSoup(requests.get(URL, headers=headers).text, "lxml")
names = [n.getText(strip=True) for n in soup.select("table tr td font a")]
prices = [
p.getText(strip=True).split("Price:")[-1] for p
in soup.select("table tr td font font")
]
names[:] = [" ".join(n.split()) for n in names if not n.startswith("[")]
prices[:] = [p for p in prices if p]
with open("Minerals.txt", "a+", encoding='utf-8') as file:
for name, price in zip(names, prices):
# print(f"{name}\n{price}")
# print("-" * 50)
filename = str(name)+" "+str(price)+"\n"
split1 = filename.split(' / ')
cutted1 = split1.pop(0)
split2 = cutted1.split(": ")
try:
cutted2 = split2.pop(1)
except IndexError:
continue
two_prices = cutted2+" "+split1.pop(0)+"\n"
file.write(two_prices)
URL1 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=0"
URL2 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=25"
URL3 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=50"
URL4 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=75"
URL5 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=100"
URL6 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=125"
URL7 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=150"
URL8 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=175"
URL9 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=200"
URL10 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=225"
URL11 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=250"
URL12 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=275"
URL13 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=300"
URL14 = "https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First=325"
collecter(URL1)
collecter(URL2)
collecter(URL3)
collecter(URL4)
collecter(URL5)
collecter(URL6)
collecter(URL7)
collecter(URL8)
collecter(URL9)
collecter(URL10)
collecter(URL11)
collecter(URL12)
collecter(URL13)
collecter(URL14)
EDIT: Berikut ialah kod yang berfungsi sepenuhnya di bawah, terima kasih kepada pembantu!
import requests
from bs4 import BeautifulSoup
import re
for URL in range(0,2569,25):
headers = {"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"}
soup = BeautifulSoup(requests.get(f'https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First={URL}', headers=headers).text, "lxml")
names = [n.getText(strip=True) for n in soup.select("table tr td font>a")]
prices = [p.getText(strip=True).split("Price:")[-1] for p in soup.select("table tr td font>font")]
names[:] = [" ".join(n.split()) for n in names if not n.startswith("[") ]
prices[:] = [p for p in prices if p]
with open("MineralsList.txt", "a+", encoding='utf-8') as file:
for name, price in zip(names, prices):
# print(f"{name}\n{price}")
# print("-" * 50)
filename = str(name)+" "+str(price)+"\n"
split1 = filename.split(' / ')
cutted1 = split1.pop(0)
split2 = cutted1.split(": ")
cutted2 = split2.pop(1)
try:
two_prices = cutted2+" "+split1.pop(0)+"\n"
except IndexError:
two_prices = cutted2+"\n"
file.write(two_prices)
Tetapi selepas beberapa perubahan ia berhenti dengan ralat baharu - ia tidak dapat mencari rentetan oleh harta yang diberikan, maka ralat "IndexError: muncul dari senarai kosong"... malah soup.select("table tr td font>font" ) memberikan bantuan seperti yang dilakukan seperti yang dilakukan dalam "nama"
![[Web front-end] Permulaan pantas Node.js](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
Anda boleh mencuba contoh seterusnya bersama penomboran
import requests from bs4 import BeautifulSoup for URL in range(0,100,25): headers = {"User-Agent": "Mozilla/5.0"} soup = BeautifulSoup(requests.get(f'https://www.fabreminerals.com/search_results.php?LANG=EN&SearchTerms=&submit=Buscar&MineralSpeciment=&Country=&Locality=&PriceRange=&checkbox=enventa&First={URL}', headers=headers).text, "lxml") names = [ x.get_text(strip=True) for x in soup.select('table tr td font a')][:25] print(names) prices = [ x.get_text(strip=True) for x in soup.select('table tr td font:nth-child(3)')][:25] print(prices) # with open("Minerals.txt", "a+", encoding='utf-8') as file: # for name, price in zip(names, prices): # # print(f"{name}\n{price}") # # print("-" * 50) # filename = str(name)+" "+str(price)+"\n" # split1 = filename.split(' / ') # cutted1 = split1.pop(0) # split2 = cutted1.split(": ") # try: # cutted2 = split2.pop(1) # except IndexError: # continue # two_prices = cutted2+" "+split1.pop(0)+"\n" # file.write(two_prices)Output:
Anda hanya perlu menjadikan pemilih CSS lebih khusus supaya ia hanya mengenal pasti pautan yang berada terus di dalam elemen fon (bukan beberapa lapisan ke bawah):
soup.select("table tr td font>a")Menambah syarat lanjut bahawa pautan menghala ke satu item dan bukannya pautan halaman seterusnya/sebelumnya di bahagian bawah halaman juga akan membantu:
soup.select("table tr td font>a[href*='CODE']")