今天不詳細講代碼邏輯，只講如何入庫
mongo的入庫

from pyspider.libs.base_handler import * from lxml import etree import pymongoclass Handler(BaseHandler):crawl_config = {}def __init__(self):connection = pymongo.MongoClient(host='192.168.180.128',port=27017)client = connection['lianjia']self.db = client['items']@every(minutes=24 * 60)def on_start(self):for i in range(1,101):self.crawl('https://dg.lianjia.com/ershoufang/pg{}'.format(i), callback=self.index_page)@config(age=10 * 24 * 60 * 60)def index_page(self, response):seletor = etree.HTML(response.text)urls =seletor.xpath('//li[@class ="clear LOGCLICKDATA"]/a/@href')for url in urls :self.crawl(url, callback=self.detail_page)@config(priority=2)def detail_page(self,

總結

以上是生活随笔為你收集整理的pyspider爬链家网入库遇到的坑的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。