閑來無事，玩了個成語接龍，于是就想用python爬取下成語網(wǎng)站上的成語，直接上代碼:

#coding=utf-8

import requests
from bs4 import BeautifulSoup
import sqlite3
import uuid

conn = sqlite3.connect("idiombase.db3")  #創(chuàng)建sqlite.db數(shù)據(jù)庫
print ("open database success")
conn.execute("drop table IF EXISTS idiom")
query = """create table IF NOT EXISTS idiom(
    id VARCHAR(50),
    word VARCHAR(50)
);"""
conn.execute(query)
print ("Table created successfully")

all_url = 'http://chengyu.t086.com/'


#http請求頭
Hostreferer = {
    'User-Agent':'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)',
    'Referer':'http://chengyu.t086.com/'
}

word=['A','B','C','D','E','F','G','H','J','K','L','M','N','O','P','Q','R','S','T','W','X','Y','Z']

for w in word:

    for n in range(1,100):

        url=all_url+'list/'+w+'_'+str(n)+'.html'
       
        start_html = requests.get(url,headers = Hostreferer)
        if(start_html.status_code==404):
            break
        start_html.encoding='gb2312'
        soup = BeautifulSoup(start_html.text,"html.parser")

        listw = soup.find('div',class_='listw')
        
        lista = listw.find_all('a')
        for p in lista:
            print(p.text)
            ids=str(uuid.uuid1())
            query = "insert into idiom (id,word) values ('"+ids+"','"+p.text+"');"
            conn.execute(query)
            conn.commit()

取到數(shù)據(jù)后，保存在里sqlite數(shù)據(jù)庫中，以后想用的時候隨便取

總結(jié)

以上是生活随笔為你收集整理的[代码]--python爬虫联系--爬取成语的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。