#目標：獲取上交所和深交所所有股票的名稱和交易信息 #輸出：保存到文件中 #技術路線：requests-bs4-re#候選網站選取原則： # 股票信息靜態存在與html界面中，非js代碼生成，沒有robots協議限制 #選取心態： # 不要糾結于某個網站，多找信息源進行嘗試#程序結構程序設計 #步驟1：從東方財富網獲取股票列表 #步驟2：根據股票列表逐個到百度股票獲取個股信息 #步驟3：將結果存儲到文件中import requests from bs4 import BeautifulSoup import traceback import redef getHtTMLText(url):try:r = requests.get(url)r.raise_for_status()r.encoding = r.apparent_encodingreturn r.text except:return ''def getStockList(lst,stockURL):html = getHtTMLText(stockURL)soup = BeautifulSoup(html,'html.parser')a = soup.find_all('a')for i in a:try:href = i.attrs['href']number = re.findall(r'[s][hz]\d{6}',href)[0]if number is not '':lst.append(number)except:continuedef getStockInfo(lst,stockURL,fpath):count = 0for stock in lst:url = stockURL + stock + '.html'html = getHtTMLText(url)try:if html == '':continue infoDict = {}soup = BeautifulSoup(html,'html.parser')stockInfo = soup.find('div',attrs = {'class':'stock-bets'})name = stockInfo.find_all(attrs = {'class':'bets-name'})[0]infoDict.update({'股票名稱':name.text.split()[0]})print('\n'+url)print({'股票名稱':name.text.split()[0]}) keyList = stockInfo.find_all('dt')valueList = stockInfo.find_all('dd')for i in range(len(keyList)):key = keyList[i].string.strip()val = valueList[i].string.strip()infoDict[key] = valprint('\t' + key + ':' + val)with open(fpath,'a',encoding = 'utf-8') as f:f.write(str(infoDict)+'\n')count = count + 1print('\r當前進度：{:.2f}%'.format(count*100/len(lst)),end = '')except:count = count + 1print('\r當前進度：{:.2f}%'.format(count*100/len(lst)),end = '')#traceback.print_exc()continuedef main():stock_list_url = 'http://quote.eastmoney.com/stocklist.html'stock_info_url = 'http://gupiao.baidu.com/stock/'output_file = 'D://BaiduStockInfo.txt'slist = []getStockList(slist,stock_list_url)getStockInfo(slist,stock_info_url,output_file)main()

?

轉載于:https://www.cnblogs.com/zhanghaijie/p/8418264.html

總結

以上是生活随笔為你收集整理的爬取股票信息的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。