四要素：

首先，確定首頁(yè)URL? ? ?【谷歌瀏覽器 F12】【https://new.qq.com/ch/ent/】

?

---

?

代碼展示：

實(shí)體類News?:

public class News {public String appId;//app_idpublic String title;//titlepublic String intro;//intropublic String source;//sourcepublic String url; //urlpublic String updateTime ;//update_time@Overridepublic String toString() {return "News{" +"appId='" + appId + '\'' +", title='" + title + '\'' +", intro='" + intro + '\'' +", source='" + source + '\'' +", url='" + url + '\'' +", updateTime='" + updateTime + '\'' +'}';}public String getTitle() {return title;}public void setTitle(String title) {this.title = title;}public String getIntro() {return intro;}public void setIntro(String intro) {this.intro = intro;}public String getSource() {return source;}public void setSource(String source) {this.source = source;}public String getUrl() {return url;}public void setUrl(String url) {this.url = url;}@JSONField(name="update_time",format = "yyyy-MM-dd HH:mm:ss")public String getUpdateTime() {return updateTime;}@JSONField(name="update_time",format = "yyyy-MM-dd HH:mm:ss")public void setUpdateTime(String updateTime) {this.updateTime = updateTime;}@JSONField(name="app_id")public String getAppId() {return appId;}@JSONField(name="app_id")public void setAppId(String appId) {this.appId = appId;} }

---

集合類：

public class NewsCollection {//data與json中data相對(duì)應(yīng)public List<News> data;public List<News> getData() {return data;}public void setData(List<News> data) {this.data = data;} }

---

?爬蟲(chóng)類：

?

import com.alibaba.fastjson.JSON; import org.apache.http.HttpEntity; import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpGet; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.util.EntityUtils; import org.jsoup.Jsoup; import org.jsoup.nodes.Document; import org.jsoup.nodes.Element; import org.jsoup.select.Elements; import org.junit.Test;import java.io.File; import java.io.IOException; import java.util.List;public class TestQQ {@Test //根據(jù)標(biāo)簽名稱public void jsoupParse01() throws IOException {String url="https://pacaio.match.qq.com/irs/rcd?cid=146&token=49cbb2154853ef1a74ff4e53723372ce&ext=ent" +"&page=1&callback=__jp6";CloseableHttpClient closeableHttpClient = HttpClients.createDefault();HttpGet httpGet=new HttpGet(url);//執(zhí)行請(qǐng)求 user center pageCloseableHttpResponse closeableHttpResponse = closeableHttpClient.execute(httpGet);HttpEntity entity = closeableHttpResponse.getEntity();String content = EntityUtils.toString(entity, "GB2312");String jsonContent= StringToJson(content);List<News> news=jsonToObject(jsonContent);}/*** string轉(zhuǎn)換成json工具方法* @param srcJson* @return*/public String StringToJson(String srcJson){int start=srcJson.indexOf('(')+1;int end=srcJson.lastIndexOf(')');String jsonContent= srcJson.substring(start,end);return jsonContent;}public List<News> jsonToObject(String srcJson){//通過(guò)fastjson把json字符串轉(zhuǎn)換成java對(duì)象NewsCollection newsCollection = JSON.parseObject(srcJson, NewsCollection.class);List<News> news = newsCollection.getData();for (News newsMessage : news) {System.out.println(newsMessage);System.out.println("---------------------------------------------------------");}return news;} }

?結(jié)果展示：

---

【注】根據(jù)不同頁(yè)數(shù)爬取多頁(yè)信息?

?

?

?

?

總結(jié)

以上是生活随笔為你收集整理的爬虫 spider07——爬取腾讯娱乐新闻的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。