HttpClient

網(wǎng)絡(luò)爬蟲就是用程序幫助我們訪問網(wǎng)絡(luò)上的資源，我們一直以來都是使用HTTP協(xié)議訪問互聯(lián)網(wǎng)的網(wǎng)頁，網(wǎng)絡(luò)爬蟲需要編寫程序，在這里使用同樣的HTTP協(xié)議訪問網(wǎng)頁。

這里使用Java的HTTP協(xié)議客戶端 HttpClient這個技術(shù)，來實現(xiàn)抓取網(wǎng)頁數(shù)據(jù)。

---

GET請求?

package cn.itcast.crawler.test;import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpGet; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.util.EntityUtils;import java.io.IOException;public class HttpGetTest {public static void main(String[] args) {//創(chuàng)建HttpClient對象CloseableHttpClient httpClient = HttpClients.createDefault();//創(chuàng)建HttpGet對象，設(shè)置url訪問地址HttpGet httpGet = new HttpGet("http://www.itcast.cn");CloseableHttpResponse response = null;try {//使用HttpClient發(fā)起請求，獲取responseresponse = httpClient.execute(httpGet);//解析響應(yīng)if (response.getStatusLine().getStatusCode() == 200) {String content = EntityUtils.toString(response.getEntity(), "utf8");System.out.println(content.length());}} catch (IOException e) {e.printStackTrace();}finally {//關(guān)閉responsetry {response.close();} catch (IOException e) {e.printStackTrace();}try {httpClient.close();} catch (IOException e) {e.printStackTrace();}}} }

---

---

GET請求

帶參數(shù)的GET請求

---

package cn.itcast.crawler.test;import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpGet; import org.apache.http.client.utils.URIBuilder; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.util.EntityUtils;import java.io.IOException; import java.net.URISyntaxException;public class HttpGetParamTest {public static void main(String[] args) throws Exception {//創(chuàng)建HttpClient對象CloseableHttpClient httpClient = HttpClients.createDefault();//設(shè)置請求地址是：http://yun.itheima.com/search?keys=Java//創(chuàng)建URIBuilderURIBuilder uriBuilder = new URIBuilder("http://yun.itheima.com/search");//設(shè)置參數(shù)uriBuilder.setParameter("keys","Java");//創(chuàng)建HttpGet對象，設(shè)置url訪問地址HttpGet httpGet = new HttpGet(uriBuilder.build());System.out.println("發(fā)起請求的信息："+httpGet);CloseableHttpResponse response = null;try {//使用HttpClient發(fā)起請求，獲取responseresponse = httpClient.execute(httpGet);//解析響應(yīng)if (response.getStatusLine().getStatusCode() == 200) {String content = EntityUtils.toString(response.getEntity(), "utf8");System.out.println(content.length());}} catch (IOException e) {e.printStackTrace();}finally {//關(guān)閉responsetry {response.close();} catch (IOException e) {e.printStackTrace();}try {httpClient.close();} catch (IOException e) {e.printStackTrace();}}} }

總結(jié)

以上是生活随笔為你收集整理的HttpClient ||GET请求||带参数的GET请求的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。