網絡爬蟲+模擬瀏覽器(獲取有權限網站資源)：

獲取URL

下載資源

分析

處理

public class http {

public static void main(String[]args) throws Exception

{

//http+s更安全

//URL.openStream()打開于URL的連接，并返回一個InputStream用于從連接中讀取數據

//獲取URL

URL url=new URL("https://www.jd.com");

//下載資源

InputStream is = url.openStream();

BufferedReader br=new BufferedReader(new InputStreamReader(is,"UTF-8"));;

String msg=null;

while((msg=br.readLine())!=null)

{

System.out.println(msg);

}

br.close();

}

}

獲取有權限網絡資源:

public class http {

public static void main(String[]args) throws Exception

{

//.openConnectio，,返回一個URLConnection實例表示由所引用的遠程對象的連接URL

//URLConnection的子類有HttpURLConnection和JarURLConnection

URL url=new URL("https://www.jd.com");

//下載資源

HttpURLConnection conn=(HttpURLConnection)url.openConnection();

conn.setRequestMethod("GET");//模擬瀏覽器得get請求

conn.setRequestProperty( "User-Agent","Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/18.17763");

BufferedReader br=new BufferedReader(new InputStreamReader(conn.getInputStream(),"UTF-8"));

String msg=null;

while((msg=br.readLine())!=null)

{

System.out.println(msg);

}

br.close();

}

}

總結

以上是生活随笔為你收集整理的java 浏览器 爬虫_java 网络编程-爬虫+模拟浏览器的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。