Cnbolg Crawl

a). 加載用到的R包

##library packages needed in this case
library(proto) library(gsubfn)

## Warning in doTryCatch(return(expr), name, parentenv, handler): 無法載入共享目標對象‘/Library/Frameworks/R.framework/Resources/modules//R_X11.so’：:
##   dlopen(/Library/Frameworks/R.framework/Resources/modules//R_X11.so, 6): Library not loaded: /opt/X11/lib/libSM.6.dylib
##   Referenced from: /Library/Frameworks/R.framework/Resources/modules//R_X11.so
##   Reason: image not found

## Could not load tcltk.  Will use slower R code instead.

library(bitops)
library(rvest) library(stringr) library(DBI) library(RSQLite) library(sqldf) library(RCurl) library(ggplot2) library(sp) library(raster) ##由于我們的電腦一般是中文環境，但是我想要Monday，Tuesday，所以，這時需要增加設置參數 ##來告知系統采用英文（北美）環境用法。 Sys.setlocale("LC_TIME", "C")

## [1] "C"

b). 自定義一個函數，后續用于爬取信息。

## Create a function,the parameter 'i' means page number.
getdata <- function(i){ url <- paste0("www.cnblogs.com/p",i)##generate url combined_info <- url%>%html_session()%>%html_nodes("div.post_item div.post_item_foot")%>%html_text()%>%strsplit(split="\r\n") post_date <- sapply(combined_info, function(v) return(v[3]))%>%str_sub(9,24)%>%as.POSIXlt()##get the date post_year <- post_date$year+1900 post_month <- post_date$mon+1 post_day <- post_date$mday post_hour <- post_date$hour post_weekday <- weekdays(post_date) title <- url%>%html_session()%>%html_nodes("div.post_item h3")%>%html_text()%>%as.character()%>%trim() link <- url%>%html_session()%>%html_nodes("div.post_item a.titlelnk")%>%html_attr("href")%>%as.character() author <- url%>%html_session()%>%html_nodes("div.post_item a.lightblue")%>%html_text()%>%as.character()%>%trim() author_hp <- url%>%html_session()%>%html_nodes("div.post_item a.lightblue")%>%html_attr("href")%>%as.character() recommendation <- url%>%html_session()%>%html_nodes("div.post_item span.diggnum")%>%html_text()%>%trim()%>%as.numeric() article_view <- url%>%html_session()%>%html_nodes("div.post_item span.article_view")%>%html_text()%>%str_sub(4,20) article_view <- gsub(")","",article_view)%>%trim

轉載于:https://www.cnblogs.com/litao1105/p/4415665.html

總結

以上是生活随笔為你收集整理的R 语言爬虫 之 cnblog博文爬取的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。