二手車交易價(jià)格預(yù)測(cè)-Task4 建模調(diào)參

四、建模與調(diào)參

Tip:此部分為零基礎(chǔ)入門數(shù)據(jù)挖掘的 Task4 建模調(diào)參 部分，帶你來(lái)了解各種模型以及模型的評(píng)價(jià)和調(diào)參策略，歡迎大家后續(xù)多多交流。

賽題：零基礎(chǔ)入門數(shù)據(jù)挖掘 - 二手車交易價(jià)格預(yù)測(cè)

地址：https://tianchi.aliyun.com/competition/entrance/231784/introduction?spm=5176.12281957.1004.1.38b02448ausjSX

5.1 學(xué)習(xí)目標(biāo)

• 了解常用的機(jī)器學(xué)習(xí)模型，并掌握機(jī)器學(xué)習(xí)模型的建模與調(diào)參流程
• 完成相應(yīng)學(xué)習(xí)打卡任務(wù)

5.2 內(nèi)容介紹

線性回歸模型：

• 線性回歸對(duì)于特征的要求；
• 處理長(zhǎng)尾分布；
• 理解線性回歸模型；

模型性能驗(yàn)證：

• 評(píng)價(jià)函數(shù)與目標(biāo)函數(shù)；
• 交叉驗(yàn)證方法；
• 留一驗(yàn)證方法；
• 針對(duì)時(shí)間序列問(wèn)題的驗(yàn)證；
• 繪制學(xué)習(xí)率曲線；
• 繪制驗(yàn)證曲線；

嵌入式特征選擇：

• Lasso回歸；
• Ridge回歸；
• 決策樹；

模型對(duì)比：

• 常用線性模型；
• 常用非線性模型；

模型調(diào)參：

• 貪心調(diào)參方法；
• 網(wǎng)格調(diào)參方法；
• 貝葉斯調(diào)參方法；

5.3 相關(guān)原理介紹與推薦

由于相關(guān)算法原理篇幅較長(zhǎng)，本文推薦了一些博客與教材供初學(xué)者們進(jìn)行學(xué)習(xí)。

5.3.1 線性回歸模型

https://zhuanlan.zhihu.com/p/49480391

5.3.2 決策樹模型

https://zhuanlan.zhihu.com/p/65304798

5.3.3 GBDT模型

https://zhuanlan.zhihu.com/p/45145899

5.3.4 XGBoost模型

https://zhuanlan.zhihu.com/p/86816771

5.3.5 LightGBM模型

https://zhuanlan.zhihu.com/p/89360721

5.3.6 推薦教材：

• 《機(jī)器學(xué)習(xí)》 https://book.douban.com/subject/26708119/
• 《統(tǒng)計(jì)學(xué)習(xí)方法》 https://book.douban.com/subject/10590856/
• 《Python大戰(zhàn)機(jī)器學(xué)習(xí)》 https://book.douban.com/subject/26987890/
• 《面向機(jī)器學(xué)習(xí)的特征工程》 https://book.douban.com/subject/26826639/
• 《數(shù)據(jù)科學(xué)家訪談錄》 https://book.douban.com/subject/30129410/

5.4 代碼示例

5.4.1 讀取數(shù)據(jù)

import pandas as pd import numpy as np import warnings warnings.filterwarnings('ignore')

reduce_mem_usage 函數(shù)通過(guò)調(diào)整數(shù)據(jù)類型，幫助我們減少數(shù)據(jù)在內(nèi)存中占用的空間

def reduce_mem_usage(df):""" iterate through all the columns of a dataframe and modify the data typeto reduce memory usage. """start_mem = df.memory_usage().sum() print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))for col in df.columns:col_type = df[col].dtypeif col_type != object:c_min = df[col].min()c_max = df[col].max()if str(col_type)[:3] == 'int':if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:df[col] = df[col].astype(np.int8)elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:df[col] = df[col].astype(np.int16)elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:df[col] = df[col].astype(np.int32)elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:df[col] = df[col].astype(np.int64) else:if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:df[col] = df[col].astype(np.float16)elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:df[col] = df[col].astype(np.float32)else:df[col] = df[col].astype(np.float64)else:df[col] = df[col].astype('category')end_mem = df.memory_usage().sum() print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))return df sample_feature = reduce_mem_usage(pd.read_csv('data_for_tree.csv')) Memory usage of dataframe is 60507328.00 MB Memory usage after optimization is: 15724107.00 MB Decreased by 74.0% continuous_feature_names = [x for x in sample_feature.columns if x not in ['price','brand','model','brand']]

5.4.2 線性回歸 & 五折交叉驗(yàn)證 & 模擬真實(shí)業(yè)務(wù)情況

sample_feature = sample_feature.dropna().replace('-', 0).reset_index(drop=True) sample_feature['notRepairedDamage'] = sample_feature['notRepairedDamage'].astype(np.float32) train = sample_feature[continuous_feature_names + ['price']]train_X = train[continuous_feature_names] train_y = train['price']

5.4.2 - 1 簡(jiǎn)單建模

from sklearn.linear_model import LinearRegression model = LinearRegression(normalize=True) model = model.fit(train_X, train_y)

查看訓(xùn)練的線性回歸模型的截距（intercept）與權(quán)重(coef)

'intercept:'+ str(model.intercept_)sorted(dict(zip(continuous_feature_names, model.coef_)).items(), key=lambda x:x[1], reverse=True) [('v_6', 3342612.384537345),('v_8', 684205.534533214),('v_9', 178967.94192530424),('v_7', 35223.07319016895),('v_5', 21917.550249749802),('v_3', 12782.03250792227),('v_12', 11654.925634146672),('v_13', 9884.194615297649),('v_11', 5519.182176035517),('v_10', 3765.6101415594258),('gearbox', 900.3205339198406),('fuelType', 353.5206495542567),('bodyType', 186.51797317460046),('city', 45.17354204168846),('power', 31.163045441455335),('brand_price_median', 0.535967111869784),('brand_price_std', 0.4346788365040235),('brand_amount', 0.15308295553300566),('brand_price_max', 0.003891831020467389),('seller', -1.2684613466262817e-06),('offerType', -4.759058356285095e-06),('brand_price_sum', -2.2430642281682917e-05),('name', -0.00042591632723759166),('used_time', -0.012574429533889028),('brand_price_average', -0.414105722833381),('brand_price_min', -2.3163823428971835),('train', -5.392535065078232),('power_bin', -59.24591853031839),('v_14', -233.1604256172217),('kilometer', -372.96600915402496),('notRepairedDamage', -449.29703564695365),('v_0', -1490.6790578168238),('v_4', -14219.648899108111),('v_2', -16528.55239086934),('v_1', -42869.43976200439)] from matplotlib import pyplot as plt subsample_index = np.random.randint(low=0, high=len(train_y), size=50)

繪制特征v_9的值與標(biāo)簽的散點(diǎn)圖，圖片發(fā)現(xiàn)模型的預(yù)測(cè)結(jié)果（藍(lán)色點(diǎn)）與真實(shí)標(biāo)簽（黑色點(diǎn)）的分布差異較大，且部分預(yù)測(cè)值出現(xiàn)了小于0的情況，說(shuō)明我們的模型存在一些問(wèn)題

plt.scatter(train_X['v_9'][subsample_index], train_y[subsample_index], color='black') plt.scatter(train_X['v_9'][subsample_index], model.predict(train_X.loc[subsample_index]), color='blue') plt.xlabel('v_9') plt.ylabel('price') plt.legend(['True Price','Predicted Price'],loc='upper right') print('The predicted price is obvious different from true price') plt.show() The predicted price is obvious different from true price

通過(guò)作圖我們發(fā)現(xiàn)數(shù)據(jù)的標(biāo)簽（price）呈現(xiàn)長(zhǎng)尾分布，不利于我們的建模預(yù)測(cè)。原因是很多模型都假設(shè)數(shù)據(jù)誤差項(xiàng)符合正態(tài)分布，而長(zhǎng)尾分布的數(shù)據(jù)違背了這一假設(shè)。參考博客：https://blog.csdn.net/Noob_daniel/article/details/76087829

import seaborn as sns print('It is clear to see the price shows a typical exponential distribution') plt.figure(figsize=(15,5)) plt.subplot(1,2,1) sns.distplot(train_y) plt.subplot(1,2,2) sns.distplot(train_y[train_y < np.quantile(train_y, 0.9)]) It is clear to see the price shows a typical exponential distribution<matplotlib.axes._subplots.AxesSubplot at 0x1b33efb2f98>

在這里我們對(duì)標(biāo)簽進(jìn)行了 $log(x+1)$ 變換，使標(biāo)簽貼近于正態(tài)分布

train_y_ln = np.log(train_y + 1) import seaborn as sns print('The transformed price seems like normal distribution') plt.figure(figsize=(15,5)) plt.subplot(1,2,1) sns.distplot(train_y_ln) plt.subplot(1,2,2) sns.distplot(train_y_ln[train_y_ln < np.quantile(train_y_ln, 0.9)]) The transformed price seems like normal distribution<matplotlib.axes._subplots.AxesSubplot at 0x1b33f077160>

model = model.fit(train_X, train_y_ln)print('intercept:'+ str(model.intercept_)) sorted(dict(zip(continuous_feature_names, model.coef_)).items(), key=lambda x:x[1], reverse=True) intercept:23.515920686637713[('v_9', 6.043993029165403),('v_12', 2.0357439855551394),('v_11', 1.3607608712255672),('v_1', 1.3079816298861897),('v_13', 1.0788833838535354),('v_3', 0.9895814429387444),('gearbox', 0.009170812023421397),('fuelType', 0.006447089787635784),('bodyType', 0.004815242907679581),('power_bin', 0.003151801949447194),('power', 0.0012550361843629999),('train', 0.0001429273782925814),('brand_price_min', 2.0721302299502698e-05),('brand_price_average', 5.308179717783439e-06),('brand_amount', 2.8308531339942507e-06),('brand_price_max', 6.764442596115763e-07),('offerType', 1.6765966392995324e-10),('seller', 9.308109838457312e-12),('brand_price_sum', -1.3473184925468486e-10),('name', -7.11403461065247e-08),('brand_price_median', -1.7608143661053008e-06),('brand_price_std', -2.7899058266986454e-06),('used_time', -5.6142735899344175e-06),('city', -0.0024992974087053223),('v_14', -0.012754139659375262),('kilometer', -0.013999175312751872),('v_0', -0.04553774829634237),('notRepairedDamage', -0.273686961116076),('v_7', -0.7455902679730504),('v_4', -0.9281349233755761),('v_2', -1.2781892166433606),('v_5', -1.5458846136756323),('v_10', -1.8059217242413748),('v_8', -42.611729973490604),('v_6', -241.30992120503035)]

再次進(jìn)行可視化，發(fā)現(xiàn)預(yù)測(cè)結(jié)果與真實(shí)值較為接近，且未出現(xiàn)異常狀況

plt.scatter(train_X['v_9'][subsample_index], train_y[subsample_index], color='black') plt.scatter(train_X['v_9'][subsample_index], np.exp(model.predict(train_X.loc[subsample_index])), color='blue') plt.xlabel('v_9') plt.ylabel('price') plt.legend(['True Price','Predicted Price'],loc='upper right') print('The predicted price seems normal after np.log transforming') plt.show() The predicted price seems normal after np.log transforming

5.4.2 - 2 五折交叉驗(yàn)證

在使用訓(xùn)練集對(duì)參數(shù)進(jìn)行訓(xùn)練的時(shí)候，經(jīng)常會(huì)發(fā)現(xiàn)人們通常會(huì)將一整個(gè)訓(xùn)練集分為三個(gè)部分（比如mnist手寫訓(xùn)練集）。一般分為：訓(xùn)練集（train_set），評(píng)估集（valid_set），測(cè)試集（test_set）這三個(gè)部分。這其實(shí)是為了保證訓(xùn)練效果而特意設(shè)置的。其中測(cè)試集很好理解，其實(shí)就是完全不參與訓(xùn)練的數(shù)據(jù)，僅僅用來(lái)觀測(cè)測(cè)試效果的數(shù)據(jù)。而訓(xùn)練集和評(píng)估集則牽涉到下面的知識(shí)了。

因?yàn)樵趯?shí)際的訓(xùn)練中，訓(xùn)練的結(jié)果對(duì)于訓(xùn)練集的擬合程度通常還是挺好的（初始條件敏感），但是對(duì)于訓(xùn)練集之外的數(shù)據(jù)的擬合程度通常就不那么令人滿意了。因此我們通常并不會(huì)把所有的數(shù)據(jù)集都拿來(lái)訓(xùn)練，而是分出一部分來(lái)（這一部分不參加訓(xùn)練）對(duì)訓(xùn)練集生成的參數(shù)進(jìn)行測(cè)試，相對(duì)客觀的判斷這些參數(shù)對(duì)訓(xùn)練集之外的數(shù)據(jù)的符合程度。這種思想就稱為交叉驗(yàn)證（Cross Validation）

from sklearn.model_selection import cross_val_score from sklearn.metrics import mean_absolute_error, make_scorer def log_transfer(func):def wrapper(y, yhat):result = func(np.log(y), np.nan_to_num(np.log(yhat)))return resultreturn wrapper scores = cross_val_score(model, X=train_X, y=train_y, verbose=1, cv = 5, scoring=make_scorer(log_transfer(mean_absolute_error))) [Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers. [Parallel(n_jobs=1)]: Done 5 out of 5 | elapsed: 1.1s finished

使用線性回歸模型，對(duì)未處理標(biāo)簽的特征數(shù)據(jù)進(jìn)行五折交叉驗(yàn)證（Error 1.36）

print('AVG:', np.mean(scores)) AVG: 1.3641908155886227

使用線性回歸模型，對(duì)處理過(guò)標(biāo)簽的特征數(shù)據(jù)進(jìn)行五折交叉驗(yàn)證（Error 0.19）

scores = cross_val_score(model, X=train_X, y=train_y_ln, verbose=1, cv = 5, scoring=make_scorer(mean_absolute_error)) [Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers. [Parallel(n_jobs=1)]: Done 5 out of 5 | elapsed: 1.1s finished print('AVG:', np.mean(scores)) AVG: 0.19382863663604424 scores = pd.DataFrame(scores.reshape(1,-1)) scores.columns = ['cv' + str(x) for x in range(1, 6)] scores.index = ['MAE'] scores cv1cv2cv3cv4cv5MAE

0.191642	0.194986	0.192737	0.195329	0.19445

5.4.2 - 3 模擬真實(shí)業(yè)務(wù)情況

但在事實(shí)上，由于我們并不具有預(yù)知未來(lái)的能力，五折交叉驗(yàn)證在某些與時(shí)間相關(guān)的數(shù)據(jù)集上反而反映了不真實(shí)的情況。通過(guò)2018年的二手車價(jià)格預(yù)測(cè)2017年的二手車價(jià)格，這顯然是不合理的，因此我們還可以采用時(shí)間順序?qū)?shù)據(jù)集進(jìn)行分隔。在本例中，我們選用靠前時(shí)間的4/5樣本當(dāng)作訓(xùn)練集，靠后時(shí)間的1/5當(dāng)作驗(yàn)證集，最終結(jié)果與五折交叉驗(yàn)證差距不大

import datetime sample_feature = sample_feature.reset_index(drop=True) split_point = len(sample_feature) // 5 * 4 train = sample_feature.loc[:split_point].dropna() val = sample_feature.loc[split_point:].dropna()train_X = train[continuous_feature_names] train_y_ln = np.log(train['price'] + 1) val_X = val[continuous_feature_names] val_y_ln = np.log(val['price'] + 1) model = model.fit(train_X, train_y_ln) mean_absolute_error(val_y_ln, model.predict(val_X)) 0.19443858353490887

5.4.2 - 4 繪制學(xué)習(xí)率曲線與驗(yàn)證曲線

from sklearn.model_selection import learning_curve, validation_curve ? learning_curve def plot_learning_curve(estimator, title, X, y, ylim=None, cv=None,n_jobs=1, train_size=np.linspace(.1, 1.0, 5 )): plt.figure() plt.title(title) if ylim is not None: plt.ylim(*ylim) plt.xlabel('Training example') plt.ylabel('score') train_sizes, train_scores, test_scores = learning_curve(estimator, X, y, cv=cv, n_jobs=n_jobs, train_sizes=train_size, scoring = make_scorer(mean_absolute_error)) train_scores_mean = np.mean(train_scores, axis=1) train_scores_std = np.std(train_scores, axis=1) test_scores_mean = np.mean(test_scores, axis=1) test_scores_std = np.std(test_scores, axis=1) plt.grid()#區(qū)域 plt.fill_between(train_sizes, train_scores_mean - train_scores_std, train_scores_mean + train_scores_std, alpha=0.1, color="r") plt.fill_between(train_sizes, test_scores_mean - test_scores_std, test_scores_mean + test_scores_std, alpha=0.1, color="g") plt.plot(train_sizes, train_scores_mean, 'o-', color='r', label="Training score") plt.plot(train_sizes, test_scores_mean,'o-',color="g", label="Cross-validation score") plt.legend(loc="best") return plt plot_learning_curve(LinearRegression(), 'Liner_model', train_X[:1000], train_y_ln[:1000], ylim=(0.0, 0.5), cv=5, n_jobs=1) <module 'matplotlib.pyplot' from 'C:\\ProgramData\\Anaconda3\\lib\\site-packages\\matplotlib\\pyplot.py'>

5.4.3 多種模型對(duì)比

train = sample_feature[continuous_feature_names + ['price']].dropna()train_X = train[continuous_feature_names] train_y = train['price'] train_y_ln = np.log(train_y + 1)

5.4.3 - 1 線性模型 & 嵌入式特征選擇

本章節(jié)默認(rèn)，學(xué)習(xí)者已經(jīng)了解關(guān)于過(guò)擬合、模型復(fù)雜度、正則化等概念。否則請(qǐng)尋找相關(guān)資料或參考如下連接：

• 用簡(jiǎn)單易懂的語(yǔ)言描述「過(guò)擬合 overfitting」？ https://www.zhihu.com/question/32246256/answer/55320482
• 模型復(fù)雜度與模型的泛化能力 http://yangyingming.com/article/434/
• 正則化的直觀理解 https://blog.csdn.net/jinping_shi/article/details/52433975

在過(guò)濾式和包裹式特征選擇方法中，特征選擇過(guò)程與學(xué)習(xí)器訓(xùn)練過(guò)程有明顯的分別。而嵌入式特征選擇在學(xué)習(xí)器訓(xùn)練過(guò)程中自動(dòng)地進(jìn)行特征選擇。嵌入式選擇最常用的是L1正則化與L2正則化。在對(duì)線性回歸模型加入兩種正則化方法后，他們分別變成了嶺回歸與Lasso回歸。

from sklearn.linear_model import LinearRegression from sklearn.linear_model import Ridge from sklearn.linear_model import Lasso models = [LinearRegression(),Ridge(),Lasso()] result = dict() for model in models:model_name = str(model).split('(')[0]scores = cross_val_score(model, X=train_X, y=train_y_ln, verbose=0, cv = 5, scoring=make_scorer(mean_absolute_error))result[model_name] = scoresprint(model_name + ' is finished') LinearRegression is finished Ridge is finished Lasso is finished

對(duì)三種方法的效果對(duì)比

result = pd.DataFrame(result) result.index = ['cv' + str(x) for x in range(1, 6)] result LinearRegressionRidgeLassocv1cv2cv3cv4cv5

0.191642	0.195665	0.382708
0.194986	0.198841	0.383916
0.192737	0.196629	0.380754
0.195329	0.199255	0.385683
0.194450	0.198173	0.383555

model = LinearRegression().fit(train_X, train_y_ln) print('intercept:'+ str(model.intercept_)) sns.barplot(abs(model.coef_), continuous_feature_names) intercept:23.515984499017883<matplotlib.axes._subplots.AxesSubplot at 0x1feb933ca58>

L2正則化在擬合過(guò)程中通常都傾向于讓權(quán)值盡可能小，最后構(gòu)造一個(gè)所有參數(shù)都比較小的模型。因?yàn)橐话阏J(rèn)為參數(shù)值小的模型比較簡(jiǎn)單，能適應(yīng)不同的數(shù)據(jù)集，也在一定程度上避免了過(guò)擬合現(xiàn)象。可以設(shè)想一下對(duì)于一個(gè)線性回歸方程，若參數(shù)很大，那么只要數(shù)據(jù)偏移一點(diǎn)點(diǎn)，就會(huì)對(duì)結(jié)果造成很大的影響；但如果參數(shù)足夠小，數(shù)據(jù)偏移得多一點(diǎn)也不會(huì)對(duì)結(jié)果造成什么影響，專業(yè)一點(diǎn)的說(shuō)法是『抗擾動(dòng)能力強(qiáng)』

model = Ridge().fit(train_X, train_y_ln) print('intercept:'+ str(model.intercept_)) sns.barplot(abs(model.coef_), continuous_feature_names) intercept:5.901527844424091<matplotlib.axes._subplots.AxesSubplot at 0x1fea9056860>

L1正則化有助于生成一個(gè)稀疏權(quán)值矩陣，進(jìn)而可以用于特征選擇。如下圖，我們發(fā)現(xiàn)power與userd_time特征非常重要。

model = Lasso().fit(train_X, train_y_ln) print('intercept:'+ str(model.intercept_)) sns.barplot(abs(model.coef_), continuous_feature_names) intercept:8.674427764003347<matplotlib.axes._subplots.AxesSubplot at 0x1fea90b69b0>

除此之外，決策樹通過(guò)信息熵或GINI指數(shù)選擇分裂節(jié)點(diǎn)時(shí)，優(yōu)先選擇的分裂特征也更加重要，這同樣是一種特征選擇的方法。XGBoost與LightGBM模型中的model_importance指標(biāo)正是基于此計(jì)算的

5.4.3 - 2 非線性模型

除了線性模型以外，還有許多我們常用的非線性模型如下，在此篇幅有限不再一一講解原理。我們選擇了部分常用模型與線性模型進(jìn)行效果比對(duì)。

from sklearn.linear_model import LinearRegression from sklearn.svm import SVC from sklearn.tree import DecisionTreeRegressor from sklearn.ensemble import RandomForestRegressor from sklearn.ensemble import GradientBoostingRegressor from sklearn.neural_network import MLPRegressor from xgboost.sklearn import XGBRegressor from lightgbm.sklearn import LGBMRegressor models = [LinearRegression(),DecisionTreeRegressor(),RandomForestRegressor(),GradientBoostingRegressor(),MLPRegressor(solver='lbfgs', max_iter=100), XGBRegressor(n_estimators = 100, objective='reg:squarederror'), LGBMRegressor(n_estimators = 100)] result = dict() for model in models:model_name = str(model).split('(')[0]scores = cross_val_score(model, X=train_X, y=train_y_ln, verbose=0, cv = 5, scoring=make_scorer(mean_absolute_error))result[model_name] = scoresprint(model_name + ' is finished') LinearRegression is finished DecisionTreeRegressor is finished RandomForestRegressor is finished GradientBoostingRegressor is finished MLPRegressor is finished XGBRegressor is finished LGBMRegressor is finished result = pd.DataFrame(result) result.index = ['cv' + str(x) for x in range(1, 6)] result LinearRegressionDecisionTreeRegressorRandomForestRegressorGradientBoostingRegressorMLPRegressorXGBRegressorLGBMRegressorcv1cv2cv3cv4cv5

0.191642	0.184566	0.136266	0.168626	124.299426	0.168698	0.141159
0.194986	0.187029	0.139693	0.171905	257.886236	0.172258	0.143363
0.192737	0.184839	0.136871	0.169553	236.829589	0.168604	0.142137
0.195329	0.182605	0.138689	0.172299	130.197264	0.172474	0.143461
0.194450	0.186626	0.137420	0.171206	268.090236	0.170898	0.141921

可以看到隨機(jī)森林模型在每一個(gè)fold中均取得了更好的效果

5.4.4 模型調(diào)參

在此我們介紹了三種常用的調(diào)參方法如下：

• 貪心算法 https://www.jianshu.com/p/ab89df9759c8
• 網(wǎng)格調(diào)參 https://blog.csdn.net/weixin_43172660/article/details/83032029
• 貝葉斯調(diào)參 https://blog.csdn.net/linxid/article/details/81189154

## LGB的參數(shù)集合：objective = ['regression', 'regression_l1', 'mape', 'huber', 'fair']num_leaves = [3,5,10,15,20,40, 55] max_depth = [3,5,10,15,20,40, 55] bagging_fraction = [] feature_fraction = [] drop_rate = []

5.4.4 - 1 貪心調(diào)參

best_obj = dict() for obj in objective:model = LGBMRegressor(objective=obj)score = np.mean(cross_val_score(model, X=train_X, y=train_y_ln, verbose=0, cv = 5, scoring=make_scorer(mean_absolute_error)))best_obj[obj] = scorebest_leaves = dict() for leaves in num_leaves:model = LGBMRegressor(objective=min(best_obj.items(), key=lambda x:x[1])[0], num_leaves=leaves)score = np.mean(cross_val_score(model, X=train_X, y=train_y_ln, verbose=0, cv = 5, scoring=make_scorer(mean_absolute_error)))best_leaves[leaves] = scorebest_depth = dict() for depth in max_depth:model = LGBMRegressor(objective=min(best_obj.items(), key=lambda x:x[1])[0],num_leaves=min(best_leaves.items(), key=lambda x:x[1])[0],max_depth=depth)score = np.mean(cross_val_score(model, X=train_X, y=train_y_ln, verbose=0, cv = 5, scoring=make_scorer(mean_absolute_error)))best_depth[depth] = score sns.lineplot(x=['0_initial','1_turning_obj','2_turning_leaves','3_turning_depth'], y=[0.143 ,min(best_obj.values()), min(best_leaves.values()), min(best_depth.values())]) <matplotlib.axes._subplots.AxesSubplot at 0x1fea93f6080>

5.4.4 - 2 Grid Search 調(diào)參

from sklearn.model_selection import GridSearchCV parameters = {'objective': objective , 'num_leaves': num_leaves, 'max_depth': max_depth} model = LGBMRegressor() clf = GridSearchCV(model, parameters, cv=5) clf = clf.fit(train_X, train_y) clf.best_params_ {'max_depth': 15, 'num_leaves': 55, 'objective': 'regression'} model = LGBMRegressor(objective='regression',num_leaves=55,max_depth=15) np.mean(cross_val_score(model, X=train_X, y=train_y_ln, verbose=0, cv = 5, scoring=make_scorer(mean_absolute_error))) 0.13626164479243302

5.4.4 - 3 貝葉斯調(diào)參

from bayes_opt import BayesianOptimization def rf_cv(num_leaves, max_depth, subsample, min_child_samples):val = cross_val_score(LGBMRegressor(objective = 'regression_l1',num_leaves=int(num_leaves),max_depth=int(max_depth),subsample = subsample,min_child_samples = int(min_child_samples)),X=train_X, y=train_y_ln, verbose=0, cv = 5, scoring=make_scorer(mean_absolute_error)).mean()return 1 - val rf_bo = BayesianOptimization(rf_cv,{'num_leaves': (2, 100),'max_depth': (2, 100),'subsample': (0.1, 1),'min_child_samples' : (2, 100)} ) rf_bo.maximize() | iter | target | max_depth | min_ch... | num_le... | subsample | ------------------------------------------------------------------------- | [0m 1 [0m | [0m 0.8649 [0m | [0m 89.57 [0m | [0m 47.3 [0m | [0m 55.13 [0m | [0m 0.1792 [0m | | [0m 2 [0m | [0m 0.8477 [0m | [0m 99.86 [0m | [0m 60.91 [0m | [0m 15.35 [0m | [0m 0.4716 [0m | | [95m 3 [0m | [95m 0.8698 [0m | [95m 81.74 [0m | [95m 83.32 [0m | [95m 92.59 [0m | [95m 0.9559 [0m | | [0m 4 [0m | [0m 0.8627 [0m | [0m 90.2 [0m | [0m 8.754 [0m | [0m 43.34 [0m | [0m 0.7772 [0m | | [0m 5 [0m | [0m 0.8115 [0m | [0m 10.07 [0m | [0m 86.15 [0m | [0m 4.109 [0m | [0m 0.3416 [0m | | [95m 6 [0m | [95m 0.8701 [0m | [95m 99.15 [0m | [95m 9.158 [0m | [95m 99.47 [0m | [95m 0.494 [0m | | [0m 7 [0m | [0m 0.806 [0m | [0m 2.166 [0m | [0m 2.416 [0m | [0m 97.7 [0m | [0m 0.224 [0m | | [0m 8 [0m | [0m 0.8701 [0m | [0m 98.57 [0m | [0m 97.67 [0m | [0m 99.87 [0m | [0m 0.3703 [0m | | [95m 9 [0m | [95m 0.8703 [0m | [95m 99.87 [0m | [95m 43.03 [0m | [95m 99.72 [0m | [95m 0.9749 [0m | | [0m 10 [0m | [0m 0.869 [0m | [0m 10.31 [0m | [0m 99.63 [0m | [0m 99.34 [0m | [0m 0.2517 [0m | | [95m 11 [0m | [95m 0.8703 [0m | [95m 52.27 [0m | [95m 99.56 [0m | [95m 98.97 [0m | [95m 0.9641 [0m | | [0m 12 [0m | [0m 0.8669 [0m | [0m 99.89 [0m | [0m 8.846 [0m | [0m 66.49 [0m | [0m 0.1437 [0m | | [0m 13 [0m | [0m 0.8702 [0m | [0m 68.13 [0m | [0m 75.28 [0m | [0m 98.71 [0m | [0m 0.153 [0m | | [0m 14 [0m | [0m 0.8695 [0m | [0m 84.13 [0m | [0m 86.48 [0m | [0m 91.9 [0m | [0m 0.7949 [0m | | [0m 15 [0m | [0m 0.8702 [0m | [0m 98.09 [0m | [0m 59.2 [0m | [0m 99.65 [0m | [0m 0.3275 [0m | | [0m 16 [0m | [0m 0.87 [0m | [0m 68.97 [0m | [0m 98.62 [0m | [0m 98.93 [0m | [0m 0.2221 [0m | | [0m 17 [0m | [0m 0.8702 [0m | [0m 99.85 [0m | [0m 63.74 [0m | [0m 99.63 [0m | [0m 0.4137 [0m | | [0m 18 [0m | [0m 0.8703 [0m | [0m 45.87 [0m | [0m 99.05 [0m | [0m 99.89 [0m | [0m 0.3238 [0m | | [0m 19 [0m | [0m 0.8702 [0m | [0m 79.65 [0m | [0m 46.91 [0m | [0m 98.61 [0m | [0m 0.8999 [0m | | [0m 20 [0m | [0m 0.8702 [0m | [0m 99.25 [0m | [0m 36.73 [0m | [0m 99.05 [0m | [0m 0.1262 [0m | | [0m 21 [0m | [0m 0.8702 [0m | [0m 85.51 [0m | [0m 85.34 [0m | [0m 99.77 [0m | [0m 0.8917 [0m | | [0m 22 [0m | [0m 0.8696 [0m | [0m 99.99 [0m | [0m 38.51 [0m | [0m 89.13 [0m | [0m 0.9884 [0m | | [0m 23 [0m | [0m 0.8701 [0m | [0m 63.29 [0m | [0m 97.93 [0m | [0m 99.94 [0m | [0m 0.9585 [0m | | [0m 24 [0m | [0m 0.8702 [0m | [0m 93.04 [0m | [0m 71.42 [0m | [0m 99.94 [0m | [0m 0.9646 [0m | | [0m 25 [0m | [0m 0.8701 [0m | [0m 99.73 [0m | [0m 16.21 [0m | [0m 99.38 [0m | [0m 0.9778 [0m | | [0m 26 [0m | [0m 0.87 [0m | [0m 86.28 [0m | [0m 58.1 [0m | [0m 99.47 [0m | [0m 0.107 [0m | | [0m 27 [0m | [0m 0.8703 [0m | [0m 47.28 [0m | [0m 99.83 [0m | [0m 99.65 [0m | [0m 0.4674 [0m | | [0m 28 [0m | [0m 0.8703 [0m | [0m 68.29 [0m | [0m 99.51 [0m | [0m 99.4 [0m | [0m 0.2757 [0m | | [0m 29 [0m | [0m 0.8701 [0m | [0m 76.49 [0m | [0m 73.41 [0m | [0m 99.86 [0m | [0m 0.9394 [0m | | [0m 30 [0m | [0m 0.8695 [0m | [0m 37.27 [0m | [0m 99.87 [0m | [0m 89.87 [0m | [0m 0.7588 [0m | ========================================================================= 1 - rf_bo.max['target'] 0.1296693644053145

總結(jié)

在本章中，我們完成了建模與調(diào)參的工作，并對(duì)我們的模型進(jìn)行了驗(yàn)證。此外，我們還采用了一些基本方法來(lái)提高預(yù)測(cè)的精度，提升如下圖所示。

plt.figure(figsize=(13,5)) sns.lineplot(x=['0_origin','1_log_transfer','2_L1_&_L2','3_change_model','4_parameter_turning'], y=[1.36 ,0.19, 0.19, 0.14, 0.13]) <matplotlib.axes._subplots.AxesSubplot at 0x1feac73ceb8>

總結(jié)

以上是生活随笔為你收集整理的【算法竞赛学习】二手车交易价格预测-Task4建模调参的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。