思想極度簡單

應用數學知識少

效果好(缺點？)

可以解釋機器學習算法使用過程中的很多細節問題

更完整的刻畫機器學習應用的流程

?

?

distances = [] for x_train in X_train:d=sqrt(np.sum((x_train-x)**2))distances.append(d) View Code distances=[sqrt(np.sum((x_train-x)**2)) for x_train in X_train]

?可以說kNN是一個不需要訓練過程的算法

?K近鄰算法是非常特殊的，可以被認為是沒有模型的算法

為了和其他算法統一，可以認為訓練數據集就是模型本身

kNN： from sklearn.neighbors import KNeighborsClassifier kNN_classifier=KNeighborsClassifier(n_neighbors=6) kNN_classifier.fit(X_train,y_train) kNN_classifier.predict(x)

有關K近鄰算法

解決分類問題

天然可以解決多分類問題

思想簡單，效果強大

使用k近鄰算法解決回歸問題

KNeighborsRegressor

kNN： from sklearn.neighbors import KNeighborsClassifier kNN_classifier=KNeighborsClassifier(n_neighbors=6) kNN_classifier.fit(X_train,y_train) kNN_classifier.predict(x)

?

?

?

?

?

須考慮距離的權重！通常是將距離的倒數作為權重

?

?

?

?

?

相當于因為距離又獲得了一個超參數

?

尋找最好的k,調參 best_score = 0.0 besk_k = -1 for k in range(1,11):knn_clf = KNeighborsClassifier(n_neighbors=k)knn_clf.fit(X_train,y_train)score = knn_clf.score(X_test,y_test)if score>best_score:best_k=kbest_score=scoreprint('best_k=',best_k) print('best_score=',best_score)考慮距離？ best_method = '' best_score = 0.0 besk_k = -1 for method in ['uniform','distance']:for k in range(1,11):knn_clf = KNeighborsClassifier(n_neighbors=k,weights=method)knn_clf.fit(X_train,y_train)score = knn_clf.score(X_test,y_test)if score>best_score:best_k=kbest_score=scorebest_method = method print('best_k=',best_k) print('best_score=',best_score) print('best_method',best_method)搜索明可夫斯基距離相應的p %%time best_p = -1 best_score = 0.0 besk_k = -1 for k in range(1,11):for p in range(1,6):knn_clf = KNeighborsClassifier(n_neighbors=k,weights='distance',p = p)knn_clf.fit(X_train,y_train)score = knn_clf.score(X_test,y_test)if score>best_score:best_k=kbest_score=scorebest_p=p print('best_k=',best_k) print('best_score=',best_score) print('best_p=',best_p) View Code

?

缺點2：高度數據相關

缺點3:預測的結果不具有可解釋性

缺點4：維數災難

隨著維度的增加，‘看似相近’的的兩個點之間的距離越來越大

解決方法：降維(PCA)

# coding=utf-8 import numpy as np from sklearn import datasets from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn.neighbors import KNeighborsClassifier from sklearn.metrics import accuracy_score # 分類的準確度 from sklearn.model_selection import GridSearchCViris = datasets.load_iris() X = iris.data y = iris.target X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2, random_state=666) standardScaler = StandardScaler() # 創建實例 standardScaler.fit(X_train) # standardScaler.mean_ # standardScaler.scale_ X_train = standardScaler.transform(X_train) # 使用transform方法進行歸一化 X_test_standard = standardScaler.transform(X_test)# 尋找最好的參數K # param_grid = [ # { # 'weights': ['uniform'], # 'n_neighbors': [i for i in range(1, 11)] # }, # { # 'weights': ['distance'], # 'n_neighbors': [i for i in range(1, 11)], # 'p': [i for i in range(1, 6)] # } # ] # knn_clf = KNeighborsClassifier() # grid_search = GridSearchCV(knn_clf, param_grid) # grid_search.fit(X_train, y_train) # print(grid_search.best_estimator_, grid_search.best_params_, grid_search.best_score_) # knn_clf.predict(X_test) # knn_clf.score(X_test, y_test)knn_clf = KNeighborsClassifier(n_neighbors=3) knn_clf.fit(X_train, y_train) # X_train已經進行了歸一化 print(knn_clf.score(X_test_standard, y_test)) # 或者 y_predict = knn_clf.predict(X_test_standard) print(accuracy_score(y_test, y_predict)) knn_clf.score(X_test_standard, y_test) 個人整個流程代碼

總結

以上是生活随笔為你收集整理的第4章 最基础的分类算法-k近邻算法的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。