車輛檢測在車輛感知模塊中是非常重要的功能，本節我們的目標如下：

• 在標記的圖像訓練集上進行面向梯度的直方圖（HOG）特征提取并訓練分類器線性SVM分類器

• 應用顏色轉換，并將分箱的顏色特征以及顏色的直方圖添加到HOG特征矢量中

• 對于上面兩個步驟，不要忘記標準化您的功能，并隨機選擇一個用于訓練和測試的選項

• 實施滑動窗口技術，并使用您訓練的分類器搜索圖像中的車輛

• 在視頻流上運行流水線（從test_video.mp4開始，稍后在完整的project_video.mp4中實現），并逐幀創建循環檢測的熱圖，以拒絕異常值并跟蹤檢測到的車輛

• 估算檢測到的車輛的邊界框

定向梯度直方圖（HOG）

定向梯度直方圖（HOG）是計算機視覺和圖像處理中用于目標檢測的特征描述符。該技術計算圖像的局部部分中梯度定向的發生。這種方法類似于邊緣方向直方圖，尺度不變特征變換描述符和形狀上下文，但不同之處在于它是在均勻間隔的單元的密集網格上計算的，并使用重疊的局部對比度歸一化來提高準確性。

從訓練圖像中提取HOG特征

此步驟的代碼包含在方法“get_hog_features”中的文件vehicle_detection.py中

# Define a function to return HOG features and visualizationdef get_hog_features(self, img, orient, pix_per_cell, cell_per_block, vis=False, feature_vec=True):# Call with two outputs if vis==Trueif vis == True:features, hog_image = hog(img, orientations=orient, pixels_per_cell=(pix_per_cell, pix_per_cell),cells_per_block=(cell_per_block, cell_per_block), transform_sqrt=True, visualise=vis, feature_vector=feature_vec) ? ?return features, hog_image# Otherwise call with one outputelse: ? ? ?features = hog(img, orientations=orient, pixels_per_cell=(pix_per_cell, pix_per_cell),cells_per_block=(cell_per_block, cell_per_block), transform_sqrt=True, visualise=vis, feature_vector=feature_vec) ? ?return features

我開始閱讀所有的車輛和非車輛圖像。這里是每一個中的一個的一個例子vehicle和non-vehicle類：

然后我探索不同的色彩空間和不同的skimage.hog()參數（orientations，pixels_per_cell，和cells_per_block）。我從兩個類的每一個中抓取隨機圖像，并顯示它們，以感受skimage.hog()輸出的樣子。使用后的圖像以不同的對比度和光度多次試驗，它的工作最好使用YCrCb相結合的色彩空間與HOG提取的特征orientations=9，pixels_per_cell=(8, 8)和cells_per_block=(2, 2)。

這是一個使用YCrCb色彩空間和HOG參數的例子orientations=9，pixels_per_cell=(8, 8)并且cells_per_block=(2, 2)：

提取顏色特征的空間分級

為了使算法在識別汽車時更加穩健，HOG特征還增加了一種新的特征。除非你確切地知道你的目標對象是什么樣子，否則模板匹配不是一個特別可靠的尋找車輛的方法。但是，原始像素值在搜索汽車中包含在您的特征向量中仍然非常有用。

雖然包含全分辨率圖像的三個顏色通道可能很麻煩，但是我們可以對圖像執行空間分級，并且仍然保留足夠的信息來幫助查找車輛。

正如你在下面的例子中看到的那樣，即使一路下降到32×32像素分辨率，汽車本身仍然可以被眼睛清楚地識別，這意味著相關特征仍然保留在這個分辨率下。

OpenCV的cv2.resize（）是一個方便的縮小圖像分辨率的函數。

# Define a function to compute binned color features ?def bin_spatial(self, img, size=(32, 32)):# Use cv2.resize().ravel() to create the feature vector#features = cv2.resize(img, size).ravel() # Return the feature vector#return featurescolor1 = cv2.resize(img[:,:,0], size).ravel()color2 = cv2.resize(img[:,:,1], size).ravel()color3 = cv2.resize(img[:,:,2], size).ravel() ? ? ? ?return np.hstack((color1, color2, color3))

提取顏色特征的直方圖

在這個項目中使用的另一個技術，使更多的功能是顏色強度的直方圖，如下圖所示。

并執行如下所示：

# Define a function to compute color histogram features # NEED TO CHANGE bins_range if reading .png files with mpimg!def color_hist(self, img, nbins=32, bins_range=(0, 256)):# Compute the histogram of the color channels separatelychannel1_hist = np.histogram(img[:,:,0], bins=nbins, range=bins_range)channel2_hist = np.histogram(img[:,:,1], bins=nbins, range=bins_range)channel3_hist = np.histogram(img[:,:,2], bins=nbins, range=bins_range) ? ? ? ?# Concatenate the histograms into a single feature vectorhist_features = np.concatenate((channel1_hist[0], channel2_hist[0], channel3_hist[0])) ? ? ? ?# Return the individual histograms, bin_centers and feature vectorreturn hist_features

合并和規范化的功能

現在我們的工具箱中已經有了幾個特征提取方法，我們幾乎已經準備好對分類器進行訓練了，但是首先，就像在任何機器學習應用程序中一樣，我們需要規范化數據。Python的sklearn包為您提供了StandardScaler（）方法來完成這個任務。

將單個圖像的所有不同特征組合為一組特征：

def convert_color(self, image, color_space='RGB'):if color_space == 'HSV':image = cv2.cvtColor(image, cv2.COLOR_RGB2HSV) ? ? ? ?elif color_space == 'LUV':image = cv2.cvtColor(image, cv2.COLOR_RGB2LUV) ? ? ? ?elif color_space == 'HLS':image = cv2.cvtColor(image, cv2.COLOR_RGB2HLS) ? ? ? ?elif color_space == 'YUV':image = cv2.cvtColor(image, cv2.COLOR_RGB2YUV) ? ? ? ?elif color_space == 'YCrCb':image = cv2.cvtColor(image, cv2.COLOR_RGB2YCrCb) ? ? ? ?return image ? ?# Define a function to extract features from a list of images# Have this function call bin_spatial() and color_hist()def extract_features(self, image, color_space='RGB', spatial_size=(32, 32),hist_bins=32, orient=9, pix_per_cell=8, cell_per_block=2, hog_channel=0,spatial_feat=True, hist_feat=True, hog_feat=True):file_features = [] ? ? ? ?# apply color conversion if other than 'RGB'if color_space != 'RGB':feature_image = self.convert_color(image, color_space) ? ? ? ?else: feature_image = np.copy(image) ? ? ?if spatial_feat == True:spatial_features = self.bin_spatial(feature_image, size=spatial_size)file_features.append(spatial_features) ? ? ? ?if hist_feat == True: ? ? ? ? ? ?# Apply color_hist()hist_features = self.color_hist(feature_image, nbins=hist_bins)file_features.append(hist_features) ? ? ? ?if hog_feat == True: ? ? ? ?# Call get_hog_features() with vis=False, feature_vec=Trueif hog_channel == 'ALL':hog_features = [] ? ? ? ? ? ? ? ?for channel in range(feature_image.shape[2]):hog_features.append(self.get_hog_features(feature_image[:,:,channel], orient, pix_per_cell, cell_per_block, vis=False, feature_vec=True))hog_features = np.ravel(hog_features) ? ? ? ?else:hog_features = self.get_hog_features(feature_image[:,:,hog_channel], orient, pix_per_cell, cell_per_block, vis=False, feature_vec=True) ? ? ? ? ? ?# Append the new feature vector to the features listfile_features.append(hog_features) ? ? ? ?return file_features

規范化是需要避免一些功能類型更重要的其他：

# Extract features of from all not-car imagesnotcar_features = [] ? ? ? ?for file in notcar_filenames:# Read in each one by oneimage = mpimg.imread(file)features = self.extract_features(image, color_space=self.color_space, spatial_size=self.spatial_size, hist_bins=self.hist_bins, orient=self.orient, pix_per_cell=self.pix_per_cell, cell_per_block=self.cell_per_block, hog_channel=self.hog_channel, spatial_feat=self.spatial_feat, hist_feat=self.hist_feat, hog_feat=self.hog_feat)notcar_features.append(np.concatenate(features))X = np.vstack((car_features, notcar_features)).astype(np.float64) ? ? ? ? ? ? ? ? ? ? ? ?# Fit a per-column scaler ? ? ? ?self.X_scaler = StandardScaler().fit(X)# Apply the scaler to Xscaled_X = self.X_scaler.transform(X)

使用規范化的功能來訓練分類器

我使用2類圖像車輛和非車輛圖像訓練了一個線性支持向量機。首先加載圖像，然后提取歸一化的特征，并在2個數據集中訓練（80％）和測試（20％）中的混洗和分裂。在使用StandardScaler（）訓練分類器之前，將特征縮放到零均值和單位方差。源代碼可以在vechicle_detection.py中找到

def __train(self):print ('Training the model ...') ? ? ? ?# Read in and make a list of calibration imagescar_filenames = glob.glob(self.__train_directory+'/vehicles/*/*') ?notcar_filenames = glob.glob(self.__train_directory+'/non-vehicles/*/*') ? ? ? ? ? ? ? # Extract features of from all car imagescar_features = [] ? ? ? ?for file in car_filenames:# Read in each one by oneimage = mpimg.imread(file)features = self.extract_features(image, color_space=self.color_space, spatial_size=self.spatial_size, hist_bins=self.hist_bins, orient=self.orient, pix_per_cell=self.pix_per_cell, cell_per_block=self.cell_per_block, hog_channel=self.hog_channel, spatial_feat=self.spatial_feat, hist_feat=self.hist_feat, hog_feat=self.hog_feat)car_features.append(np.concatenate(features)) ? ? ? ?# Extract features of from all not-car imagesnotcar_features = [] ? ? ? ?for file in notcar_filenames:# Read in each one by oneimage = mpimg.imread(file)features = self.extract_features(image, color_space=self.color_space, spatial_size=self.spatial_size, hist_bins=self.hist_bins, orient=self.orient, pix_per_cell=self.pix_per_cell, cell_per_block=self.cell_per_block, hog_channel=self.hog_channel, spatial_feat=self.spatial_feat, hist_feat=self.hist_feat, hog_feat=self.hog_feat)notcar_features.append(np.concatenate(features)) ? ? ? ?X = np.vstack((car_features, notcar_features)).astype(np.float64) ? ? ? ? ? ? ? ? ? ? ? ?# Fit a per-column scalerself.X_scaler = StandardScaler().fit(X) ? ? ? ?# Apply the scaler to Xscaled_X = self.X_scaler.transform(X) ? ? ? ?# Define the labels vectory = np.hstack((np.ones(len(car_features)), np.zeros(len(notcar_features)))) ? ? ? ?# Split up data into randomized training and test setsrand_state = np.random.randint(0, 100) ? ? ? ?X_train, X_test, y_train, y_test = train_test_split(scaled_X, y, test_size=0.2, random_state=rand_state)print('Using:',self.orient,'orientations',self.pix_per_cell, ? ? ? ? ? ?'pixels per cell and', self.cell_per_block,'cells per block')print('Feature vector length:', len(X_train[0])) ? ? ? ?# Use a linear SVC self.svc = LinearSVC() ? ? ? ?self.svc.fit(X_train, y_train) ? ? ? ?# Check the score of the SVCprint('Test Accuracy of SVC = ', round(self.svc.score(X_test, y_test), 4)) ? ? ? ?# Pickle to save time for subsequent runsbinary = {}binary["svc"] = self.svcbinary["X_scaler"] = self.X_scalerpickle.dump(binary, open(self.__train_directory + '/' + self.__binary_filename, "wb")) ? ?def __load_binary(self):'''Load previously computed trained classifier'''with open(self.__train_directory + '/' + self.__binary_filename, mode='rb') as f:binary = pickle.load(f) ? ? ? ?self.svc = binary['svc'] ? ? ? ?self.X_scaler = binary['X_scaler'] ? ?def get_data(self):'''Getter for the trained data. At the first call it gerenates it.'''if os.path.isfile(self.__train_directory + '/' + self.__binary_filename):self.__load_binary() ? ? ? ?else:self.__train() ? ? ? ?return self.svc, self.X_scaler

整個數據集（列車+測試）在車輛和非車輛之間均勻分布有17.767個項目。訓練完成后，train.p被保存在子文件夾列中的磁盤上，供以后重新使用。訓練好的線性支持向量機分類器在測試數據集上的準確性相當高?0.989

滑動窗口搜索

我決定使用重疊的滑動窗口搜索來搜索圖像下部的車輛。只需要搜索下面的部分，以避免搜索天空中的車輛，并使算法更快。窗口大小為64像素，每個單元8個單元和8個像素。在每張幻燈片窗戶移動2個單元向右或向下。為了避免每個窗口反復提取特征，搜索速度更快，特征提取只進行一次，滑動窗口只使用該部分圖像。如果窗戶在長短途容納所有車輛時具有不同的比例尺，則檢測也可以更加穩健。

該實現可以在vehicle_detection.py中找到：

# Define a single function that can extract features using hog sub-sampling and make predictionsdef find_cars(self, img, plot=False):bbox_list = [] draw_img = np.copy(img)img = img.astype(np.float32)/255img_tosearch = img[self.ystart:self.ystop,:,:]ctrans_tosearch = self.convert_color(img_tosearch, color_space='YCrCb') ? ? ? ?if self.scale != 1:imshape = ctrans_tosearch.shapectrans_tosearch = cv2.resize(ctrans_tosearch, (np.int(imshape[1]/self.scale), np.int(imshape[0]/self.scale)))ch1 = ctrans_tosearch[:,:,0]ch2 = ctrans_tosearch[:,:,1]ch3 = ctrans_tosearch[:,:,2] ? ? ? ?# Define blocks and steps as abovenxblocks = (ch1.shape[1] // self.pix_per_cell)-1nyblocks = (ch1.shape[0] // self.pix_per_cell)-1 nfeat_per_block = self.orient*self.cell_per_block**2# 64 was the orginal sampling rate, with 8 cells and 8 pix per cellwindow = 64nblocks_per_window = (window // self.pix_per_cell)-1 cells_per_step = 2 ?# Instead of overlap, define how many cells to stepnxsteps = (nxblocks - nblocks_per_window) // cells_per_stepnysteps = (nyblocks - nblocks_per_window) // cells_per_step# Compute individual channel HOG features for the entire imagehog1 = self.get_hog_features(ch1, self.orient, self.pix_per_cell, self.cell_per_block, feature_vec=False)hog2 = self.get_hog_features(ch2, self.orient, self.pix_per_cell, self.cell_per_block, feature_vec=False)hog3 = self.get_hog_features(ch3, self.orient, self.pix_per_cell, self.cell_per_block, feature_vec=False)bbox_all_list = [] for xb in range(nxsteps+1): ? ? ? ? ? ?for yb in range(nysteps):ypos = yb*cells_per_stepxpos = xb*cells_per_step ? ? ? ? ? ? ? ?# Extract HOG for this patchhog_feat1 = hog1[ypos:ypos+nblocks_per_window, xpos:xpos+nblocks_per_window].ravel() hog_feat2 = hog2[ypos:ypos+nblocks_per_window, xpos:xpos+nblocks_per_window].ravel() hog_feat3 = hog3[ypos:ypos+nblocks_per_window, xpos:xpos+nblocks_per_window].ravel() hog_features = np.concatenate((hog_feat1, hog_feat2, hog_feat3))xleft = xpos*self.pix_per_cellytop = ypos*self.pix_per_cell ? ? ? ? ? ? ? ?# Extract the image patchsubimg = cv2.resize(ctrans_tosearch[ytop:ytop+window, xleft:xleft+window], (64,64)) ? ? ? ? ? ? ? ?# Get color featuresspatial_features = self.bin_spatial(subimg, size=self.spatial_size)hist_features = self.color_hist(subimg, nbins=self.hist_bins) ? ? ? ? ? ? ? ?# Scale features and make a predictiontest_features = self.X_scaler.transform(np.hstack((spatial_features, hist_features, hog_features)).reshape(1, -1)) ? ?test_prediction = self.svc.predict(test_features) ? ? ? ? ? ? ? ?# compute current seize of the windowxbox_left = np.int(xleft*self.scale)ytop_draw = np.int(ytop*self.scale)win_draw = np.int(window*self.scale)bbox = ((xbox_left, ytop_draw+self.ystart),(xbox_left+win_draw,ytop_draw+win_draw+self.ystart)) ? ? ? ? ? ? ? ?if test_prediction == 1:bbox_list.append(bbox)bbox_all_list.append(bbox) ? ? ? ?if(plot==True):draw_img_detected = np.copy(draw_img) ? ? ? ? ? ?# draw all all searched windowsfor bbox in bbox_all_list:cv2.rectangle(draw_img, bbox[0], bbox[1], (0,0,255), 3) for bbox in bbox_list:cv2.rectangle(draw_img_detected, bbox[0], bbox[1], (0,0,255), 3) fig = plt.figure()plt.subplot(121)plt.imshow(draw_img)plt.title('Searched sliding windows')plt.subplot(122)plt.imshow(draw_img_detected, cmap='hot')plt.title('Detected vechicle windows')fig.tight_layout()plt.show() ? ? ? ?return bbox_listdef draw_labeled_bboxes(self, img, labels): ? ? ? ?# Iterate through all detected carsfor car_number in range(1, labels[1]+1): ? ? ? ? ? ?# Find pixels with each car_number label valuenonzero = (labels[0] == car_number).nonzero() ? ? ? ? ? ?# Identify x and y values of those pixelsnonzeroy = np.array(nonzero[0])nonzerox = np.array(nonzero[1]) ? ? ? ? ? ?# Define a bounding box based on min/max x and ybbox = ((np.min(nonzerox), np.min(nonzeroy)), (np.max(nonzerox), np.max(nonzeroy))) ? ? ? ? ? ?# Draw the box on the imagecv2.rectangle(img, bbox[0], bbox[1], (0,0,255), 6) ? ? ? ?# Return the imagereturn img

從圖中可以看出，2輛車正確檢測到，但也有一些誤報。

為了避免誤報，使用熱圖。點擊地圖加窗，重疊的窗口有更高的價值。超過一定的閾值的值保持為真正的正值。

def add_heat(self, heatmap, bbox_list):# Iterate through list of bboxesfor box in bbox_list: ? ? ? ? ? ?# Add += 1 for all pixels inside each bbox# Assuming each "box" takes the form ((x1, y1), (x2, y2))heatmap[box[0][1]:box[1][1], box[0][0]:box[1][0]] += 1# Return updated heatmapreturn heatmap# Iterate through list of bboxesdef apply_threshold(self, heatmap, threshold):# Zero out pixels below the thresholdheatmap[heatmap <= threshold] = 0# Return thresholded mapreturn heatmap box_list = vehicle_detector.find_cars(image, plot=plot)heat = np.zeros_like(image[:,:,0]).astype(np.float)# Add heat to each box in box listheat = vehicle_detector.add_heat(heat, box_list)# Apply threshold to help remove false positivesheat = vehicle_detector.apply_threshold(heat,1)# Visualize the heatmap when displaying ? ?heatmap = np.clip(heat, 0, 255)

要從熱圖找到最終的框，使用標簽功能。

from scipy.ndimage.measurements import label # Find final boxes from heatmap using label functionlabels = label(heatmap)if(plot==True):#print(labels[1], 'cars found')plt.imshow(labels[0], cmap='gray')plt.show()

管道處理一個圖像

如下面的代碼所示，首先我們提取邊界框，包括真和假的正面。然后使用熱圖我們丟棄誤報。在使用該scipy.ndimage.measurements.label()方法計算最終的框之后。最后，這些框被渲染。

def process_image(image, plot=False):box_list = vehicle_detector.find_cars(image, plot=plot)heat = np.zeros_like(image[:,:,0]).astype(np.float) ? ?# Add heat to each box in box listheat = vehicle_detector.add_heat(heat, box_list) ? ?# Apply threshold to help remove false positivesheat = vehicle_detector.apply_threshold(heat,1) ? ?# Visualize the heatmap when displaying ? ?heatmap = np.clip(heat, 0, 255) ? ?# Find final boxes from heatmap using label functionlabels = label(heatmap) ? ?if(plot==True): ? ? ? ?#print(labels[1], 'cars found')plt.imshow(labels[0], cmap='gray')plt.show()new_image = vehicle_detector.draw_labeled_bboxes(image, labels) ? ?if(plot==True):fig = plt.figure()plt.subplot(121)plt.imshow(new_image)plt.title('Car Positions')plt.subplot(122)plt.imshow(heatmap, cmap='hot')plt.title('Heat Map')fig.tight_layout()plt.show() ? ?return new_imagedef process_test_images(vehicle_detector, plot=False):test_filenames = glob.glob(TEST_DIRECTORY+'/'+TEST_FILENAME) # Process each test imagefor image_filename in test_filenames: ? ? ? ?# Read in each imageimage = cv2.imread(image_filename)image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # RGB is standard in matlibplotimage = process_image(image, plot)image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR) # RGB is standard in matlibplotcv2.imwrite(OUTPUT_DIRECTORY+'/'+image_filename.split('/')[-1], image)

這是測試圖像之一的流水線的結果：

管道處理一個視頻

process_image(image, plot=False)在視頻處理中使用了用于處理一個圖像的相同流水線。每個幀都從視頻中提取，由圖像管道處理，并使用VideoFileClip和ffmpeg合并到最終的視頻中

from moviepy.editor import VideoFileClip def process_video(video_filename, vehicle_detector, plot=False):video_input = VideoFileClip(video_filename + ".mp4")video_output = video_input.fl_image(process_image)video_output.write_videofile(video_filename + "_output.mp4", audio=False)process_test_images(vehicle_detector, plot=False)

結論

當前使用SVM分類器的實現對于測試的圖像和視頻來說工作良好，這主要是因為圖像和視頻被記錄在類似的環境中。用一個非常不同的環境測試這個分類器不會有類似的好結果。使用深度學習和卷積神經網絡的更健壯的分類器將更好地推廣到未知數據。

當前實現的另一個問題是在視頻處理流水線中不考慮后續幀。保持連續幀之間的熱圖將更好地丟棄誤報。

目前的實施還有一個更大的改進是多尺寸滑動窗口，這將更好地概括查找短距離和長距離的車輛。

轉自：https://mp.weixin.qq.com/s/mNkLu15ZrnssnlrPvXSg7w

總結

以上是生活随笔為你收集整理的OpenCV中的HOG+SVM在自动驾驶车辆检测中的应用实例的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。