struts實(shí)現(xiàn)分頁(yè)

If you want to get started on 3D Object Detection and more specifically on Point Pillars, I have a series of posts written on it just for that purpose. Here’s the link. Also, going through the Point Pillars paper directly will be really helpful to understand this post better, however, it’s not a prerequisite.

如果您想開始使用3D對(duì)象檢測(cè)，更具體地說是關(guān)于“點(diǎn)Struts”，那么我就為此目的撰寫了一系列文章。 這是鏈接 。 另外，直接閱讀Point Pillars論文對(duì)于更好地理解此文章確實(shí)很有幫助，但是，這不是前提條件。

Github Repo for this tutorial: https://github.com/tyagi-iiitv/PointPillars. Thanks to Francesco Ferroni for the initial code.

本教程的Github Repo： https : //github.com/tyagi-iiitv/PointPillars 。 感謝Francesco Ferroni的初始代碼。

內(nèi)容 (Contents)

• Defining Initial Configurations
  定義初始配置
• Building Point Pillars Network
  建筑點(diǎn)Struts網(wǎng)絡(luò)
• Loss functions
  損失函數(shù)
• Data Readers for Kitti Dataset
  Kitti數(shù)據(jù)集的數(shù)據(jù)讀取器
• Training
  訓(xùn)練
• Inference
  推理

定義初始配置(config.py) (Defining Initial Configurations (config.py))

We define initial configurations that include the limits for x,y,z dimensions. These limits define the region of interest in which we will be predicting the bounding boxes. Since the range of LiDARs is generally very high, we focus on a smaller region in this space which is defined by these limits. Other configurations include the maximum number of Pillars, maximum points per pillar, etc.

我們定義了包括x，y，z尺寸限制的初始配置。 這些限制定義了我們將在其中預(yù)測(cè)邊界框的感興趣區(qū)域。 由于LiDAR的范圍通常很高，因此我們將重點(diǎn)放在由這些限制定義的該空間中的一個(gè)較小區(qū)域。 其他配置包括最大Struts數(shù)量，每個(gè)Struts的最大點(diǎn)數(shù)等。

Configurations about the Point Pillars Network.有關(guān)點(diǎn)Struts網(wǎng)絡(luò)的配置。

These configurations are generally used to transfer the point cloud from 3D coordinates to Pillar coordinates in the Point Pillars detection pipeline as discussed in the next section. For now, just remember that all the configurations are defined in this file.

這些配置通常用于將點(diǎn)云從3D坐標(biāo)傳遞到點(diǎn)Struts檢測(cè)管道中的Struts坐標(biāo)，如下一節(jié)所述。 現(xiàn)在，請(qǐng)記住，所有配置都在此文件中定義。

建立點(diǎn)Struts網(wǎng)絡(luò)(network.py) (Building the Point Pillars Network (network.py))

The overall architecture of Point Pillars contains three components: A pillar feature net, A Backbone 2D CNN, and a Detection Head. We will discuss these three components individually.

點(diǎn)Struts的整體體系結(jié)構(gòu)包含三個(gè)組件：Struts特征網(wǎng)，骨干2D CNN和檢測(cè)頭。 我們將分別討論這三個(gè)組成部分。

Source)來源 )

Struts特征網(wǎng) (Pillar Feature Net)

Pillar Feature Net architecture.Struts功能網(wǎng)體系結(jié)構(gòu)。

• pillars/input (Input Layer) (4,12000,100,7) refers to — Batch Size (4), Maximum Number of Pillars (12000), Maximum number of Points per Pillar (100), 7 dim vector: [x,y,z,intensity,Xc,Yc,Zc]. X,y,z are the 3D coordinates of a point inside a point cloud and its intensity as measured by the LiDAR device is stored inside “intensity”, Xc, Yc, Zc stores the distance of that point from the Pillar Center in each dimension.

  Struts/輸入(輸入層)(4,12000,100,7)指代— 批大小(4)，最大Struts數(shù)(12000)，每個(gè)Struts的最大點(diǎn)數(shù)(100)，7個(gè)暗矢量：[x， y，z，強(qiáng)度，Xc，Yc，Zc]。 X，y，z是點(diǎn)云中點(diǎn)的3D坐標(biāo)，由LiDAR設(shè)備測(cè)量的點(diǎn)的強(qiáng)度存儲(chǔ)在“強(qiáng)度”內(nèi)部，Xc，Yc，Zc存儲(chǔ)該點(diǎn)到各個(gè)維度中距柱心的距離。

• This input is now transformed from 7 dimensions to a 64-dimensional vector with features extracted using a CNN.
  現(xiàn)在，此輸入已從7維轉(zhuǎn)換為具有使用CNN提取的特征的64維矢量。

• pillars/indices (InputLayer) (4,12000,3) refers to — Batch Size, Maximum number of Pillars, and the center of each pillar, i.e. the x,y and z coordinates. This layer contains the center of each pillar, which is used to map the features learned for each pillar back to the x-y grid in pillars/scatter_nd (Lambda) layer.

  Struts/索引(InputLayer)(4,12000,3)指- 批大小，最大Struts數(shù)以及每個(gè)Struts的中心，即x，y和z坐標(biāo)。 該層包含每個(gè)Struts的中心，該中心用于將為每個(gè)Struts學(xué)習(xí)的特征映射回Struts/ scatter_nd(Lambda)層中的xy網(wǎng)格。

• pillars/scatter_nd (Lambda) — Using the x,y coordinates from the above layer, the point cloud space is divided into (504 X 504) x-y grid with each grid cell containing the 64 features of the corresponding pillar.
  Struts/散點(diǎn)圖(Lambda)-使用來自上一層的x，y坐標(biāo)，將點(diǎn)云空間劃分為(504 X 504)xy柵格，每個(gè)柵格單元包含對(duì)應(yīng)Struts的64個(gè)要素。

骨干2D CNN (Backbone 2D CNN)

This is a practically simple layer with input being the pillar features encoded in an x-y grid, as described in the previous section. This grid is treated as an image and is transformed into different scales with features extracted at those scales and finally concatenated. We are trying to capture features from different scales of the point cloud and concatenate them into a single tensor.

如上一節(jié)所述，這是一個(gè)實(shí)際上簡(jiǎn)單的圖層，輸入是在xy網(wǎng)格中編碼的Struts特征。 該網(wǎng)格被視為圖像，并轉(zhuǎn)換為不同的比例，并以這些比例提取特征并最終進(jìn)行連接。 我們正在嘗試從不同比例的點(diǎn)云中捕獲特征，并將它們連接為單個(gè)張量。

Backbone 2D CNN. The x-y grid with encoded pillar features is transformed into different scales with their features extracted and concatenated.骨干2D CNN。 具有已編碼Struts特征的xy網(wǎng)格會(huì)轉(zhuǎn)換為不同的比例，并提取并連接其特征。

檢測(cè)頭 (Detection Head)

Detection Head network architecture.檢測(cè)頭網(wǎng)絡(luò)架構(gòu)。

In this part, we are predicting the regression targets for the anchor boxes as described below. All of these predictions are for each grid cell in the x-y coordinates (252 X 252). The bounding boxes are not directly predicted in this case, but the initial anchor boxes are resized to generate the predicted bounding box. The initial anchor boxes are provided inside the config.py file.

在這一部分中，我們將預(yù)測(cè)錨框的回歸目標(biāo)，如下所述。 所有這些預(yù)測(cè)都是針對(duì)xy坐標(biāo)(252 X 252)中的每個(gè)網(wǎng)格單元。 在這種情況下，不會(huì)直接預(yù)測(cè)邊界框，但是會(huì)調(diào)整初始錨點(diǎn)框的大小以生成預(yù)測(cè)的邊界框。 初始錨框位于config.py文件中。

• Occupancy — Whether the predicted label contains an element. 1 — Yes and No Otherwise. So we only need to check other predictions (loc, size, angle, heading, and clf) only when occupancy is 1.
  占用率—預(yù)測(cè)的標(biāo)簽是否包含元素。 1-是，否則，否。 因此，僅當(dāng)占用率為1時(shí)，我們才需要檢查其他預(yù)測(cè)(位置，大小，角度，航向和clf)。
• Loc — Contains the delta x,y,z for the corresponding anchor box.
  Loc —包含相應(yīng)錨點(diǎn)框的增量x，y，z。
• Size — Contains delta length, width, and height for the corresponding anchor box.
  大小-包含相應(yīng)錨點(diǎn)框的增量長(zhǎng)度，寬度和高度。
• Angle — Contains the delta yaw value.
  角度—包含增量偏航值。
• Heading — Contains if Angle is > 90 to check the heading angle of the object.
  航向—包含“角度”是否大于90，以檢查對(duì)象的航向角。
• Clf — Contains the classification id and the confidence score for each prediction.
  Clf —包含每個(gè)預(yù)測(cè)的分類ID和置信度分?jǐn)?shù)。

Note that the Detection head predicts 4 bounding boxes per grid cell, and each bounding box has the properties listed above.

請(qǐng)注意，檢測(cè)頭會(huì)為每個(gè)網(wǎng)格單元預(yù)測(cè)4個(gè)邊界框，并且每個(gè)邊界框都具有上面列出的屬性。

損失函數(shù)(loss.py) (Loss Functions (loss.py))

Once we have decided on the network architecture, we create the loss functions to complete the training process definitions. As you can see in the loss.py file, we are calculating the loss values for each of our regression targets and adding them with weights assigned to each error. So, the final loss value will be a weighted sum of each loss value.

一旦確定了網(wǎng)絡(luò)體系結(jié)構(gòu)，就創(chuàng)建損失函數(shù)以完成訓(xùn)練過程定義。 正如您在loss.py文件中看到的那樣，我們正在計(jì)算每個(gè)回歸目標(biāo)的損失值，并將它們與分配給每個(gè)錯(cuò)誤的權(quán)重相加。 因此，最終損失值將是每個(gè)損失值的加權(quán)和。

Weights assigned to each loss value. (Config.py file)分配給每個(gè)損失值的權(quán)重。 (Config.py文件) focal_weight*focal_loss + loc_weight*loc_loss + size_weight*size_loss + angle_weight*angle_loss +
...

Kitti數(shù)據(jù)集的數(shù)據(jù)讀取器(readers.py) (Data Readers for Kitti Dataset (readers.py))

We have designed a class to read the files in the Kitti Dataset and extract the 3D bounding box dimensions from the label (txt) files. Every row in the labels file contain the following information:

我們?cè)O(shè)計(jì)了一個(gè)類來讀取Kitti數(shù)據(jù)集中的文件，并從標(biāo)簽(txt)文件中提取3D邊框尺寸。 標(biāo)簽文件中的每一行都包含以下信息：

Label files in Kitti datasetKitti數(shù)據(jù)集中的標(biāo)簽文件

Type of object (Car, Truck, etc.)

對(duì)象類型(汽車，卡車等)

Truncated Float, 0 means truncated and 1 means non-truncated

截?cái)喔↑c(diǎn)，0表示被截?cái)?#xff0c;1表示不截?cái)?

Occlusion State, 0=Fully Visible, 1=Partly Occluded, 2=Largely Occluded, 3=Unknown

遮擋狀態(tài)，0 =完全可見，1 =部分遮擋，2 =大遮擋，3 =未知

Alpha Observation angle

阿爾法觀察角

4 2D bounding box parameters (center, length, width)

4個(gè)2D邊界框參數(shù)(中心，長(zhǎng)度，寬度)

3D object dimensions (height, width, length)

3D對(duì)象尺寸(高度，寬度，長(zhǎng)度)

3D object location (x,y,z) in camera coordinates

相機(jī)坐標(biāo)中的3D對(duì)象位置(x，y，z)

Yaw angle

偏航角

培訓(xùn)(point_pillars_training_run.py，processors.py，src / point_pillars.cpp) (Training (point_pillars_training_run.py, processors.py, src/point_pillars.cpp))

After defining loss functions, configurations, and data generators for each data point in the Kitti dataset, the SimpleDataGenerator in the processor.py file, generates the training examples. The main steps in this process are:

在為Kitti數(shù)據(jù)集中的每個(gè)數(shù)據(jù)點(diǎn)定義損失函數(shù)，配置和數(shù)據(jù)生成器之后， processor.py文件中的SimpleDataGenerator會(huì)生成訓(xùn)練示例。 此過程的主要步驟是：

• Generate Pillars and Pillar Ids (DataProcessor.make_point_pillars inside processor.py file)

  生成Struts和StrutsID( processor.py文件中的DataProcessor.make_point_pillars )

• Generate the regression targets (discussed above) from the label files and given anchor boxes in the config.py file. (DataProcessor.make_ground_truth inside processor.py file)

  從標(biāo)簽文件和config.py文件中給定的錨定框生成回歸目標(biāo)(如上所述)。 (processor.py文件中的DataProcessor.make_ground_truth )

The helper functions for these two tasks are provided inside the src/point_pillars.cpp file which is included with the current python code using Pybind.

這兩個(gè)任務(wù)的幫助程序功能在src / point_pillars.cpp文件中提供，該文件包含在使用Pybind的當(dāng)前python代碼中。

• createPillars (src/point_pillars.cpp) — Generates the maximum number of pillars and assign pillar centers to each pillar. Use zero-padding if the number of points inside a pillar is less than the maximum points and random sampling otherwise. Each pillar will have 100 points and 7 values for each point (x, y, z, intensity, Xc, Yc, Zc). We start from the (0,0) LiDAR coordinates and expand radially in the field of view until we reach a maximum number of pillars.
  createPillars(src / point_pillars.cpp)-生成最大數(shù)量的Struts并將Struts中心分配給每個(gè)Struts。 如果Struts內(nèi)的點(diǎn)數(shù)少于最大點(diǎn)數(shù)，請(qǐng)使用零填充，否則使用隨機(jī)抽樣。 每個(gè)Struts將具有100個(gè)點(diǎn)和每個(gè)點(diǎn)7個(gè)值(x，y，z，強(qiáng)度，Xc，Yc，Zc)。 我們從(0,0)LiDAR坐標(biāo)開始，在視場(chǎng)中徑向擴(kuò)展，直到達(dá)到最大柱數(shù)為止。

We expand from a starting point in the field of view to generate pillars for each grid cell. This process is repeated until we have encoded the maximum number of pillars.我們從視場(chǎng)的起點(diǎn)擴(kuò)展為每個(gè)網(wǎng)??格單元生成Struts。 重復(fù)此過程，直到我們編碼了最大數(shù)量的Struts為止。

• createPillarsTarget (src/point_pillars.cpp) — Generates the regression targets which can be used to calculate the loss values later. For each 3D bounding box given inside the labels file, it is compared to the anchor boxes. First, a search radius is created around the original bounding box based on object dimensions (red circle on the grid below). Then for each grid cell inside that radius, the anchor boxes are placed and IoU is calculated between each anchor box and the original bounding box.
  createPillarsTarget(src / point_pillars.cpp)-生成可用于以后計(jì)算損失值的回歸目標(biāo)。 對(duì)于標(biāo)簽文件中給定的每個(gè)3D邊界框，將其與錨定框進(jìn)行比較。 首先，根據(jù)對(duì)象尺寸(下面的網(wǎng)格上的紅色圓圈)在原始邊界框周圍創(chuàng)建搜索半徑。 然后，對(duì)于該半徑內(nèi)的每個(gè)網(wǎng)格單元，放置錨框，并在每個(gè)錨框和原始邊界框之間計(jì)算IoU。

Process of generating regression targets given an actual bounding box.給定實(shí)際邊界框的生成回歸目標(biāo)的過程。 Calculating Intersection over Union. This is a key step in generating regression targets by comparing the anchor boxes with actual bounding boxes during training. (Source: pyimagesearch.com)計(jì)算聯(lián)合的交點(diǎn)。 這是通過在訓(xùn)練期間將錨點(diǎn)框與實(shí)際邊界框進(jìn)行比較來生成回歸目標(biāo)的關(guān)鍵步驟。 (來源：pyimagesearch.com)

If the IoU value is above a certain threshold, then the targets (delta - x, y, z, length, width height, yaw, etc.) are stored for that bounding box. Otherwise, if no anchor passed the threshold, the best anchor based on IoU scores is chosen for that grid cell and is stored. These values are then used to calculate the loss values as described above.

如果IoU值高于某個(gè)閾值，則將為該邊界框存儲(chǔ)目標(biāo)(δ-x，y，z，長(zhǎng)度，寬度，高度，偏航等)。 否則，如果沒有錨點(diǎn)超過閾值，則為該網(wǎng)格單元選擇基于IoU分?jǐn)?shù)的最佳錨點(diǎn)并將其存儲(chǔ)。 然后，將這些值用于計(jì)算如上所述的損耗值。

推論(point_pillars_prediction.py) (Inference (point_pillars_prediction.py))

The last part of this codebase is the point_pillars_prediction.py file used for inferencing. After we have trained the network based on the loss functions described before, during the inference stage, the output of the network is the regression targets, occupancy index, and the classification confidence value for each prediction. You can verify that for each grid cell, the network outputs 4 bounding boxes with their respective parameters.

該代碼庫(kù)的最后一部分是用于推理的point_pillars_prediction.py文件。 在根據(jù)之前所述的損失函數(shù)對(duì)網(wǎng)絡(luò)進(jìn)行訓(xùn)練之后，在推理階段，網(wǎng)絡(luò)的輸出是每個(gè)預(yù)測(cè)的回歸目標(biāo)，占用指數(shù)和分類置信度值 。 您可以驗(yàn)證網(wǎng)絡(luò)是否為每個(gè)網(wǎng)格單元輸出4個(gè)邊界框及其各自的參數(shù)。

We can remove most of the predicted boxes based on the occupancy index since we only want the boxes with occupancy 1, which means that the network has actually predicted a bounding box on that grid cell. Still, there can be many bounding boxes that are predicted by the network for the same object. To filter these, we can use Non-Maximum Suppression and come up with final bounding box predictions. Non-Maximum Suppression ranks the bounding boxes on the basis of IoU and their classification confidence values and selects the one with the best value as the final bounding box. Refer to this post to read more on Non-Maximum Suppression.

由于我們只希望占用率為1的框，因此我們可以根據(jù)占用率索引刪除大多數(shù)預(yù)測(cè)框，這意味著網(wǎng)絡(luò)實(shí)際上已經(jīng)預(yù)測(cè)了該網(wǎng)格單元上的邊界框。 但是，網(wǎng)絡(luò)仍然可以為同一對(duì)象預(yù)測(cè)許多邊界框。 為了過濾這些，我們可以使用非最大抑制，并給出最終的邊界框預(yù)測(cè)。 非最大抑制基于IoU及其分類置信度對(duì)邊界框進(jìn)行排序，并選擇具有最佳值的邊界框作為最終邊界框。 請(qǐng)參閱這篇文章，以了解有關(guān)非最大抑制的更多信息。

Non-Maximum Suppression to converge to a final bounding box when multiple bounding boxes are predicted by the object detection algorithms. (Source: towardsdatascience.com)當(dāng)對(duì)象檢測(cè)算法預(yù)測(cè)到多個(gè)邊界框時(shí)，非最大抑制將收斂到最終邊界框。 (來源：toondatascience.com)

結(jié)論 (Conclusion)

This post covered the details of Point Pillars implementation on Tensorflow. All the code details including configuration files, model implementation, data generators, training, and inference are explained with code examples. The code is available in this Github repo and can be used to experiment with the details given in this article. As the next steps, we are working on generating inference results on the Kitti data which will be posted on the Github repo.

這篇文章涵蓋了在Tensorflow上實(shí)現(xiàn)Point Pillars的細(xì)節(jié)。 所有代碼細(xì)節(jié)(包括配置文件，模型實(shí)現(xiàn)，數(shù)據(jù)生成器，培訓(xùn)和推斷)均通過代碼示例進(jìn)行了說明。 該代碼可在該Github存儲(chǔ)庫(kù)中使用 ，并可用于試驗(yàn)本文給出的詳細(xì)信息。 下一步，我們正在根據(jù)Kitti數(shù)據(jù)生成推斷結(jié)果，該結(jié)果將發(fā)布在Github存儲(chǔ)庫(kù)中。

If you are interested in contributing to this project, please check out the issues on the GitHub repo.

如果您有興趣為該項(xiàng)目做貢獻(xiàn)，請(qǐng)?jiān)贕itHub存儲(chǔ)庫(kù)上查看問題。

翻譯自: https://towardsdatascience.com/implementing-point-pillars-in-tensorflow-c38d10e9286

struts實(shí)現(xiàn)分頁(yè)

總結(jié)

以上是生活随笔為你收集整理的struts实现分页_在TensorFlow中实现点Struts的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。