边缘计算边缘计算edge_Edge AI-边缘上的计算机视觉推理
邊緣計算邊緣計算edge
Carsten Moenning, Ph.D.
卡斯滕·莫寧(Carsten Moenning)博士
This is the final part of a two-part series on running AI applications on the edge in general and Computer Vision inference on an Intel Neural Compute Stick 2-accelerated Raspberry Pi 3, in particular.
?他是在對英特爾的神經計算棒2加速樹莓派3一般與計算機視覺推理邊緣延伸AI的應用，特別是兩部分系列的最后一部分。
Have a look at part 1 to obtain:
看一下第1部分 ，獲得：
This second part builds on steps 2 and 3. Whilst part 1 finished with an OpenVINO face detection demo application running on a Raspberry Pi, this post deals with developing your very own Computer Vision custom solution on such an edge device. More specifically, I will walk you through the custom implementation of a YOLOv3-tiny object detection model on a Raspberry Pi pre-configured as described in part 1. (Details on the YOLOv3 object detection model can be found here.) But, first, let’s have a closer look at the open source Intel OpenVINO toolkit in general and the OpenVINO toolkit for Raspbian OS, in particular. 🗹
第二部分基于第2步和第3步。第1部分完成了在Raspberry Pi上運行的OpenVINO人臉檢測演示應用程序，而本文則涉及在這種邊緣設備上開發自己的Computer Vision定制解決方案。 更具體地說，我將引導您完成在第1部分中所述預先配置的Raspberry Pi上的YOLOv3-tiny對象檢測模型的自定義實現。(有關YOLOv3對象檢測模型的詳細信息，請參見此處 。)但是，首先，讓我們仔細了解一下開放源代碼的Intel OpenVINO工具包 ，特別是Raspbian OS的OpenVINO工具包 。 🗹
The Intel OpenVINO toolkit
英特爾OpenVINO工具包
To quote the official OpenVINO toolkit documentation: “The OpenVINO? toolkit is a comprehensive toolkit for quickly developing applications and solutions that emulate human vision. Based on Convolutional Neural Networks (CNNs), the toolkit extends CV workloads across Intel? hardware, maximizing performance.”
引用官方的OpenVINO工具包文檔：“ OpenVINO?工具包是一種綜合工具包，用于快速開發可模擬人類視覺的應用程序和解決方案。 該工具包基于卷積神經網絡(CNN)，可在英特爾?硬件上擴展CV工作負載，從而最大限度地提高性能。”
OpenVINO)OpenVINO )To quote further, the toolkit:
進一步引用該工具包：
- “enables CNN-based deep learning inference on the edge “在邊緣啟用基于CNN的深度學習推理
- supports heterogeneous execution across an Intel? CPU, Intel? Integrated Graphics, Intel? FPGA, Intel? Neural Compute Stick 2 and Intel? Vision Accelerator Design with Intel? Movidius? VPUs 支持通過英特爾?Movidius?VPU在英特爾?CPU，英特爾?集成顯卡，英特爾?FPGA，英特爾?神經計算棒2和英特爾?視覺加速器設計之間進行異構執行
- speeds time-to-market via an easy-to-use library of computer vision functions and pre-optimized kernels 通過易于使用的計算機視覺功能和預優化內核庫，加快了產品上市時間
- includes optimized calls for computer vision standards, including OpenCV and OpenCL?.” 包括對計算機視覺標準(包括OpenCV和OpenCL?)的優化呼吁。”
Amongst other things, this implies that the toolkit is particularly well-suited for the development of Computer Vision inference applications on edge devices such as a Raspberry Pi enhanced by an Intel Neural Compute Stick 2 USB accelerator stick featuring the Intel Movidius VPU.
除其他外，這意味著該工具包特別適合在邊緣設備(例如通過具有Intel Movidius VPU的Intel Neural Compute Stick 2 USB加速棒增強的Raspberry Pi)上開發計算機視覺推理應用程序。
? In case you might be wondering: “OpenVINO” stands for “Open Visual Inference and Neural Network Optimization”, indicating its Deep Learning Computer Vision focus. ?
?也許您會感到奇怪：“ OpenVINO”代表“開放式視覺推理和神經網絡優化”，表示其深度學習計算機視覺的重點。 ?
The main elements of the OpenVINO toolkit consist of:
OpenVINO工具包的主要元素包括：
- the Model Optimizer, 模型優化器
- the Inference Engine and 推理引擎和
- the Open Model Zoo. 開放模型動物園。
The Model Optimizer takes Deep Learning Computer Vision models pre-trained in the Caffe, TensorFlow, MXNet, Kaldi or ONNX frameworks and converts them into the OpenVINO Intermediate Representation (IR), a streamlined model representation optimised for model execution using the Inference Engine running on an edge device. The Inference Engine loads and infers these IR files using a common API for both CPU and GPU as well as VPU hardware.
模型優化器采用在Caffe，TensorFlow，MXNet，Kaldi或ONNX框架中預先訓練的深度學習計算機視覺模型，并將其轉換為OpenVINO中間表示(IR)，這是一種簡化的模型表示，可使用運行在以下平臺上的推理引擎進行模型執行優化。邊緣設備。 推理引擎使用針對CPU和GPU以及VPU硬件的通用API加載和推斷這些IR文件。
The Open Model Zoo provides pre-trained models for free download that can be used to speed up the development process by not having to train one’s own model first, which would then need to be converted into the Intermediate Representation using the Model Optimizer next before being ready for use with the Inference Engine on the edge device under consideration.
Open Model Zoo提供了免費的預訓練模型，可以免費下載，無需先訓練自己的模型即可用于加速開發過程，然后需要先使用Model Optimizer將其轉換為中間表示形式，然后再進行建模準備與正在考慮的邊緣設備上的推理引擎一起使用。
When not using a pre-trained model from the Open Model Zoo, the typical workflow for deploying a deep learning model on the edge using the OpenVINO toolkit is shown below.
當不使用Open Model Zoo中的預訓練模型時，使用OpenVINO工具包在邊緣部署深度學習模型的典型工作流程如下所示。
OpenVINO)OpenVINO )So, Open Model Zoo models come with the advantage of already having been converted into the OpenVINO Intermediate Representation, but what is the Intermediate Representation?
因此，Open Model Zoo模型的優點是已經轉換為OpenVINO中間表示，但是什么是中間表示？
The IR describes the result of an input model, which has been optimised for edge inference with the help of one or multiple of the following techniques:
IR描述了輸入模型的結果，借助以下一種或多種技術，該模型已針對邊沿推理進行了優化：
- quantization, i.e. the reduction in numerical precision of the model weights and bias, 量化，即模型權重和偏差的數值精度降低，
- model layer fusion, i.e. the combination of multiple model layers into one, 模型層融合，即將多個模型層組合成一個層，
- freezing, i.e. the removal of metadata and operations which were only useful for model training (in the case of TensorFlow models only). 凍結，即刪除僅對模型訓練有用的元數據和操作(僅在TensorFlow模型的情況下)。
You are also perfectly free to use models from other sources, of course. These will have to go through step two of the OpenVINO workflow, though. That is, you will have to apply the Model Optimizer to the input model to produce its inference-optimised IR.
當然，您也可以完全自由地使用其他來源的模型。 不過，這些步驟必須經過OpenVINO工作流程的第二步。 也就是說，您必須將Model Optimizer應用于輸入模型，以產生其推理優化的IR。
Once the IR is generated, its .xml (optimised model topology) and .bin (optimised model weights and biases) files can be fed into the OpenVINO Inference Engine running on the edge device for performance-optimised inference on the edge.
生成IR后，可以將其.xml(優化的模型拓撲)和.bin(優化的模型權重和偏差)文件輸入到在邊緣設備上運行的OpenVINO推理引擎中，以對邊緣進行性能優化的推理。
In fact, we already made our way through this workflow the easy way during the face detection example in part 1 of this two-part blog series: We downloaded a pre-trained face detection model from the Open Model Zoo. More specifically, since the OpenVINO Model Downloader is not available within the OpenVINO toolkit for Raspbian OS, we downloaded the IR files manually onto the Raspberry Pi and fed them into the Inference Engine for bounding box generation on an input image.
實際上，在這個由兩部分組成的博客系列的第1部分中 ，我們已經通過臉部檢測示例中的簡單方法來完成此工作流程：我們從Open Model Zoo下載了預先訓練的臉部檢測模型。 更具體地說，由于OpenVINO模型下載器在Raspbian OS的OpenVINO工具包中不可用，因此我們將IR文件手動下載到Raspberry Pi上，并將其輸入到推理引擎中，以在輸入圖像上生成邊界框。
We will now take the slightly more challenging route of converting a pre-trained public model, the “tiny” version of YOLOv3 [2], into the IR via the Model Optimizer. This gets us to a crucial point: Since the OpenVINO Inference Engine represents the only OpenVINO main element included within the OpenVINO toolkit for Raspbian OS, the model conversion needs to be done outside of the Raspberry Pi environment, for example, on your main machine, which means that you will need to install OpenVINO on your main machine as well. With the help of the OpenVINO Linux, macOS and Windows installation guides, this should prove not to be too much of a challenge.
現在，我們將采取更具挑戰性的方法，即通過模型優化器將經過預訓練的公共模型YOLOv3 [2]的“微型”版本轉換為IR。 這將我們帶到了關鍵點：由于OpenVINO推理引擎代表Raspbian OS的OpenVINO工具包中包含的唯一OpenVINO主要元素，因此模型轉換需要在Raspberry Pi環境之外進行，例如，在您的主機上，這意味著您還需要在您的主機上安裝OpenVINO。 在OpenVINO Linux ， macOS和Windows安裝指南的幫助下，這應該證明不是太大的挑戰。
? In the case of the OpenVINO toolkit for Windows, you could do without the Microsoft Visual Studio and CMake installation steps, since, in the following, we will be using the Model Optimizer only. However, if you want to have a fully-fledged OpenVINO installation on your Windows machine, it is, of course, strongly recommended to complete all of the installation steps. ?
?對于Windows的OpenVINO工具包，您可以不用Microsoft Visual Studio和CMake安裝步驟，因為在下面，我們將僅使用模型優化器。 但是，如果要在Windows機器上進行完整的OpenVINO安裝，則強烈建議您完成所有安裝步驟。 ?
You are now all set for the custom implementation of the YOLOv3-tiny [2] model on your Raspberry Pi.
現在，您已經準備好在Raspberry Pi上自定義實現YOLOv3-tiny [2]模型。
Converting a pre-trained YOLOv3-tiny model into the OpenVINO Intermediate Representation
將預先訓練的YOLOv3-tiny模型轉換為OpenVINO中間表示
With the OpenVINO installation on your main machine out of the way, let’s generate an IR of the YOLOv3-tiny model. The network architecture of YOLOv3-tiny is execution-optimised for low-performance devices at the expense of reduced prediction accuracy: It uses only 19 convolutional layers instead of the 53 convolutional layers of the standard YOLOv3 model. As a consequence, it is much more execution-efficient than the standard YOLOv3 model, but also much less accurate with a mean average precision (mAP) of 33.1% compared to a mAP value of 51–57% of the standard YOLOv3 model.
通過在主機上安裝OpenVINO，讓我們生成YOLOv3-tiny模型的IR。 YOLOv3-tiny的網絡體系結構針對低性能設備進行了執行優化，以降低預測精度為代價：它僅使用19個卷積層，而不是標準YOLOv3模型的53個卷積層。 因此，它比標準YOLOv3模型的執行效率高得多，但與標準YOLOv3模型的51-57％的mAP值相比，其平均平均精度(mAP)為33.1％時精度要低得多。
To convert the YOLOv3-tiny model into the IR format, we will largely follow the OpenVINO guide for converting YOLO models.
要將YOLOv3-tiny模型轉換為IR格式，我們將在很大程度上遵循OpenVINO 指南來轉換YOLO模型 。
Step 1: Clone the following GitHub repository to obtain a TensorFlow-based YOLOv3-tiny implementation.
步驟1 ：克隆以下GitHub存儲庫以獲得基于TensorFlow的YOLOv3-tiny實現。
? In principle, you can also use any other TensorFlow-based YOLOv3-tiny model implementation, but you may run into difficulties when trying to convert it into the Intermediate Representation using the steps below. ?
?原則上，您也可以使用任何其他基于TensorFlow的YOLOv3-tiny模型實現，但是當您嘗試使用以下步驟將其轉換為中間表示形式時，您可能會遇到困難。 ?
git clone https://github.com/mystic123/tensorflow-yolo-v3.git
git clone https://github.com/mystic123/tensorflow-yolo-v3.git
YOLOv3 repository cloningYOLOv3存儲庫克隆Step 2: Download a set of class labels in the form of, for example, coco.names, which includes 80 common object classes ranging from people to cars and animals, or provide your own set of class labels you want to do inference on using YOLOv3-tiny, subject to YOLOv3-tiny having been pre-trained on your classes of choice.
第2步 ：以例如coco.names的形式下載一組類標簽，其中包括80種常見的對象類，范圍從人到汽車和動物，或者提供您自己的一組要推斷使用的類標簽YOLOv3-tiny，請按照您選擇的課程預先訓練YOLOv3-tiny。
YOLOv3 repository contents and coco.names fileYOLOv3存儲庫內容和coco.names文件Step 3: Download pre-trained YOLOv3-tiny weights, or train YOLOv3-tiny yourself and use the resulting model weights.
步驟3 ：下載經過預先??訓練的YOLOv3-tiny權重 ，或者自己訓練YOLOv3-tiny權重 ，然后使用生成的模型權重。
Pre-trained YOLOv3-tiny weights (yellow) and the frozen model definition (red)預先訓練的YOLOv3-小重量(黃色)和凍結的模型定義(紅色)Step 4: Convert the YOLOv3 model to the protocol buffer file format (.pb) to obtain a simplified, i.e. “frozen”, model definition for inference.
步驟4 ：將YOLOv3模型轉換為協議緩沖區文件格式(.pb)，以獲取簡化的模型(即“凍結”)模型定義以進行推斷。
python convert_weights_pb.py --class_names coco.names --data_format NHWC --weights_file yolov3-tiny.weights --tiny
python convert_weights_pb.py --class_names coco.names --data_format NHWC --weights_file yolov3-tiny.weights --tiny
The --tiny parameter at the end of the statemenet instructs the converter to generate the YOLOv3-tiny version of the frozen TensorFlow graph.
--tiny末尾的--tiny參數指示轉換器生成凍結的TensorFlow圖的YOLOv3-tiny版本。
Step 5: Convert the frozen model defintion to its IR format. Don’t forget to execute the OpenVINO setupvars environment setup script located in $OPENVINO_INSTALL_DIR/bin/ prior to this step.
步驟5 ：將凍結的模型定義轉換為其IR格式。 在執行此步驟之前，請不要忘記執行位于$OPENVINO_INSTALL_DIR/bin/的OpenVINO setupvars環境設置腳本。
python mo_tf.py --input_model frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config <path/>yolo_v3_tiny.json --batch 1 --generate_deprecated_IR_V7
python mo_tf.py --input_model frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config <path/>yolo_v3_tiny.json --batch 1 --generate_deprecated_IR_V7
where the OpenVINO <path/> to the yolo_v3_tiny.json model configuration file should look something like this:
yolo_v3_tiny.json模型配置文件的OpenVINO <path/>應該看起來像這樣：
$OPENVINO_INSTALL_DIR/deployment_tools/model_optimizer/extensions/front/tf/
$OPENVINO_INSTALL_DIR/deployment_tools/model_optimizer/extensions/front/tf/
? The --generate_deprecated_IR_V7 parameter forces the Model Optimizer to generate the older IR version 7. This is due to the compatability bug associated with the IR version 10 already mentioned in part 1. If not using this parameter, your converted model will fail during inference with somewhat intractable error messages. ?
?-- --generate_deprecated_IR_V7參數強制模型優化器生成較舊的IR版本7。這是由于與在第1部分中已經提到的IR版本10相關的兼容性錯誤 。 如果不使用此參數，則轉換后的模型將在推理過程中失敗，并帶有一些棘手的錯誤消息。 ?
? If you want to run OpenVINO demos using your converted model, please note that you may need to add the --reverse_input_channel option, since OpenVINO demos typically expect the colour channels to be in BGR rather than the usual RGB order. Consult the official OpenVINO demo documentation to check the demo’s specific input requirements. ?
?如果要使用轉換后的模型運行OpenVINO演示，請注意，您可能需要添加--reverse_input_channel選項，因為OpenVINO演示通常期望顏色通道為BGR，而不是通常的RGB順序。 請查閱OpenVINO官方演示文檔，以檢查演示的特定輸入要求。 ?
As a result, we obtain the edge-optimised YOLOv3-tiny network topology (.xml) and weight and bias terms (.bin) IR files.
結果，我們獲得了邊緣優化的YOLOv3-tiny網絡拓撲(.xml)以及權重和偏差項(.bin)IR文件。
YOLOv3-tiny Intermediate Representation filesYOLOv3-tiny中級表示文件To test the validity of the generated IR files, you may want to run the OpenVINO object detection YOLOv3 demo on your main machine. Please take into account the --reverse_input_channel remark above in this context.
為了測試所生成的IR文件的有效性，您可能需要在主機上運行OpenVINO 對象檢測YOLOv3演示 。 在這種情況下，請考慮上面的--reverse_input_channel備注。
The result for a sample input video can be shown below. As to be expected when using YOLOv3-tiny for inference, the frame rate easily hits near real-time inference throughout on a non-Edge device at the expense of partially rather poor car detection and bounding box accuracies.
樣本輸入視頻的結果如下所示。 可以預期，使用YOLOv3-tiny進行推理時 ，整個非Edge設備上的幀速率很容易達到接近實時推理的水平，但代價是汽車檢測和邊界框的準確性很差。
To conclude this preparation stage, simply transfer the IR files to your Raspberry Pi. 🗹
要結束此準備階段，只需將IR文件傳輸到Raspberry Pi。 🗹
This leaves us with incorporating the converted YOLOv3-tiny model in a custom-developed application running on the Raspberry Pi.
這使我們可以將轉換后的YOLOv3-tiny模型整合到在Raspberry Pi上運行的定制開發的應用程序中。
Object detection application using YOLOv3-tiny on the edge
在邊緣使用YOLOv3-tiny的物體檢測應用程序
Our application consists of three main elements, which are, generally speaking, representative for most OpenVINO-based edge applications:
我們的應用程序包含三個主要元素，通常來說，它們是大多數基于OpenVINO的邊緣應用程序的代表：
- Inference Engine model loading 推理機模型加載
- App-based input processing and inference request 基于應用程序的輸入處理和推理請求
- App-based inference result and output processing 基于應用程序的推理結果和輸出處理
This is reflected in the following code structure of our Raspberry PI application:
這反映在我們的Raspberry PI應用程序的以下代碼結構中：
We will use the Python wrapper of the Inference Engine API. The codebase can be found here on GitHub. The elements concerning YOLO inference processing largely coincide with, for example, the corresponding parts within object_detection_demo_yolov3_async.py from the OpenVINO YOLO showcase. So, to keep things easy and to the point, in the following, I discuss the OpenVINO-specific elements only.
我們將使用推理引擎API的Python包裝器。 代碼庫可以發現這里在GitHub上。 與YOLO推理處理有關的元素，例如，與OpenVINO YOLO展示中的 object_detection_demo_yolov3_async.py的相應部分基本一致。 因此，為了使事情變得簡單易懂，在下文中，我僅討論OpenVINO特定的元素。
Let’s have a quick look at the YOLOv3-tiny parameterisation in the form of Python class YoloParams first. Its values correspond to the original darknet parameterisation, so no surprises here in principle. Choosing the “right” set of anchors (line 9–10), however, is as much an art as it is a science and the particular set of anchor values used here is likely not to be the best choice for the particular object detection scenarios under consideration further below. So, in case you, too, have been wondering about YOLO anchor parameter selection, please have a look here.
讓我們以Python class YoloParams的形式快速瀏覽YOLOv3-tiny參數化。 它的值對應于原始的Darknet參數化，因此原則上在此不會感到驚訝。 然而，選擇“正確的”錨點集(第9-10行)既是一門科學，也是一門藝術，這里使用的特定錨點值集可能不是特定對象檢測場景的最佳選擇。正在進一步考慮中。 因此，如果您也想知道YOLO錨點參數的選擇，請在此處查看 。
The OpenVINO-specific methods provided by the helper class Network are more relevant in the context of this two-part blog series: The load_model method instantiates an object of the Inference Engine Core class, IECore (line 24 below). This class provides an abstraction layer across the supported devices hiding any device-specifics. As a consequence, the methods of this class typically expect the device one is actually working with as a parameter, which is why a device parameter is included in the load_model list of parameters. Since we are dealing with the Intel Neural Compute Stick 2 as inference device, this parameter’s default value is set to “MYRIAD”.
在這個分為兩部分的博客系列中，助手class Network提供的OpenVINO特定方法更加相關： load_model方法實例化推理引擎核心類IECore的對象(下面的第24行)。 此類在受支持的設備上提供了一個抽象層，隱藏了所有設備特定信息。 結果，此類的方法通常期望設備實際與之一起使用作為參數，這就是為什么將設備參數包含在參數的load_model列表中的原因。 由于我們將Intel Neural Compute Stick 2作為推理設備，因此此參數的默認值設置為“ MYRIAD”。
The YOLOv3-tiny object detection model in the form of the previously created IR files are then read into an IENetwork object (line 29). The IENetwork class supports the reading and manipulation of model parameters such as batch size, numerical precision, model shape and various layer properties. This functionality is used to set the batch size to 1 (line 31), since we will be performing inference on one frame at a time. This single input requirement is also asserted in line 33 by looking at the model’s input topology. If successful, an executable version of the model is instantiated in line 36 and loaded into the MYRIAD device for inference. Finally, lines 39–40 create iterable objects for the model input and output layers with the help of the IENetwork object.
然后將先前創建的IR文件形式的YOLOv3-tiny對象檢測模型讀入IENetwork對象(第29行)。 IENetwork類支持讀取和操縱模型參數，例如批處理大小，數值精度，模型形狀和各種圖層屬性。 此功能用于將批處理大小設置為1(第31行)，因為我們將一次對一幀執行推理。 通過查看模型的輸入拓撲，在第33行中也可以斷言這種單一輸入要求。 如果成功，則在第36行實例化模型的可執行版本，并將其加載到MYRIAD設備中以進行推斷。 最后，第39–40行借助IENetwork對象為模型輸入和輸出層創建可迭代對象。
? Since we are dealing with a MYRIAD device, to keep things simple, I have opted against the usual safety check for unsupported network layers, which is required when using a CPU device for visual inference: Since the Model Optimizer preprocessing going into the IR file generation is device-agnostic, it is possible that the optimized neural network might contain layers not supported by the specific device CPU being used. It therefore needs to be checked for any unsupported layers prior to doing CPU-based inference and further processing is actually to be exited to prevent arbitrary results or system crashes if unsupported layers are encountered. Since the the above lines of code support CPU-based inference also and if you want to experiment with inference using the Raspberry Pi’s CPU instead of an Intel Neural Compute Stick 2, have a look at, for example, object_detection_demo_yolov3_async.py for the additional layer support-related code lines required. ?
?由于我們正在處理MYRIAD設備，為簡單起見，我選擇了針對不支持的網絡層的常規安全檢查，這在使用CPU設備進行視覺推理時是必需的：由于Model Optimizer預處理已進入IR文件生成與設備無關，優化的神經網絡可能包含所使用的特定設備CPU不支持的層。 因此，在進行基于CPU的推理之前，需要檢查所有不受支持的層，并且實際上會退出進一步的處理，以防止在遇到不受支持的層時產生任意結果或系統崩潰。 由于上述代碼行還支持基于CPU的推理，并且如果您想使用Raspberry Pi的CPU代替Intel Neural Compute Stick 2進行推理實驗，請查看例如object_detection_demo_yolov3_async.py作為附加層需要與支持相關的代碼行。 ?
As far as the other methods within class Network are concerned, the get_input_shape method uses the IENetwork object to return the shape of the model’s input layer.
就class Network的其他方法而言， get_input_shape方法使用IENetwork對象返回模型輸入層的形狀。
Method async_inference takes an input request ID and performs an asynchronous inference for that request. Since we are not doing truly asynchronous processing, but rather push one frame at a time to the Inference Engine, the request ID is set to 0. Truly asynchronous processing would require different IDs for different inference requests pushing additional frames to the Inference Engine, whilst it might still be processing previous requests. These difference request IDs then allow to distinguish between the various inference results and to perform the individually required further processing.
方法async_inference接受輸入請求ID并對該請求執行異步推斷。 由于我們不是在進行真正的異步處理，而是一次將一幀推送到推理引擎，因此請求ID設置為0。對于不同的推理請求，真正的異步處理將需要不同的ID，從而將其他幀推送到推理引擎，而它可能仍在處理先前的請求。 然后，這些差異請求ID可以區分各種推斷結果，并執行單獨需要的進一步處理。
Since we are performing asynchronous processing, the wait method supports waiting for this process to be completed prior to trying to retrieve any inference result. Note that this status check needs to be done with reference to the correct request ID, so, in this case, 0.
由于我們正在執行異步處理，因此wait方法支持在嘗試檢索任何推理結果之前等待此過程完成。 請注意，此狀態檢查需要參考正確的請求ID進行，因此在這種情況下為0。
Method extract_output then returns the inference output generated by an inference request against the executable network. The request index again needs to be set to 0 to reference the correct inference request. The wait function’s value -1 makes sure that the status is returned once the process is completed. It effectively blocks processing until a certain timeout elapses or the result becomes available, whichever comes first.
然后方法extract_output返回由針對可執行網絡的推理請求生成的推理輸出。 再次需要將請求索引設置為0，以引用正確的推理請求。 等待函數的值-1確保該過程完成后返回狀態。 它有效地阻止了處理，直到經過一定的超時或結果變為可用(以先到者為準)為止。
With the various OpenVINO-related helper methods out of the way, this leaves the discussion of the video inference core routine, infer_video, within the tinyYOLOv3.py file of the GitHub repository, which uses the helper methods to take video or camera input in order to generate on-screen and video file inference results.
擺脫了與OpenVINO相關的各種輔助方法，這使得對視頻推理核心例程infer_video的討論infer_video在了GitHub存儲庫的tinyYOLOv3.py文件中，該文件使用輔助方法按順序進行視頻或攝像機輸入。生成屏幕和視頻文件推斷結果。
The OpenVINO-related initialisation contents of infer_video consists of the Inference Engine initialisation with the help of the Network class (line 7 below), followed by the model’s input shape retrieval (line 18).
的OpenVINO相關初始化內容infer_video由推理引擎初始化與所述的幫助下的Network class (下面第7行)，然后該模型的輸入形狀檢索(線18)。
The net object is subsequently used to launch an asynchronous inference request on an individual, pre-processed video frame (line 3 below) and, having waited until the asynchronous inference is completed (line 8), retrieves the inference result using the extract_output helper method (line 9). This output is then parsed as applicable in the YOLOv3 model case.
net對象隨后用于在經過預處理的單個視頻幀上發起異步推理請求(下面的第3行)，并等待異步推理完成(第8行)，然后使用extract_output helper方法檢索推理結果。 (第9行)。 然后將此輸出解析為適用于YOLOv3模型的情況。
? To write the inference result to video later on within the infer_video method, I use the OpenCV VideoWriter class with an .avi output file extension and the MP42 codec: codec = cv2.VideoWriter_fourcc("M","P","4","2"). The search for the “right” combination of codec and file extension for a particular device setup can be somewhat frustrating. See, for example, this article by Adrian Rosebrock of Pyimagesearch on this matter. It gives you a number of pointers as to what might be potentially valid combinations for your particular setup. Also, it recommends installing the Python bindings for FFMPEG on your Raspberry Pi. This has worked rather well in my case. Still, the fact that an .avi file extension in combination with the MP42 codec worked for my setup after having installed the FFMPEG library does not mean that it will work for yours and you might find yourself having to go through a number of combinations until you obtain valid video output. ?
?為了稍后在infer_video方法內將推斷結果寫入視頻，我使用帶有.avi輸出文件擴展名和MP42編codec = cv2.VideoWriter_fourcc("M","P","4","2").的OpenCV VideoWriter類： codec = cv2.VideoWriter_fourcc("M","P","4","2"). 對特定設備設置的編解碼器和文件擴展名的“正確”組合的搜索可能會有些令人沮喪。 見，例如， 本文通過Pyimagesearch的阿德里安Rosebrock在這個問題上。 它為您提供了一些有關特定設置可能是有效組合的指針。 另外，建議在Raspberry Pi上安裝FFMPEG的Python綁定。 就我而言，這種方法效果很好。 不過，在安裝FFMPEG庫后，.avi文件擴展名與MP42編解碼器結合使用可用于我的設置的事實并不意味著它會為您使用，并且您可能會發現自己必須經過多種組合才能使用，獲取有效的視頻輸出。 ?
? Please note that the infer_video method also makes use of the video package within the imutils package of Pyimagesearch to compute the frames per second (FPS) supported by the processing pipeline. ?
?請注意， infer_video方法還利用Pyimagesearch的imutils包中的video包來計算處理管道支持的每秒幀數(FPS)。 ?
We are now ready to give our application a test run on the Raspberry Pi using both a MP4 input video and the Raspberry Pi’s on-board camera scenario.
現在，我們準備使用MP4輸入視頻和Raspberry Pi的車載攝像頭場景在Raspberry Pi上對我們的應用程序進行測試。
In the case of a MP4 input file, the following command launches the application, reads in the input file, produces an on-screen window showing the inference results in the form of bounding boxes around detected objects, including the corresponding labels and writes these results to an .avi output file:
對于MP4輸入文件，以下命令將啟動應用程序，讀取輸入文件，生成一個屏幕上的窗口，以包圍對象(包括相應標簽)的包圍框的形式顯示推斷結果，并寫入這些結果到.avi輸出文件：
python tinyYOLOv3.py --m "frozen_darknet_yolov3_model.xml" --i "test_video.mp4" --l "coco.names"
python tinyYOLOv3.py --m "frozen_darknet_yolov3_model.xml" --i "test_video.mp4" --l "coco.names"
where you might have to add your specific paths to the model, input video and class label file parameters. The resulting output video for the test file [1] used is shown below. The Raspberry Pi manages a respectable frame rate of around 3 FPS due to the help of the Intel Neural Compute Stick 2. Without this hardware accelerator stick, processing slows down to a crawl with the Raspberry Pi CPU mostly being occupied with decoding the MP4 input. Not unexpectedly, the performance benefit of the YOLOv3-tiny model comes at the price of low object detection accuracy: Although persons and bicycles are detected reasonably well, other objects such as dogs and balls are not detected at all.
您可能需要在其中添加模型的特定路徑，輸入視頻和類標簽文件參數。 下面顯示了所使用的測試文件[1]的最終輸出視頻。 Raspberry Pi在Intel Neural Compute Stick 2的幫助下可管理約3 FPS的可觀幀率。如果沒有此硬件加速器Stick，Raspberry Pi CPU的處理速度將減慢到抓取速度，而Raspberry Pi CPU則主要用于解碼MP4輸入。 毫不意外的是，YOLOv3-tiny模型的性能優勢是以低的物體檢測精度為代價的：盡管可以很好地檢測到人和自行車，但是根本無法檢測到其他物體(例如狗和球)。
CAMPUS dataset for multi-view object tracking [1]CAMPUS數據集 [1]When it comes to inference on a live camera feed, the launch command looks rather similar with the exception of no input video file being provided, of course, so that the application defaults to camera input processing instead, i.e.:
當涉及到實時攝像機源的推理時，啟動命令看起來非常相似，除了沒有提供輸入視頻文件外，當然，因此該應用程序默認為攝像機輸入處理，即：
python tinyYOLOv3.py --m "frozen_darknet_yolov3_model.xml" --l "coco.names"
python tinyYOLOv3.py --m "frozen_darknet_yolov3_model.xml" --l "coco.names"
Again, you might have to add your specific paths to the model and class label file parameters. The frame rate increases to between 4 and 5 FPS. This is due to the MP4 input file processing being replaced with the less computationally demanding camera feed processing. Similarly to the input video scenario, certain objects such as the remote control and the coffee mug are detected pretty reliably, whilst, for example, the desk chair is not detected at all.
同樣，您可能必須將特定路徑添加到模型和類標簽文件參數。 幀速率增加到4到5 FPS之間。 這是因為MP4輸入文件處理已被對計算量要求較低的相機Feed處理所取代。 與輸入視頻場景類似，可以非常可靠地檢測到某些對象(例如遙控器和咖啡杯)，而例如根本沒有檢測到桌椅。
Depending on the application scenario, these relatively low levels of accuracy might be good enough. If not so, more accurate but frequently less execution-efficient models such as MobileNet single-shot detectors could be utilised as well. 🗹
根據應用程序的情況，這些相對較低的準確性水平可能會足夠好。 如果不是這樣，也可以使用更準確但經常執行效率較低的模型，例如MobileNet單發檢測器。 🗹
Object detection on camera frames using YOLOv3-tiny on a Raspberry Pi with an Intel Neural Compute Stick 2在帶Intel Neural Compute Stick 2的Raspberry Pi上使用YOLOv3-tiny在相機框架上進行物體檢測It’s a wrap.
包好了
Starting with the overview of Edge AI hardware accelerators and development boards and both Raspberry Pi configuration and Intel OpenVINO installation guides, including an object detection demo application, in part 1, we have managed to work our way all the way to a custom-implementation of a YOLOv3-tiny-based object detection application processing video file or camera input on a Raspberry Pi edge device featuring an Intel Neural Compute Stick 2. The codebase can be found on GitHub.
在第1部分中 ，從Edge AI硬件加速器和開發板的概述以及Raspberry Pi配置和Intel OpenVINO安裝指南(包括對象檢測演示應用程序)的概述開始，我們一直在努力實現自定義實現一個基于YOLOv3的對象檢測應用程序，可在具有Intel Neural Compute Stick 2的Raspberry Pi邊緣設備上處理視頻文件或攝像機輸入。該代碼庫可在GitHub上找到 。
Although the Intel OpenVINO package comes with the odd intricacy, overall it represents an excellent and straightforward way of implementing AI applications on edge devices, especially so when making use of pre-trained and pre-converted models from the Open Model Zoo.
盡管Intel OpenVINO軟件包具有奇特的復雜性，但總體而言，它代表了一種在邊緣設備上實現AI應用程序的出色而直接的方法，尤其是在使用Open Model Zoo中經過預訓練和預轉換的模型時。
Thanks to these fairly recent developments, a realm of Edge AI application scenarios has opened up too numerous to mention and ranging from, for example, smart home or smart drone devices to gesture recognition and visual sensor-based automation scenarios. All that is needed are relatively low-cost devices such as the Raspberry Pi and an hardware accelerator, the free OpenVINO library, some imagination and a little bit of perseverance.
得益于這些相當近期的發展，Edge AI應用場景的領域已經開放了太多，無法提及，范圍從智能家居或智能無人機設備到手勢識別和基于視覺傳感器的自動化場景。 所需要的只是相對低成本的設備，例如Raspberry Pi和硬件加速器，免費的OpenVINO庫，一些想像力和毅力。
[2] J. Redmon and A. Farhadi, “YOLOv3: An Incremental Improvement”, Technical Report, arXiv:1804.02767, April 2018
[2] J. Redmon和A. Farhadi，“ YOLOv3：漸進式改進”，技術報告，arXiv：1804.02767，2018年4月
翻譯自: https://medium.com/datadriveninvestor/edge-ai-computer-vision-inference-on-the-edge-part-2-2-aaddfae870f0
邊緣計算邊緣計算edge
總結
以上是生活随笔為你收集整理的边缘计算边缘计算edge_Edge AI-边缘上的计算机视觉推理的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： ArcGIS 10.1 for Serv
- 下一篇： cloud 部署_使用Google Cl