介紹 (Introduction)

As a data scientist at VATBox, I’ve mainly worked on projects which at their core involve building Machine Learning models. What’s nice about this project is that it purely includes building an algorithm to solve the given task. A real-world problem with a custom-made solution.

作為VATBox的數據科學家，我主要從事以構建機器學習模型為核心的項目。 這個項目的好處在于，它純粹包括構建用于解決給定任務的算法。 定制解決方案的實際問題。

問題定義 (Problem Definition)

The problem in hand is essentially matching between two images which are close to being identical (changes can be due to image size, for example). The goal here is to do so in real-time, therefore we need the algorithm to be relatively fast.

當前的問題實質上是兩個幾乎相同的圖像之間的匹配(例如，變化可能是由于圖像大小引起的)。 這里的目標是實時執行此操作，因此我們需要算法相對較快。

At VATBox our world is a world of invoices. Users upload reports, which contain images of invoices, to our platform. A single report contains two groups of images, one is a collection of separate invoices, and the other is all the collection together in one single file. For reasons which we will not go into, there may be images in one group that won’t appear in the other. The goal is to detect the images which don’t appear in the other group (if there are any). If we match between all the images which are essentially the same image, we can also identify the spare ones.

在VATBox，我們的世界就是發票的世界。 用戶將包含發票圖像的報告上載到我們的平臺。 單個報告包含兩組圖像，一組是單獨發票的集合，另一組是所有集合在一起的一個文件。 由于我們不愿討論的原因，一組中的圖像可能不會出現在另一組中。 目的是檢測未出現在另一組中的圖像(如果有)。 如果我們在本質上是相同圖像的所有圖像之間匹配，我們也可以識別備用圖像。

We want to detect the spare images (marked in red on the left and blue on the right)我們要檢測備用圖像(左側標記為紅色，右側標記為藍色)

解 (Solution)

So to find the spare images, we need to perform a comparison between every two images (or do we..? we’ll get to that..), one image being from the first group and the other from the second group. Once that is done, we can match between images, and then see which images are left without any match. We can build a Comparison Matrix which holds these comparisons.

因此，要查找備用圖像，我們需要在每兩個圖像之間進行比較(否則我們將進行比較。)，一個圖像來自第一組，另一圖像來自第二組。 完成后，我們可以在圖像之間進行匹配，然后查看剩下的圖像沒有任何匹配。 我們可以建立一個包含這些比較的比較矩陣。

Comparison Matrix比較矩陣

During my research, I mostly encountered solutions that are based on features extracted from images to match between images, but that’s computationally expensive for our case, as we’re interested in doing this for every pair of images. Therefore we are in need of a different approach.

在研究過程中，我經常遇到基于從圖像中提取的特征以在圖像之間進行匹配的解決方案，但這對于我們的案例而言在計算上是昂貴的，因為我們有興趣對每對圖像進行此操作。 因此，我們需要一種不同的方法。

As described, the images that actually match, are close to being identical. So we can expect their pixel values to be relatively similar. So can we perhaps calculate the correlation between the two given signals? But how can this be done efficiently with images?

如上所述，實際匹配的圖像幾乎是相同的。 因此，我們可以期望它們的像素值相對相似。 那么也許我們可以計算兩個給定信號之間的相關性嗎？ 但是，如何使用圖像有效地做到這一點呢？

OpenCV has a Template Matching module. The purpose of this module is to find a given template within a (larger) image. The module enables us to “swipe” a template (T) across an image (I) and perform calculations efficiently (similarly to how a convolutional kernel is swiped on an image in a CNN).

OpenCV具有模板匹配模塊。 該模塊的目的是在(較大)圖像中找到給定的模板。 該模塊使我們能夠在圖像(I)上“滑動”模板(T)并高效地執行計算(類似于卷積核如何在CNN中的圖像上滑動)。

pexels.compexels.com

This can be great for our needs! We can use this to swipe an image from the first group on an image of the other group efficently, as OpenCV’s implementation is optimal. Of course, since we’re looking for two images that match (and not a template in another image) the swiping is just done a single time.

這可以滿足我們的需求！ 因為OpenCV實現是最佳的，所以我們可以使用它從第一組的圖像有效地刷到另一組的圖像上。 當然，由于我們要查找匹配的兩個圖像(而不是另一個圖像中的模板)，因此只需一次刷卡即可。

The Template Matching module includes different methods of calculating correlations, one of them being a normalized calculation, returning values between 0 (no similarity) to 1 (completely identical images):

模板匹配模塊包括計算相關性的不同方法 ，其中之一是歸一化計算，返回0(無相似性)到1(完全相同的圖像)之間的值：

corr = cv2.matchTemplate(group1_image, group2_image,
cv2.TM_CCOEFF_NORMED)[0][0]

The indexing shows that we’re accessing a matrix. As mentioned, we’re essentially performing a single swipe, therefore we’re just grabbing the single value from a matrix with a single element.

索引顯示我們正在訪問矩陣。 如前所述，我們實際上是在執行一次滑動，因此我們只是從具有單個元素的矩陣中獲取單個值。

A word about the calculation. Prior to the actual correlation calculation, the images are normalized, resulting in a matrix (or tensor) with negative values as well as positive values. When performing the correlation calculation, similar regions will get positive values within the correlation calculation, whereas dissimilar regions will get negative values (resulting in lower correlation).

關于計算的一句話。 在實際的相關性計算之前，將圖像標準化，從而得到具有負值和正值的矩陣(或張量)。 當執行相關計算時，相似區域在相關計算中將獲得正值，而不同區域將獲得負值(導致較低的相關性)。

復雜 (Complexity)

Since we’re comparing every pair of images, the complexity of the algorithm is O(n*m), n being the number of images in group one, and m being the number of images in group two. Can we make the algorithm more efficient?

由于我們正在比較每對圖像，因此算法的復雜度為O(n * m)，n為第一組中的圖像數量，m為第二組中的圖像數量。 我們可以使算法更有效嗎？

As the above function returns values close to 1 for similar images, we can set a threshold (e.g. 0.9) for which there’s no need to keep on searching for a match if one is found. In addition, if a match is found, we’ll remove the matched images from our search. For example, if in the first iteration we are comparing an image from group one with 4 images from group two, on the second iteration we’ll do a comparison with only 3 images, and so on.

由于上述函數針對相似圖像返回的值接近1，因此我們可以設置一個閾值(例如0.9)，如果找到該閾值，則無需繼續搜索匹配項。 此外，如果找到匹配項，我們將從搜索中刪除匹配的圖像。 例如，如果在第一次迭代中我們將來自第一組的圖像與來自第二組的4個圖像進行比較，那么在第二次迭代中，我們將僅對3個圖像進行比較，依此類推。

摘要 (Summary)

In this article, we reviewed a somewhat out of the ordinary real-world data science problem which required a tailor-made solution. We dug into OpenCV’s Template Matching and understood how it can be used to fulfill the requirements of the problem at hand. Hope you enjoyed it!

在本文中，我們回顧了一個與眾不同的現實世界數據科學問題，該問題需要量身定制的解決方案。 我們深入研究了OpenCV模板匹配，并了解了如何將其用于滿足當前問題的要求。 希望你喜歡！

翻譯自: https://towardsdatascience.com/image-matching-with-opencvs-template-matching-5df577a3ce2e

總結

以上是生活随笔為你收集整理的图像匹配与OpenCV模板匹配的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。