实时模板匹配-OpenCV,C ++
c++
computer-vision
image-processing
opencv
5
0

我正在尝试使用模板实施实时跟踪。我希望每帧都更新模板。我所做的主要修改是:

1)将模板匹配和minmaxLoc分离到单独的模块中,分别是TplMatch()minmax()函数。

2)在track()函数内部,select_flag始终保持为true,以便每次迭代都将新模板复制到“ myTemplate”。

3)函数track( )的最后三行用于更新模板(roiImg)。

4) 另外,我删除了track()函数的所有参数,因为imgroiImg是全局变量,因此不需要将它们传递给函数。

以下是代码:

#include <iostream>
#include "opencv2/opencv.hpp"
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/objdetect/objdetect.hpp>

#include <sstream>


using namespace cv;
using namespace std;

Point point1, point2; /* vertical points of the bounding box */
int drag = 0;
Rect rect; /* bounding box */
Mat img, roiImg; /* roiImg - the part of the image in the bounding box */
int select_flag = 0;
bool go_fast = false;

Mat mytemplate;


///------- template matching -----------------------------------------------------------------------------------------------

Mat TplMatch( Mat &img, Mat &mytemplate )
{
  Mat result;

  matchTemplate( img, mytemplate, result, CV_TM_SQDIFF_NORMED );
  normalize( result, result, 0, 1, NORM_MINMAX, -1, Mat() );

  return result;
}


///------- Localizing the best match with minMaxLoc ------------------------------------------------------------------------

Point minmax( Mat &result )
{
  double minVal, maxVal;
  Point  minLoc, maxLoc, matchLoc;

  minMaxLoc( result, &minVal, &maxVal, &minLoc, &maxLoc, Mat() );
  matchLoc = minLoc;

  return matchLoc;
}


///------- tracking --------------------------------------------------------------------------------------------------------

void track()
{
    if (select_flag)
    {
        roiImg.copyTo(mytemplate);
//         select_flag = false;
        go_fast = true;
    }

//     imshow( "mytemplate", mytemplate ); waitKey(0);

    Mat result  =  TplMatch( img, mytemplate );
    Point match =  minmax( result ); 

    rectangle( img, match, Point( match.x + mytemplate.cols , match.y + mytemplate.rows ), CV_RGB(255, 255, 255), 0.5 );

    std::cout << "match: " << match << endl;

    /// latest match is the new template
    Rect ROI = cv::Rect( match.x, match.y, mytemplate.cols, mytemplate.rows );
    roiImg = img( ROI );
    imshow( "roiImg", roiImg ); //waitKey(0);
}


///------- MouseCallback function ------------------------------------------------------------------------------------------

void mouseHandler(int event, int x, int y, int flags, void *param)
{
    if (event == CV_EVENT_LBUTTONDOWN && !drag)
    {
        /// left button clicked. ROI selection begins
        point1 = Point(x, y);
        drag = 1;
    }

    if (event == CV_EVENT_MOUSEMOVE && drag)
    {
        /// mouse dragged. ROI being selected
        Mat img1 = img.clone();
        point2 = Point(x, y);
        rectangle(img1, point1, point2, CV_RGB(255, 0, 0), 3, 8, 0);
        imshow("image", img1);
    }

    if (event == CV_EVENT_LBUTTONUP && drag)
    {
        point2 = Point(x, y);
        rect = Rect(point1.x, point1.y, x - point1.x, y - point1.y);
        drag = 0;
        roiImg = img(rect);
//  imshow("MOUSE roiImg", roiImg); waitKey(0);
    }

    if (event == CV_EVENT_LBUTTONUP)
    {
        /// ROI selected
        select_flag = 1;
        drag = 0;
    }

}



///------- Main() ----------------------------------------------------------------------------------------------------------

int main()
{
    int k;
/*    
///open webcam
    VideoCapture cap(0);
    if (!cap.isOpened())
      return 1;*/

    ///open video file
    VideoCapture cap;
    cap.open( "Megamind.avi" );
    if ( !cap.isOpened() )
    {   cout << "Unable to open video file" << endl;    return -1;    }
/*    
    /// Set video to 320x240
     cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);
     cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);*/

    cap >> img;
    GaussianBlur( img, img, Size(7,7), 3.0 );
    imshow( "image", img );

    while (1)
    {
        cap >> img;
        if ( img.empty() )
            break;

    // Flip the frame horizontally and add blur
    cv::flip( img, img, 1 );
    GaussianBlur( img, img, Size(7,7), 3.0 );

        if ( rect.width == 0 && rect.height == 0 )
            cvSetMouseCallback( "image", mouseHandler, NULL );
        else
            track();

        imshow("image", img);
//  waitKey(100);   k = waitKey(75);
    k = waitKey(go_fast ? 30 : 10000);
        if (k == 27)
            break;
    }

    return 0;
}

不会跟踪更新的模板。我无法弄清楚为什么会发生这种情况,因为我每次迭代都更新模板(roiImg)。 minmax()函数的match值每次都返回相同的点(坐标)。可以在以下位置获得测试视频: http : //www.youtube.com/watch?v =vpnkk7N2E0Q&feature=youtu.be请仔细研究并进行指导...非常感谢!

参考资料:
Stack Overflow
收藏
评论
共 2 个回答
高赞 时间 活跃

您可以按照OpenCV教程“模板匹配”进行操作 。您的track功能可能包含用于在当前帧中查找模板的代码;一个简单的代码基于matchTemplateminMaxLoc函数。

与您问题的“实时”部分相关的有趣问题是,在当前帧与下一帧之间的时间内成功找到匹配项(如果存在)。

编辑

以下快捷方式代码和http://www.youtube.com/watch?v=vpnkk7N2E0Q&feature=youtu.be上的视频显示了我要跟踪的意思。

由于我没有网络摄像头,因此我对您的代码进行了少许修改,以仅使用视频,该https://code.ros.org/trac/opencv/export/7237/trunk/opencv/samples/cpp/tutorial_code/HighGUI/ video-input-psnr-ssim / video / Megamind.avi

然后,我添加track功能和一些逻辑来减慢视频播放速度,直到选择ROI,然后以正常速度播放视频。

#include <iostream>
#include "opencv2/opencv.hpp"
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/objdetect/objdetect.hpp>

#include <sstream>


using namespace cv;
using namespace std;

Point point1, point2; /* vertical points of the bounding box */
int drag = 0;
Rect rect; /* bounding box */
Mat img, roiImg; /* roiImg - the part of the image in the bounding box */
int select_flag = 0;
bool go_fast = false;

Mat mytemplate;

void track(cv::Mat &img, const cv::Mat &templ, const cv::Rect &r )
{
    static int n = 0;

    if (select_flag)
    {
        templ.copyTo(mytemplate);
        select_flag = false;
        go_fast = true;
    }


    cv::Mat result;
    /// Do the Matching and Normalize
    matchTemplate( img, mytemplate, result, CV_TM_SQDIFF_NORMED );
    normalize( result, result, 0, 1, NORM_MINMAX, -1, Mat() );

    /// Localizing the best match with minMaxLoc
    double minVal; double maxVal; Point minLoc; Point maxLoc;
    Point matchLoc;

    minMaxLoc( result, &minVal, &maxVal, &minLoc, &maxLoc, Mat() );
    matchLoc = minLoc;

    rectangle( img, matchLoc, Point( matchLoc.x + mytemplate.cols , matchLoc.y + mytemplate.rows ), CV_RGB(255, 255, 255), 3 );

    std::cout << matchLoc << "\n";
}

///MouseCallback function

void mouseHandler(int event, int x, int y, int flags, void *param)
{
    if (event == CV_EVENT_LBUTTONDOWN && !drag)
    {
        /* left button clicked. ROI selection begins */
        point1 = Point(x, y);
        drag = 1;
    }

    if (event == CV_EVENT_MOUSEMOVE && drag)
    {
        /* mouse dragged. ROI being selected */
        Mat img1 = img.clone();
        point2 = Point(x, y);
        rectangle(img1, point1, point2, CV_RGB(255, 0, 0), 3, 8, 0);
        imshow("image", img1);
    }

    if (event == CV_EVENT_LBUTTONUP && drag)
    {
        point2 = Point(x, y);
        rect = Rect(point1.x, point1.y, x - point1.x, y - point1.y);
        drag = 0;
        roiImg = img(rect);
    }

    if (event == CV_EVENT_LBUTTONUP)
    {
        /* ROI selected */
        select_flag = 1;
        drag = 0;
    }

}


///Main function

int main()
{
    int k;
    /*
        VideoCapture cap(0);
        if (!cap.isOpened())
        return 1;
    */
    VideoCapture cap;
    //cap.open("~/Downloads/opencv-2.4.4/samples/cpp/tutorial_code/HighGUI/video-input-psnr-ssim/video/Megamind.avi");
    cap.open("./Megamind.avi");
    if (!cap.isOpened())
    {
        printf("Unable to open video file\n");
        return -1;
    }

    /*
        // Set video to 320x240
        cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);
        cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);
        */

    cap >> img;
    imshow("image", img);

    while (1)
    {
        cap >> img;
        if (img.empty())
            break;

        if (rect.width == 0 && rect.height == 0)
            cvSetMouseCallback("image", mouseHandler, NULL);
        else
            track(img, roiImg, rect);

        if (select_flag == 1)
            imshow("Template", roiImg);

        imshow("image", img);
        k = waitKey(go_fast ? 30 : 10000);
        if (k == 27)
            break;

    }


    return 0;
}
收藏
评论

我从这个问题的修订版中获取了原始代码: https : //stackoverflow.com/revisions/20180073/3

我对您的原始代码进行了最小的更改,得到的代码如下:

#include <iostream>
#include "opencv2/opencv.hpp"
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/objdetect/objdetect.hpp>

#include <sstream>


using namespace cv;
using namespace std;

Point point1, point2; /* vertical points of the bounding box */
int drag = 0;
Rect rect; /* bounding box */
Mat img, roiImg; /* roiImg - the part of the image in the bounding box */
int select_flag = 0;
bool go_fast = false;

Mat mytemplate;


///------- template matching -----------------------------------------------------------------------------------------------

Mat TplMatch( Mat &img, Mat &mytemplate )
{
  Mat result;

  matchTemplate( img, mytemplate, result, CV_TM_SQDIFF_NORMED );
  normalize( result, result, 0, 1, NORM_MINMAX, -1, Mat() );

  return result;
}


///------- Localizing the best match with minMaxLoc ------------------------------------------------------------------------

Point minmax( Mat &result )
{
  double minVal, maxVal;
  Point  minLoc, maxLoc, matchLoc;

  minMaxLoc( result, &minVal, &maxVal, &minLoc, &maxLoc, Mat() );
  matchLoc = minLoc;

  return matchLoc;
}


///------- tracking --------------------------------------------------------------------------------------------------------

void track()
{
    if (select_flag)
    {
        //roiImg.copyTo(mytemplate);
//         select_flag = false;
        go_fast = true;
    }

//     imshow( "mytemplate", mytemplate ); waitKey(0);

    Mat result  =  TplMatch( img, mytemplate );
    Point match =  minmax( result ); 

    rectangle( img, match, Point( match.x + mytemplate.cols , match.y + mytemplate.rows ), CV_RGB(255, 255, 255), 0.5 );

    std::cout << "match: " << match << endl;

    /// latest match is the new template
    Rect ROI = cv::Rect( match.x, match.y, mytemplate.cols, mytemplate.rows );
    roiImg = img( ROI );
    roiImg.copyTo(mytemplate);
    imshow( "roiImg", roiImg ); //waitKey(0);
}


///------- MouseCallback function ------------------------------------------------------------------------------------------

void mouseHandler(int event, int x, int y, int flags, void *param)
{
    if (event == CV_EVENT_LBUTTONDOWN && !drag)
    {
        /// left button clicked. ROI selection begins
        point1 = Point(x, y);
        drag = 1;
    }

    if (event == CV_EVENT_MOUSEMOVE && drag)
    {
        /// mouse dragged. ROI being selected
        Mat img1 = img.clone();
        point2 = Point(x, y);
        rectangle(img1, point1, point2, CV_RGB(255, 0, 0), 3, 8, 0);
        imshow("image", img1);
    }

    if (event == CV_EVENT_LBUTTONUP && drag)
    {
        point2 = Point(x, y);
        rect = Rect(point1.x, point1.y, x - point1.x, y - point1.y);
        drag = 0;
        roiImg = img(rect);
        roiImg.copyTo(mytemplate);
//  imshow("MOUSE roiImg", roiImg); waitKey(0);
    }

    if (event == CV_EVENT_LBUTTONUP)
    {
        /// ROI selected
        select_flag = 1;
        drag = 0;
    }

}



///------- Main() ----------------------------------------------------------------------------------------------------------

int main()
{
    int k;
/*    
///open webcam
    VideoCapture cap(0);
    if (!cap.isOpened())
      return 1;*/

    ///open video file
    VideoCapture cap;
    cap.open( "Megamind.avi" );
    if ( !cap.isOpened() )
    {   cout << "Unable to open video file" << endl;    return -1;    }
/*    
    /// Set video to 320x240
     cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);
     cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);*/

    cap >> img;
    GaussianBlur( img, img, Size(7,7), 3.0 );
    imshow( "image", img );

    while (1)
    {
        cap >> img;
        if ( img.empty() )
            break;

    // Flip the frame horizontally and add blur
    cv::flip( img, img, 1 );
    GaussianBlur( img, img, Size(7,7), 3.0 );

        if ( rect.width == 0 && rect.height == 0 )
            cvSetMouseCallback( "image", mouseHandler, NULL );
        else
            track();

        imshow("image", img);
//  waitKey(100);   k = waitKey(75);
    k = waitKey(go_fast ? 30 : 10000);
        if (k == 27)
            break;
    }

    return 0;
}

https://www.youtube.com/watch?v=rBCopeneCos上的视频显示了对上述程序的测试。

我会避免使用全局变量,因为我认为它们无助于理解问题所在。此外,我还将关注OpenCV的Mat类的浅层副本与深层副本,如1''在他的回答中写道:

OpenCV的Mat类只是实际图像数据的标题,它包含一个指向的指针。 operator=复制指针(以及标题中的其他信息,如图像尺寸),以便两个Mat共享相同的数据。这意味着修改一个Mat中的数据也会更改另一个Mat中的数据。这被称为“浅”副本,因为仅复制顶层(标题),而不复制较低层(数据)。

要复制基础数据(称为“深层复制”),请使用clone()方法。您可以在链接到的页面上找到有关它的信息。

编辑有关漂移的内容:在注释中实时模板匹配-OpenCV,C ++学习者询问有关跟踪漂移的信息。观看视频https://www.youtube.com/watch?v=rBCopeneCos,我们看到在视频的开头,该程序正在跟踪女孩的右眼,而在0:15时,它开始跟踪女孩的眉毛。 0:19,它开始跟踪男孩的眉毛,并且不再跟踪女孩的眼睛,例如,在0:27,它跟踪女孩的右眉,而女孩的右眼在图像中清晰可见。

在我发布的简单代码中,从跟踪眼睛到跟踪眉毛这是正常现象,并且解释也很简单:请参见https://www.youtube.com/watch?v=sGHEu3u9XvI上的视频首先从跟踪纸牌(黑色矩形的内容)开始,然后从场景中移除纸牌,然后将跟踪的黑色矩形“漂移”到场景的左下角;毕竟,我们一直在不断更新模板,因此行为是正确的:程序停止跟踪纸牌并开始跟踪白色背景,因此您具有“漂移” ...换句话说,就是您的TplMatch()函数将始终返回有效的result图像,而您当前的minmax()将始终返回有效的最小值。

收藏
评论
新手导航
  • 社区规范
  • 提出问题
  • 进行投票
  • 个人资料
  • 优化问题
  • 回答问题

关于我们

常见问题

内容许可

联系我们

@2020 AskGo
京ICP备20001863号