Commit 3431d8ae authored by Vadim Pisarevsky's avatar Vadim Pisarevsky

Merge pull request #286 from Jiaolong:dpm_cascade

parents ffa6637b c0858100
set(the_description "Object Detection")
#uncomment the following line to enable parallel computing
#add_definitions(-DHAVE_TBB)
ocv_define_module(dpm opencv_core opencv_imgproc opencv_objdetect OPTIONAL opencv_highgui WRAP python)
ocv_warnings_disable(CMAKE_CXX_FLAGS /wd4512) # disable warning on Win64
Cascade object detection with deformable part models
====================================================
The object detector described below has been initially proposed by P.F. Felzenszwalb in [1]. It is based on a Dalal-Triggs detector that uses a single filter on histogram of oriented gradients (HOG) features to represent an object category. This detector uses a sliding window approach, where a filter is applied at all positions and scales of an image. The first innovation is enriching the Dalal-Triggs model using a star-structured part-based model defined by a "root" filter (analogous to the Dalal-Triggs filter) plus a set of parts filters and associated deformation models. The score of one of star models at a particular position and scale within an image is the score of the root filter at the given location plus the sum over parts of the maximum, over placements of that part, of the part filter score on its location minus a deformation cost easuring the deviation of the part from its ideal location relative to the root. Both root and part filter scores are defined by the dot product between a filter (a set of weights) and a subwindow of a feature pyramid computed from the input image. Another improvement is a representation of the class of models by a mixture of star models. The score of a mixture model at a particular position and scale is the maximum over components, of the score of that component model at the given location.
The detector was dramatically speeded-up with cascade algorithm proposed by P.F. Felzenszwalb in [2]. The algorithm prunes partial hypotheses using thresholds on their scores. The basic idea of the algorithm is to use a hierarchy of models defined by an ordering of the original model's parts. For a model with (n+1) parts, including the root, a sequence of (n+1) models is obtained. The i-th model in this sequence is defined by the first i parts from the original model.
Using this hierarchy, low scoring hypotheses can be pruned after looking at the best configuration of a subset of the parts. Hypotheses that score high under a weak model are evaluated further using a richer model.
In OpenCV there is an C++ implementation of DPM cascade detector.
Usage
-----
```
// load model from model_path
cv::Ptr<DPMDetector> detector = DPMDetector::create(vector<string>(1, model_path));
// read image from image_path
Mat image = imread(image_path);
// detection
vector<DPMDetector::ObjectDetection> ds;
detector->detect(image, ds);
```
Examples
----------
```
// detect using web camera
./example_dpm_cascade_detect_camera <model_path>
// detect for an image sequence
./example_dpm_cascade_detect_sequence <model_path> <image_dir>
```
References
----------
[1]: P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan Object Detection with Discriminatively Trained Part Based Models IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, No. 9, Sep. 2010.
[2]: P. Felzenszwalb, R. Girshick, D. McAllester Cascade Object Detection with Deformable Part Models IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010.
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......@@ -37,7 +37,8 @@
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
// SVM implementation authors:
// Implementation authors:
// Jiaolong Xu - jiaolongxu@gmail.com
// Evgeniy Kozinov - evgeniy.kozinov@gmail.com
// Valentina Kustikova - valentina.kustikova@gmail.com
// Nikolai Zolotykh - Nikolai.Zolotykh@gmail.com
......@@ -55,7 +56,7 @@
#include <vector>
#include <string>
/** @defgroup latentsvm Latent SVM
/** @defgroup dpm Deformable Part-based Models
Discriminatively Trained Part Based Models for Object Detection
---------------------------------------------------------------
......@@ -85,19 +86,19 @@ Using this hierarchy, low scoring hypotheses can be pruned after looking at the
of a subset of the parts. Hypotheses that score high under a weak model are evaluated further using
a richer model.
In OpenCV there is an C++ implementation of Latent SVM.
In OpenCV there is an C++ implementation of DPM cascade detector.
*/
namespace cv
{
namespace lsvm
namespace dpm
{
/** @brief This is a C++ abstract class, it provides external user API to work with Latent SVM.
/** @brief This is a C++ abstract class, it provides external user API to work with DPM.
*/
class CV_EXPORTS_W LSVMDetector
class CV_EXPORTS_W DPMDetector
{
public:
......@@ -116,10 +117,8 @@ public:
(models) and corresponding confidence levels.
@param image An image.
@param objects The detections: rectangulars, scores and class IDs.
@param overlapThreshold Threshold for the non-maximum suppression algorithm.
*/
virtual void detect(cv::Mat const &image, CV_OUT std::vector<ObjectDetection> &objects,
float overlapThreshold=0.5f ) = 0;
*/
virtual void detect(cv::Mat &image, CV_OUT std::vector<ObjectDetection> &objects) = 0;
/** @brief Return the class (model) names that were passed in constructor or method load or extracted from
models filenames in those methods.
......@@ -130,20 +129,20 @@ public:
*/
virtual size_t getClassCount() const = 0;
/** @brief Load the trained models from given .xml files and return cv::Ptr\<LSVMDetector\>.
/** @brief Load the trained models from given .xml files and return cv::Ptr\<DPMDetector\>.
@param filenames A set of filenames storing the trained detectors (models). Each file contains one
model. See examples of such files here `/opencv_extra/testdata/cv/LSVMDetector/models_VOC2007/`.
model. See examples of such files here `/opencv_extra/testdata/cv/dpm/VOC2007_Cascade/`.
@param classNames A set of trained models names. If it's empty then the name of each model will be
constructed from the name of file containing the model. E.g. the model stored in
"/home/user/cat.xml" will get the name "cat".
*/
static cv::Ptr<LSVMDetector> create(std::vector<std::string> const &filenames,
std::vector<std::string> const &classNames = std::vector<std::string>());
static cv::Ptr<DPMDetector> create(std::vector<std::string> const &filenames,
std::vector<std::string> const &classNames = std::vector<std::string>());
virtual ~LSVMDetector(){}
virtual ~DPMDetector(){}
};
} // namespace lsvm
} // namespace dpm
} // namespace cv
#endif
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......@@ -37,93 +37,118 @@
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
// Author: Jiaolong Xu <jiaolongxu AT gmail.com>
//M*/
#include "precomp.hpp"
#include "_lsvmc_routine.h"
namespace cv
{
namespace lsvm
#include <opencv2/dpm.hpp>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/videoio.hpp>
#include <opencv2/videoio/videoio_c.h>
#include <stdio.h>
#include <iostream>
using namespace cv;
using namespace cv::dpm;
using namespace std;
static void help()
{
int allocFilterObject(CvLSVMFilterObjectCascade **obj, const int sizeX,
const int sizeY, const int numFeatures)
cout << "\nThis is a demo of \"Deformable Part-based Model (DPM) cascade detection API\" using web camera.\n"
"Call:\n"
"./example_dpm_cascade_detect_camera <model_path>\n"
<< endl;
}
void drawBoxes(Mat &frame,
vector<DPMDetector::ObjectDetection> ds,
Scalar color,
string text);
int main( int argc, char** argv )
{
int i;
(*obj) = (CvLSVMFilterObjectCascade *)malloc(sizeof(CvLSVMFilterObjectCascade));
(*obj)->sizeX = sizeX;
(*obj)->sizeY = sizeY;
(*obj)->numFeatures = numFeatures;
(*obj)->fineFunction[0] = 0.0f;
(*obj)->fineFunction[1] = 0.0f;
(*obj)->fineFunction[2] = 0.0f;
(*obj)->fineFunction[3] = 0.0f;
(*obj)->V.x = 0;
(*obj)->V.y = 0;
(*obj)->V.l = 0;
(*obj)->H = (float *) malloc(sizeof (float) *
(sizeX * sizeY * numFeatures));
for(i = 0; i < sizeX * sizeY * numFeatures; i++)
const char* keys =
{
(*obj)->H[i] = 0.0f;
"{@model_path | | Path of the DPM cascade model}"
};
CommandLineParser parser(argc, argv, keys);
string model_path(parser.get<string>(0));
if( model_path.empty() )
{
help();
return -1;
}
return LATENT_SVM_OK;
}
int freeFilterObject (CvLSVMFilterObjectCascade **obj)
{
if(*obj == NULL) return LATENT_SVM_MEM_NULL;
free((*obj)->H);
free(*obj);
(*obj) = NULL;
return LATENT_SVM_OK;
}
int allocFeatureMapObject(CvLSVMFeatureMapCascade **obj, const int sizeX,
const int sizeY, const int numFeatures)
{
int i;
(*obj) = (CvLSVMFeatureMapCascade *)malloc(sizeof(CvLSVMFeatureMapCascade));
(*obj)->sizeX = sizeX;
(*obj)->sizeY = sizeY;
(*obj)->numFeatures = numFeatures;
(*obj)->map = (float *) malloc(sizeof (float) *
(sizeX * sizeY * numFeatures));
for(i = 0; i < sizeX * sizeY * numFeatures; i++)
cv::Ptr<DPMDetector> detector = \
DPMDetector::create(vector<string>(1, model_path));
// use web camera
VideoCapture capture(0);
capture.set(CV_CAP_PROP_FRAME_WIDTH, 320);
capture.set(CV_CAP_PROP_FRAME_HEIGHT, 240);
if ( !capture.isOpened() )
{
(*obj)->map[i] = 0.0f;
cerr << "Fail to open default camera (0)!" << endl;
return -1;
}
return LATENT_SVM_OK;
}
int freeFeatureMapObject (CvLSVMFeatureMapCascade **obj)
{
if(*obj == NULL) return LATENT_SVM_MEM_NULL;
free((*obj)->map);
free(*obj);
(*obj) = NULL;
return LATENT_SVM_OK;
}
int allocFeaturePyramidObject(CvLSVMFeaturePyramidCascade **obj,
const int numLevels)
{
(*obj) = (CvLSVMFeaturePyramidCascade *)malloc(sizeof(CvLSVMFeaturePyramidCascade));
(*obj)->numLevels = numLevels;
(*obj)->pyramid = (CvLSVMFeatureMapCascade **)malloc(
sizeof(CvLSVMFeatureMapCascade *) * numLevels);
return LATENT_SVM_OK;
#ifdef HAVE_TBB
cout << "Running with TBB" << endl;
#else
#ifdef _OPENMP
cout << "Running with OpenMP" << endl;
#else
cout << "Running without OpenMP and without TBB" << endl;
#endif
#endif
Mat frame;
namedWindow("DPM Cascade Detection", 1);
// the color of the rectangle
Scalar color(0, 255, 255); // yellow
while( capture.read(frame) )
{
vector<DPMDetector::ObjectDetection> ds;
Mat image;
frame.copyTo(image);
double t = (double) getTickCount();
// detection
detector->detect(image, ds);
// compute frame per second (fps)
t = ((double) getTickCount() - t)/getTickFrequency();//elapsed time
// draw boxes
string text = format("%0.1f fps", 1.0/t);
drawBoxes(frame, ds, color, text);
imshow("DPM Cascade Detection", frame);
if ( waitKey(30) >= 0)
break;
}
return 0;
}
int freeFeaturePyramidObject (CvLSVMFeaturePyramidCascade **obj)
void drawBoxes(Mat &frame,
vector<DPMDetector::ObjectDetection> ds,
Scalar color,
string text)
{
int i;
if(*obj == NULL) return LATENT_SVM_MEM_NULL;
for(i = 0; i < (*obj)->numLevels; i++)
for (unsigned int i = 0; i < ds.size(); i++)
{
freeFeatureMapObject(&((*obj)->pyramid[i]));
rectangle(frame, ds[i].rect, color, 2);
}
free((*obj)->pyramid);
free(*obj);
(*obj) = NULL;
return LATENT_SVM_OK;
}
}
// draw text on image
Scalar textColor(0,0,250);
putText(frame, text, Point(10,50), FONT_HERSHEY_PLAIN, 2, textColor, 2);
}
/*M///////////////////////////////////////////////////////////////////////////////////////
//
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
//
// By downloading, copying, installing or using the software you agree to this license.
// If you do not agree to this license, do not download, install,
// copy or use the software.
//
//
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
// are permitted provided that the following conditions are met:
//
// * Redistribution's of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// * Redistribution's in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// * The name of the copyright holders may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
// and on any theory of liability, whether in contract, strict liability,
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
// Author: Jiaolong Xu <jiaolongxu AT gmail.com>
//M*/
#include <opencv2/dpm.hpp>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <stdio.h>
#include <iostream>
#include <fstream>
using namespace cv;
using namespace cv::dpm;
using namespace std;
int save_results(const string id, const vector<DPMDetector::ObjectDetection> ds, ofstream &out);
static void help()
{
cout << "\nThis example shows object detection on image sequences using \"Deformable Part-based Model (DPM) cascade detection API\n"
"Call:\n"
"./example_dpm_cascade_detect_sequence <model_path> <image_dir>\n"
"The image names has to be provided in \"files.txt\" under <image_dir>.\n"
<< endl;
}
static bool readImageLists( const string &file, vector<string> &imgFileList)
{
ifstream in(file.c_str(), ios::binary);
if (in.is_open())
{
while (in)
{
string line;
getline(in, line);
imgFileList.push_back(line);
}
return true;
}
else
{
cerr << "Invalid image index file: " << file << endl;
return false;
}
}
void drawBoxes(Mat &frame,
vector<DPMDetector::ObjectDetection> ds,
Scalar color,
string text);
int main( int argc, char** argv )
{
const char* keys =
{
"{@model_path | | Path of the DPM cascade model}"
"{@image_dir | | Directory of the images }"
};
CommandLineParser parser(argc, argv, keys);
string model_path(parser.get<string>(0));
string image_dir(parser.get<string>(1));
string image_list = image_dir + "/files.txt";
if( model_path.empty() || image_dir.empty() )
{
help();
return -1;
}
vector<string> imgFileList;
if ( !readImageLists(image_list, imgFileList) )
return -1;
#ifdef HAVE_TBB
cout << "Running with TBB" << endl;
#else
#ifdef _OPENMP
cout << "Running with OpenMP" << endl;
#else
cout << "Running without OpenMP and without TBB" << endl;
#endif
#endif
cv::Ptr<DPMDetector> detector = \
DPMDetector::create(vector<string>(1, model_path));
namedWindow("DPM Cascade Detection", 1);
// the color of the rectangle
Scalar color(0, 255, 255); // yellow
Mat frame;
for (size_t i = 0; i < imgFileList.size(); i++)
{
double t = (double) getTickCount();
vector<DPMDetector::ObjectDetection> ds;
Mat image = imread(image_dir + "/" + imgFileList[i]);
frame = image.clone();
if (image.empty()) {
cerr << "\nInvalid image:\n" << imgFileList[i] << endl;
return -1;
}
// detection
detector->detect(image, ds);
// compute frame per second (fps)
t = ((double) getTickCount() - t)/getTickFrequency();//elapsed time
// draw boxes
string text = format("%0.1f fps", 1.0/t);
drawBoxes(frame, ds, color, text);
// show detections
imshow("DPM Cascade Detection", frame);
if ( waitKey(30) >= 0)
break;
}
return 0;
}
void drawBoxes(Mat &frame, \
vector<DPMDetector::ObjectDetection> ds, Scalar color, string text)
{
for (unsigned int i = 0; i < ds.size(); i++)
{
rectangle(frame, ds[i].rect, color, 2);
}
// draw text on image
Scalar textColor(0,0,250);
putText(frame, text, Point(10,50), FONT_HERSHEY_PLAIN, 2, textColor, 2);
}
This source diff could not be displayed because it is too large. You can view the blob instead.
This diff is collapsed.
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......@@ -39,92 +39,120 @@
//
//M*/
/*****************************************************************************/
/* Matching procedure API */
/*****************************************************************************/
//
#ifndef _LSVM_MATCHING_H_
#define _LSVM_MATCHING_H_
#ifndef __DPM_CASCADE__
#define __DPM_CASCADE__
#include "dpm_model.hpp"
#include "dpm_feature.hpp"
#include "dpm_convolution.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/core.hpp"
#include "_lsvmc_latentsvm.h"
#include "_lsvmc_error.h"
#include "_lsvmc_routine.h"
#include <string>
#include <vector>
namespace cv
{
namespace lsvm
namespace dpm
{
/** @brief This class is the main process of DPM cascade
*/
class DPMCascade
{
private:
// pyramid level offset for covolution
std::vector< int > convLevelOffset;
// pyramid level offset for distance transform
std::vector< int > dtLevelOffset;
// convolution values
std::vector< double > convValues;
std::vector< double > pcaConvValues;
// distance transform values
std::vector< double > dtValues;
std::vector< double > pcaDtValues;
// distance transform argmax, x dimension
std::vector< int > dtArgmaxX;
std::vector< int > pcaDtArgmaxX;
// distance transform argmax, y dimension
std::vector< int > dtArgmaxY;
std::vector< int > pcaDtArgmaxY;
// half-width of distance transform window
static const int halfWindowSize = 4;
// the amount of temporary storage of cascade
int tempStorageSize;
// precomputed deformation costs
std::vector< std::vector< double > > defCostCacheX;
std::vector< std::vector< double > > defCostCacheY;
// DPM cascade model
CascadeModel model;
// feature process
Feature feature;
// feature pyramid
std::vector< Mat > pyramid;
// projected (PCA) pyramid;
std::vector< Mat > pcaPyramid;
// number of positions in each pyramid level
std::vector< int > featDimsProd;
// convolution engine
ConvolutionEngine convolutionEngine;
public:
// constructor
DPMCascade () {}
// destructor
virtual ~DPMCascade () {}
/*
// Computation border size for feature map
//
// API
// int computeBorderSize(int maxXBorder, int maxYBorder, int *bx, int *by);
// INPUT
// maxXBorder - the largest root filter size (X-direction)
// maxYBorder - the largest root filter size (Y-direction)
// OUTPUT
// bx - border size (X-direction)
// by - border size (Y-direction)
// RESULT
// Error status
*/
int computeBorderSize(int maxXBorder, int maxYBorder, int *bx, int *by);
/*
// Addition nullable border to the feature map
//
// API
// int addNullableBorder(featureMap *map, int bx, int by);
// INPUT
// map - feature map
// bx - border size (X-direction)
// by - border size (Y-direction)
// OUTPUT
// RESULT
// Error status
*/
int addNullableBorder(CvLSVMFeatureMapCascade *map, int bx, int by);
/*
// Perform non-maximum suppression algorithm (described in original paper)
// to remove "similar" bounding boxes
//
// API
// int nonMaximumSuppression(int numBoxes, const CvPoint *points,
const CvPoint *oppositePoints, const float *score,
float overlapThreshold,
int *numBoxesout, CvPoint **pointsOut,
CvPoint **oppositePointsOut, float **scoreOut);
// INPUT
// numBoxes - number of bounding boxes
// points - array of left top corner coordinates
// oppositePoints - array of right bottom corner coordinates
// score - array of detection scores
// overlapThreshold - threshold: bounding box is removed if overlap part
is greater than passed value
// OUTPUT
// numBoxesOut - the number of bounding boxes algorithm returns
// pointsOut - array of left top corner coordinates
// oppositePointsOut - array of right bottom corner coordinates
// scoreOut - array of detection scores
// RESULT
// Error status
*/
int nonMaximumSuppression(int numBoxes, const CvPoint *points,
const CvPoint *oppositePoints, const float *score,
float overlapThreshold,
int *numBoxesOut, CvPoint **pointsOut,
CvPoint **oppositePointsOut, float **scoreOut);
int getMaxFilterDims(const CvLSVMFilterObjectCascade **filters, int kComponents,
const int *kPartFilters,
unsigned int *maxXBorder, unsigned int *maxYBorder);
//}
int getMaxFilterDims(const CvLSVMFilterObjectCascade **filters, int kComponents,
const int *kPartFilters,
unsigned int *maxXBorder, unsigned int *maxYBorder);
}
}
// load cascade mode and initialize cascade
void loadCascadeModel(const std::string &modelPath);
// compute feature pyramid and projected feature pyramid
void computeFeatures(const Mat &im);
// compute root PCA scores
void computeRootPCAScores(std::vector< std::vector< Mat > > &rootScores);
// lookup or compute the score of a part at a location
double computePartScore(int plevel, int pId, int px, int py, bool isPCA, double defThreshold);
// compute location scores
void computeLocationScores(std::vector< std::vector< double > > &locctionScores);
// initialization pre-allocate storage
void initDPMCascade();
// cascade process
void process(std::vector< std::vector<double> > &detections);
// detect object from image
std::vector< std::vector<double> > detect(Mat &image);
};
#ifdef HAVE_TBB
/** @brief This class convolves root PCA feature pyramid
* and root PCA filters in parallel using Intel Threading
* Building Blocks (TBB)
*/
class ParalComputeRootPCAScores : public ParallelLoopBody
{
public:
// constructor
ParalComputeRootPCAScores(const std::vector< Mat > &pcaPyramid, const Mat &filter,\
int dim, std::vector< Mat > &scores);
// parallel loop body
void operator() (const Range &range) const;
ParalComputeRootPCAScores(const ParalComputeRootPCAScores &pComp);
private:
const std::vector< Mat > &pcaPyramid;
const Mat &filter;
int pcaDim;
std::vector< Mat > &scores;
};
#endif
} // namespace dpm
} // namespace cv
#endif // __DPM_CASCADE_
/*M///////////////////////////////////////////////////////////////////////////////////////
//
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
//
// By downloading, copying, installing or using the software you agree to this license.
// If you do not agree to this license, do not download, install,
// copy or use the software.
//
//
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
// are permitted provided that the following conditions are met:
//
// * Redistribution's of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// * Redistribution's in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// * The name of the copyright holders may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
// and on any theory of liability, whether in contract, strict liability,
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
//M*/
#include "precomp.hpp"
#include "dpm_cascade.hpp"
using namespace std;
namespace cv
{
namespace dpm
{
class DPMDetectorImpl : public DPMDetector
{
public:
DPMDetectorImpl( const vector<string>& filenames, const vector<string>& classNames=vector<string>() );
~DPMDetectorImpl();
bool isEmpty() const;
void detect(Mat &image, CV_OUT vector<ObjectDetection>& objects);
const vector<string>& getClassNames() const;
size_t getClassCount() const;
string extractModelName( const string& filename );
private:
vector< Ptr<DPMCascade> > detectors;
vector<string> classNames;
};
Ptr<DPMDetector> DPMDetector::create(vector<string> const &filenames,
vector<string> const &classNames)
{
return makePtr<DPMDetectorImpl>(filenames, classNames);
}
DPMDetectorImpl::ObjectDetection::ObjectDetection()
: score(0.f), classID(-1) {}
DPMDetectorImpl::ObjectDetection::ObjectDetection( const Rect& _rect, float _score, int _classID )
: rect(_rect), score(_score), classID(_classID) {}
DPMDetectorImpl::DPMDetectorImpl( const vector<string>& filenames,
const vector<string>& _classNames )
{
for( size_t i = 0; i < filenames.size(); i++ )
{
const string filename = filenames[i];
if( filename.length() < 5 || filename.substr(filename.length()-4, 4) != ".xml" )
continue;
Ptr<DPMCascade> detector = makePtr<DPMCascade>();
// initialization
detector->loadCascadeModel( filename.c_str() );
if( detector )
{
detectors.push_back( detector );
if( _classNames.empty() )
{
classNames.push_back( extractModelName(filename));
}
else
classNames.push_back( _classNames[i] );
}
}
}
DPMDetectorImpl::~DPMDetectorImpl()
{
}
bool DPMDetectorImpl::isEmpty() const
{
return detectors.empty();
}
const vector<string>& DPMDetectorImpl::getClassNames() const
{
return classNames;
}
size_t DPMDetectorImpl::getClassCount() const
{
return classNames.size();
}
string DPMDetectorImpl::extractModelName( const string& filename )
{
size_t startPos = filename.rfind('/');
if( startPos == string::npos )
startPos = filename.rfind('\\');
if( startPos == string::npos )
startPos = 0;
else
startPos++;
const int extentionSize = 4; //.xml
int substrLength = (int)(filename.size() - startPos - extentionSize);
return filename.substr(startPos, substrLength);
}
void DPMDetectorImpl::detect( Mat &image,
vector<ObjectDetection> &objectDetections)
{
objectDetections.clear();
for( size_t classID = 0; classID < detectors.size(); classID++ )
{
// detect objects
vector< vector<double> > detections;
detections = detectors[classID]->detect(image);
for (unsigned int i = 0; i < detections.size(); i++)
{
ObjectDetection ds = ObjectDetection();
int s = (int)detections[i].size() - 1;
ds.score = (float)detections[i][s];
int x1 = (int)detections[i][0];
int y1 = (int)detections[i][1];
int w = (int)detections[i][2] - x1 + 1;
int h = (int)detections[i][3] - y1 + 1;
ds.rect = Rect(x1, y1, w, h);
ds.classID = (int)classID;
objectDetections.push_back(ds);
}
}
}
} // namespace cv
}
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......@@ -39,29 +39,43 @@
//
//M*/
#ifndef __OPENCV_PRECOMP_H__
#define __OPENCV_PRECOMP_H__
#include "dpm_convolution.hpp"
#ifdef HAVE_CVCONFIG_H
#include "cvconfig.h"
#endif
namespace cv
{
namespace dpm
{
double ConvolutionEngine::convolve(const Mat &feat, const Mat &filter,
int dimHOG, int x, int y)
{
double val = 0;
for (int xp = 0; xp < filter.cols; xp++)
{
for (int yp = 0; yp < filter.rows; yp++)
val += filter.at<double>(yp, xp)
* feat.at<double>(y + yp, x * dimHOG + xp);
}
#ifdef __cplusplus
#include <map>
#include <deque>
#endif
return val;
}
#include "opencv2/latentsvm.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/imgproc/imgproc_c.h"
#include "opencv2/core/core_c.h"
#include "opencv2/opencv_modules.hpp"
#include "opencv2/highgui/highgui_c.h"
#ifdef HAVE_TEGRA_OPTIMIZATION
#include "opencv2/objdetect/objdetect_tegra.hpp"
#endif
#endif
void ConvolutionEngine::convolve(const Mat &feat, const Mat &filter,
int dimHOG, Mat &result)
{
for (int x = 0; x < result.cols; x++)
{
for (int y = 0; y < result.rows; y++)
{
double val = 0;
for (int xp = 0; xp < filter.cols; xp++)
{
for (int yp = 0; yp < filter.rows; yp++)
val += feat.at<double>(y + yp, x*dimHOG + xp)
* filter.at<double>(yp, xp);
} // xp
result.at<double>(y, x) = val;
} // y
} // x
}
} // namespace cv
} // namespace dpm
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......@@ -39,20 +39,36 @@
//
//M*/
#ifndef FUNCTION_SC
#define FUNCTION_SC
#ifndef __DPM_CONVOLUTION__
#define __DPM_CONVOLUTION__
#include "_lsvmc_types.h"
#include "opencv2/core.hpp"
#include <vector>
namespace cv
{
namespace lsvm
namespace dpm
{
/** @brief This class contains DPM model parameters
*/
class ConvolutionEngine
{
public:
// constructor
ConvolutionEngine() {}
// destructor
~ConvolutionEngine() {}
// compute convolution value at a fixed location
double convolve(const Mat &feat, const Mat &filter,
int dimHOG, int x, int y);
float calcM (int k,int di,int dj, const CvLSVMFeaturePyramidCascade * H, const CvLSVMFilterObjectCascade *filter);
float calcM_PCA (int k,int di,int dj, const CvLSVMFeaturePyramidCascade * H, const CvLSVMFilterObjectCascade *filter);
float calcM_PCA_cash(int k,int di,int dj, const CvLSVMFeaturePyramidCascade * H, const CvLSVMFilterObjectCascade *filter, float * cashM, int * maskM, int step);
float calcFine (const CvLSVMFilterObjectCascade *filter, int di, int dj);
}
}
#endif
// compute convolution of a feature map and multiple filters
// sum the filter convolution values into results
void convolve(const Mat &feat, const Mat &filter,
int dimHOG, Mat &result);
};
} // namespace dpm
} // namespace cv
#endif //__DPM_CONVOLUTION__
This diff is collapsed.
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......@@ -39,38 +39,135 @@
//
//M*/
#ifndef _LSVM_ROUTINE_H_
#define _LSVM_ROUTINE_H_
#ifndef __DPM_FEATURE__
#define __DPM_FEATURE__
#include "_lsvmc_types.h"
#include "_lsvmc_error.h"
#include "opencv2/core.hpp"
#include "opencv2/core/core_c.h"
#include "opencv2/imgproc.hpp"
#include <string>
#include <vector>
namespace cv
{
namespace lsvm
namespace dpm
{
// parameters of the feature pyramid
class PyramidParameter
{
public:
// number of levels per octave in feature pyramid
int interval;
// HOG cell size
int binSize;
// horizontal padding (in cells)
int padx;
// vertical padding (in cells)
int pady;
// scale factor
double sfactor;
// maximum number of scales in the pyramid
int maxScale;
// scale of each level
std::vector< double > scales;
public:
PyramidParameter()
{
// default parameters
interval = 10;
binSize = 8;
padx = 0;
pady = 0;
sfactor = 1.0;
maxScale = 0;
}
~PyramidParameter() {}
};
/** @brief This class contains DPM model parameters
*/
class Feature
{
public:
// dimension of the HOG features in a sigle cell
static const int dimHOG = 32;
// top dimPCA PCA eigenvectors
int dimPCA;
// set pyramid parameter
void setPyramidParameters(PyramidParameter val)
{
params = val;
}
// returns pyramid parameters
PyramidParameter getPyramidParameters()
{
return params;
}
// constructor
Feature ();
// constructor with parameters
Feature (PyramidParameter p);
//////////////////////////////////////////////////////////////
// Memory management routines
// All paramaters names correspond to previous data structures description
// All "alloc" functions return allocated memory for 1 object
// with all fields including arrays
// Error status is return value
//////////////////////////////////////////////////////////////
int allocFilterObject(CvLSVMFilterObjectCascade **obj, const int sizeX, const int sizeY,
const int p);
int freeFilterObject (CvLSVMFilterObjectCascade **obj);
// destrcutor
~Feature () {}
int allocFeatureMapObject(CvLSVMFeatureMapCascade **obj, const int sizeX, const int sizeY,
const int p);
int freeFeatureMapObject (CvLSVMFeatureMapCascade **obj);
// compute feature pyramid
void computeFeaturePyramid(const Mat &imageM, std::vector< Mat > &pyramid);
int allocFeaturePyramidObject(CvLSVMFeaturePyramidCascade **obj,
const int countLevel);
// project the feature pyramid with PCA coefficient matrix
void projectFeaturePyramid(const Mat &pcaCoeff, const std::vector< Mat > &pyramid, std::vector< Mat > &projPyramid);
int freeFeaturePyramidObject (CvLSVMFeaturePyramidCascade **obj);
// compute 32 dimension HOG as described in
// "Object Detection with Discriminatively Trained Part-based Models"
// by Felzenszwalb, Girshick, McAllester and Ramanan, PAMI 2010
static void computeHOG32D(const Mat &imageM, Mat &featM, const int sbin, const int padx, const int pady);
}
}
// compute location features
void computeLocationFeatures(const int numLevels, Mat &locFeature);
private:
PyramidParameter params;
};
#ifdef HAVE_TBB
/** @brief This class computes feature pyramid in parallel
* using Intel Threading Building Blocks (TBB)
*/
class ParalComputePyramid : public ParallelLoopBody
{
public:
// constructor
ParalComputePyramid(const Mat &inputImage, \
std::vector< Mat > &outputPyramid,\
PyramidParameter &p);
// initializate parameters
void initialize();
// parallel loop body
void operator() (const Range &range) const;
private:
// image to compute feature pyramid
const Mat &imageM;
// image size
Size_<double> imSize;
// output feature pyramid
std::vector< Mat > &pyramid;
// pyramid parameters
PyramidParameter &params;
};
#endif
} // namespace dpm
} // namespace cv
#endif // __DPM_FEATURE_
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......@@ -39,90 +39,110 @@
//
//M*/
#ifndef LSVM_PARSER
#define LSVM_PARSER
#ifndef __DPM_MODEL__
#define __DPM_MODEL__
#include "_lsvmc_types.h"
#include "opencv2/core.hpp"
#define MODEL 1
#define P 2
#define COMP 3
#define SCORE 4
#define RFILTER 100
#define PFILTERs 101
#define PFILTER 200
#define SIZEX 150
#define SIZEY 151
#define WEIGHTS 152
#define TAGV 300
#define Vx 350
#define Vy 351
#define TAGD 400
#define Dx 451
#define Dy 452
#define Dxx 453
#define Dyy 454
#define BTAG 500
#include <string>
#include <vector>
#define PCA 5
#define WEIGHTSPCA 162
#define CASCADE_Th 163
#define HYPOTHES_PCA 164
#define DEFORM_PCA 165
#define HYPOTHES 166
#define DEFORM 167
namespace cv
{
namespace dpm
{
/** @brief This class contains DPM model parameters
*/
class Model
{
public:
// size of HOG feature cell (e.g., 8 pixels)
int sBin;
// number of levels per octave in feature pyramid
int interval;
// maximum width of the detection window
int maxSizeX;
// maximum height of the detection window
int maxSizeY;
// dimension of HOG features
int numFeatures;
// number of components in the model
int numComponents;
// number of parts per component
std::vector<int> numParts;
// size of root filters
std::vector< Size > rootFilterDims;
// size of part filters
std::vector< Size > partFilterDims;
// root filters
std::vector< Mat > rootFilters;
// part filters
std::vector< Mat > partFilters;
// global detecion threshold
float scoreThresh;
// component indexed array of part orderings
std::vector< std::vector<int> > partOrder;
// component indexed offset (a.k.a. bias) values
std::vector<float> bias;
// location/scale weight
std::vector< std::vector< double > > locationWeight;
// idea relative positions for each deformation model
std::vector< std::vector< double > > anchors;
// array of deformation models
std::vector< std::vector< double > > defs;
#define PCACOEFF 6
// map: pFind[component][part] => part filter index
std::vector< std::vector<int> > pFind;
#define STEP_END 1000
private:
// number of part filters and deformation model
int numPartFilters;
int numDefParams;
#define EMODEL (STEP_END + MODEL)
#define EP (STEP_END + P)
#define ECOMP (STEP_END + COMP)
#define ESCORE (STEP_END + SCORE)
#define ERFILTER (STEP_END + RFILTER)
#define EPFILTERs (STEP_END + PFILTERs)
#define EPFILTER (STEP_END + PFILTER)
#define ESIZEX (STEP_END + SIZEX)
#define ESIZEY (STEP_END + SIZEY)
#define EWEIGHTS (STEP_END + WEIGHTS)
#define ETAGV (STEP_END + TAGV)
#define EVx (STEP_END + Vx)
#define EVy (STEP_END + Vy)
#define ETAGD (STEP_END + TAGD)
#define EDx (STEP_END + Dx)
#define EDy (STEP_END + Dy)
#define EDxx (STEP_END + Dxx)
#define EDyy (STEP_END + Dyy)
#define EBTAG (STEP_END + BTAG)
public:
Model () {}
virtual ~Model () {}
#define EPCA (STEP_END + PCA)
#define EWEIGHTSPCA (STEP_END + WEIGHTSPCA)
#define ECASCADE_Th (STEP_END + CASCADE_Th)
#define EHYPOTHES_PCA (STEP_END + HYPOTHES_PCA)
#define EDEFORM_PCA (STEP_END + DEFORM_PCA)
#define EHYPOTHES (STEP_END + HYPOTHES)
#define EDEFORM (STEP_END + DEFORM)
// get number of part filters
int getNumPartFilters()
{
return (int) partFilters.size();
}
#define EPCACOEFF (STEP_END + PCACOEFF)
// get number of deformation parameters
int getNumDefParams()
{
return (int) defs.size();
}
namespace cv
{
namespace lsvm
virtual void initModel() {};
virtual bool serialize(const std::string &filename) const = 0;
virtual bool deserialize(const std::string &filename) = 0;
};
class CascadeModel : public Model
{
public:
// PCA coefficient matrix
Mat pcaCoeff;
// number of dimensions used for the PCA projection
int pcaDim;
// component indexed arrays of pruning threshold
std::vector< std::vector< double > > prunThreshold;
// root pca filters
std::vector< Mat > rootPCAFilters;
// part PCA filters
std::vector< Mat > partPCAFilters;
public:
CascadeModel() {}
~CascadeModel() {}
void initModel();
bool serialize(const std::string &filename) const;
bool deserialize(const std::string &filename);
};
} // namespace lsvm
} // namespace cv
int loadModel(
// input parametr
const char *modelPath,// model path
// output parametrs
CvLSVMFilterObjectCascade ***filters,
int *kFilters,
int *kComponents,
int **kPartFilters,
float **b,
float *scoreThreshold,
float ** PCAcoeff);
}
}
#endif
#endif // __DPM_MODEL_
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......@@ -39,89 +39,114 @@
//
//M*/
#include "precomp.hpp"
#include "_lsvmc_function.h"
#include "dpm_nms.hpp"
using namespace std;
namespace cv
{
namespace lsvm
namespace dpm
{
float calcM (int k,int di,int dj, const CvLSVMFeaturePyramidCascade * H, const CvLSVMFilterObjectCascade *filter){
int i, j;
float m = 0.0f;
for(j = dj; j < dj + filter->sizeY; j++){
for(i = di * H->pyramid[k]->numFeatures; i < (di + filter->sizeX) * H->pyramid[k]->numFeatures; i++){
m += H->pyramid[k]->map[(j * H->pyramid[k]->sizeX ) * H->pyramid[k]->numFeatures + i] *
filter ->H [((j - dj) * filter->sizeX - di) * H->pyramid[k]->numFeatures + i];
void NonMaximumSuppression::sort(const vector< double > x, vector< int > &indices)
{
for (unsigned int i = 0; i < x.size(); i++)
{
for (unsigned int j = i + 1; j < x.size(); j++)
{
if (x[indices[j]] < x[indices[i]])
{
int tmp = indices[i];
indices[i] = indices[j];
indices[j] = tmp;
}
}
}
return m;
}
float calcM_PCA(int k,int di,int dj, const CvLSVMFeaturePyramidCascade * H, const CvLSVMFilterObjectCascade *filter){
int i, j;
float m = 0.0f;
for(j = dj; j < dj + filter->sizeY; j++){
for(i = di * H->pyramid[k]->numFeatures; i < (di + filter->sizeX) * H->pyramid[k]->numFeatures; i++){
m += H->pyramid[k]->map[(j * H->pyramid[k]->sizeX ) * H->pyramid[k]->numFeatures + i] *
filter ->H_PCA [((j - dj) * filter->sizeX - di) * H->pyramid[k]->numFeatures + i];
}
}
return m;
}
float calcM_PCA_cash(int k,int di,int dj, const CvLSVMFeaturePyramidCascade * H, const CvLSVMFilterObjectCascade *filter, float * cashM, int * maskM, int step){
int i, j, n;
float m = 0.0f;
float tmp1, tmp2, tmp3, tmp4;
float res;
int pos;
float *a, *b;
void NonMaximumSuppression::process(vector< vector< double > > &detections, double overlapThreshold)
{
int numBoxes = (int) detections.size();
if (numBoxes <= 0)
return;
vector< double > area(numBoxes);
vector< double > score(numBoxes);
vector< int > indices(numBoxes);
for (int i = 0; i < numBoxes; i++)
{
indices[i] = i;
int s = (int)detections[i].size();
double x1 = detections[i][0];
double y1 = detections[i][1];
double x2 = detections[i][2];
double y2 = detections[i][3];
double sc = detections[i][s-1];
score[i] = sc;
area[i] = (x2 - x1 + 1) * ( y2 - y1 + 1);
}
pos = dj * step + di;
// sort boxes by score
sort(score, indices);
vector< int > pick;
vector< int > suppress;
if(!((maskM[pos / (sizeof(int) * 8)]) & (1 << pos % (sizeof(int) * 8))))
while (indices.size() > 0)
{
for(j = dj; j < dj + filter->sizeY; j++)
int last = (int) indices.size() - 1;
int i = indices[last];
pick.push_back(i);
suppress.clear();
suppress.push_back(last);
for (int k = 0; k <= last - 1; k++)
{
a = H->pyramid[k]->map + (j * H->pyramid[k]->sizeX) * H->pyramid[k]->numFeatures
+ di * H->pyramid[k]->numFeatures;
b = filter ->H_PCA + (j - dj) * filter->sizeX * H->pyramid[k]->numFeatures;
n = ((di + filter->sizeX) * H->pyramid[k]->numFeatures) -
(di * H->pyramid[k]->numFeatures);
res = 0.0f;
tmp1 = 0.0f; tmp2 = 0.0f; tmp3 = 0.0f; tmp4 = 0.0f;
for (i = 0; i < (n >> 2); ++i)
int j = indices[k];
double xx1 = max(detections[i][0], detections[j][0]);
double yy1 = max(detections[i][1], detections[j][1]);
double xx2 = min(detections[i][2], detections[j][2]);
double yy2 = min(detections[i][3], detections[j][3]);
double w = xx2 - xx1 + 1;
double h = yy2 - yy1 + 1;
if (w > 0 && h > 0)
{
tmp1 += a[4 * i + 0] * b[4 * i + 0];
tmp2 += a[4 * i + 1] * b[4 * i + 1];
tmp3 += a[4 * i + 2] * b[4 * i + 2];
tmp4 += a[4 * i + 3] * b[4 * i + 3];
// compute overlap
double o = w*h / area[j];
if (o > overlapThreshold)
suppress.push_back(k);
}
for (i = (n >> 2) << 2; i < n; ++i) //?
} // k
// remove suppressed indices
vector< int > newIndices;
for (unsigned int n = 0; n < indices.size(); n++)
{
bool isSuppressed = false;
for (unsigned int r = 0; r < suppress.size(); r++)
{
res += a[i] * b[i];
if (n == (unsigned int)suppress[r])
{
isSuppressed = true;
break;
}
}
res += tmp1 + tmp2 + tmp3 + tmp4;
m += res;
if (!isSuppressed)
newIndices.push_back(indices[n]);
}
indices = newIndices;
} // while
cashM[pos ] = m;
maskM[pos / (sizeof(int) * 8)] |= 1 << pos % (sizeof(int) * 8);
}
else
{
m = cashM[pos];
}
return m;
}
float calcFine (const CvLSVMFilterObjectCascade *filter, int di, int dj){
return filter->fineFunction[0] * di + filter->fineFunction[1] * dj +
filter->fineFunction[2] * di * di + filter->fineFunction[3] * dj * dj;
}
}
vector< vector< double > > newDetections(pick.size());
for (unsigned int i = 0; i < pick.size(); i++)
newDetections[i] = detections[pick[i]];
detections = newDetections;
}
} // dpm
} // cv
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......@@ -38,34 +38,29 @@
// the use of this software, even if advised of the possibility of such damage.
//
//M*/
#include "precomp.hpp"
#include "_lsvmc_resizeimg.h"
#include <stdio.h>
#include <assert.h>
#include <math.h>
#ifndef __DPM_NMS__
#define __DPM_NMS__
#include <vector>
namespace cv
{
namespace lsvm
namespace dpm
{
IplImage* resize_opencv(IplImage* img, float scale)
/** @brief Non-maximum suppression
* Greedily select high-scoring detections and skip
* detections that are significantly covered by a
* previously selected detection.
*/
class NonMaximumSuppression
{
IplImage* imgTmp;
int W, H, tW, tH;
W = img->width;
H = img->height;
tW = (int)(((float)W) * scale + 0.5);
tH = (int)(((float)H) * scale + 0.5);
public:
NonMaximumSuppression() {}
~NonMaximumSuppression() {}
imgTmp = cvCreateImage(cvSize(tW , tH), img->depth, img->nChannels);
cvResize(img, imgTmp, CV_INTER_AREA);
void sort(const std::vector< double > x, std::vector< int > &indices);
return imgTmp;
}
}
}
void process(std::vector< std::vector< double > > &detections, double overlapThreshold);
};
} // dpm
} // cv
#endif // __DPM_NMS__
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......
......@@ -10,7 +10,7 @@
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Copyright (C) 2015, Itseez Inc, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
......@@ -29,7 +29,7 @@
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// In no event shall the Itseez Inc or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
......@@ -39,18 +39,17 @@
//
//M*/
#ifndef _LSVM_RESIZEIMG_H_
#define _LSVM_RESIZEIMG_H_
#ifndef __OPENCV_PRECOMP_H__
#define __OPENCV_PRECOMP_H__
#include "_lsvmc_types.h"
#ifdef HAVE_CVCONFIG_H
#include "cvconfig.h"
#endif
namespace cv
{
namespace lsvm
{
#include "opencv2/dpm.hpp"
IplImage * resize_opencv (IplImage * img, float scale);
}
}
#ifdef HAVE_TEGRA_OPTIMIZATION
#include "opencv2/objdetect/objdetect_tegra.hpp"
#endif
#endif
set(the_description "Object Detection")
ocv_define_module(latentsvm opencv_core opencv_imgproc opencv_objdetect OPTIONAL opencv_highgui WRAP python)
Implementation of the LatentSVM detector algorithm
==================================================
#include "perf_precomp.hpp"
#include <opencv2/imgproc.hpp>
using namespace std;
using namespace cv;
using namespace perf;
using std::tr1::make_tuple;
using std::tr1::get;
typedef std::tr1::tuple<std::string, int> ImageName_MinSize_t;
typedef perf::TestBaseWithParam<ImageName_MinSize_t> ImageName_MinSize;
PERF_TEST_P(ImageName_MinSize, CascadeClassifierLBPFrontalFace,
testing::Combine(testing::Values( std::string("cv/shared/lena.png"),
std::string("cv/shared/1_itseez-0000289.png"),
std::string("cv/shared/1_itseez-0000492.png"),
std::string("cv/shared/1_itseez-0000573.png")),
testing::Values(24, 30, 40, 50, 60, 70, 80, 90)
)
)
{
const string filename = get<0>(GetParam());
int min_size = get<1>(GetParam());
Size minSize(min_size, min_size);
CascadeClassifier cc(getDataPath("cv/cascadeandhog/cascades/lbpcascade_frontalface.xml"));
if (cc.empty())
FAIL() << "Can't load cascade file";
Mat img = imread(getDataPath(filename), 0);
if (img.empty())
FAIL() << "Can't load source image";
vector<Rect> faces;
equalizeHist(img, img);
declare.in(img);
while(next())
{
faces.clear();
startTimer();
cc.detectMultiScale(img, faces, 1.1, 3, 0, minSize);
stopTimer();
}
std::sort(faces.begin(), faces.end(), comparators::RectLess());
SANITY_CHECK(faces, 3.001 * faces.size());
}
\ No newline at end of file
#include "perf_precomp.hpp"
CV_PERF_TEST_MAIN(objdetect)
#ifdef __GNUC__
# pragma GCC diagnostic ignored "-Wmissing-declarations"
# if defined __clang__ || defined __APPLE__
# pragma GCC diagnostic ignored "-Wmissing-prototypes"
# pragma GCC diagnostic ignored "-Wextra"
# endif
#endif
#ifndef __OPENCV_PERF_PRECOMP_HPP__
#define __OPENCV_PERF_PRECOMP_HPP__
#include "opencv2/ts.hpp"
#include "opencv2/objdetect.hpp"
#include "opencv2/highgui.hpp"
#ifdef GTEST_CREATE_SHARED_LIBRARY
#error no modules except ts should have GTEST_CREATE_SHARED_LIBRARY defined
#endif
#endif
/*M///////////////////////////////////////////////////////////////////////////////////////
//
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
//
// By downloading, copying, installing or using the software you agree to this license.
// If you do not agree to this license, do not download, install,
// copy or use the software.
//
//
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
// are permitted provided that the following conditions are met:
//
// * Redistribution's of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// * Redistribution's in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// * The name of the copyright holders may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
// and on any theory of liability, whether in contract, strict liability,
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
//M*/
#ifndef LSVM_ERROR
#define LSVM_ERROR
#define LATENT_SVM_OK 0
#define LATENT_SVM_MEM_NULL 2
#define DISTANCE_TRANSFORM_OK 1
#define DISTANCE_TRANSFORM_GET_INTERSECTION_ERROR -1
#define DISTANCE_TRANSFORM_ERROR -2
#define DISTANCE_TRANSFORM_EQUAL_POINTS -3
#define LATENT_SVM_GET_FEATURE_PYRAMID_FAILED -4
#define LATENT_SVM_SEARCH_OBJECT_FAILED -5
#define LATENT_SVM_FAILED_SUPERPOSITION -6
#define FILTER_OUT_OF_BOUNDARIES -7
#define LATENT_SVM_TBB_SCHEDULE_CREATION_FAILED -8
#define LATENT_SVM_TBB_NUMTHREADS_NOT_CORRECT -9
#define FFT_OK 2
#define FFT_ERROR -10
#define LSVM_PARSER_FILE_NOT_FOUND -11
#endif
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
/*M///////////////////////////////////////////////////////////////////////////////////////
//
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
//
// By downloading, copying, installing or using the software you agree to this license.
// If you do not agree to this license, do not download, install,
// copy or use the software.
//
//
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2010-2013, University of Nizhny Novgorod, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
// are permitted provided that the following conditions are met:
//
// * Redistribution's of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// * Redistribution's in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// * The name of the copyright holders may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
// and on any theory of liability, whether in contract, strict liability,
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
//M*/
#include "precomp.hpp"
#include "_lsvmc_matching.h"
#include <stdio.h>
#ifndef max
#define max(a,b) (((a) > (b)) ? (a) : (b))
#endif
#ifndef min
#define min(a,b) (((a) < (b)) ? (a) : (b))
#endif
namespace cv
{
namespace lsvm
{
void sort(int n, const float* x, int* indices);
/*
// Computation border size for feature map
//
// API
// int computeBorderSize(int maxXBorder, int maxYBorder, int *bx, int *by);
// INPUT
// maxXBorder - the largest root filter size (X-direction)
// maxYBorder - the largest root filter size (Y-direction)
// OUTPUT
// bx - border size (X-direction)
// by - border size (Y-direction)
// RESULT
// Error status
*/
int computeBorderSize(int maxXBorder, int maxYBorder, int *bx, int *by)
{
*bx = (int)ceilf(((float) maxXBorder) / 2.0f + 1.0f);
*by = (int)ceilf(((float) maxYBorder) / 2.0f + 1.0f);
return LATENT_SVM_OK;
}
/*
// Addition nullable border to the feature map
//
// API
// int addNullableBorder(featureMap *map, int bx, int by);
// INPUT
// map - feature map
// bx - border size (X-direction)
// by - border size (Y-direction)
// OUTPUT
// RESULT
// Error status
*/
int addNullableBorder(CvLSVMFeatureMapCascade *map, int bx, int by)
{
int sizeX, sizeY, i, j, k;
float *new_map;
sizeX = map->sizeX + 2 * bx;
sizeY = map->sizeY + 2 * by;
new_map = (float *)malloc(sizeof(float) * sizeX * sizeY * map->numFeatures);
for (i = 0; i < sizeX * sizeY * map->numFeatures; i++)
{
new_map[i] = 0.0;
}
for (i = by; i < map->sizeY + by; i++)
{
for (j = bx; j < map->sizeX + bx; j++)
{
for (k = 0; k < map->numFeatures; k++)
{
new_map[(i * sizeX + j) * map->numFeatures + k] =
map->map[((i - by) * map->sizeX + j - bx) * map->numFeatures + k];
}
}
}
map->sizeX = sizeX;
map->sizeY = sizeY;
free(map->map);
map->map = new_map;
return LATENT_SVM_OK;
}
/*
// Computation maximum filter size for each dimension
//
// API
// int getMaxFilterDims(const CvLSVMFilterObjectCascade **filters, int kComponents,
const int *kPartFilters,
unsigned int *maxXBorder, unsigned int *maxYBorder);
// INPUT
// filters - a set of filters (at first root filter, then part filters
and etc. for all components)
// kComponents - number of components
// kPartFilters - number of part filters for each component
// OUTPUT
// maxXBorder - maximum of filter size at the horizontal dimension
// maxYBorder - maximum of filter size at the vertical dimension
// RESULT
// Error status
*/
int getMaxFilterDims(const CvLSVMFilterObjectCascade **filters, int kComponents,
const int *kPartFilters,
unsigned int *maxXBorder, unsigned int *maxYBorder)
{
int i, componentIndex;
*maxXBorder = filters[0]->sizeX;
*maxYBorder = filters[0]->sizeY;
componentIndex = kPartFilters[0] + 1;
for (i = 1; i < kComponents; i++)
{
if (unsigned(filters[componentIndex]->sizeX) > *maxXBorder)
{
*maxXBorder = filters[componentIndex]->sizeX;
}
if (unsigned(filters[componentIndex]->sizeY) > *maxYBorder)
{
*maxYBorder = filters[componentIndex]->sizeY;
}
componentIndex += (kPartFilters[i] + 1);
}
return LATENT_SVM_OK;
}
void sort(int n, const float* x, int* indices)
{
int i, j;
for (i = 0; i < n; i++)
for (j = i + 1; j < n; j++)
{
if (x[indices[j]] > x[indices[i]])
{
//float x_tmp = x[i];
int index_tmp = indices[i];
//x[i] = x[j];
indices[i] = indices[j];
//x[j] = x_tmp;
indices[j] = index_tmp;
}
}
}
/*
// Perform non-maximum suppression algorithm (described in original paper)
// to remove "similar" bounding boxes
//
// API
// int nonMaximumSuppression(int numBoxes, const CvPoint *points,
const CvPoint *oppositePoints, const float *score,
float overlapThreshold,
int *numBoxesOut, CvPoint **pointsOut,
CvPoint **oppositePointsOut, float **scoreOut);
// INPUT
// numBoxes - number of bounding boxes
// points - array of left top corner coordinates
// oppositePoints - array of right bottom corner coordinates
// score - array of detection scores
// overlapThreshold - threshold: bounding box is removed if overlap part
is greater than passed value
// OUTPUT
// numBoxesOut - the number of bounding boxes algorithm returns
// pointsOut - array of left top corner coordinates
// oppositePointsOut - array of right bottom corner coordinates
// scoreOut - array of detection scores
// RESULT
// Error status
*/
int nonMaximumSuppression(int numBoxes, const CvPoint *points,
const CvPoint *oppositePoints, const float *score,
float overlapThreshold,
int *numBoxesOut, CvPoint **pointsOut,
CvPoint **oppositePointsOut, float **scoreOut)
{
int i, j, index;
float* box_area = (float*)malloc(numBoxes * sizeof(float));
int* indices = (int*)malloc(numBoxes * sizeof(int));
int* is_suppressed = (int*)malloc(numBoxes * sizeof(int));
for (i = 0; i < numBoxes; i++)
{
indices[i] = i;
is_suppressed[i] = 0;
box_area[i] = (float)( (oppositePoints[i].x - points[i].x + 1) *
(oppositePoints[i].y - points[i].y + 1));
}
sort(numBoxes, score, indices);
for (i = 0; i < numBoxes; i++)
{
if (!is_suppressed[indices[i]])
{
for (j = i + 1; j < numBoxes; j++)
{
if (!is_suppressed[indices[j]])
{
int x1max = max(points[indices[i]].x, points[indices[j]].x);
int x2min = min(oppositePoints[indices[i]].x, oppositePoints[indices[j]].x);
int y1max = max(points[indices[i]].y, points[indices[j]].y);
int y2min = min(oppositePoints[indices[i]].y, oppositePoints[indices[j]].y);
int overlapWidth = x2min - x1max + 1;
int overlapHeight = y2min - y1max + 1;
if (overlapWidth > 0 && overlapHeight > 0)
{
float overlapPart = (overlapWidth * overlapHeight) / box_area[indices[j]];
if (overlapPart > overlapThreshold)
{
is_suppressed[indices[j]] = 1;
}
}
}
}
}
}
*numBoxesOut = 0;
for (i = 0; i < numBoxes; i++)
{
if (!is_suppressed[i]) (*numBoxesOut)++;
}
*pointsOut = (CvPoint *)malloc((*numBoxesOut) * sizeof(CvPoint));
*oppositePointsOut = (CvPoint *)malloc((*numBoxesOut) * sizeof(CvPoint));
*scoreOut = (float *)malloc((*numBoxesOut) * sizeof(float));
index = 0;
for (i = 0; i < numBoxes; i++)
{
if (!is_suppressed[indices[i]])
{
(*pointsOut)[index].x = points[indices[i]].x;
(*pointsOut)[index].y = points[indices[i]].y;
(*oppositePointsOut)[index].x = oppositePoints[indices[i]].x;
(*oppositePointsOut)[index].y = oppositePoints[indices[i]].y;
(*scoreOut)[index] = score[indices[i]];
index++;
}
}
free(indices);
free(box_area);
free(is_suppressed);
return LATENT_SVM_OK;
}
}
}
This diff is collapsed.
#include "test_precomp.hpp"
CV_TEST_MAIN("cv")
#ifdef __GNUC__
# pragma GCC diagnostic ignored "-Wmissing-declarations"
# if defined __clang__ || defined __APPLE__
# pragma GCC diagnostic ignored "-Wmissing-prototypes"
# pragma GCC diagnostic ignored "-Wextra"
# endif
#endif
#ifndef __OPENCV_TEST_PRECOMP_HPP__
#define __OPENCV_TEST_PRECOMP_HPP__
#include "opencv2/ts.hpp"
#include "opencv2/latentsvm.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"
#endif
This diff is collapsed.
<?xml version="1.0"?>
<opencv_storage>
<detections1_cat>
0 0 340 485 -0.837739 0</detections1_cat>
<detections12_cat>
0 0 340 485 -0.837739 0 129 0 181 155 -0.795819 1</detections12_cat>
<detections12_cars>
218 24 218 121 2.36436 1 0 285 233 129 1.93423 1 0 21 190 105 1.7496 1 202 183 202
73 1.57262 1 0 171 171 68 1.49932 1 238 312 165 91 0.504801 1 0 181 226 90 0.404986
1 0 0 240 171 0.158534 1 207 155 233 129 -0.1988589 1 195 278 250 139 -0.50933 1
89 0 328 119 -0.570692 1 0 295 422 154 -0.922104 1</detections12_cars>
</opencv_storage>
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment