Merge pull request #56 from vpisarev/added_xfeatures2d

added xfeatures2d (made of opencv/nonfree and a part of opencv/features2d)

modules/face/doc/facerec/facerec_tutorial.rst

@@ -375,7 +375,7 @@ Now real life isn't perfect. You simply can't guarantee perfect light settings i
 
 So in order to get good recognition rates you'll need at least 8(+-1) images for each person and the Fisherfaces method doesn't really help here. The above experiment is a 10-fold cross validated result carried out with the facerec framework at: `https://github.com/bytefish/facerec <https://github.com/bytefish/facerec>`_. This is not a publication, so I won't back these figures with a deep mathematical analysis. Please have a look into [KM01]_ for a detailed analysis of both methods, when it comes to small training datasets.
 
-So some research concentrated on extracting local features from images. The idea is to not look at the whole image as a high-dimensional vector, but describe only local features of an object. The features you extract this way will have a low-dimensionality implicitly. A fine idea! But you'll soon observe the image representation we are given doesn't only suffer from illumination variations. Think of things like scale, translation or rotation in images - your local description has to be at least a bit robust against those things. Just like :ocv:class:`SIFT`, the Local Binary Patterns methodology has its roots in 2D texture analysis. The basic idea of Local Binary Patterns is to summarize the local structure in an image by comparing each pixel with its neighborhood. Take a pixel as center and threshold its neighbors against. If the intensity of the center pixel is greater-equal its neighbor, then denote it with 1 and 0 if not. You'll end up with a binary number for each pixel, just like 11001111. So with 8 surrounding pixels you'll end up with 2^8 possible combinations, called *Local Binary Patterns* or sometimes referred to as *LBP codes*. The first LBP operator described in literature actually used a fixed 3 x 3 neighborhood just like this:
+So some research concentrated on extracting local features from images. The idea is to not look at the whole image as a high-dimensional vector, but describe only local features of an object. The features you extract this way will have a low-dimensionality implicitly. A fine idea! But you'll soon observe the image representation we are given doesn't only suffer from illumination variations. Think of things like scale, translation or rotation in images - your local description has to be at least a bit robust against those things. Just like ``SIFT``, the Local Binary Patterns methodology has its roots in 2D texture analysis. The basic idea of Local Binary Patterns is to summarize the local structure in an image by comparing each pixel with its neighborhood. Take a pixel as center and threshold its neighbors against. If the intensity of the center pixel is greater-equal its neighbor, then denote it with 1 and 0 if not. You'll end up with a binary number for each pixel, just like 11001111. So with 8 surrounding pixels you'll end up with 2^8 possible combinations, called *Local Binary Patterns* or sometimes referred to as *LBP codes*. The first LBP operator described in literature actually used a fixed 3 x 3 neighborhood just like this:
 
 .. image:: img/lbp/lbp.png
     :scale: 80%

modules/xfeatures2d/CMakeLists.txt

+set(the_description "Contributed/Experimental Algorithms for Salient 2D Features Detection")
+ocv_define_module(xfeatures2d opencv_core opencv_imgproc opencv_features2d opencv_calib3d opencv_shape opencv_highgui opencv_videoio opencv_ml)

modules/xfeatures2d/doc/extra_features.rst

+Non-free 2D Features Algorithms
+=================================
+
+This section describes experimental algorithms for 2d feature detection.
+
+StarFeatureDetector
+-------------------
+.. ocv:class:: StarFeatureDetector : public FeatureDetector
+
+The class implements the keypoint detector introduced by [Agrawal08]_, synonym of ``StarDetector``.  ::
+
+    class StarFeatureDetector : public FeatureDetector
+    {
+    public:
+        StarFeatureDetector( int maxSize=16, int responseThreshold=30,
+                             int lineThresholdProjected = 10,
+                             int lineThresholdBinarized=8, int suppressNonmaxSize=5 );
+        virtual void read( const FileNode& fn );
+        virtual void write( FileStorage& fs ) const;
+    protected:
+        ...
+    };
+
+.. [Agrawal08] Agrawal, M., Konolige, K., & Blas, M. R. (2008). Censure: Center surround extremas for realtime feature detection and matching. In Computer Vision–ECCV 2008 (pp. 102-115). Springer Berlin Heidelberg.
+
+
+BriefDescriptorExtractor
+------------------------
+.. ocv:class:: BriefDescriptorExtractor : public DescriptorExtractor
+
+Class for computing BRIEF descriptors described in a paper of Calonder M., Lepetit V.,
+Strecha C., Fua P. *BRIEF: Binary Robust Independent Elementary Features* ,
+11th European Conference on Computer Vision (ECCV), Heraklion, Crete. LNCS Springer, September 2010. ::
+
+    class BriefDescriptorExtractor : public DescriptorExtractor
+    {
+    public:
+        static const int PATCH_SIZE = 48;
+        static const int KERNEL_SIZE = 9;
+
+        // bytes is a length of descriptor in bytes. It can be equal 16, 32 or 64 bytes.
+        BriefDescriptorExtractor( int bytes = 32 );
+
+        virtual void read( const FileNode& );
+        virtual void write( FileStorage& ) const;
+        virtual int descriptorSize() const;
+        virtual int descriptorType() const;
+        virtual int defaultNorm() const;
+    protected:
+        ...
+    };
+
+.. note::
+
+   * A complete BRIEF extractor sample can be found at opencv_source_code/samples/cpp/brief_match_test.cpp
+
+
+FREAK
+-----
+.. ocv:class:: FREAK : public DescriptorExtractor
+
+Class implementing the FREAK (*Fast Retina Keypoint*) keypoint descriptor, described in [AOV12]_. The algorithm propose a novel keypoint descriptor inspired by the human visual system and more precisely the retina, coined Fast Retina Key- point (FREAK). A cascade of binary strings is computed by efficiently comparing image intensities over a retinal sampling pattern. FREAKs are in general faster to compute with lower memory load and also more robust than SIFT, SURF or BRISK. They are competitive alternatives to existing keypoints in particular for embedded applications.
+
+.. [AOV12] A. Alahi, R. Ortiz, and P. Vandergheynst. FREAK: Fast Retina Keypoint. In IEEE Conference on Computer Vision and Pattern Recognition, 2012. CVPR 2012 Open Source Award Winner.
+
+.. note::
+
+   * An example on how to use the FREAK descriptor can be found at opencv_source_code/samples/cpp/freak_demo.cpp
+
+FREAK::FREAK
+------------
+The FREAK constructor
+
+.. ocv:function:: FREAK::FREAK( bool orientationNormalized=true, bool scaleNormalized=true, float patternScale=22.0f, int nOctaves=4, const vector<int>& selectedPairs=vector<int>() )
+
+    :param orientationNormalized: Enable orientation normalization.
+    :param scaleNormalized: Enable scale normalization.
+    :param patternScale: Scaling of the description pattern.
+    :param nOctaves: Number of octaves covered by the detected keypoints.
+    :param selectedPairs: (Optional) user defined selected pairs indexes,
+
+FREAK::selectPairs
+------------------
+Select the 512 best description pair indexes from an input (grayscale) image set. FREAK is available with a set of pairs learned off-line. Researchers can run a training process to learn their own set of pair. For more details read section 4.2 in: A. Alahi, R. Ortiz, and P. Vandergheynst. FREAK: Fast Retina Keypoint. In IEEE Conference on Computer Vision and Pattern Recognition, 2012.
+
+We notice that for keypoint matching applications, image content has little effect on the selected pairs unless very specific what does matter is the detector type (blobs, corners,...) and the options used (scale/rotation invariance,...). Reduce corrThresh if not enough pairs are selected (43 points --> 903 possible pairs)
+
+.. ocv:function:: vector<int> FREAK::selectPairs(const vector<Mat>& images, vector<vector<KeyPoint> >& keypoints, const double corrThresh = 0.7, bool verbose = true)
+
+    :param images: Grayscale image input set.
+    :param keypoints: Set of detected keypoints
+    :param corrThresh: Correlation threshold.
+    :param verbose: Prints pair selection informations.

modules/xfeatures2d/doc/xfeatures2d.rst

+***************************************
+features2d. Extra 2D Features Framework
+***************************************
+
+.. highlight:: cpp
+
+.. toctree::
+    :maxdepth: 2
+
+    extra_features
+    nonfree_features

modules/xfeatures2d/include/opencv2/xfeatures2d.hpp

+/*
+By downloading, copying, installing or using the software you agree to this
+license. If you do not agree to this license, do not download, install,
+copy or use the software.
+
+                          License Agreement
+               For Open Source Computer Vision Library
+                       (3-clause BSD License)
+
+Copyright (C) 2013, OpenCV Foundation, all rights reserved.
+Third party copyrights are property of their respective owners.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+  * Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+
+  * Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+
+  * Neither the names of the copyright holders nor the names of the contributors
+    may be used to endorse or promote products derived from this software
+    without specific prior written permission.
+
+This software is provided by the copyright holders and contributors "as is" and
+any express or implied warranties, including, but not limited to, the implied
+warranties of merchantability and fitness for a particular purpose are
+disclaimed. In no event shall copyright holders or contributors be liable for
+any direct, indirect, incidental, special, exemplary, or consequential damages
+(including, but not limited to, procurement of substitute goods or services;
+loss of use, data, or profits; or business interruption) however caused
+and on any theory of liability, whether in contract, strict liability,
+or tort (including negligence or otherwise) arising in any way out of
+the use of this software, even if advised of the possibility of such damage.
+*/
+
+#ifndef __OPENCV_XFEATURES2D_HPP__
+#define __OPENCV_XFEATURES2D_HPP__
+
+#include "opencv2/features2d.hpp"
+#include "opencv2/xfeatures2d/nonfree.hpp"
+
+namespace cv
+{
+namespace xfeatures2d
+{
+
+CV_EXPORTS bool initModule_xfeatures2d(void);
+
+/*!
+ FREAK implementation
+ */
+class CV_EXPORTS FREAK : public DescriptorExtractor
+{
+public:
+    /** Constructor
+     * @param orientationNormalized enable orientation normalization
+     * @param scaleNormalized enable scale normalization
+     * @param patternScale scaling of the description pattern
+     * @param nbOctave number of octaves covered by the detected keypoints
+     * @param selectedPairs (optional) user defined selected pairs
+     */
+    explicit FREAK( bool orientationNormalized = true,
+                   bool scaleNormalized = true,
+                   float patternScale = 22.0f,
+                   int nOctaves = 4,
+                   const std::vector<int>& selectedPairs = std::vector<int>());
+    FREAK( const FREAK& rhs );
+    FREAK& operator=( const FREAK& );
+
+    virtual ~FREAK();
+
+    /** returns the descriptor length in bytes */
+    virtual int descriptorSize() const;
+
+    /** returns the descriptor type */
+    virtual int descriptorType() const;
+
+    /** returns the default norm type */
+    virtual int defaultNorm() const;
+
+    /** select the 512 "best description pairs"
+     * @param images grayscale images set
+     * @param keypoints set of detected keypoints
+     * @param corrThresh correlation threshold
+     * @param verbose print construction information
+     * @return list of best pair indexes
+     */
+    std::vector<int> selectPairs( const std::vector<Mat>& images, std::vector<std::vector<KeyPoint> >& keypoints,
+                                 const double corrThresh = 0.7, bool verbose = true );
+
+    AlgorithmInfo* info() const;
+
+    enum
+    {
+        NB_SCALES = 64, NB_PAIRS = 512, NB_ORIENPAIRS = 45
+    };
+
+protected:
+    virtual void computeImpl( InputArray image, std::vector<KeyPoint>& keypoints, OutputArray descriptors ) const;
+    void buildPattern();
+
+    template <typename imgType, typename iiType>
+    imgType meanIntensity( InputArray image, InputArray integral, const float kp_x, const float kp_y,
+                          const unsigned int scale, const unsigned int rot, const unsigned int point ) const;
+
+    template <typename srcMatType, typename iiMatType>
+    void computeDescriptors( InputArray image, std::vector<KeyPoint>& keypoints, OutputArray descriptors ) const;
+
+    template <typename srcMatType>
+    void extractDescriptor(srcMatType *pointsValue, void ** ptr) const;
+
+    bool orientationNormalized; //true if the orientation is normalized, false otherwise
+    bool scaleNormalized; //true if the scale is normalized, false otherwise
+    double patternScale; //scaling of the pattern
+    int nOctaves; //number of octaves
+    bool extAll; // true if all pairs need to be extracted for pairs selection
+
+    double patternScale0;
+    int nOctaves0;
+    std::vector<int> selectedPairs0;
+
+    struct PatternPoint
+    {
+        float x; // x coordinate relative to center
+        float y; // x coordinate relative to center
+        float sigma; // Gaussian smoothing sigma
+    };
+
+    struct DescriptionPair
+    {
+        uchar i; // index of the first point
+        uchar j; // index of the second point
+    };
+
+    struct OrientationPair
+    {
+        uchar i; // index of the first point
+        uchar j; // index of the second point
+        int weight_dx; // dx/(norm_sq))*4096
+        int weight_dy; // dy/(norm_sq))*4096
+    };
+    
+    std::vector<PatternPoint> patternLookup; // look-up table for the pattern points (position+sigma of all points at all scales and orientation)
+    int patternSizes[NB_SCALES]; // size of the pattern at a specific scale (used to check if a point is within image boundaries)
+    DescriptionPair descriptionPairs[NB_PAIRS];
+    OrientationPair orientationPairs[NB_ORIENPAIRS];
+};
+
+
+/*!
+ The "Star" Detector.
+
+ The class implements the keypoint detector introduced by K. Konolige.
+ */
+class CV_EXPORTS_W StarDetector : public FeatureDetector
+{
+public:
+    //! the full constructor
+    CV_WRAP StarDetector(int _maxSize=45, int _responseThreshold=30,
+                         int _lineThresholdProjected=10,
+                         int _lineThresholdBinarized=8,
+                         int _suppressNonmaxSize=5);
+
+    //! finds the keypoints in the image
+    CV_WRAP_AS(detect) void operator()(const Mat& image,
+                                       CV_OUT std::vector<KeyPoint>& keypoints) const;
+
+    AlgorithmInfo* info() const;
+
+protected:
+    void detectImpl( InputArray image, std::vector<KeyPoint>& keypoints, InputArray mask=noArray() ) const;
+    
+    int maxSize;
+    int responseThreshold;
+    int lineThresholdProjected;
+    int lineThresholdBinarized;
+    int suppressNonmaxSize;
+};
+
+typedef StarDetector StarFeatureDetector;
+
+/*
+ * BRIEF Descriptor
+ */
+class CV_EXPORTS BriefDescriptorExtractor : public DescriptorExtractor
+{
+public:
+    static const int PATCH_SIZE = 48;
+    static const int KERNEL_SIZE = 9;
+
+    // bytes is a length of descriptor in bytes. It can be equal 16, 32 or 64 bytes.
+    BriefDescriptorExtractor( int bytes = 32 );
+
+    virtual void read( const FileNode& );
+    virtual void write( FileStorage& ) const;
+
+    virtual int descriptorSize() const;
+    virtual int descriptorType() const;
+    virtual int defaultNorm() const;
+
+    /// @todo read and write for brief
+
+    AlgorithmInfo* info() const;
+
+protected:
+    virtual void computeImpl(InputArray image, std::vector<KeyPoint>& keypoints, OutputArray descriptors) const;
+
+    typedef void(*PixelTestFn)(InputArray, const std::vector<KeyPoint>&, OutputArray);
+    
+    int bytes_;
+    PixelTestFn test_fn_;
+};
+    
+}
+}
+
+#endif

modules/xfeatures2d/include/opencv2/xfeatures2d/cuda.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+//
+//  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+//
+//  By downloading, copying, installing or using the software you agree to this license.
+//  If you do not agree to this license, do not download, install,
+//  copy or use the software.
+//
+//
+//                           License Agreement
+//                For Open Source Computer Vision Library
+//
+// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
+// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
+// Third party copyrights are property of their respective owners.
+//
+// Redistribution and use in source and binary forms, with or without modification,
+// are permitted provided that the following conditions are met:
+//
+//   * Redistribution's of source code must retain the above copyright notice,
+//     this list of conditions and the following disclaimer.
+//
+//   * Redistribution's in binary form must reproduce the above copyright notice,
+//     this list of conditions and the following disclaimer in the documentation
+//     and/or other materials provided with the distribution.
+//
+//   * The name of the copyright holders may not be used to endorse or promote products
+//     derived from this software without specific prior written permission.
+//
+// This software is provided by the copyright holders and contributors "as is" and
+// any express or implied warranties, including, but not limited to, the implied
+// warranties of merchantability and fitness for a particular purpose are disclaimed.
+// In no event shall the Intel Corporation or contributors be liable for any direct,
+// indirect, incidental, special, exemplary, or consequential damages
+// (including, but not limited to, procurement of substitute goods or services;
+// loss of use, data, or profits; or business interruption) however caused
+// and on any theory of liability, whether in contract, strict liability,
+// or tort (including negligence or otherwise) arising in any way out of
+// the use of this software, even if advised of the possibility of such damage.
+//
+//M*/
+
+#ifndef __OPENCV_NONFREE_CUDA_HPP__
+#define __OPENCV_NONFREE_CUDA_HPP__
+
+#include "opencv2/core/cuda.hpp"
+
+namespace cv { namespace cuda {
+
+class CV_EXPORTS SURF_CUDA
+{
+public:
+    enum KeypointLayout
+    {
+        X_ROW = 0,
+        Y_ROW,
+        LAPLACIAN_ROW,
+        OCTAVE_ROW,
+        SIZE_ROW,
+        ANGLE_ROW,
+        HESSIAN_ROW,
+        ROWS_COUNT
+    };
+
+    //! the default constructor
+    SURF_CUDA();
+    //! the full constructor taking all the necessary parameters
+    explicit SURF_CUDA(double _hessianThreshold, int _nOctaves=4,
+         int _nOctaveLayers=2, bool _extended=false, float _keypointsRatio=0.01f, bool _upright = false);
+
+    //! returns the descriptor size in float's (64 or 128)
+    int descriptorSize() const;
+    //! returns the default norm type
+    int defaultNorm() const;
+
+    //! upload host keypoints to device memory
+    void uploadKeypoints(const std::vector<KeyPoint>& keypoints, GpuMat& keypointsGPU);
+    //! download keypoints from device to host memory
+    void downloadKeypoints(const GpuMat& keypointsGPU, std::vector<KeyPoint>& keypoints);
+
+    //! download descriptors from device to host memory
+    void downloadDescriptors(const GpuMat& descriptorsGPU, std::vector<float>& descriptors);
+
+    //! finds the keypoints using fast hessian detector used in SURF
+    //! supports CV_8UC1 images
+    //! keypoints will have nFeature cols and 6 rows
+    //! keypoints.ptr<float>(X_ROW)[i] will contain x coordinate of i'th feature
+    //! keypoints.ptr<float>(Y_ROW)[i] will contain y coordinate of i'th feature
+    //! keypoints.ptr<float>(LAPLACIAN_ROW)[i] will contain laplacian sign of i'th feature
+    //! keypoints.ptr<float>(OCTAVE_ROW)[i] will contain octave of i'th feature
+    //! keypoints.ptr<float>(SIZE_ROW)[i] will contain size of i'th feature
+    //! keypoints.ptr<float>(ANGLE_ROW)[i] will contain orientation of i'th feature
+    //! keypoints.ptr<float>(HESSIAN_ROW)[i] will contain response of i'th feature
+    void operator()(const GpuMat& img, const GpuMat& mask, GpuMat& keypoints);
+    //! finds the keypoints and computes their descriptors.
+    //! Optionally it can compute descriptors for the user-provided keypoints and recompute keypoints direction
+    void operator()(const GpuMat& img, const GpuMat& mask, GpuMat& keypoints, GpuMat& descriptors,
+        bool useProvidedKeypoints = false);
+
+    void operator()(const GpuMat& img, const GpuMat& mask, std::vector<KeyPoint>& keypoints);
+    void operator()(const GpuMat& img, const GpuMat& mask, std::vector<KeyPoint>& keypoints, GpuMat& descriptors,
+        bool useProvidedKeypoints = false);
+
+    void operator()(const GpuMat& img, const GpuMat& mask, std::vector<KeyPoint>& keypoints, std::vector<float>& descriptors,
+        bool useProvidedKeypoints = false);
+
+    void releaseMemory();
+
+    // SURF parameters
+    double hessianThreshold;
+    int nOctaves;
+    int nOctaveLayers;
+    bool extended;
+    bool upright;
+
+    //! max keypoints = min(keypointsRatio * img.size().area(), 65535)
+    float keypointsRatio;
+
+    GpuMat sum, mask1, maskSum, intBuffer;
+
+    GpuMat det, trace;
+
+    GpuMat maxPosBuffer;
+};
+
+}} // namespace cv { namespace cuda {
+
+#endif // __OPENCV_NONFREE_CUDA_HPP__

modules/xfeatures2d/include/opencv2/xfeatures2d/nonfree.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+//
+//  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+//
+//  By downloading, copying, installing or using the software you agree to this license.
+//  If you do not agree to this license, do not download, install,
+//  copy or use the software.
+//
+//
+//                           License Agreement
+//                For Open Source Computer Vision Library
+//
+// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
+// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
+// Third party copyrights are property of their respective owners.
+//
+// Redistribution and use in source and binary forms, with or without modification,
+// are permitted provided that the following conditions are met:
+//
+//   * Redistribution's of source code must retain the above copyright notice,
+//     this list of conditions and the following disclaimer.
+//
+//   * Redistribution's in binary form must reproduce the above copyright notice,
+//     this list of conditions and the following disclaimer in the documentation
+//     and/or other materials provided with the distribution.
+//
+//   * The name of the copyright holders may not be used to endorse or promote products
+//     derived from this software without specific prior written permission.
+//
+// This software is provided by the copyright holders and contributors "as is" and
+// any express or implied warranties, including, but not limited to, the implied
+// warranties of merchantability and fitness for a particular purpose are disclaimed.
+// In no event shall the Intel Corporation or contributors be liable for any direct,
+// indirect, incidental, special, exemplary, or consequential damages
+// (including, but not limited to, procurement of substitute goods or services;
+// loss of use, data, or profits; or business interruption) however caused
+// and on any theory of liability, whether in contract, strict liability,
+// or tort (including negligence or otherwise) arising in any way out of
+// the use of this software, even if advised of the possibility of such damage.
+//
+//M*/
+
+#ifndef __OPENCV_NONFREE_FEATURES_2D_HPP__
+#define __OPENCV_NONFREE_FEATURES_2D_HPP__
+
+#include "opencv2/features2d.hpp"
+
+namespace cv
+{
+namespace xfeatures2d
+{
+
+/*!
+ SIFT implementation.
+
+ The class implements SIFT algorithm by D. Lowe.
+*/
+class CV_EXPORTS_W SIFT : public Feature2D
+{
+public:
+    CV_WRAP explicit SIFT( int nfeatures = 0, int nOctaveLayers = 3,
+          double contrastThreshold = 0.04, double edgeThreshold = 10,
+          double sigma = 1.6);
+
+    //! returns the descriptor size in floats (128)
+    CV_WRAP int descriptorSize() const;
+
+    //! returns the descriptor type
+    CV_WRAP int descriptorType() const;
+
+    //! returns the default norm type
+    CV_WRAP int defaultNorm() const;
+
+    //! finds the keypoints using SIFT algorithm
+    void operator()(InputArray img, InputArray mask,
+                    std::vector<KeyPoint>& keypoints) const;
+    //! finds the keypoints and computes descriptors for them using SIFT algorithm.
+    //! Optionally it can compute descriptors for the user-provided keypoints
+    void operator()(InputArray img, InputArray mask,
+                    std::vector<KeyPoint>& keypoints,
+                    OutputArray descriptors,
+                    bool useProvidedKeypoints = false) const;
+
+    AlgorithmInfo* info() const;
+
+    void buildGaussianPyramid( const Mat& base, std::vector<Mat>& pyr, int nOctaves ) const;
+    void buildDoGPyramid( const std::vector<Mat>& pyr, std::vector<Mat>& dogpyr ) const;
+    void findScaleSpaceExtrema( const std::vector<Mat>& gauss_pyr, const std::vector<Mat>& dog_pyr,
+                                std::vector<KeyPoint>& keypoints ) const;
+
+protected:
+    void detectImpl( InputArray image, std::vector<KeyPoint>& keypoints, InputArray mask = noArray() ) const;
+    void computeImpl( InputArray image, std::vector<KeyPoint>& keypoints, OutputArray descriptors ) const;
+
+    CV_PROP_RW int nfeatures;
+    CV_PROP_RW int nOctaveLayers;
+    CV_PROP_RW double contrastThreshold;
+    CV_PROP_RW double edgeThreshold;
+    CV_PROP_RW double sigma;
+};
+
+typedef SIFT SiftFeatureDetector;
+typedef SIFT SiftDescriptorExtractor;
+
+/*!
+ SURF implementation.
+
+ The class implements SURF algorithm by H. Bay et al.
+ */
+class CV_EXPORTS_W SURF : public Feature2D
+{
+public:
+    //! the default constructor
+    CV_WRAP SURF();
+    //! the full constructor taking all the necessary parameters
+    explicit CV_WRAP SURF(double hessianThreshold,
+                  int nOctaves = 4, int nOctaveLayers = 2,
+                  bool extended = true, bool upright = false);
+
+    //! returns the descriptor size in float's (64 or 128)
+    CV_WRAP int descriptorSize() const;
+
+    //! returns the descriptor type
+    CV_WRAP int descriptorType() const;
+
+    //! returns the descriptor type
+    CV_WRAP int defaultNorm() const;
+
+    //! finds the keypoints using fast hessian detector used in SURF
+    void operator()(InputArray img, InputArray mask,
+                    CV_OUT std::vector<KeyPoint>& keypoints) const;
+    //! finds the keypoints and computes their descriptors. Optionally it can compute descriptors for the user-provided keypoints
+    void operator()(InputArray img, InputArray mask,
+                    CV_OUT std::vector<KeyPoint>& keypoints,
+                    OutputArray descriptors,
+                    bool useProvidedKeypoints = false) const;
+
+    AlgorithmInfo* info() const;
+
+    CV_PROP_RW double hessianThreshold;
+    CV_PROP_RW int nOctaves;
+    CV_PROP_RW int nOctaveLayers;
+    CV_PROP_RW bool extended;
+    CV_PROP_RW bool upright;
+
+protected:
+    void detectImpl( InputArray image, std::vector<KeyPoint>& keypoints, InputArray mask = noArray() ) const;
+    void computeImpl( InputArray image, std::vector<KeyPoint>& keypoints, OutputArray descriptors ) const;
+};
+
+typedef SURF SurfFeatureDetector;
+typedef SURF SurfDescriptorExtractor;
+
+}
+} /* namespace cv */
+
+#endif

modules/xfeatures2d/perf/perf_main.cpp

+#include "perf_precomp.hpp"
+#include "opencv2/ts/cuda_perf.hpp"
+
+static const char * impls[] = {
+#ifdef HAVE_CUDA
+    "cuda",
+#endif
+    "plain"
+};
+
+CV_PERF_TEST_MAIN_WITH_IMPLS(nonfree, impls, perf::printCudaInfo())

modules/xfeatures2d/perf/perf_precomp.hpp

+#ifdef __GNUC__
+#  pragma GCC diagnostic ignored "-Wmissing-declarations"
+#  if defined __clang__ || defined __APPLE__
+#    pragma GCC diagnostic ignored "-Wmissing-prototypes"
+#    pragma GCC diagnostic ignored "-Wextra"
+#  endif
+#endif
+
+#ifndef __OPENCV_PERF_PRECOMP_HPP__
+#define __OPENCV_PERF_PRECOMP_HPP__
+
+#include "cvconfig.h"
+
+#include "opencv2/ts.hpp"
+#include "opencv2/xfeatures2d.hpp"
+#include "opencv2/imgcodecs.hpp"
+
+#include "opencv2/opencv_modules.hpp"
+
+#ifdef HAVE_OPENCV_OCL
+#  include "opencv2/nonfree/ocl.hpp"
+#endif
+
+#ifdef HAVE_CUDA
+#  include "opencv2/nonfree/cuda.hpp"
+#endif
+
+#ifdef GTEST_CREATE_SHARED_LIBRARY
+#error no modules except ts should have GTEST_CREATE_SHARED_LIBRARY defined
+#endif
+
+#endif

modules/xfeatures2d/perf/perf_surf.cpp

+#include "perf_precomp.hpp"
+
+using namespace std;
+using namespace cv;
+using namespace cv::xfeatures2d;
+using namespace perf;
+using std::tr1::make_tuple;
+using std::tr1::get;
+
+typedef perf::TestBaseWithParam<std::string> surf;
+
+#define SURF_IMAGES \
+    "cv/detectors_descriptors_evaluation/images_datasets/leuven/img1.png",\
+    "stitching/a3.png"
+
+PERF_TEST_P(surf, detect, testing::Values(SURF_IMAGES))
+{
+    string filename = getDataPath(GetParam());
+    Mat frame = imread(filename, IMREAD_GRAYSCALE);
+    ASSERT_FALSE(frame.empty()) << "Unable to load source image " << filename;
+
+    Mat mask;
+    declare.in(frame).time(90);
+    SURF detector;
+    vector<KeyPoint> points;
+
+    TEST_CYCLE() detector(frame, mask, points);
+
+    SANITY_CHECK_KEYPOINTS(points, 1e-3);
+}
+
+PERF_TEST_P(surf, extract, testing::Values(SURF_IMAGES))
+{
+    string filename = getDataPath(GetParam());
+    Mat frame = imread(filename, IMREAD_GRAYSCALE);
+    ASSERT_FALSE(frame.empty()) << "Unable to load source image " << filename;
+
+    Mat mask;
+    declare.in(frame).time(90);
+
+    SURF detector;
+    vector<KeyPoint> points;
+    vector<float> descriptors;
+    detector(frame, mask, points);
+
+    TEST_CYCLE() detector(frame, mask, points, descriptors, true);
+
+    SANITY_CHECK(descriptors, 1e-4);
+}
+
+PERF_TEST_P(surf, full, testing::Values(SURF_IMAGES))
+{
+    string filename = getDataPath(GetParam());
+    Mat frame = imread(filename, IMREAD_GRAYSCALE);
+    ASSERT_FALSE(frame.empty()) << "Unable to load source image " << filename;
+
+    Mat mask;
+    declare.in(frame).time(90);
+    SURF detector;
+    vector<KeyPoint> points;
+    vector<float> descriptors;
+
+    TEST_CYCLE() detector(frame, mask, points, descriptors, false);
+
+    SANITY_CHECK_KEYPOINTS(points, 1e-3);
+    SANITY_CHECK(descriptors, 1e-4);
+}

modules/xfeatures2d/perf/perf_surf.cuda.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+//
+//  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+//
+//  By downloading, copying, installing or using the software you agree to this license.
+//  If you do not agree to this license, do not download, install,
+//  copy or use the software.
+//
+//
+//                           License Agreement
+//                For Open Source Computer Vision Library
+//
+// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
+// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
+// Third party copyrights are property of their respective owners.
+//
+// Redistribution and use in source and binary forms, with or without modification,
+// are permitted provided that the following conditions are met:
+//
+//   * Redistribution's of source code must retain the above copyright notice,
+//     this list of conditions and the following disclaimer.
+//
+//   * Redistribution's in binary form must reproduce the above copyright notice,
+//     this list of conditions and the following disclaimer in the documentation
+//     and/or other materials provided with the distribution.
+//
+//   * The name of the copyright holders may not be used to endorse or promote products
+//     derived from this software without specific prior written permission.
+//
+// This software is provided by the copyright holders and contributors "as is" and
+// any express or implied warranties, including, but not limited to, the implied
+// warranties of merchantability and fitness for a particular purpose are disclaimed.
+// In no event shall the Intel Corporation or contributors be liable for any direct,
+// indirect, incidental, special, exemplary, or consequential damages
+// (including, but not limited to, procurement of substitute goods or services;
+// loss of use, data, or profits; or business interruption) however caused
+// and on any theory of liability, whether in contract, strict liability,
+// or tort (including negligence or otherwise) arising in any way out of
+// the use of this software, even if advised of the possibility of such damage.
+//
+//M*/
+
+#include "perf_precomp.hpp"
+
+#ifdef HAVE_CUDA
+
+#include "opencv2/ts/cuda_perf.hpp"
+
+using namespace std;
+using namespace testing;
+using namespace perf;
+
+//////////////////////////////////////////////////////////////////////
+// SURF
+
+#ifdef HAVE_OPENCV_CUDAARITHM
+
+DEF_PARAM_TEST_1(Image, string);
+
+PERF_TEST_P(Image, CUDA_SURF,
+            Values<std::string>("gpu/perf/aloe.png"))
+{
+    declare.time(50.0);
+
+    const cv::Mat img = readImage(GetParam(), cv::IMREAD_GRAYSCALE);
+    ASSERT_FALSE(img.empty());
+
+    if (PERF_RUN_CUDA())
+    {
+        cv::cuda::SURF_CUDA d_surf;
+
+        const cv::cuda::GpuMat d_img(img);
+        cv::cuda::GpuMat d_keypoints, d_descriptors;
+
+        TEST_CYCLE() d_surf(d_img, cv::cuda::GpuMat(), d_keypoints, d_descriptors);
+
+        std::vector<cv::KeyPoint> gpu_keypoints;
+        d_surf.downloadKeypoints(d_keypoints, gpu_keypoints);
+
+        cv::Mat gpu_descriptors(d_descriptors);
+
+        sortKeyPoints(gpu_keypoints, gpu_descriptors);
+
+        SANITY_CHECK_KEYPOINTS(gpu_keypoints);
+        SANITY_CHECK(gpu_descriptors, 1e-3);
+    }
+    else
+    {
+        cv::SURF surf;
+
+        std::vector<cv::KeyPoint> cpu_keypoints;
+        cv::Mat cpu_descriptors;
+
+        TEST_CYCLE() surf(img, cv::noArray(), cpu_keypoints, cpu_descriptors);
+
+        SANITY_CHECK_KEYPOINTS(cpu_keypoints);
+        SANITY_CHECK(cpu_descriptors);
+    }
+}
+
+#endif // HAVE_OPENCV_CUDAARITHM
+
+#endif // HAVE_CUDA

modules/xfeatures2d/perf/perf_surf.ocl.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+//
+//  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+//
+//  By downloading, copying, installing or using the software you agree to this license.
+//  If you do not agree to this license, do not download, install,
+//  copy or use the software.
+//
+//
+//                           License Agreement
+//                For Open Source Computer Vision Library
+//
+// Copyright (C) 2010-2012, Multicoreware, Inc., all rights reserved.
+// Copyright (C) 2010-2012, Advanced Micro Devices, Inc., all rights reserved.
+// Third party copyrights are property of their respective owners.
+//
+// @Authors
+//    Peng Xiao, pengxiao@multicorewareinc.com
+//
+// Redistribution and use in source and binary forms, with or without modification,
+// are permitted provided that the following conditions are met:
+//
+//   * Redistribution's of source code must retain the above copyright notice,
+//     this list of conditions and the following disclaimer.
+//
+//   * Redistribution's in binary form must reproduce the above copyright notice,
+//     this list of conditions and the following disclaimer in the documentation
+//     and/or other materials provided with the distribution.
+//
+//   * The name of the copyright holders may not be used to endorse or promote products
+//     derived from this software without specific prior written permission.
+//
+// This software is provided by the copyright holders and contributors as is and
+// any express or implied warranties, including, but not limited to, the implied
+// warranties of merchantability and fitness for a particular purpose are disclaimed.
+// In no event shall the Intel Corporation or contributors be liable for any direct,
+// indirect, incidental, special, exemplary, or consequential damages
+// (including, but not limited to, procurement of substitute goods or services;
+// loss of use, data, or profits; or business interruption) however caused
+// and on any theory of liability, whether in contract, strict liability,
+// or tort (including negligence or otherwise) arising in any way out of
+// the use of this software, even if advised of the possibility of such damage.
+//
+//M*/
+
+#include "perf_precomp.hpp"
+
+#ifdef HAVE_OPENCV_OCL
+
+using namespace cv;
+using namespace cv::ocl;
+using namespace std;
+
+typedef perf::TestBaseWithParam<std::string> OCL_SURF;
+
+#define SURF_IMAGES \
+    "cv/detectors_descriptors_evaluation/images_datasets/leuven/img1.png",\
+    "stitching/a3.png"
+
+PERF_TEST_P(OCL_SURF, DISABLED_with_data_transfer, testing::Values(SURF_IMAGES))
+{
+    string filename = getDataPath(GetParam());
+    Mat img = imread(filename, IMREAD_GRAYSCALE);
+    ASSERT_FALSE(img.empty());
+
+    SURF_OCL d_surf;
+    oclMat d_keypoints;
+    oclMat d_descriptors;
+    Mat cpu_kp;
+    Mat cpu_dp;
+
+    declare.time(60);
+
+    TEST_CYCLE()
+    {
+        oclMat d_src(img);
+
+        d_surf(d_src, oclMat(), d_keypoints, d_descriptors);
+
+        d_keypoints.download(cpu_kp);
+        d_descriptors.download(cpu_dp);
+    }
+
+    SANITY_CHECK(cpu_kp, 1);
+    SANITY_CHECK(cpu_dp, 1);
+}
+
+PERF_TEST_P(OCL_SURF, DISABLED_without_data_transfer, testing::Values(SURF_IMAGES))
+{
+    string filename = getDataPath(GetParam());
+    Mat img = imread(filename, IMREAD_GRAYSCALE);
+    ASSERT_FALSE(img.empty());
+
+    SURF_OCL d_surf;
+    oclMat d_keypoints;
+    oclMat d_descriptors;
+    oclMat d_src(img);
+
+    declare.time(60);
+
+    TEST_CYCLE() d_surf(d_src, oclMat(), d_keypoints, d_descriptors);
+
+    Mat cpu_kp;
+    Mat cpu_dp;
+    d_keypoints.download(cpu_kp);
+    d_descriptors.download(cpu_dp);
+    SANITY_CHECK(cpu_kp, 1);
+    SANITY_CHECK(cpu_dp, 1);
+}
+
+#endif // HAVE_OPENCV_OCL

modules/xfeatures2d/samples/shape_transformation.cpp

+/*
+ * shape_context.cpp -- Shape context demo for shape matching
+ */
+
+#include "opencv2/shape.hpp"
+#include "opencv2/imgcodecs.hpp"
+#include "opencv2/highgui.hpp"
+#include "opencv2/imgproc.hpp"
+#include "opencv2/features2d.hpp"
+#include "opencv2/xfeatures2d.hpp"
+#include <opencv2/core/utility.hpp>
+#include <iostream>
+#include <string>
+
+using namespace std;
+using namespace cv;
+using namespace cv::xfeatures2d;
+
+static void help()
+{
+    printf("\nThis program demonstrates how to use common interface for shape transformers\n"
+           "Call\n"
+           "shape_transformation [image1] [image2]\n");
+}
+
+int main(int argc, char** argv)
+{
+    help();
+    Mat img1 = imread(argv[1], IMREAD_GRAYSCALE);
+    Mat img2 = imread(argv[2], IMREAD_GRAYSCALE);
+    if(img1.empty() || img2.empty() || argc<2)
+    {
+        printf("Can't read one of the images\n");
+        return -1;
+    }
+
+    // detecting keypoints
+    SurfFeatureDetector detector(5000);
+    vector<KeyPoint> keypoints1, keypoints2;
+    detector.detect(img1, keypoints1);
+    detector.detect(img2, keypoints2);
+
+    // computing descriptors
+    SurfDescriptorExtractor extractor;
+    Mat descriptors1, descriptors2;
+    extractor.compute(img1, keypoints1, descriptors1);
+    extractor.compute(img2, keypoints2, descriptors2);
+
+    // matching descriptors
+    BFMatcher matcher(extractor.defaultNorm());
+    vector<DMatch> matches;
+    matcher.match(descriptors1, descriptors2, matches);
+
+    // drawing the results
+    namedWindow("matches", 1);
+    Mat img_matches;
+    drawMatches(img1, keypoints1, img2, keypoints2, matches, img_matches);
+    imshow("matches", img_matches);
+
+    // extract points
+    vector<Point2f> pts1, pts2;
+    for (size_t ii=0; ii<keypoints1.size(); ii++)
+        pts1.push_back( keypoints1[ii].pt );
+    for (size_t ii=0; ii<keypoints2.size(); ii++)
+        pts2.push_back( keypoints2[ii].pt );
+
+    // Apply TPS
+    Ptr<ThinPlateSplineShapeTransformer> mytps = createThinPlateSplineShapeTransformer(25000); //TPS with a relaxed constraint
+    mytps->estimateTransformation(pts1, pts2, matches);
+    mytps->warpImage(img2, img2);
+
+    imshow("Tranformed", img2);
+    waitKey(0);
+
+    return 0;
+}

modules/xfeatures2d/samples/surf_matcher.cpp

+#include <iostream>
+#include <stdio.h>
+#include "opencv2/core.hpp"
+#include "opencv2/core/utility.hpp"
+#include "opencv2/core/ocl.hpp"
+#include "opencv2/imgcodecs.hpp"
+#include "opencv2/highgui.hpp"
+#include "opencv2/features2d.hpp"
+#include "opencv2/calib3d.hpp"
+#include "opencv2/imgproc.hpp"
+#include "opencv2/xfeatures2d.hpp"
+
+using namespace cv;
+using namespace cv::xfeatures2d;
+
+const int LOOP_NUM = 10;
+const int GOOD_PTS_MAX = 50;
+const float GOOD_PORTION = 0.15f;
+
+int64 work_begin = 0;
+int64 work_end = 0;
+
+static void workBegin()
+{
+    work_begin = getTickCount();
+}
+
+static void workEnd()
+{
+    work_end = getTickCount() - work_begin;
+}
+
+static double getTime()
+{
+    return work_end /((double)getTickFrequency() )* 1000.;
+}
+
+template<class KPDetector>
+struct SURFDetector
+{
+    KPDetector surf;
+    SURFDetector(double hessian = 800.0)
+        :surf(hessian)
+    {
+    }
+    template<class T>
+    void operator()(const T& in, const T& mask, std::vector<cv::KeyPoint>& pts, T& descriptors, bool useProvided = false)
+    {
+        surf(in, mask, pts, descriptors, useProvided);
+    }
+};
+
+template<class KPMatcher>
+struct SURFMatcher
+{
+    KPMatcher matcher;
+    template<class T>
+    void match(const T& in1, const T& in2, std::vector<cv::DMatch>& matches)
+    {
+        matcher.match(in1, in2, matches);
+    }
+};
+
+static Mat drawGoodMatches(
+    const Mat& img1,
+    const Mat& img2,
+    const std::vector<KeyPoint>& keypoints1,
+    const std::vector<KeyPoint>& keypoints2,
+    std::vector<DMatch>& matches,
+    std::vector<Point2f>& scene_corners_
+    )
+{
+    //-- Sort matches and preserve top 10% matches
+    std::sort(matches.begin(), matches.end());
+    std::vector< DMatch > good_matches;
+    double minDist = matches.front().distance;
+    double maxDist = matches.back().distance;
+
+    const int ptsPairs = std::min(GOOD_PTS_MAX, (int)(matches.size() * GOOD_PORTION));
+    for( int i = 0; i < ptsPairs; i++ )
+    {
+        good_matches.push_back( matches[i] );
+    }
+    std::cout << "\nMax distance: " << maxDist << std::endl;
+    std::cout << "Min distance: " << minDist << std::endl;
+
+    std::cout << "Calculating homography using " << ptsPairs << " point pairs." << std::endl;
+
+    // drawing the results
+    Mat img_matches;
+
+    drawMatches( img1, keypoints1, img2, keypoints2,
+                 good_matches, img_matches, Scalar::all(-1), Scalar::all(-1),
+                 std::vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS  );
+
+    //-- Localize the object
+    std::vector<Point2f> obj;
+    std::vector<Point2f> scene;
+
+    for( size_t i = 0; i < good_matches.size(); i++ )
+    {
+        //-- Get the keypoints from the good matches
+        obj.push_back( keypoints1[ good_matches[i].queryIdx ].pt );
+        scene.push_back( keypoints2[ good_matches[i].trainIdx ].pt );
+    }
+    //-- Get the corners from the image_1 ( the object to be "detected" )
+    std::vector<Point2f> obj_corners(4);
+    obj_corners[0] = Point(0,0);
+    obj_corners[1] = Point( img1.cols, 0 );
+    obj_corners[2] = Point( img1.cols, img1.rows );
+    obj_corners[3] = Point( 0, img1.rows );
+    std::vector<Point2f> scene_corners(4);
+
+    Mat H = findHomography( obj, scene, RANSAC );
+    perspectiveTransform( obj_corners, scene_corners, H);
+
+    scene_corners_ = scene_corners;
+
+    //-- Draw lines between the corners (the mapped object in the scene - image_2 )
+    line( img_matches,
+          scene_corners[0] + Point2f( (float)img1.cols, 0), scene_corners[1] + Point2f( (float)img1.cols, 0),
+          Scalar( 0, 255, 0), 2, LINE_AA );
+    line( img_matches,
+          scene_corners[1] + Point2f( (float)img1.cols, 0), scene_corners[2] + Point2f( (float)img1.cols, 0),
+          Scalar( 0, 255, 0), 2, LINE_AA );
+    line( img_matches,
+          scene_corners[2] + Point2f( (float)img1.cols, 0), scene_corners[3] + Point2f( (float)img1.cols, 0),
+          Scalar( 0, 255, 0), 2, LINE_AA );
+    line( img_matches,
+          scene_corners[3] + Point2f( (float)img1.cols, 0), scene_corners[0] + Point2f( (float)img1.cols, 0),
+          Scalar( 0, 255, 0), 2, LINE_AA );
+    return img_matches;
+}
+
+////////////////////////////////////////////////////
+// This program demonstrates the usage of SURF_OCL.
+// use cpu findHomography interface to calculate the transformation matrix
+int main(int argc, char* argv[])
+{
+    const char* keys =
+        "{ h help     | false            | print help message  }"
+        "{ l left     | box.png          | specify left image  }"
+        "{ r right    | box_in_scene.png | specify right image }"
+        "{ o output   | SURF_output.jpg  | specify output save path }"
+        "{ m cpu_mode | false            | run without OpenCL }";
+
+    CommandLineParser cmd(argc, argv, keys);
+    if (cmd.has("help"))
+    {
+        std::cout << "Usage: surf_matcher [options]" << std::endl;
+        std::cout << "Available options:" << std::endl;
+        cmd.printMessage();
+        return EXIT_SUCCESS;
+    }
+    if (cmd.has("cpu_mode"))
+    {
+        ocl::setUseOpenCL(false);
+        std::cout << "OpenCL was disabled" << std::endl;
+    }
+
+    UMat img1, img2;
+
+    std::string outpath = cmd.get<std::string>("o");
+
+    std::string leftName = cmd.get<std::string>("l");
+    imread(leftName, IMREAD_GRAYSCALE).copyTo(img1);
+    if(img1.empty())
+    {
+        std::cout << "Couldn't load " << leftName << std::endl;
+        cmd.printMessage();
+        return EXIT_FAILURE;
+    }
+
+    std::string rightName = cmd.get<std::string>("r");
+    imread(rightName, IMREAD_GRAYSCALE).copyTo(img2);
+    if(img2.empty())
+    {
+        std::cout << "Couldn't load " << rightName << std::endl;
+        cmd.printMessage();
+        return EXIT_FAILURE;
+    }
+
+    double surf_time = 0.;
+
+    //declare input/output
+    std::vector<KeyPoint> keypoints1, keypoints2;
+    std::vector<DMatch> matches;
+
+    UMat _descriptors1, _descriptors2;
+    Mat descriptors1 = _descriptors1.getMat(ACCESS_RW),
+        descriptors2 = _descriptors2.getMat(ACCESS_RW);
+
+    //instantiate detectors/matchers
+    SURFDetector<SURF> surf;
+
+    SURFMatcher<BFMatcher> matcher;
+
+    //-- start of timing section
+
+    for (int i = 0; i <= LOOP_NUM; i++)
+    {
+        if(i == 1) workBegin();
+        surf(img1.getMat(ACCESS_READ), Mat(), keypoints1, descriptors1);
+        surf(img2.getMat(ACCESS_READ), Mat(), keypoints2, descriptors2);
+        matcher.match(descriptors1, descriptors2, matches);
+    }
+    workEnd();
+    std::cout << "FOUND " << keypoints1.size() << " keypoints on first image" << std::endl;
+    std::cout << "FOUND " << keypoints2.size() << " keypoints on second image" << std::endl;
+
+    surf_time = getTime();
+    std::cout << "SURF run time: " << surf_time / LOOP_NUM << " ms" << std::endl<<"\n";
+
+
+    std::vector<Point2f> corner;
+    Mat img_matches = drawGoodMatches(img1.getMat(ACCESS_READ), img2.getMat(ACCESS_READ), keypoints1, keypoints2, matches, corner);
+
+    //-- Show detected matches
+
+    namedWindow("surf matches", 0);
+    imshow("surf matches", img_matches);
+    imwrite(outpath, img_matches);
+
+    waitKey(0);
+    return EXIT_SUCCESS;
+}

modules/xfeatures2d/samples/video_homography.cpp

+/*
+* video_homography.cpp
+*
+*  Created on: Oct 18, 2010
+*      Author: erublee
+*/
+
+#include "opencv2/calib3d.hpp"
+#include "opencv2/videoio.hpp"
+#include "opencv2/highgui.hpp"
+#include "opencv2/imgproc.hpp"
+#include "opencv2/features2d.hpp"
+#include "opencv2/xfeatures2d.hpp"
+#include <iostream>
+#include <list>
+#include <vector>
+
+using namespace std;
+using namespace cv;
+using namespace cv::xfeatures2d;
+
+static void help(char **av)
+{
+    cout << "\nThis program demonstrated the use of features2d with the Fast corner detector and brief descriptors\n"
+        << "to track planar objects by computing their homography from the key (training) image to the query (test) image\n\n" << endl;
+    cout << "usage: " << av[0] << " <video device number>\n" << endl;
+    cout << "The following keys do stuff:" << endl;
+    cout << "  t : grabs a reference frame to match against" << endl;
+    cout << "  l : makes the reference frame new every frame" << endl;
+    cout << "  q or escape: quit" << endl;
+}
+
+namespace
+{
+    void drawMatchesRelative(const vector<KeyPoint>& train, const vector<KeyPoint>& query,
+        std::vector<cv::DMatch>& matches, Mat& img, const vector<unsigned char>& mask = vector<
+        unsigned char> ())
+    {
+        for (int i = 0; i < (int)matches.size(); i++)
+        {
+            if (mask.empty() || mask[i])
+            {
+                Point2f pt_new = query[matches[i].queryIdx].pt;
+                Point2f pt_old = train[matches[i].trainIdx].pt;
+
+                cv::line(img, pt_new, pt_old, Scalar(125, 255, 125), 1);
+                cv::circle(img, pt_new, 2, Scalar(255, 0, 125), 1);
+
+            }
+        }
+    }
+
+    //Takes a descriptor and turns it into an xy point
+    void keypoints2points(const vector<KeyPoint>& in, vector<Point2f>& out)
+    {
+        out.clear();
+        out.reserve(in.size());
+        for (size_t i = 0; i < in.size(); ++i)
+        {
+            out.push_back(in[i].pt);
+        }
+    }
+
+    //Takes an xy point and appends that to a keypoint structure
+    void points2keypoints(const vector<Point2f>& in, vector<KeyPoint>& out)
+    {
+        out.clear();
+        out.reserve(in.size());
+        for (size_t i = 0; i < in.size(); ++i)
+        {
+            out.push_back(KeyPoint(in[i], 1));
+        }
+    }
+
+    //Uses computed homography H to warp original input points to new planar position
+    void warpKeypoints(const Mat& H, const vector<KeyPoint>& in, vector<KeyPoint>& out)
+    {
+        vector<Point2f> pts;
+        keypoints2points(in, pts);
+        vector<Point2f> pts_w(pts.size());
+        Mat m_pts_w(pts_w);
+        perspectiveTransform(Mat(pts), m_pts_w, H);
+        points2keypoints(pts_w, out);
+    }
+
+    //Converts matching indices to xy points
+    void matches2points(const vector<KeyPoint>& train, const vector<KeyPoint>& query,
+        const std::vector<cv::DMatch>& matches, std::vector<cv::Point2f>& pts_train,
+        std::vector<Point2f>& pts_query)
+    {
+
+        pts_train.clear();
+        pts_query.clear();
+        pts_train.reserve(matches.size());
+        pts_query.reserve(matches.size());
+
+        size_t i = 0;
+
+        for (; i < matches.size(); i++)
+        {
+
+            const DMatch & dmatch = matches[i];
+
+            pts_query.push_back(query[dmatch.queryIdx].pt);
+            pts_train.push_back(train[dmatch.trainIdx].pt);
+
+        }
+
+    }
+
+    void resetH(Mat&H)
+    {
+        H = Mat::eye(3, 3, CV_32FC1);
+    }
+}
+
+int main(int ac, char ** av)
+{
+
+    if (ac != 2)
+    {
+        help(av);
+        return 1;
+    }
+
+    BriefDescriptorExtractor brief(32);
+
+    VideoCapture capture;
+    capture.open(atoi(av[1]));
+    if (!capture.isOpened())
+    {
+        help(av);
+        cout << "capture device " << atoi(av[1]) << " failed to open!" << endl;
+        return 1;
+    }
+
+    cout << "following keys do stuff:" << endl;
+    cout << "t : grabs a reference frame to match against" << endl;
+    cout << "l : makes the reference frame new every frame" << endl;
+    cout << "q or escape: quit" << endl;
+
+    Mat frame;
+
+    vector<DMatch> matches;
+
+    BFMatcher desc_matcher(brief.defaultNorm());
+
+    vector<Point2f> train_pts, query_pts;
+    vector<KeyPoint> train_kpts, query_kpts;
+    vector<unsigned char> match_mask;
+
+    Mat gray;
+
+    bool ref_live = true;
+
+    Mat train_desc, query_desc;
+    FastFeatureDetector detector(10, true);
+
+    Mat H_prev = Mat::eye(3, 3, CV_32FC1);
+    for (;;)
+    {
+        capture >> frame;
+        if (frame.empty())
+            break;
+
+        cvtColor(frame, gray, COLOR_RGB2GRAY);
+
+        detector.detect(gray, query_kpts); //Find interest points
+
+        brief.compute(gray, query_kpts, query_desc); //Compute brief descriptors at each keypoint location
+
+        if (!train_kpts.empty())
+        {
+
+            vector<KeyPoint> test_kpts;
+            warpKeypoints(H_prev.inv(), query_kpts, test_kpts);
+
+            //Mat mask = windowedMatchingMask(test_kpts, train_kpts, 25, 25);
+            desc_matcher.match(query_desc, train_desc, matches, Mat());
+            drawKeypoints(frame, test_kpts, frame, Scalar(255, 0, 0), DrawMatchesFlags::DRAW_OVER_OUTIMG);
+
+            matches2points(train_kpts, query_kpts, matches, train_pts, query_pts);
+
+            if (matches.size() > 5)
+            {
+                Mat H = findHomography(train_pts, query_pts, RANSAC, 4, match_mask);
+                if (countNonZero(Mat(match_mask)) > 15)
+                {
+                    H_prev = H;
+                }
+                else
+                    resetH(H_prev);
+                drawMatchesRelative(train_kpts, query_kpts, matches, frame, match_mask);
+            }
+            else
+                resetH(H_prev);
+
+        }
+        else
+        {
+            H_prev = Mat::eye(3, 3, CV_32FC1);
+            Mat out;
+            drawKeypoints(gray, query_kpts, out);
+            frame = out;
+        }
+
+        imshow("frame", frame);
+
+        if (ref_live)
+        {
+            train_kpts = query_kpts;
+            query_desc.copyTo(train_desc);
+        }
+        char key = (char)waitKey(2);
+        switch (key)
+        {
+        case 'l':
+            ref_live = true;
+            resetH(H_prev);
+            break;
+        case 't':
+            ref_live = false;
+            train_kpts = query_kpts;
+            query_desc.copyTo(train_desc);
+            resetH(H_prev);
+            break;
+        case 27:
+        case 'q':
+            return 0;
+            break;
+        }
+
+    }
+    return 0;
+}

modules/xfeatures2d/src/brief.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+//
+//  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+//
+//  By downloading, copying, installing or using the software you agree to this license.
+//  If you do not agree to this license, do not download, install,
+//  copy or use the software.
+//
+//
+//                           License Agreement
+//                For Open Source Computer Vision Library
+//
+// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
+// Copyright (C) 2009-2010, Willow Garage Inc., all rights reserved.
+// Third party copyrights are property of their respective owners.
+//
+// Redistribution and use in source and binary forms, with or without modification,
+// are permitted provided that the following conditions are met:
+//
+//   * Redistribution's of source code must retain the above copyright notice,
+//     this list of conditions and the following disclaimer.
+//
+//   * Redistribution's in binary form must reproduce the above copyright notice,
+//     this list of conditions and the following disclaimer in the documentation
+//     and/or other materials provided with the distribution.
+//
+//   * The name of the copyright holders may not be used to endorse or promote products
+//     derived from this software without specific prior written permission.
+//
+// This software is provided by the copyright holders and contributors "as is" and
+// any express or implied warranties, including, but not limited to, the implied
+// warranties of merchantability and fitness for a particular purpose are disclaimed.
+// In no event shall the Intel Corporation or contributors be liable for any direct,
+// indirect, incidental, special, exemplary, or consequential damages
+// (including, but not limited to, procurement of substitute goods or services;
+// loss of use, data, or profits; or business interruption) however caused
+// and on any theory of liability, whether in contract, strict liability,
+// or tort (including negligence or otherwise) arising in any way out of
+// the use of this software, even if advised of the possibility of such damage.
+//
+//M*/
+
+#include "precomp.hpp"
+#include <algorithm>
+#include <vector>
+
+#include <iostream>
+#include <iomanip>
+
+namespace cv
+{
+namespace xfeatures2d
+{
+
+inline int smoothedSum(const Mat& sum, const KeyPoint& pt, int y, int x)
+{
+    static const int HALF_KERNEL = BriefDescriptorExtractor::KERNEL_SIZE / 2;
+
+    int img_y = (int)(pt.pt.y + 0.5) + y;
+    int img_x = (int)(pt.pt.x + 0.5) + x;
+    return   sum.at<int>(img_y + HALF_KERNEL + 1, img_x + HALF_KERNEL + 1)
+           - sum.at<int>(img_y + HALF_KERNEL + 1, img_x - HALF_KERNEL)
+           - sum.at<int>(img_y - HALF_KERNEL, img_x + HALF_KERNEL + 1)
+           + sum.at<int>(img_y - HALF_KERNEL, img_x - HALF_KERNEL);
+}
+
+static void pixelTests16(InputArray _sum, const std::vector<KeyPoint>& keypoints, OutputArray _descriptors)
+{
+    Mat sum = _sum.getMat(), descriptors = _descriptors.getMat();
+    for (int i = 0; i < (int)keypoints.size(); ++i)
+    {
+        uchar* desc = descriptors.ptr(i);
+        const KeyPoint& pt = keypoints[i];
+#include "generated_16.i"
+    }
+}
+
+static void pixelTests32(InputArray _sum, const std::vector<KeyPoint>& keypoints, OutputArray _descriptors)
+{
+    Mat sum = _sum.getMat(), descriptors = _descriptors.getMat();
+    for (int i = 0; i < (int)keypoints.size(); ++i)
+    {
+        uchar* desc = descriptors.ptr(i);
+        const KeyPoint& pt = keypoints[i];
+
+#include "generated_32.i"
+    }
+}
+
+static void pixelTests64(InputArray _sum, const std::vector<KeyPoint>& keypoints, OutputArray _descriptors)
+{
+    Mat sum = _sum.getMat(), descriptors = _descriptors.getMat();
+    for (int i = 0; i < (int)keypoints.size(); ++i)
+    {
+        uchar* desc = descriptors.ptr(i);
+        const KeyPoint& pt = keypoints[i];
+
+#include "generated_64.i"
+    }
+}
+
+BriefDescriptorExtractor::BriefDescriptorExtractor(int bytes) :
+    bytes_(bytes), test_fn_(NULL)
+{
+    switch (bytes)
+    {
+        case 16:
+            test_fn_ = pixelTests16;
+            break;
+        case 32:
+            test_fn_ = pixelTests32;
+            break;
+        case 64:
+            test_fn_ = pixelTests64;
+            break;
+        default:
+            CV_Error(Error::StsBadArg, "bytes must be 16, 32, or 64");
+    }
+}
+
+int BriefDescriptorExtractor::descriptorSize() const
+{
+    return bytes_;
+}
+
+int BriefDescriptorExtractor::descriptorType() const
+{
+    return CV_8UC1;
+}
+
+int BriefDescriptorExtractor::defaultNorm() const
+{
+    return NORM_HAMMING;
+}
+
+void BriefDescriptorExtractor::read( const FileNode& fn)
+{
+    int dSize = fn["descriptorSize"];
+    switch (dSize)
+    {
+        case 16:
+            test_fn_ = pixelTests16;
+            break;
+        case 32:
+            test_fn_ = pixelTests32;
+            break;
+        case 64:
+            test_fn_ = pixelTests64;
+            break;
+        default:
+            CV_Error(Error::StsBadArg, "descriptorSize must be 16, 32, or 64");
+    }
+    bytes_ = dSize;
+}
+
+void BriefDescriptorExtractor::write( FileStorage& fs) const
+{
+    fs << "descriptorSize" << bytes_;
+}
+
+void BriefDescriptorExtractor::computeImpl(InputArray image, std::vector<KeyPoint>& keypoints, OutputArray descriptors) const
+{
+    // Construct integral image for fast smoothing (box filter)
+    Mat sum;
+
+    Mat grayImage = image.getMat();
+    if( image.type() != CV_8U ) cvtColor( image, grayImage, COLOR_BGR2GRAY );
+
+    ///TODO allow the user to pass in a precomputed integral image
+    //if(image.type() == CV_32S)
+    //  sum = image;
+    //else
+
+    integral( grayImage, sum, CV_32S);
+
+    //Remove keypoints very close to the border
+    KeyPointsFilter::runByImageBorder(keypoints, image.size(), PATCH_SIZE/2 + KERNEL_SIZE/2);
+
+    descriptors.create((int)keypoints.size(), bytes_, CV_8U);
+    descriptors.setTo(Scalar::all(0));
+    test_fn_(sum, keypoints, descriptors);
+}
+
+}
+} // namespace cv

modules/xfeatures2d/src/generated_16.i

+// Code generated with '$ scripts/generate_code.py src/test_pairs.txt 16'
+#define SMOOTHED(y,x) smoothedSum(sum, pt, y, x)
+    desc[0] = (uchar)(((SMOOTHED(-2, -1) < SMOOTHED(7, -1)) << 7) + ((SMOOTHED(-14, -1) < SMOOTHED(-3, 3)) << 6) + ((SMOOTHED(1, -2) < SMOOTHED(11, 2)) << 5) + ((SMOOTHED(1, 6) < SMOOTHED(-10, -7)) << 4) + ((SMOOTHED(13, 2) < SMOOTHED(-1, 0)) << 3) + ((SMOOTHED(-14, 5) < SMOOTHED(5, -3)) << 2) + ((SMOOTHED(-2, 8) < SMOOTHED(2, 4)) << 1) + ((SMOOTHED(-11, 8) < SMOOTHED(-15, 5)) << 0));
+    desc[1] = (uchar)(((SMOOTHED(-6, -23) < SMOOTHED(8, -9)) << 7) + ((SMOOTHED(-12, 6) < SMOOTHED(-10, 8)) << 6) + ((SMOOTHED(-3, -1) < SMOOTHED(8, 1)) << 5) + ((SMOOTHED(3, 6) < SMOOTHED(5, 6)) << 4) + ((SMOOTHED(-7, -6) < SMOOTHED(5, -5)) << 3) + ((SMOOTHED(22, -2) < SMOOTHED(-11, -8)) << 2) + ((SMOOTHED(14, 7) < SMOOTHED(8, 5)) << 1) + ((SMOOTHED(-1, 14) < SMOOTHED(-5, -14)) << 0));
+    desc[2] = (uchar)(((SMOOTHED(-14, 9) < SMOOTHED(2, 0)) << 7) + ((SMOOTHED(7, -3) < SMOOTHED(22, 6)) << 6) + ((SMOOTHED(-6, 6) < SMOOTHED(-8, -5)) << 5) + ((SMOOTHED(-5, 9) < SMOOTHED(7, -1)) << 4) + ((SMOOTHED(-3, -7) < SMOOTHED(-10, -18)) << 3) + ((SMOOTHED(4, -5) < SMOOTHED(0, 11)) << 2) + ((SMOOTHED(2, 3) < SMOOTHED(9, 10)) << 1) + ((SMOOTHED(-10, 3) < SMOOTHED(4, 9)) << 0));
+    desc[3] = (uchar)(((SMOOTHED(0, 12) < SMOOTHED(-3, 19)) << 7) + ((SMOOTHED(1, 15) < SMOOTHED(-11, -5)) << 6) + ((SMOOTHED(14, -1) < SMOOTHED(7, 8)) << 5) + ((SMOOTHED(7, -23) < SMOOTHED(-5, 5)) << 4) + ((SMOOTHED(0, -6) < SMOOTHED(-10, 17)) << 3) + ((SMOOTHED(13, -4) < SMOOTHED(-3, -4)) << 2) + ((SMOOTHED(-12, 1) < SMOOTHED(-12, 2)) << 1) + ((SMOOTHED(0, 8) < SMOOTHED(3, 22)) << 0));
+    desc[4] = (uchar)(((SMOOTHED(-13, 13) < SMOOTHED(3, -1)) << 7) + ((SMOOTHED(-16, 17) < SMOOTHED(6, 10)) << 6) + ((SMOOTHED(7, 15) < SMOOTHED(-5, 0)) << 5) + ((SMOOTHED(2, -12) < SMOOTHED(19, -2)) << 4) + ((SMOOTHED(3, -6) < SMOOTHED(-4, -15)) << 3) + ((SMOOTHED(8, 3) < SMOOTHED(0, 14)) << 2) + ((SMOOTHED(4, -11) < SMOOTHED(5, 5)) << 1) + ((SMOOTHED(11, -7) < SMOOTHED(7, 1)) << 0));
+    desc[5] = (uchar)(((SMOOTHED(6, 12) < SMOOTHED(21, 3)) << 7) + ((SMOOTHED(-3, 2) < SMOOTHED(14, 1)) << 6) + ((SMOOTHED(5, 1) < SMOOTHED(-5, 11)) << 5) + ((SMOOTHED(3, -17) < SMOOTHED(-6, 2)) << 4) + ((SMOOTHED(6, 8) < SMOOTHED(5, -10)) << 3) + ((SMOOTHED(-14, -2) < SMOOTHED(0, 4)) << 2) + ((SMOOTHED(5, -7) < SMOOTHED(-6, 5)) << 1) + ((SMOOTHED(10, 4) < SMOOTHED(4, -7)) << 0));
+    desc[6] = (uchar)(((SMOOTHED(22, 0) < SMOOTHED(7, -18)) << 7) + ((SMOOTHED(-1, -3) < SMOOTHED(0, 18)) << 6) + ((SMOOTHED(-4, 22) < SMOOTHED(-5, 3)) << 5) + ((SMOOTHED(1, -7) < SMOOTHED(2, -3)) << 4) + ((SMOOTHED(19, -20) < SMOOTHED(17, -2)) << 3) + ((SMOOTHED(3, -10) < SMOOTHED(-8, 24)) << 2) + ((SMOOTHED(-5, -14) < SMOOTHED(7, 5)) << 1) + ((SMOOTHED(-2, 12) < SMOOTHED(-4, -15)) << 0));
+    desc[7] = (uchar)(((SMOOTHED(4, 12) < SMOOTHED(0, -19)) << 7) + ((SMOOTHED(20, 13) < SMOOTHED(3, 5)) << 6) + ((SMOOTHED(-8, -12) < SMOOTHED(5, 0)) << 5) + ((SMOOTHED(-5, 6) < SMOOTHED(-7, -11)) << 4) + ((SMOOTHED(6, -11) < SMOOTHED(-3, -22)) << 3) + ((SMOOTHED(15, 4) < SMOOTHED(10, 1)) << 2) + ((SMOOTHED(-7, -4) < SMOOTHED(15, -6)) << 1) + ((SMOOTHED(5, 10) < SMOOTHED(0, 24)) << 0));
+    desc[8] = (uchar)(((SMOOTHED(3, 6) < SMOOTHED(22, -2)) << 7) + ((SMOOTHED(-13, 14) < SMOOTHED(4, -4)) << 6) + ((SMOOTHED(-13, 8) < SMOOTHED(-18, -22)) << 5) + ((SMOOTHED(-1, -1) < SMOOTHED(-7, 3)) << 4) + ((SMOOTHED(-19, -12) < SMOOTHED(4, 3)) << 3) + ((SMOOTHED(8, 10) < SMOOTHED(13, -2)) << 2) + ((SMOOTHED(-6, -1) < SMOOTHED(-6, -5)) << 1) + ((SMOOTHED(2, -21) < SMOOTHED(-3, 2)) << 0));
+    desc[9] = (uchar)(((SMOOTHED(4, -7) < SMOOTHED(0, 16)) << 7) + ((SMOOTHED(-6, -5) < SMOOTHED(-12, -1)) << 6) + ((SMOOTHED(1, -1) < SMOOTHED(9, 18)) << 5) + ((SMOOTHED(-7, 10) < SMOOTHED(-11, 6)) << 4) + ((SMOOTHED(4, 3) < SMOOTHED(19, -7)) << 3) + ((SMOOTHED(-18, 5) < SMOOTHED(-4, 5)) << 2) + ((SMOOTHED(4, 0) < SMOOTHED(-20, 4)) << 1) + ((SMOOTHED(7, -11) < SMOOTHED(18, 12)) << 0));
+    desc[10] = (uchar)(((SMOOTHED(-20, 17) < SMOOTHED(-18, 7)) << 7) + ((SMOOTHED(2, 15) < SMOOTHED(19, -11)) << 6) + ((SMOOTHED(-18, 6) < SMOOTHED(-7, 3)) << 5) + ((SMOOTHED(-4, 1) < SMOOTHED(-14, 13)) << 4) + ((SMOOTHED(17, 3) < SMOOTHED(2, -8)) << 3) + ((SMOOTHED(-7, 2) < SMOOTHED(1, 6)) << 2) + ((SMOOTHED(17, -9) < SMOOTHED(-2, 8)) << 1) + ((SMOOTHED(-8, -6) < SMOOTHED(-1, 12)) << 0));
+    desc[11] = (uchar)(((SMOOTHED(-2, 4) < SMOOTHED(-1, 6)) << 7) + ((SMOOTHED(-2, 7) < SMOOTHED(6, 8)) << 6) + ((SMOOTHED(-8, -1) < SMOOTHED(-7, -9)) << 5) + ((SMOOTHED(8, -9) < SMOOTHED(15, 0)) << 4) + ((SMOOTHED(0, 22) < SMOOTHED(-4, -15)) << 3) + ((SMOOTHED(-14, -1) < SMOOTHED(3, -2)) << 2) + ((SMOOTHED(-7, -4) < SMOOTHED(17, -7)) << 1) + ((SMOOTHED(-8, -2) < SMOOTHED(9, -4)) << 0));
+    desc[12] = (uchar)(((SMOOTHED(5, -7) < SMOOTHED(7, 7)) << 7) + ((SMOOTHED(-5, 13) < SMOOTHED(-8, 11)) << 6) + ((SMOOTHED(11, -4) < SMOOTHED(0, 8)) << 5) + ((SMOOTHED(5, -11) < SMOOTHED(-9, -6)) << 4) + ((SMOOTHED(2, -6) < SMOOTHED(3, -20)) << 3) + ((SMOOTHED(-6, 2) < SMOOTHED(6, 10)) << 2) + ((SMOOTHED(-6, -6) < SMOOTHED(-15, 7)) << 1) + ((SMOOTHED(-6, -3) < SMOOTHED(2, 1)) << 0));
+    desc[13] = (uchar)(((SMOOTHED(11, 0) < SMOOTHED(-3, 2)) << 7) + ((SMOOTHED(7, -12) < SMOOTHED(14, 5)) << 6) + ((SMOOTHED(0, -7) < SMOOTHED(-1, -1)) << 5) + ((SMOOTHED(-16, 0) < SMOOTHED(6, 8)) << 4) + ((SMOOTHED(22, 11) < SMOOTHED(0, -3)) << 3) + ((SMOOTHED(19, 0) < SMOOTHED(5, -17)) << 2) + ((SMOOTHED(-23, -14) < SMOOTHED(-13, -19)) << 1) + ((SMOOTHED(-8, 10) < SMOOTHED(-11, -2)) << 0));
+    desc[14] = (uchar)(((SMOOTHED(-11, 6) < SMOOTHED(-10, 13)) << 7) + ((SMOOTHED(1, -7) < SMOOTHED(14, 0)) << 6) + ((SMOOTHED(-12, 1) < SMOOTHED(-5, -5)) << 5) + ((SMOOTHED(4, 7) < SMOOTHED(8, -1)) << 4) + ((SMOOTHED(-1, -5) < SMOOTHED(15, 2)) << 3) + ((SMOOTHED(-3, -1) < SMOOTHED(7, -10)) << 2) + ((SMOOTHED(3, -6) < SMOOTHED(10, -18)) << 1) + ((SMOOTHED(-7, -13) < SMOOTHED(-13, 10)) << 0));
+    desc[15] = (uchar)(((SMOOTHED(1, -1) < SMOOTHED(13, -10)) << 7) + ((SMOOTHED(-19, 14) < SMOOTHED(8, -14)) << 6) + ((SMOOTHED(-4, -13) < SMOOTHED(7, 1)) << 5) + ((SMOOTHED(1, -2) < SMOOTHED(12, -7)) << 4) + ((SMOOTHED(3, -5) < SMOOTHED(1, -5)) << 3) + ((SMOOTHED(-2, -2) < SMOOTHED(8, -10)) << 2) + ((SMOOTHED(2, 14) < SMOOTHED(8, 7)) << 1) + ((SMOOTHED(3, 9) < SMOOTHED(8, 2)) << 0));
+#undef SMOOTHED

modules/xfeatures2d/src/precomp.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+//
+//  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+//
+//  By downloading, copying, installing or using the software you agree to this license.
+//  If you do not agree to this license, do not download, install,
+//  copy or use the software.
+//
+//
+//                          License Agreement
+//                For Open Source Computer Vision Library
+//
+// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
+// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
+// Third party copyrights are property of their respective owners.
+//
+// Redistribution and use in source and binary forms, with or without modification,
+// are permitted provided that the following conditions are met:
+//
+//   * Redistribution's of source code must retain the above copyright notice,
+//     this list of conditions and the following disclaimer.
+//
+//   * Redistribution's in binary form must reproduce the above copyright notice,
+//     this list of conditions and the following disclaimer in the documentation
+//     and/or other materials provided with the distribution.
+//
+//   * The name of the copyright holders may not be used to endorse or promote products
+//     derived from this software without specific prior written permission.
+//
+// This software is provided by the copyright holders and contributors "as is" and
+// any express or implied warranties, including, but not limited to, the implied
+// warranties of merchantability and fitness for a particular purpose are disclaimed.
+// In no event shall the Intel Corporation or contributors be liable for any direct,
+// indirect, incidental, special, exemplary, or consequential damages
+// (including, but not limited to, procurement of substitute goods or services;
+// loss of use, data, or profits; or business interruption) however caused
+// and on any theory of liability, whether in contract, strict liability,
+// or tort (including negligence or otherwise) arising in any way out of
+// the use of this software, even if advised of the possibility of such damage.
+//
+//M*/
+
+#ifndef __OPENCV_PRECOMP_H__
+#define __OPENCV_PRECOMP_H__
+
+#include "opencv2/xfeatures2d.hpp"
+#include "opencv2/imgproc.hpp"
+
+#include "opencv2/core/utility.hpp"
+#include "opencv2/core/private.hpp"
+
+//#include "opencv2/nonfree/cuda.hpp"
+//#include "opencv2/core/private.cuda.hpp"
+
+#include "opencv2/core/ocl.hpp"
+
+#include "opencv2/opencv_modules.hpp"
+
+#ifdef HAVE_OPENCV_CUDAARITHM
+#  include "opencv2/cudaarithm.hpp"
+#endif
+
+#include "opencv2/core/private.hpp"
+
+#endif

modules/xfeatures2d/src/surf.hpp

+///////////// see LICENSE.txt in the OpenCV root directory //////////////
+
+#ifndef __OPENCV_NONFREE_SURF_HPP__
+#define __OPENCV_NONFREE_SURF_HPP__
+
+namespace cv
+{
+namespace xfeatures2d
+{
+
+//! Speeded up robust features, port from CUDA module.
+////////////////////////////////// SURF //////////////////////////////////////////
+
+class SURF_OCL
+{
+public:
+    enum KeypointLayout
+    {
+        X_ROW = 0,
+        Y_ROW,
+        LAPLACIAN_ROW,
+        OCTAVE_ROW,
+        SIZE_ROW,
+        ANGLE_ROW,
+        HESSIAN_ROW,
+        ROWS_COUNT
+    };
+
+    //! the full constructor taking all the necessary parameters
+    SURF_OCL();
+
+    bool init(const SURF* params);
+
+    //! returns the descriptor size in float's (64 or 128)
+    int descriptorSize() const { return params->extended ? 128 : 64; }
+
+    void uploadKeypoints(const std::vector<KeyPoint> &keypoints, UMat &keypointsGPU);
+    void downloadKeypoints(const UMat &keypointsGPU, std::vector<KeyPoint> &keypoints);
+
+    //! finds the keypoints using fast hessian detector used in SURF
+    //! supports CV_8UC1 images
+    //! keypoints will have nFeature cols and 6 rows
+    //! keypoints.ptr<float>(X_ROW)[i] will contain x coordinate of i'th feature
+    //! keypoints.ptr<float>(Y_ROW)[i] will contain y coordinate of i'th feature
+    //! keypoints.ptr<float>(LAPLACIAN_ROW)[i] will contain laplacian sign of i'th feature
+    //! keypoints.ptr<float>(OCTAVE_ROW)[i] will contain octave of i'th feature
+    //! keypoints.ptr<float>(SIZE_ROW)[i] will contain size of i'th feature
+    //! keypoints.ptr<float>(ANGLE_ROW)[i] will contain orientation of i'th feature
+    //! keypoints.ptr<float>(HESSIAN_ROW)[i] will contain response of i'th feature
+    bool detect(InputArray img, InputArray mask, UMat& keypoints);
+    //! finds the keypoints and computes their descriptors.
+    //! Optionally it can compute descriptors for the user-provided keypoints and recompute keypoints direction
+    bool detectAndCompute(InputArray img, InputArray mask, UMat& keypoints,
+                          OutputArray descriptors, bool useProvidedKeypoints = false);
+
+protected:
+    bool setImage(InputArray img, InputArray mask);
+
+    // kernel callers declarations
+    bool calcLayerDetAndTrace(int octave, int layer_rows);
+
+    bool findMaximaInLayer(int counterOffset, int octave, int layer_rows, int layer_cols);
+
+    bool interpolateKeypoint(int maxCounter, UMat &keypoints, int octave, int layer_rows, int maxFeatures);
+
+    bool calcOrientation(UMat &keypoints);
+
+    bool setUpRight(UMat &keypoints);
+
+    bool computeDescriptors(const UMat &keypoints, OutputArray descriptors);
+
+    bool detectKeypoints(UMat &keypoints);
+
+    const SURF* params;
+
+    //! max keypoints = min(keypointsRatio * img.size().area(), 65535)
+    UMat sum, intBuffer;
+    UMat det, trace;
+    UMat maxPosBuffer;
+
+    int img_cols, img_rows;
+
+    int maxCandidates;
+    int maxFeatures;
+
+    UMat img, counters;
+
+    // texture buffers
+    ocl::Image2D imgTex, sumTex;
+    bool haveImageSupport;
+    String kerOpts;
+
+    int status;
+};
+
+/*
+template<typename _Tp> void copyVectorToUMat(const std::vector<_Tp>& v, UMat& um)
+{
+    if(v.empty())
+        um.release();
+    else
+        Mat(1, (int)(v.size()*sizeof(v[0])), CV_8U, (void*)&v[0]).copyTo(um);
+}
+
+template<typename _Tp> void copyUMatToVector(const UMat& um, std::vector<_Tp>& v)
+{
+    if(um.empty())
+        v.clear();
+    else
+    {
+        size_t sz = um.total()*um.elemSize();
+        CV_Assert(um.isContinuous() && (sz % sizeof(_Tp) == 0));
+        v.resize(sz/sizeof(_Tp));
+        Mat m(um.size(), um.type(), &v[0]);
+        um.copyTo(m);
+    }
+}*/
+
+}
+}
+
+#endif

modules/xfeatures2d/src/xfeatures2d_init.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+//
+//  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+//
+//  By downloading, copying, installing or using the software you agree to this license.
+//  If you do not agree to this license, do not download, install,
+//  copy or use the software.
+//
+//
+//                          License Agreement
+//                For Open Source Computer Vision Library
+//
+// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
+// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
+// Third party copyrights are property of their respective owners.
+//
+// Redistribution and use in source and binary forms, with or without modification,
+// are permitted provided that the following conditions are met:
+//
+//   * Redistribution's of source code must retain the above copyright notice,
+//     this list of conditions and the following disclaimer.
+//
+//   * Redistribution's in binary form must reproduce the above copyright notice,
+//     this list of conditions and the following disclaimer in the documentation
+//     and/or other materials provided with the distribution.
+//
+//   * The name of the copyright holders may not be used to endorse or promote products
+//     derived from this software without specific prior written permission.
+//
+// This software is provided by the copyright holders and contributors "as is" and
+// any express or implied warranties, including, but not limited to, the implied
+// warranties of merchantability and fitness for a particular purpose are disclaimed.
+// In no event shall the Intel Corporation or contributors be liable for any direct,
+// indirect, incidental, special, exemplary, or consequential damages
+// (including, but not limited to, procurement of substitute goods or services;
+// loss of use, data, or profits; or business interruption) however caused
+// and on any theory of liability, whether in contract, strict liability,
+// or tort (including negligence or otherwise) arising in any way out of
+// the use of this software, even if advised of the possibility of such damage.
+//
+//M*/
+
+#include "precomp.hpp"
+
+namespace cv
+{
+namespace xfeatures2d
+{
+
+///////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+CV_INIT_ALGORITHM(BriefDescriptorExtractor, "Feature2D.BRIEF",
+                  obj.info()->addParam(obj, "bytes", obj.bytes_))
+
+///////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+CV_INIT_ALGORITHM(StarDetector, "Feature2D.STAR",
+                  obj.info()->addParam(obj, "maxSize", obj.maxSize);
+                  obj.info()->addParam(obj, "responseThreshold", obj.responseThreshold);
+                  obj.info()->addParam(obj, "lineThresholdProjected", obj.lineThresholdProjected);
+                  obj.info()->addParam(obj, "lineThresholdBinarized", obj.lineThresholdBinarized);
+                  obj.info()->addParam(obj, "suppressNonmaxSize", obj.suppressNonmaxSize))
+
+///////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+CV_INIT_ALGORITHM(FREAK, "Feature2D.FREAK",
+                  obj.info()->addParam(obj, "orientationNormalized", obj.orientationNormalized);
+                  obj.info()->addParam(obj, "scaleNormalized", obj.scaleNormalized);
+                  obj.info()->addParam(obj, "patternScale", obj.patternScale);
+                  obj.info()->addParam(obj, "nbOctave", obj.nOctaves))
+
+///////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+CV_INIT_ALGORITHM(SURF, "Feature2D.SURF",
+                  obj.info()->addParam(obj, "hessianThreshold", obj.hessianThreshold);
+                  obj.info()->addParam(obj, "nOctaves", obj.nOctaves);
+                  obj.info()->addParam(obj, "nOctaveLayers", obj.nOctaveLayers);
+                  obj.info()->addParam(obj, "extended", obj.extended);
+                  obj.info()->addParam(obj, "upright", obj.upright))
+
+///////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+CV_INIT_ALGORITHM(SIFT, "Feature2D.SIFT",
+                  obj.info()->addParam(obj, "nFeatures", obj.nfeatures);
+                  obj.info()->addParam(obj, "nOctaveLayers", obj.nOctaveLayers);
+                  obj.info()->addParam(obj, "contrastThreshold", obj.contrastThreshold);
+                  obj.info()->addParam(obj, "edgeThreshold", obj.edgeThreshold);
+                  obj.info()->addParam(obj, "sigma", obj.sigma))
+
+///////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+bool initModule_xfeatures2d(void)
+{
+    Ptr<Algorithm> sift = createSIFT_ptr_hidden(), surf = createSURF_ptr_hidden();
+    return sift->info() != 0 && surf->info() != 0;
+}
+
+}
+}

modules/xfeatures2d/test/test_keypoints.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+//
+//  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+//
+//  By downloading, copying, installing or using the software you agree to this license.
+//  If you do not agree to this license, do not download, install,
+//  copy or use the software.
+//
+//
+//                        Intel License Agreement
+//                For Open Source Computer Vision Library
+//
+// Copyright (C) 2000, Intel Corporation, all rights reserved.
+// Third party copyrights are property of their respective owners.
+//
+// Redistribution and use in source and binary forms, with or without modification,
+// are permitted provided that the following conditions are met:
+//
+//   * Redistribution's of source code must retain the above copyright notice,
+//     this list of conditions and the following disclaimer.
+//
+//   * Redistribution's in binary form must reproduce the above copyright notice,
+//     this list of conditions and the following disclaimer in the documentation
+//     and/or other materials provided with the distribution.
+//
+//   * The name of Intel Corporation may not be used to endorse or promote products
+//     derived from this software without specific prior written permission.
+//
+// This software is provided by the copyright holders and contributors "as is" and
+// any express or implied warranties, including, but not limited to, the implied
+// warranties of merchantability and fitness for a particular purpose are disclaimed.
+// In no event shall the Intel Corporation or contributors be liable for any direct,
+// indirect, incidental, special, exemplary, or consequential damages
+// (including, but not limited to, procurement of substitute goods or services;
+// loss of use, data, or profits; or business interruption) however caused
+// and on any theory of liability, whether in contract, strict liability,
+// or tort (including negligence or otherwise) arising in any way out of
+// the use of this software, even if advised of the possibility of such damage.
+//
+//M*/
+
+#include "test_precomp.hpp"
+#include "opencv2/highgui.hpp"
+
+using namespace std;
+using namespace cv;
+
+const string FEATURES2D_DIR = "features2d";
+const string IMAGE_FILENAME = "tsukuba.png";
+
+/****************************************************************************************\
+*                                     Test for KeyPoint                                  *
+\****************************************************************************************/
+
+class CV_FeatureDetectorKeypointsTest : public cvtest::BaseTest
+{
+public:
+    CV_FeatureDetectorKeypointsTest(const Ptr<FeatureDetector>& _detector) :
+        detector(_detector) {}
+
+protected:
+    virtual void run(int)
+    {
+        cv::initModule_features2d();
+        CV_Assert(detector);
+        string imgFilename = string(ts->get_data_path()) + FEATURES2D_DIR + "/" + IMAGE_FILENAME;
+
+        // Read the test image.
+        Mat image = imread(imgFilename);
+        if(image.empty())
+        {
+            ts->printf(cvtest::TS::LOG, "Image %s can not be read.\n", imgFilename.c_str());
+            ts->set_failed_test_info(cvtest::TS::FAIL_INVALID_TEST_DATA);
+            return;
+        }
+
+        vector<KeyPoint> keypoints;
+        detector->detect(image, keypoints);
+
+        if(keypoints.empty())
+        {
+            ts->printf(cvtest::TS::LOG, "Detector can't find keypoints in image.\n");
+            ts->set_failed_test_info(cvtest::TS::FAIL_INVALID_OUTPUT);
+            return;
+        }
+
+        Rect r(0, 0, image.cols, image.rows);
+        for(size_t i = 0; i < keypoints.size(); i++)
+        {
+            const KeyPoint& kp = keypoints[i];
+
+            if(!r.contains(kp.pt))
+            {
+                ts->printf(cvtest::TS::LOG, "KeyPoint::pt is out of image (x=%f, y=%f).\n", kp.pt.x, kp.pt.y);
+                ts->set_failed_test_info(cvtest::TS::FAIL_INVALID_OUTPUT);
+                return;
+            }
+
+            if(kp.size <= 0.f)
+            {
+                ts->printf(cvtest::TS::LOG, "KeyPoint::size is not positive (%f).\n", kp.size);
+                ts->set_failed_test_info(cvtest::TS::FAIL_INVALID_OUTPUT);
+                return;
+            }
+
+            if((kp.angle < 0.f && kp.angle != -1.f) || kp.angle >= 360.f)
+            {
+                ts->printf(cvtest::TS::LOG, "KeyPoint::angle is out of range [0, 360). It's %f.\n", kp.angle);
+                ts->set_failed_test_info(cvtest::TS::FAIL_INVALID_OUTPUT);
+                return;
+            }
+        }
+        ts->set_failed_test_info(cvtest::TS::OK);
+    }
+
+    Ptr<FeatureDetector> detector;
+};
+
+
+// Registration of tests
+
+TEST(Features2d_Detector_Keypoints_SURF, validation)
+{
+    CV_FeatureDetectorKeypointsTest test(Algorithm::create<FeatureDetector>("Feature2D.SURF"));
+    test.safe_run();
+}
+
+TEST(Features2d_Detector_Keypoints_SIFT, validation)
+{
+    CV_FeatureDetectorKeypointsTest test(FeatureDetector::create("SIFT"));
+    test.safe_run();
+}
+
+TEST(Features2d_Detector_Keypoints_Star, validation)
+{
+    CV_FeatureDetectorKeypointsTest test(Algorithm::create<FeatureDetector>("Feature2D.STAR"));
+    test.safe_run();
+}

modules/xfeatures2d/test/test_main.cpp

+#include "test_precomp.hpp"
+
+CV_TEST_MAIN("cv")