Merge pull request #39 from fpuja/saliencyModuleDevelop

Saliency module develop

modules/saliency/CMakeLists.txt

+set(the_description "Saliency API")
+ocv_define_module(saliency opencv_imgproc opencv_highgui opencv_features2d)

modules/saliency/README.md

+# Saliency Module
+
+The purpose of this module is to create, group and make available to the users, different saliency algorithms, belonging to different categories.
+

modules/saliency/doc/common_interfaces_saliency.rst

+Common Interfaces of Saliency
+=============================
+
+.. highlight:: cpp
+
+
+Saliency : Algorithm
+--------------------
+
+.. ocv:class:: Saliency
+
+Base abstract class for Saliency algorithms::
+
+   class CV_EXPORTS Saliency : public virtual Algorithm
+   {
+     public:
+ 
+     virtual ~Saliency();
+
+     static Ptr<Saliency> create( const String& saliencyType );
+
+     bool computeSaliency( const InputArray image, OutputArray saliencyMap );
+
+     String getClassName() const;
+   };
+
+
+Saliency::create
+----------------
+
+Creates a specialized saliency algorithm by its name.
+
+.. ocv:function::  static Ptr<Saliency> Saliency::create( const String& saliencyType )
+
+   :param saliencyType: saliency Type
+
+The following saliency types are now supported:
+
+* ``"SPECTRAL_RESIDUAL"`` -- :ocv:class:`StaticSaliencySpectralResidual`
+
+* ``"BING"`` -- :ocv:class:`ObjectnessBING`
+
+
+Saliency::computeSaliency
+-------------------------
+
+Performs all the operations, according to the specific algorithm created, to obtain the saliency map.
+
+.. ocv:function:: bool Saliency::computeSaliency( const InputArray image, OutputArray saliencyMap )
+ 
+   :param image: image or set of input images. According to InputArray proxy and to the needs of different algorithms (currently plugged),  the param image may be *Mat* or *vector<Mat>*
+   :param saliencyMap: saliency map. According to OutputArray proxy and to the results given by different algorithms (currently plugged), the saliency map may be a *Mat* or *vector<Vec4i>* (BING results).
+
+Saliency::getClassName
+----------------------
+
+Get the name of the specific Saliency Algorithm.
+
+.. ocv:function:: String Saliency::getClassName() const
+
+
+
+
+
+
+
+

misc @ 16789e8d

+Subproject commit 16789e8dcad170f502c81ea2b9714a4dd6bff0c1

modules/saliency/doc/motion_saliency_algorithms.rst

+Motion Saliency Algorithms
+============================
+
+.. highlight:: cpp
+
+Algorithms belonging to this category, are particularly focused to detect salient objects over time (hence also over frame), then there is a temporal component sealing cosider that allows to detect "moving" objects as salient, meaning therefore also the more general sense of detection the changes in the scene.
+
+Presently, the Fast Self-tuning Background Subtraction Algorithm [BinWangApr2014]_ has been implemented.
+
+.. [BinWangApr2014] B. Wang and P. Dudek "A Fast Self-tuning Background Subtraction Algorithm", in proc of IEEE Workshop on Change Detection, 2014
+
+MotionSaliencyBinWangApr2014
+----------------------------
+.. ocv:class:: MotionSaliencyBinWangApr2014
+
+Implementation of MotionSaliencyBinWangApr2014 from :ocv:class:`MotionSaliency`::
+
+   class CV_EXPORTS MotionSaliencyBinWangApr2014 : public MotionSaliency
+   {
+    public:
+      MotionSaliencyBinWangApr2014();
+      ~MotionSaliencyBinWangApr2014();
+
+     void setImagesize( int W, int H );
+     bool init();
+
+    protected:
+     bool computeSaliencyImpl( const InputArray image, OutputArray saliencyMap );
+  
+  };
+	  
+  
+MotionSaliencyBinWangApr2014::MotionSaliencyBinWangApr2014
+----------------------------------------------------------
+
+Constructor
+
+.. ocv:function:: MotionSaliencyBinWangApr2014::MotionSaliencyBinWangApr2014()
+
+
+MotionSaliencyBinWangApr2014::setImagesize
+------------------------------------------
+This is a utility function that allows to set the correct size (taken from the input image) in the corresponding variables that will be used to size the data structures of the algorithm.  
+
+.. ocv:function:: void MotionSaliencyBinWangApr2014::setImagesize( int W, int H )
+
+   :param W: width of input image
+   :param H: height of input image
+    
+MotionSaliencyBinWangApr2014::init
+-----------------------------------
+This function allows the correct initialization of all data structures that will be used by the algorithm.
+
+.. ocv:function:: bool MotionSaliencyBinWangApr2014::init()
+
+
+MotionSaliencyBinWangApr2014::computeSaliencyImpl
+-------------------------------------------------
+Performs all the operations and calls all internal functions necessary for the accomplishment of the Fast Self-tuning Background Subtraction Algorithm algorithm.
+
+.. ocv:function:: bool MotionSaliencyBinWangApr2014::computeSaliencyImpl( const InputArray image, OutputArray saliencyMap )
+
+   :param image: input image. According to the needs of this specialized algorithm, the param image is a single *Mat*.
+   :param saliencyMap: Saliency Map. Is a binarized map that, in accordance with the nature of the algorithm, highlights the moving objects or areas of change in the scene.
+    The saliency map is given by a single *Mat* (one for each frame of an hypothetical video stream).
+

modules/saliency/doc/objectness_algorithms.rst

+Objectness Algorithms
+============================
+
+.. highlight:: cpp
+
+Objectness is usually represented as a value which reflects how likely an image window covers an object of any category. Algorithms belonging to this category, avoid making decisions early on, by proposing a small number of category-independent proposals, that are expected to cover all objects in an image. Being able to perceive objects before identifying them is closely related to bottom up visual attention (saliency)
+
+Presently, the Binarized normed gradients algorithm [BING]_ has been implemented.
+
+.. [BING] Cheng, Ming-Ming, et al. "BING: Binarized normed gradients for objectness estimation at 300fps." IEEE CVPR. 2014.
+
+ObjectnessBING
+--------------
+.. ocv:class:: ObjectnessBING
+
+Implementation of BING from :ocv:class:`Objectness`::
+
+   class CV_EXPORTS ObjectnessBING : public Objectness
+   {
+    public:
+
+     ObjectnessBING();
+     ~ObjectnessBING();
+
+     void read();
+     void write() const;
+
+     vector<float> getobjectnessValues();
+     void setTrainingPath( string trainingPath );
+     void setBBResDir( string resultsDir );
+
+   protected:
+     bool computeSaliencyImpl( const InputArray src, OutputArray dst );
+
+   };
+   
+
+ObjectnessBING::ObjectnessBING
+------------------------------
+
+Constructor
+
+.. ocv:function:: ObjectnessBING::ObjectnessBING()
+
+
+ObjectnessBING::getobjectnessValues
+-----------------------------------
+Return the list of the rectangles' objectness value, in the same order as the  *vector<Vec4i> objectnessBoundingBox* returned by the algorithm (in computeSaliencyImpl function).
+The bigger value these scores are, it is more likely to be an object window.
+
+.. ocv:function:: vector<float> ObjectnessBING::getobjectnessValues()
+
+
+ObjectnessBING::setTrainingPath
+--------------------------------
+This is a utility function that allows to set the correct path from which the algorithm will load the trained model.
+
+.. ocv:function:: void ObjectnessBING::setTrainingPath( string trainingPath )
+
+    :param trainingPath: trained model path
+
+
+ObjectnessBING::setBBResDir
+---------------------------
+This is a utility function that allows to set an arbitrary path in which the algorithm will save the optional results 
+(ie writing on file the total number and the list of rectangles returned by objectess, one for each row).
+
+.. ocv:function:: void ObjectnessBING::setBBResDir( string resultsDir )
+
+    :param setBBResDir: results' folder path
+
+
+ObjectnessBING::computeSaliencyImpl
+-----------------------------------
+Performs all the operations and calls all internal functions necessary for the accomplishment of the Binarized normed gradients algorithm.
+
+.. ocv:function:: bool ObjectnessBING::computeSaliencyImpl( const InputArray image, OutputArray objectnessBoundingBox )
+
+   :param image: input image. According to the needs of this specialized algorithm, the param image is a single *Mat*
+   :param saliencyMap: objectness Bounding Box vector. According to the result given by this specialized  algorithm, the objectnessBoundingBox is a *vector<Vec4i>*.
+    Each bounding box is represented by a *Vec4i* for (minX, minY, maxX, maxY).
+    

modules/saliency/doc/saliency.rst

+Saliency API
+============
+
+.. highlight:: cpp
+
+Many computer vision applications may benefit from understanding where humans focus given a scene. Other than cognitively understanding the way human perceive images and scenes, finding salient regions and objects in the images helps various tasks such as speeding up object detection, object recognition, object tracking and content-aware image editing.
+
+About the saliency, there is a rich literature but the development is very fragmented. The principal purpose of this API is to give a 
+unique interface, a unique framework for use and plug sever saliency algorithms, also with very different nature and methodology, but they share the same purpose, organizing algorithms into three main categories:
+
+**Static Saliency**: algorithms belonging to this category, exploit different image features that allow to detect salient objects in a non dynamic scenarios. 
+
+**Motion Saliency**: algorithms belonging to this category, are particularly focused to detect salient objects over time (hence also over frame), then there is a temporal component sealing cosider that allows to detect "moving" objects as salient, meaning therefore also the more general sense of detection the changes in the scene.
+
+**Objectness**: Objectness is usually represented as a value which reflects how likely an image window covers an object of any category. Algorithms belonging to this category, avoid making decisions early on, by proposing a small number of category-independent proposals, that are expected to cover all objects in an image. Being able to perceive objects before identifying them is closely related to bottom up visual attention (saliency).
+
+
+UML design: 
+
+-----------
+
+**Saliency diagram**
+
+.. image:: pics/saliency.png
+   :width: 80%
+   :alt: Saliency diagram
+   :align: center
+
+To see how API works, try tracker demo: https://github.com/fpuja/opencv_contrib/blob/saliencyModuleDevelop/modules/saliency/samples/computeSaliency.cpp
+
+
+.. note:: This Tracking API has been designed with PlantUML. If you modify this API please change UML files under modules/tracking/misc/
+
+
+Saliency classes:
+-----------------
+
+.. toctree::
+    :maxdepth: 2
+
+    common_interfaces_saliency
+    saliency_categories
+    static_saliency_algorithms
+    motion_saliency_algorithms
+    objectness_algorithms
+    
+

modules/saliency/doc/saliency_categories.rst

+Saliency categories
+============================
+
+.. highlight:: cpp
+
+Base classes which give a general interface for each specialized type of saliency algorithm and provide utility methods for each algorithm in its class.
+
+StaticSaliency
+--------------
+
+.. ocv:class:: StaticSaliency
+
+StaticSaliency class::
+
+   class CV_EXPORTS StaticSaliency : public virtual Saliency
+   {
+    public:
+
+     bool computeBinaryMap( const Mat& saliencyMap, Mat& binaryMap );
+
+    protected:
+     virtual bool computeSaliencyImpl( const InputArray image, OutputArray saliencyMap ) = 0;
+   };
+
+StaticSaliency::computeBinaryMap
+--------------------------------
+
+This function perform a binary map of given saliency map. This is obtained in this way:
+
+In a first step, to improve the definition of interest areas and facilitate identification of targets, a segmentation
+by clustering is performed, using *K-means algorithm*. Then, to gain a binary representation of clustered saliency map, since values of the map can vary according to the characteristics of frame under analysis, it is not convenient to use a fixed threshold.
+So, *Otsu’s algorithm* is used, which assumes that the image to be thresholded contains two classes of pixels or bi-modal histograms (e.g. foreground and back-ground pixels); later on, the algorithm calculates the optimal threshold separating those two classes, so that their
+intra-class variance is minimal.
+
+.. ocv:function:: bool computeBinaryMap( const Mat& saliencyMap, Mat& binaryMap )
+
+   :param saliencyMap: the saliency map obtained through one of the specialized algorithms
+   :param binaryMap: the binary map
+
+
+MotionSaliency
+--------------
+
+.. ocv:class:: MotionSaliency
+
+MotionSaliency class::
+
+   class CV_EXPORTS MotionSaliency : public virtual Saliency
+   {
+
+
+    protected:
+     virtual bool computeSaliencyImpl( const InputArray image, OutputArray saliencyMap ) = 0;
+   };
+
+
+Objectness
+----------
+
+.. ocv:class:: Objectness
+
+Objectness class::
+
+   class CV_EXPORTS Objectness : public virtual Saliency
+   {
+
+    protected:
+     virtual bool computeSaliencyImpl( const InputArray image, OutputArray saliencyMap ) = 0;
+   };
+
+
+

modules/saliency/doc/static_saliency_algorithms.rst

+Static Saliency algorithms
+============================
+
+.. highlight:: cpp
+
+Algorithms belonging to this category, exploit different image features that allow to detect salient objects in a non dynamic scenarios.
+
+Presently, the Spectral Residual approach [SR]_ has been implemented.
+
+.. [SR] Hou, Xiaodi, and Liqing Zhang. "Saliency detection: A spectral residual approach." Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on. IEEE, 2007.
+
+
+SpectralResidual
+----------------
+
+Starting from the principle of natural image statistics, this method simulate the behavior of pre-attentive visual search. The algorithm analyze the log spectrum of each image and obtain the spectral residual. Then transform the spectral residual to spatial domain to obtain the saliency map, which suggests the positions of proto-objects.
+
+.. ocv:class:: StaticSaliencySpectralResidual
+
+Implementation of SpectralResidual from :ocv:class:`StaticSaliency`::
+
+   class CV_EXPORTS StaticSaliencySpectralResidual : public StaticSaliency
+   {
+    public:
+
+     StaticSaliencySpectralResidual();
+     ~StaticSaliencySpectralResidual();
+
+     typedef Ptr<Size> (Algorithm::*SizeGetter)();
+     typedef void (Algorithm::*SizeSetter)( const Ptr<Size> & );
+
+     Ptr<Size> getWsize();
+     void setWsize( const Ptr<Size> &newSize );
+
+     void read( const FileNode& fn );
+     void write( FileStorage& fs ) const;
+
+    protected:
+     bool computeSaliencyImpl( const InputArray src, OutputArray dst );
+
+   };
+
+
+StaticSaliencySpectralResidual::StaticSaliencySpectralResidual
+--------------------------------------------------------------
+
+Constructor
+
+.. ocv:function:: StaticSaliencySpectralResidual::StaticSaliencySpectralResidual()
+
+
+StaticSaliencySpectralResidual::getWsize
+----------------------------------------
+Return the resized image size.
+
+.. ocv:function:: Ptr<Size> StaticSaliencySpectralResidual::getWsize()
+
+
+StaticSaliencySpectralResidual::setWsize
+----------------------------------------
+Set the dimension to which the image should be resized.
+
+.. ocv:function:: StaticSaliencySpectralResidual::void setWsize( const Ptr<Size> &newSize )
+
+    :param newSize: dimension to which the image should be resized
+    
+    
+StaticSaliencySpectralResidual::computeSaliency
+-----------------------------------------------
+    
+Performs all the operations and calls all internal functions necessary for the accomplishment of spectral residual saliency map.
+
+.. ocv:function:: bool StaticSaliencySpectralResidual::computeSaliency( const InputArray image, OutputArray saliencyMap )
+ 
+   :param image: input image. According to the needs of this specialized saliency algorithm, the param image is a single *Mat*
+   :param saliencyMap: saliency map. According to the result given by this specialized saliency algorithm, the saliency map is a single *Mat*
+    
+
+
+
+

modules/saliency/include/opencv2/saliency.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#ifndef __OPENCV_SALIENCY_HPP__
+#define __OPENCV_SALIENCY_HPP__
+
+#include "opencv2/saliency/saliencyBaseClasses.hpp"
+#include "opencv2/saliency/saliencySpecializedClasses.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+CV_EXPORTS bool initModule_saliency( void );
+}
+}
+
+#endif //__OPENCV_SALIENCY_HPP__

modules/saliency/include/opencv2/saliency/saliencyBaseClasses.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#ifndef __OPENCV_SALIENCY_BASE_CLASSES_HPP__
+#define __OPENCV_SALIENCY_BASE_CLASSES_HPP__
+
+#include "opencv2/core.hpp"
+#include <opencv2/core/persistence.hpp>
+#include "opencv2/imgproc.hpp"
+#include <iostream>
+#include <sstream>
+#include <complex>
+
+namespace cv
+{
+namespace saliency
+{
+
+/************************************ Saliency Base Class ************************************/
+
+class CV_EXPORTS Saliency : public virtual Algorithm
+{
+ public:
+  /**
+   * \brief Destructor
+   */
+  virtual ~Saliency();
+
+  /**
+   * \brief Create Saliency by saliency type.
+   */
+  static Ptr<Saliency> create( const String& saliencyType );
+
+  /**
+   * \brief Compute the saliency
+   * \param image        The image.
+   * \param saliencyMap      The computed saliency map.
+   * \return true if the saliency map is computed, false otherwise
+   */
+  bool computeSaliency( const InputArray image, OutputArray saliencyMap );
+
+  /**
+   * \brief Get the name of the specific saliency type
+   * \return The name of the tracker initializer
+   */
+  String getClassName() const;
+
+ protected:
+
+  virtual bool computeSaliencyImpl( const InputArray image, OutputArray saliencyMap ) = 0;
+  String className;
+};
+
+/************************************ Static Saliency Base Class ************************************/
+class CV_EXPORTS StaticSaliency : public virtual Saliency
+{
+ public:
+
+  bool computeBinaryMap( const Mat& saliencyMap, Mat& binaryMap );
+ protected:
+  virtual bool computeSaliencyImpl( const InputArray image, OutputArray saliencyMap )=0;
+
+};
+
+/************************************ Motion Saliency Base Class ************************************/
+class CV_EXPORTS MotionSaliency : public virtual Saliency
+{
+
+ protected:
+  virtual bool computeSaliencyImpl( const InputArray image, OutputArray saliencyMap )=0;
+
+};
+
+/************************************ Objectness Base Class ************************************/
+class CV_EXPORTS Objectness : public virtual Saliency
+{
+
+ protected:
+  virtual bool computeSaliencyImpl( const InputArray image, OutputArray saliencyMap )=0;
+
+};
+
+} /* namespace saliency */
+} /* namespace cv */
+
+#endif

modules/saliency/include/opencv2/saliency/saliencySpecializedClasses.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#ifndef __OPENCV_SALIENCY_SPECIALIZED_CLASSES_HPP__
+#define __OPENCV_SALIENCY_SPECIALIZED_CLASSES_HPP__
+
+//#include "opencv2/saliency/kyheader.hpp"
+#include <cstdio>
+#include <string>
+#include <iostream>
+#include <stdint.h>
+
+namespace cv
+{
+namespace saliency
+{
+
+/************************************ Specific Static Saliency Specialized Classes ************************************/
+
+/**
+ * \brief Saliency based on algorithms described in [1]
+ * [1]Hou, Xiaodi, and Liqing Zhang. "Saliency detection: A spectral residual approach." Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on. IEEE, 2007.
+ */
+class CV_EXPORTS StaticSaliencySpectralResidual : public StaticSaliency
+{
+public:
+
+  StaticSaliencySpectralResidual();
+  virtual ~StaticSaliencySpectralResidual();
+
+  void read( const FileNode& fn );
+  void write( FileStorage& fs ) const;
+
+protected:
+  bool computeSaliencyImpl( const InputArray image, OutputArray saliencyMap );
+  AlgorithmInfo* info() const;CV_PROP_RW
+  int resImWidth;
+  int resImHeight;
+
+};
+
+/************************************ Specific Motion Saliency Specialized Classes ************************************/
+
+/*!
+ * A Fast Self-tuning Background Subtraction Algorithm.
+ *
+ * This background subtraction algorithm is inspired to the work of B. Wang and P. Dudek [2]
+ * [2]  B. Wang and P. Dudek "A Fast Self-tuning Background Subtraction Algorithm", in proc of IEEE Workshop on Change Detection, 2014
+ *
+ */
+
+class CV_EXPORTS MotionSaliencyBinWangApr2014 : public MotionSaliency
+{
+public:
+  MotionSaliencyBinWangApr2014();
+  virtual ~MotionSaliencyBinWangApr2014();
+
+  void setImagesize( int W, int H );
+  bool init();
+
+protected:
+  bool computeSaliencyImpl( const InputArray image, OutputArray saliencyMap );
+  AlgorithmInfo* info() const;
+
+private:
+
+  // classification (and adaptation) functions
+  bool fullResolutionDetection( const Mat& image, Mat& highResBFMask );
+  bool lowResolutionDetection( const Mat& image, Mat& lowResBFMask );
+
+  // Background model maintenance functions
+  bool templateOrdering();
+  bool templateReplacement( const Mat& finalBFMask, const Mat& image );
+
+  // changing structure
+  std::vector<Ptr<Mat> > backgroundModel;// The vector represents the background template T0---TK of reference paper.
+  // Matrices are two-channel matrix. In the first layer there are the B (background value)
+  // for each pixel. In the second layer, there are the C (efficacy) value for each pixel
+  Mat potentialBackground;// Two channel Matrix. For each pixel, in the first level there are the Ba value (potential background value)
+                          // and in the secon level there are the Ca value, the counter for each potential value.
+  Mat epslonPixelsValue;  // epslon threshold
+
+  //fixed parameter
+  bool neighborhoodCheck;
+  int N_DS;// Number of template to be downsampled and used in lowResolutionDetection function
+  int imageWidth;// Width of input image
+  int imageHeight;//Height of input image
+  int K;// Number of background model template
+  int N;// NxN is the size of the block for downsampling in the lowlowResolutionDetection
+  float alpha;// Learning rate
+  int L0, L1;// Upper-bound values for C0 and C1 (efficacy of the first two template (matrices) of backgroundModel
+  int thetaL;// T0, T1 swap threshold
+  int thetaA;// Potential background value threshold
+  int gamma;// Parameter that controls the time that the newly updated long-term background value will remain in the
+            // long-term template, regardless of any subsequent background changes. A relatively large (eg gamma=3) will
+            //restrain the generation of ghosts.
+
+};
+
+/************************************ Specific Objectness Specialized Classes ************************************/
+
+/**
+ * \brief Objectness algorithms based on [3]
+ * [3] Cheng, Ming-Ming, et al. "BING: Binarized normed gradients for objectness estimation at 300fps." IEEE CVPR. 2014.
+ */
+class CV_EXPORTS ObjectnessBING : public Objectness
+{
+public:
+
+  ObjectnessBING();
+  virtual ~ObjectnessBING();
+
+  void read();
+  void write() const;
+
+  std::vector<float> getobjectnessValues();
+  void setTrainingPath( std::string trainingPath );
+  void setBBResDir( std::string resultsDir );
+
+protected:
+  bool computeSaliencyImpl( const InputArray image, OutputArray objectnessBoundingBox );
+  AlgorithmInfo* info() const;
+
+private:
+
+  class FilterTIG
+  {
+  public:
+    void update( Mat &w );
+
+    // For a W by H gradient magnitude map, find a W-7 by H-7 CV_32F matching score map
+    Mat matchTemplate( const Mat &mag1u );
+
+    float dot( const int64_t tig1, const int64_t tig2, const int64_t tig4, const int64_t tig8 );
+    void reconstruct( Mat &w );// For illustration purpose
+
+  private:
+    static const int NUM_COMP = 2;// Number of components
+    static const int D = 64;// Dimension of TIG
+    int64_t _bTIGs[NUM_COMP];// Binary TIG features
+    float _coeffs1[NUM_COMP];// Coefficients of binary TIG features
+
+    // For efficiently deals with different bits in CV_8U gradient map
+    float _coeffs2[NUM_COMP], _coeffs4[NUM_COMP], _coeffs8[NUM_COMP];
+  };
+
+  template<typename VT, typename ST>
+  struct ValStructVec
+  {
+    ValStructVec();
+    int size() const;
+    void clear();
+    void reserve( int resSz );
+    void pushBack( const VT& val, const ST& structVal );
+    const VT& operator ()( int i ) const;
+    const ST& operator []( int i ) const;
+    VT& operator ()( int i );
+    ST& operator []( int i );
+
+    void sort( bool descendOrder = true );
+    const std::vector<ST> &getSortedStructVal();
+    std::vector<std::pair<VT, int> > getvalIdxes();
+    void append( const ValStructVec<VT, ST> &newVals, int startV = 0 );
+
+    std::vector<ST> structVals;  // struct values
+    int sz;// size of the value struct vector
+    std::vector<std::pair<VT, int> > valIdxes;// Indexes after sort
+    bool smaller()
+    {
+      return true;
+    }
+    std::vector<ST> sortedStructVals;
+  };
+
+  enum
+  {
+    MAXBGR,
+    HSV,
+    G
+  };
+
+  double _base, _logBase;  // base for window size quantization
+  int _W;// As described in the paper: #Size, Size(_W, _H) of feature window.
+  int _NSS;// Size for non-maximal suppress
+  int _maxT, _minT, _numT;// The minimal and maximal dimensions of the template
+
+  int _Clr;//
+  static const char* _clrName[3];
+
+  // Names and paths to read model and to store results
+  std::string _modelName, _bbResDir, _trainingPath, _resultsDir;
+
+  std::vector<int> _svmSzIdxs;// Indexes of active size. It's equal to _svmFilters.size() and _svmReW1f.rows
+  Mat _svmFilter;// Filters learned at stage I, each is a _H by _W CV_32F matrix
+  FilterTIG _tigF;// TIG filter
+  Mat _svmReW1f;// Re-weight parameters learned at stage II.
+
+  // List of the rectangles' objectness value, in the same order as
+  // the  vector<Vec4i> objectnessBoundingBox returned by the algorithm (in computeSaliencyImpl function)
+  std::vector<float> objectnessValues;
+
+private:
+  // functions
+
+  inline static float LoG( float x, float y, float delta )
+  {
+    float d = - ( x * x + y * y ) / ( 2 * delta * delta );
+    return -1.0f / ( (float) ( CV_PI ) * pow( delta, 4 ) ) * ( 1 + d ) * exp( d );
+  }  // Laplacian of Gaussian
+
+  // Read matrix from binary file
+  static bool matRead( const std::string& filename, Mat& M );
+
+  void setColorSpace( int clr = MAXBGR );
+
+  // Load trained model.
+  int loadTrainedModel( std::string modelName = "" );// Return -1, 0, or 1 if partial, none, or all loaded
+
+  // Get potential bounding boxes, each of which is represented by a Vec4i for (minX, minY, maxX, maxY).
+  // The trained model should be prepared before calling this function: loadTrainedModel() or trainStageI() + trainStageII().
+  // Use numDet to control the final number of proposed bounding boxes, and number of per size (scale and aspect ratio)
+  void getObjBndBoxes( Mat &img3u, ValStructVec<float, Vec4i> &valBoxes, int numDetPerSize = 120 );
+  void getObjBndBoxesForSingleImage( Mat img, ValStructVec<float, Vec4i> &boxes, int numDetPerSize );
+
+  bool filtersLoaded()
+  {
+    int n = (int) _svmSzIdxs.size();
+    return n > 0 && _svmReW1f.size() == Size( 2, n ) && _svmFilter.size() == Size( _W, _W );
+  }
+  void predictBBoxSI( Mat &mag3u, ValStructVec<float, Vec4i> &valBoxes, std::vector<int> &sz, int NUM_WIN_PSZ = 100, bool fast = true );
+  void predictBBoxSII( ValStructVec<float, Vec4i> &valBoxes, const std::vector<int> &sz );
+
+  // Calculate the image gradient: center option as in VLFeat
+  void gradientMag( Mat &imgBGR3u, Mat &mag1u );
+
+  static void gradientRGB( Mat &bgr3u, Mat &mag1u );
+  static void gradientGray( Mat &bgr3u, Mat &mag1u );
+  static void gradientHSV( Mat &bgr3u, Mat &mag1u );
+  static void gradientXY( Mat &x1i, Mat &y1i, Mat &mag1u );
+
+  static inline int bgrMaxDist( const Vec3b &u, const Vec3b &v )
+  {
+    int b = abs( u[0] - v[0] ), g = abs( u[1] - v[1] ), r = abs( u[2] - v[2] );
+    b = max( b, g );
+    return max( b, r );
+  }
+  static inline int vecDist3b( const Vec3b &u, const Vec3b &v )
+  {
+    return abs( u[0] - v[0] ) + abs( u[1] - v[1] ) + abs( u[2] - v[2] );
+  }
+
+  //Non-maximal suppress
+  static void nonMaxSup( Mat &matchCost1f, ValStructVec<float, Point> &matchCost, int NSS = 1, int maxPoint = 50, bool fast = true );
+
+};
+
+}
+/* namespace saliency */
+} /* namespace cv */
+
+#endif

modules/saliency/samples/computeSaliency.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2013, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include <opencv2/core/utility.hpp>
+#include <opencv2/saliency.hpp>
+#include <opencv2/highgui.hpp>
+#include <iostream>
+
+using namespace std;
+using namespace cv;
+using namespace saliency;
+
+static const char* keys =
+{ "{@saliency_algorithm | | Saliency algorithm <saliencyAlgorithmType.[saliencyAlgorithmTypeSubType]> }"
+    "{@video_name      | | video name            }"
+    "{@start_frame     |1| Start frame           }"
+    "{@training_path   |1| Path of the folder containing the trained files}" };
+
+static void help()
+{
+  cout << "\nThis example shows the functionality of \"Saliency \""
+       "Call:\n"
+       "./example_saliency_computeSaliency <saliencyAlgorithmSubType> <video_name> <start_frame> \n"
+       << endl;
+}
+
+int main( int argc, char** argv )
+{
+  CommandLineParser parser( argc, argv, keys );
+
+  String saliency_algorithm = parser.get<String>( 0 );
+  String video_name = parser.get<String>( 1 );
+  int start_frame = parser.get<int>( 2 );
+  String training_path = parser.get<String>( 3 );
+
+  if( saliency_algorithm.empty() || video_name.empty() )
+  {
+    help();
+    return -1;
+  }
+
+  //open the capture
+  VideoCapture cap;
+  cap.open( video_name );
+  cap.set( CAP_PROP_POS_FRAMES, start_frame );
+
+  if( !cap.isOpened() )
+  {
+    help();
+    cout << "***Could not initialize capturing...***\n";
+    cout << "Current parameter's value: \n";
+    parser.printMessage();
+    return -1;
+  }
+
+  Mat frame;
+
+  //instantiates the specific Saliency
+  Ptr<Saliency> saliencyAlgorithm = Saliency::create( saliency_algorithm );
+
+  if( saliencyAlgorithm == NULL )
+  {
+    cout << "***Error in the instantiation of the saliency algorithm...***\n";
+    return -1;
+  }
+
+  Mat binaryMap;
+  Mat image;
+
+  cap >> frame;
+  if( frame.empty() )
+  {
+    return 0;
+  }
+
+  frame.copyTo( image );
+
+  if( saliency_algorithm.find( "SPECTRAL_RESIDUAL" ) == 0 )
+  {
+    Mat saliencyMap;
+    if( saliencyAlgorithm->computeSaliency( image, saliencyMap ) )
+    {
+      StaticSaliencySpectralResidual spec;
+      spec.computeBinaryMap( saliencyMap, binaryMap );
+
+      imshow( "Saliency Map", saliencyMap );
+      imshow( "Original Image", image );
+      imshow( "Binary Map", binaryMap );
+      waitKey( 0 );
+    }
+
+  }
+  else if( saliency_algorithm.find( "BING" ) == 0 )
+  {
+    if( training_path.empty() )
+    {
+
+      cout << "Path of trained files missing! " << endl;
+      return -1;
+    }
+
+    else
+    {
+      vector<Vec4i> saliencyMap;
+      saliencyAlgorithm.dynamicCast<ObjectnessBING>()->setTrainingPath( training_path );
+      saliencyAlgorithm.dynamicCast<ObjectnessBING>()->setBBResDir( training_path + "/Results" );
+
+      if( saliencyAlgorithm->computeSaliency( image, saliencyMap ) )
+      {
+        std::cout << "Objectness done" << std::endl;
+      }
+    }
+
+  }
+  else if( saliency_algorithm.find( "BinWangApr2014" ) == 0 )
+  {
+
+    //Ptr<Size> size = Ptr<Size>( new Size( image.cols, image.rows ) );
+    saliencyAlgorithm.dynamicCast<MotionSaliencyBinWangApr2014>()->setImagesize( image.cols, image.rows );
+    saliencyAlgorithm.dynamicCast<MotionSaliencyBinWangApr2014>()->init();
+
+    bool paused = false;
+    for ( ;; )
+    {
+      if( !paused )
+      {
+
+        cap >> frame;
+        cvtColor( frame, frame, COLOR_BGR2GRAY );
+
+        Mat saliencyMap;
+        if( saliencyAlgorithm->computeSaliency( frame, saliencyMap ) )
+        {
+          std::cout << "current frame motion saliency done" << std::endl;
+        }
+
+        imshow( "image", frame );
+        imshow( "saliencyMap", saliencyMap * 255 );
+      }
+
+      char c = (char) waitKey( 2 );
+      if( c == 'q' )
+        break;
+      if( c == 'p' )
+        paused = !paused;
+
+    }
+  }
+
+  return 0;
+}

modules/saliency/src/BING/CmFile.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+#include "CmFile.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+bool CmFile::MkDir( std::string &_path )
+{
+  if( _path.size() == 0 )
+    return false;
+  static char buffer[1024];
+  strcpy( buffer, _S( _path ) );
+#ifdef _WIN32
+  for (int i = 0; buffer[i] != 0; i ++)
+  {
+    if (buffer[i] == '\\' || buffer[i] == '/')
+    {
+      buffer[i] = '\0';
+      CreateDirectoryA(buffer, 0);
+      buffer[i] = '/';
+    }
+  }
+
+  CreateDirectoryA(_S(_path), 0);
+  return true;
+#else
+  for ( int i = 0; buffer[i] != 0; i++ )
+  {
+    if( buffer[i] == '\\' || buffer[i] == '/' )
+    {
+      buffer[i] = '\0';
+      mkdir( buffer, 0 );
+      buffer[i] = '/';
+    }
+  }
+  mkdir( _S( _path ), 0 );
+  return true;
+#endif
+}
+
+}  // namespace saliency
+}  // namespace cv

modules/saliency/src/BING/CmFile.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#ifndef __OPENCV_CM_FILE_HPP__
+#define __OPENCV_CM_FILE_HPP__
+
+#ifdef _WIN32
+#include <windows.h>
+#else
+#include <iostream>
+#include <stdlib.h>
+#include <sys/stat.h>
+#include <dirent.h>
+#endif
+
+#include "kyheader.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+struct CmFile
+{
+    static bool MkDir(std::string&  path);
+};
+
+}
+}
+
+#endif

modules/saliency/src/BING/CmShow.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+#include "CmShow.hpp"
+#include "opencv2/core.hpp"
+#include <opencv2/highgui.hpp>
+
+typedef std::pair<int, int> CostiIdx;
+
+namespace cv
+{
+namespace saliency
+{
+
+Mat CmShow::HistBins( CMat& color3f, CMat& val, CStr& title, bool descendShow, CMat &with )
+{
+  // Prepare data
+  int H = 300, spaceH = 6, barH = 10, n = color3f.cols;
+  CV_Assert( color3f.size() == val.size() && color3f.rows == 1 );
+  Mat binVal1i, binColor3b, width1i;
+  if( with.size() == val.size() )
+    with.convertTo( width1i, CV_32S, 400 / sum( with ).val[0] );  // Default shown width
+  else
+    width1i = Mat( 1, n, CV_32S, Scalar( 10 ) );  // Default bin width = 10
+  int W = cvRound( sum( width1i ).val[0] );
+  color3f.convertTo( binColor3b, CV_8UC3, 255 );
+  double maxVal, minVal;
+  minMaxLoc( val, &minVal, &maxVal );
+  printf( "%g\n", H / max( maxVal, -minVal ) );
+  val.convertTo( binVal1i, CV_32S, 20000 );
+  Size szShow( W, H + spaceH + barH );
+  szShow.height += minVal < 0 && !descendShow ? H + spaceH : 0;
+  Mat showImg3b( szShow, CV_8UC3, Scalar( 255, 255, 255 ) );
+  int* binH = (int*) ( binVal1i.data );
+  Vec3b* binColor = (Vec3b*) ( binColor3b.data );
+  int* binW = (int*) ( width1i.data );
+  std::vector<CostiIdx> costIdx( n );
+  if( descendShow )
+  {
+    for ( int i = 0; i < n; i++ )
+      costIdx[i] = std::make_pair( binH[i], i );
+    sort( costIdx.begin(), costIdx.end(), std::greater<CostiIdx>() );
+  }
+
+  // Show image
+  for ( int i = 0, x = 0; i < n; i++ )
+  {
+    int idx = descendShow ? costIdx[i].second : i;
+    int h = descendShow ? abs( binH[idx] ) : binH[idx];
+    Scalar color( binColor[idx] );
+    Rect reg( x, H + spaceH, binW[idx], barH );
+    showImg3b( reg ) = color;  // Draw bar
+    rectangle( showImg3b, reg, Scalar( 0 ) );
+
+    reg.height = abs( h );
+    reg.y = h >= 0 ? H - h : H + 2 * spaceH + barH;
+    showImg3b( reg ) = color;
+    rectangle( showImg3b, reg, Scalar( 0 ) );
+
+    x += binW[idx];
+  }
+  imshow( String( title.c_str() ), showImg3b );
+  return showImg3b;
+}
+
+}  // namespace saliency
+}  // namespace cv

modules/saliency/src/BING/CmShow.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#ifndef __OPENCV_CM_SHOW_HPP__
+#define __OPENCV_CM_SHOW_HPP__
+
+#include "BING/kyheader.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+class CmShow
+{
+ public:
+  static cv::Mat HistBins( CMat& color3f, CMat& val, CStr& title, bool descendShow = false, CMat &with = cv::Mat() );
+  static void showTinyMat( CStr &title, CMat &m );
+  static inline void SaveShow( CMat& img, CStr& title );
+};
+
+}
+}
+
+#endif //__OPENCV_CM_SHOW_HPP__

modules/saliency/src/BING/CmTimer.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#ifndef __OPENCV_CM_TIMER_HPP__
+#define __OPENCV_CM_TIMER_HPP__
+
+#include "BING/kyheader.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+class CmTimer
+{
+ public:
+  CmTimer( CStr t ) :
+      title( t )
+  {
+    is_started = false;
+    start_clock = 0;
+    cumulative_clock = 0;
+    n_starts = 0;
+  }
+
+  ~CmTimer()
+  {
+    if( is_started )
+      printf( "CmTimer '%s' is started and is being destroyed.\n", title.c_str() );
+  }
+
+  inline void Start();
+  inline void Stop();
+  inline void Reset();
+
+  inline bool Report();
+  inline bool StopAndReport()
+  {
+    Stop();
+    return Report();
+  }
+  inline float TimeInSeconds();
+
+ private:
+  CStr title;
+
+  CmTimer& operator=( const CmTimer& );
+
+  bool is_started;
+  clock_t start_clock;
+  clock_t cumulative_clock;
+  unsigned int n_starts;
+};
+
+/************************************************************************/
+/*                       Implementations                                */
+/************************************************************************/
+
+void CmTimer::Start()
+{
+  if( is_started )
+  {
+    printf( "CmTimer '%s' is already started. Nothing done.\n", title.c_str() );
+    return;
+  }
+
+  is_started = true;
+  n_starts++;
+  start_clock = clock();
+}
+
+void CmTimer::Stop()
+{
+  if( !is_started )
+  {
+    printf( "CmTimer '%s' is started. Nothing done\n", title.c_str() );
+    return;
+  }
+
+  cumulative_clock += clock() - start_clock;
+  is_started = false;
+}
+
+void CmTimer::Reset()
+{
+  if( is_started )
+  {
+    printf( "CmTimer '%s'is started during reset request.\n Only reset cumulative time.\n", title.c_str() );
+    return;
+  }
+  cumulative_clock = 0;
+}
+
+bool CmTimer::Report()
+{
+  if( is_started )
+  {
+    printf( "CmTimer '%s' is started.\n Cannot provide a time report.", title.c_str() );
+    return false;
+  }
+
+  float timeUsed = TimeInSeconds();
+  printf( "[%s] CumuTime: %gs, #run: %d, AvgTime: %gs\n", title.c_str(), timeUsed, n_starts, timeUsed / n_starts );
+  return true;
+}
+
+float CmTimer::TimeInSeconds()
+{
+  if( is_started )
+  {
+    printf( "CmTimer '%s' is started. Nothing done\n", title.c_str() );
+    return 0;
+  }
+  return float( cumulative_clock ) / CLOCKS_PER_SEC;
+}
+
+}  // namespace saliency
+}  // namespace cv
+
+#endif // __OPENCV_CM_TIMER_HPP__

modules/saliency/src/BING/FilterTIG.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+#include "CmShow.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+float ObjectnessBING::FilterTIG::dot( const int64_t tig1, const int64_t tig2, const int64_t tig4, const int64_t tig8 )
+{
+  int64_t bcT1 = (int64_t) POPCNT64( tig1 );
+  int64_t bcT2 = (int64_t) POPCNT64( tig2 );
+  int64_t bcT4 = (int64_t) POPCNT64( tig4 );
+  int64_t bcT8 = (int64_t) POPCNT64( tig8 );
+
+  int64_t bc01 = (int64_t) ( POPCNT64(_bTIGs[0] & tig1) << 1 ) - bcT1;
+  int64_t bc02 = (int64_t) ( ( POPCNT64(_bTIGs[0] & tig2) << 1 ) - bcT2 ) << 1;
+  int64_t bc04 = (int64_t) ( ( POPCNT64(_bTIGs[0] & tig4) << 1 ) - bcT4 ) << 2;
+  int64_t bc08 = (int64_t) ( ( POPCNT64(_bTIGs[0] & tig8) << 1 ) - bcT8 ) << 3;
+
+  int64_t bc11 = (int64_t) ( POPCNT64(_bTIGs[1] & tig1) << 1 ) - bcT1;
+  int64_t bc12 = (int64_t) ( ( POPCNT64(_bTIGs[1] & tig2) << 1 ) - bcT2 ) << 1;
+  int64_t bc14 = (int64_t) ( ( POPCNT64(_bTIGs[1] & tig4) << 1 ) - bcT4 ) << 2;
+  int64_t bc18 = (int64_t) ( ( POPCNT64(_bTIGs[1] & tig8) << 1 ) - bcT8 ) << 3;
+
+  return _coeffs1[0] * ( bc01 + bc02 + bc04 + bc08 ) + _coeffs1[1] * ( bc11 + bc12 + bc14 + bc18 );
+}
+
+void ObjectnessBING::FilterTIG::update( Mat &w1f )
+{
+  CV_Assert( w1f.cols * w1f.rows == D && w1f.type() == CV_32F && w1f.isContinuous() );
+  float b[D], residuals[D];
+  memcpy( residuals, w1f.data, sizeof(float) * D );
+  for ( int i = 0; i < NUM_COMP; i++ )
+  {
+    float avg = 0;
+    for ( int j = 0; j < D; j++ )
+    {
+      b[j] = residuals[j] >= 0.0f ? 1.0f : -1.0f;
+      avg += residuals[j] * b[j];
+    }
+    avg /= D;
+    _coeffs1[i] = avg, _coeffs2[i] = avg * 2, _coeffs4[i] = avg * 4, _coeffs8[i] = avg * 8;
+    for ( int j = 0; j < D; j++ )
+      residuals[j] -= avg * b[j];
+    uint64_t tig = 0;
+    for ( int j = 0; j < D; j++ )
+      tig = ( tig << 1 ) | ( b[j] > 0 ? 1 : 0 );
+    _bTIGs[i] = tig;
+  }
+}
+
+void ObjectnessBING::FilterTIG::reconstruct( Mat &w1f )
+{
+  w1f = Mat::zeros( 8, 8, CV_32F );
+  float *weight = (float*) w1f.data;
+  for ( int i = 0; i < NUM_COMP; i++ )
+  {
+    uint64_t tig = _bTIGs[i];
+    for ( int j = 0; j < D; j++ )
+      weight[j] += _coeffs1[i] * ( ( ( tig >> ( 63 - j ) ) & 1 ) ? 1 : -1 );
+  }
+}
+
+// For a W by H gradient magnitude map, find a W-7 by H-7 CV_32F matching score map
+// Please refer to my paper for definition of the variables used in this function
+Mat ObjectnessBING::FilterTIG::matchTemplate( const Mat &mag1u )
+{
+  const int H = mag1u.rows, W = mag1u.cols;
+  const Size sz( W + 1, H + 1 );  // Expand original size to avoid dealing with boundary conditions
+  Mat_<int64_t> Tig1 = Mat_<int64_t>::zeros( sz ), Tig2 = Mat_<int64_t>::zeros( sz );
+  Mat_<int64_t> Tig4 = Mat_<int64_t>::zeros( sz ), Tig8 = Mat_<int64_t>::zeros( sz );
+  Mat_<BYTE> Row1 = Mat_<BYTE>::zeros( sz ), Row2 = Mat_<BYTE>::zeros( sz );
+  Mat_<BYTE> Row4 = Mat_<BYTE>::zeros( sz ), Row8 = Mat_<BYTE>::zeros( sz );
+  Mat_<float> scores( sz );
+  for ( int y = 1; y <= H; y++ )
+  {
+    const BYTE* G = mag1u.ptr<BYTE>( y - 1 );
+    int64_t* T1 = Tig1.ptr<int64_t>( y );  // Binary TIG of current row
+    int64_t* T2 = Tig2.ptr<int64_t>( y );
+    int64_t* T4 = Tig4.ptr<int64_t>( y );
+    int64_t* T8 = Tig8.ptr<int64_t>( y );
+    int64_t* Tu1 = Tig1.ptr<int64_t>( y - 1 );  // Binary TIG of upper row
+    int64_t* Tu2 = Tig2.ptr<int64_t>( y - 1 );
+    int64_t* Tu4 = Tig4.ptr<int64_t>( y - 1 );
+    int64_t* Tu8 = Tig8.ptr<int64_t>( y - 1 );
+    BYTE* R1 = Row1.ptr<BYTE>( y );
+    BYTE* R2 = Row2.ptr<BYTE>( y );
+    BYTE* R4 = Row4.ptr<BYTE>( y );
+    BYTE* R8 = Row8.ptr<BYTE>( y );
+    float *s = scores.ptr<float>( y );
+    for ( int x = 1; x <= W; x++ )
+    {
+      BYTE g = G[x - 1];
+      R1[x] = ( R1[x - 1] << 1 ) | ( ( g >> 4 ) & 1 );
+      R2[x] = ( R2[x - 1] << 1 ) | ( ( g >> 5 ) & 1 );
+      R4[x] = ( R4[x - 1] << 1 ) | ( ( g >> 6 ) & 1 );
+      R8[x] = ( R8[x - 1] << 1 ) | ( ( g >> 7 ) & 1 );
+      T1[x] = ( Tu1[x] << 8 ) | R1[x];
+      T2[x] = ( Tu2[x] << 8 ) | R2[x];
+      T4[x] = ( Tu4[x] << 8 ) | R4[x];
+      T8[x] = ( Tu8[x] << 8 ) | R8[x];
+      s[x] = dot( T1[x], T2[x], T4[x], T8[x] );
+    }
+  }
+  Mat matchCost1f;
+  scores( Rect( 8, 8, W - 7, H - 7 ) ).copyTo( matchCost1f );
+  return matchCost1f;
+}
+
+}  // namespace saliency
+}  // namespace cv

modules/saliency/src/BING/ValStructVec.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+
+
+}
+}
+

modules/saliency/src/BING/kyheader.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#ifndef KYHEADER_H
+#define KYHEADER_H
+
+#include <assert.h>
+#include <string>
+#include <vector>
+#include <functional>
+#include <list>
+#include <algorithm>
+#include <iostream>
+#include <cmath>
+#include <time.h>
+#include <fstream>
+#include <stdint.h>
+
+// TODO: reference additional headers your program requires here
+
+#include "opencv2/core.hpp"
+
+#define CV_VERSION_ID CVAUX_STR(CV_MAJOR_VERSION) CVAUX_STR(CV_MINOR_VERSION) CVAUX_STR(CV_SUBMINOR_VERSION)
+#ifdef _DEBUG
+#define cvLIB(name) "opencv_" name CV_VERSION_ID "d"
+#else
+#define cvLIB(name) "opencv_" name CV_VERSION_ID
+#endif
+
+#ifdef WIN32
+/* windows stuff */
+#else
+typedef unsigned long DWORD;
+typedef unsigned short WORD;
+typedef unsigned int UNINT32;
+typedef bool BOOL;
+typedef void *HANDLE;
+#endif
+
+#ifndef _MSC_VER
+typedef unsigned char BYTE;
+#else
+#include <windows.h>
+#endif
+
+typedef std::vector<int> vecI;
+typedef const std::string CStr;
+typedef const cv::Mat CMat;
+typedef std::vector<std::string> vecS;
+typedef std::vector<cv::Mat> vecM;
+typedef std::vector<float> vecF;
+typedef std::vector<double> vecD;
+
+namespace cv
+{
+namespace saliency
+{
+
+enum
+{
+  CV_FLIP_BOTH = -1,
+  CV_FLIP_VERTICAL = 0,
+  CV_FLIP_HORIZONTAL = 1
+};
+#define _S(str) ((str).c_str())
+#define CHK_IND(p) ((p).x >= 0 && (p).x < _w && (p).y >= 0 && (p).y < _h)
+#define CV_Assert_(expr, args) \
+{\
+    if(!(expr)) {\
+    String msg = cv::format args; \
+    printf("%s in %s:%d\n", msg.c_str(), __FILE__, __LINE__); \
+    cv::error(cv::Exception(CV_StsAssert, msg, __FUNCTION__, __FILE__, __LINE__) ); }\
+}
+
+
+// Return -1 if not in the list
+template<typename T>
+static inline int findFromList( const T &word, const std::vector<T> &strList )
+{
+
+  std::vector<cv::String>::iterator it = std::find( strList.begin(), strList.end(), word );
+  if( it == strList.end() )
+  {
+    return -1;
+  }
+  else
+  {
+    int index = it - strList.begin();
+    return index;
+  }
+}
+
+template<typename T> inline T sqr( T x )
+{
+  return x * x;
+}  // out of range risk for T = byte, ...
+template<class T, int D> inline T vecSqrDist( const cv::Vec<T, D> &v1, const cv::Vec<T, D> &v2 )
+{
+  T s = 0;
+  for ( int i = 0; i < D; i++ )
+    s += sqr( v1[i] - v2[i] );
+  return s;
+}  // out of range risk for T = byte, ...
+template<class T, int D> inline T vecDist( const cv::Vec<T, D> &v1, const cv::Vec<T, D> &v2 )
+{
+  return sqrt( vecSqrDist( v1, v2 ) );
+}  // out of range risk for T = byte, ...
+
+inline cv::Rect Vec4i2Rect( cv::Vec4i &v )
+{
+  return cv::Rect( cv::Point( v[0] - 1, v[1] - 1 ), cv::Point( v[2], v[3] ) );
+}
+
+
+ #if defined(_MSC_VER)
+ # include <intrin.h>
+ # define POPCNT(x) __popcnt(x)
+ # define POPCNT64(x) __popcnt64(x)
+ #endif
+
+
+#if defined(__GNUC__)
+# define POPCNT(x) __builtin_popcount(x)
+# define POPCNT64(x) __builtin_popcountll(x)
+#endif
+
+inline int popcnt64( register uint64_t u )
+{
+  u = ( u & 0x5555555555555555 ) + ( ( u >> 1 ) & 0x5555555555555555 );
+  u = ( u & 0x3333333333333333 ) + ( ( u >> 2 ) & 0x3333333333333333 );
+  u = ( u & 0x0f0f0f0f0f0f0f0f ) + ( ( u >> 4 ) & 0x0f0f0f0f0f0f0f0f );
+  u = ( u & 0x00ff00ff00ff00ff ) + ( ( u >> 8 ) & 0x00ff00ff00ff00ff );
+  u = ( u & 0x0000ffff0000ffff ) + ( ( u >> 16 ) & 0x0000ffff0000ffff );
+  u = ( u & 0x00000000ffffffff ) + ( ( u >> 32 ) & 0x00000000ffffffff );
+  return (int)u;
+}
+
+inline int popcnt( register uint32_t u )
+{
+  u = ( u & 0x55555555 ) + ( ( u >> 1 ) & 0x55555555 );
+  u = ( u & 0x33333333 ) + ( ( u >> 2 ) & 0x33333333 );
+  u = ( u & 0x0f0f0f0f ) + ( ( u >> 4 ) & 0x0f0f0f0f );
+  u = ( u & 0x00ff00ff ) + ( ( u >> 8 ) & 0x00ff00ff );
+  u = ( u & 0x0000ffff ) + ( ( u >> 16 ) & 0x0000ffff );
+  return (int)u;
+}
+
+inline int popcnt64_nibble( register uint64_t u )
+{
+  static const uint8_t Table[] =
+  { 0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, 3, 2, 3, 3, 4 };
+
+  int c = 0;
+  while ( u )
+  {
+    c += Table[u & 0xf];
+    u >>= 4;
+  }
+  return (int)c;
+}
+
+inline int popcnt_nibble( register uint32_t u )
+{
+  static const uint8_t Table[] =
+  { 0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, 3, 2, 3, 3, 4 };
+
+  int c = 0;
+  while ( u )
+  {
+    c += Table[u & 0xf];
+    u >>= 4;
+  }
+  return (int)c;
+}
+
+inline int popcnt64_byte( register uint64_t u )
+{
+#define B2(k) k, k+1, k+1, k+2
+#define B4(k) B2(k), B2(k+1), B2(k+1), B2(k+2)
+#define B6(k) B4(k), B4(k+1), B4(k+1), B4(k+2)
+  static const uint8_t Table[] =
+  { B6( 0 ), B6( 1 ), B6( 1 ), B6( 2 ) };
+#undef B6
+#undef B4
+#undef B2
+
+  int c = 0;
+  while ( u )
+  {
+    c += Table[u & 0xff];
+    u >>= 8;
+  }
+  return (int)c;
+}
+
+inline int popcnt_byte( register uint32_t u )
+{
+#define B2(k) k, k+1, k+1, k+2
+#define B4(k) B2(k), B2(k+1), B2(k+1), B2(k+2)
+#define B6(k) B4(k), B4(k+1), B4(k+1), B4(k+2)
+  static const uint8_t Table[] =
+  { B6( 0 ), B6( 1 ), B6( 1 ), B6( 2 ) };
+#undef B6
+#undef B4
+#undef B2
+
+  int c = 0;
+  while ( u )
+  {
+    c += Table[u & 0xff];
+    u >>= 8;
+  }
+  return (int)c;
+}
+
+}
+}
+
+#endif // KYHEADER_H

modules/saliency/src/BING/objectnessBING.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+
+#include "BING/kyheader.hpp"
+#include "CmTimer.hpp"
+#include "CmFile.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+/**
+ * BING Objectness
+ */
+
+const char* ObjectnessBING::_clrName[3] =
+{ "MAXBGR", "HSV", "I" };
+
+ObjectnessBING::ObjectnessBING()
+{
+  _base = 2;  // base for window size quantization
+  _W = 8;  // feature window size (W, W)
+  _NSS = 2;  //non-maximal suppress size NSS
+  _logBase = log( _base );
+  _minT = cvCeil( log( 10. ) / _logBase );
+  _maxT = cvCeil( log( 500. ) / _logBase );
+  _numT = _maxT - _minT + 1;
+  _Clr = MAXBGR;
+
+  setColorSpace( _Clr );
+
+  className = "BING";
+}
+
+ObjectnessBING::~ObjectnessBING()
+{
+
+}
+
+void ObjectnessBING::setColorSpace( int clr )
+{
+  _Clr = clr;
+  _modelName = _trainingPath + "/" + std::string( format( "ObjNessB%gW%d%s", _base, _W, _clrName[_Clr] ).c_str() );
+  _bbResDir = _resultsDir + "/" + std::string( format( "BBoxesB%gW%d%s/", _base, _W, _clrName[_Clr] ).c_str() );
+}
+
+void ObjectnessBING::setTrainingPath( std::string trainingPath )
+{
+  _trainingPath = trainingPath;
+}
+
+void ObjectnessBING::setBBResDir( std::string resultsDir )
+{
+  _resultsDir = resultsDir;
+}
+
+int ObjectnessBING::loadTrainedModel( std::string modelName )  // Return -1, 0, or 1 if partial, none, or all loaded
+{
+  if( modelName.size() == 0 )
+    modelName = _modelName;
+  CStr s1 = modelName + ".wS1", s2 = modelName + ".wS2", sI = modelName + ".idx";
+  Mat filters1f, reW1f, idx1i, show3u;
+
+  if( !matRead( s1, filters1f ) || !matRead( sI, idx1i ) )
+  {
+    printf( "Can't load model: %s or %s\n", _S( s1 ), _S( sI ) );
+    return 0;
+  }
+
+
+  normalize( filters1f, show3u, 1, 255, NORM_MINMAX, CV_8U );
+  _tigF.update( filters1f );
+
+  _svmSzIdxs = idx1i;
+  CV_Assert( _svmSzIdxs.size() > 1 && filters1f.size() == Size(_W, _W) && filters1f.type() == CV_32F );
+  _svmFilter = filters1f;
+
+  if( !matRead( s2, _svmReW1f ) || _svmReW1f.size() != Size( 2, (int) _svmSzIdxs.size() ) )
+  {
+    _svmReW1f = Mat();
+    return -1;
+  }
+  return 1;
+}
+
+void ObjectnessBING::predictBBoxSI( Mat &img3u, ValStructVec<float, Vec4i> &valBoxes, std::vector<int> &sz, int NUM_WIN_PSZ, bool fast )
+{
+  const int numSz = (int) _svmSzIdxs.size();
+  const int imgW = img3u.cols, imgH = img3u.rows;
+  valBoxes.reserve( 10000 );
+  sz.clear();
+  sz.reserve( 10000 );
+  for ( int ir = numSz - 1; ir >= 0; ir-- )
+  {
+    int r = _svmSzIdxs[ir];
+    int height = cvRound( pow( _base, r / _numT + _minT ) ), width = cvRound( pow( _base, r % _numT + _minT ) );
+    if( height > imgH * _base || width > imgW * _base )
+      continue;
+
+    height = min( height, imgH ), width = min( width, imgW );
+    Mat im3u, matchCost1f, mag1u;
+    resize( img3u, im3u, Size( cvRound( _W * imgW * 1.0 / width ), cvRound( _W * imgH * 1.0 / height ) ) );
+    gradientMag( im3u, mag1u );
+
+    matchCost1f = _tigF.matchTemplate( mag1u );
+
+    ValStructVec<float, Point> matchCost;
+    nonMaxSup( matchCost1f, matchCost, _NSS, NUM_WIN_PSZ, fast );
+
+    // Find true locations and match values
+    double ratioX = width / _W, ratioY = height / _W;
+    int iMax = min( matchCost.size(), NUM_WIN_PSZ );
+    for ( int i = 0; i < iMax; i++ )
+    {
+      float mVal = matchCost( i );
+      Point pnt = matchCost[i];
+      Vec4i box( cvRound( pnt.x * ratioX ), cvRound( pnt.y * ratioY ) );
+      box[2] = cvRound( min( box[0] + width, imgW ) );
+      box[3] = cvRound( min( box[1] + height, imgH ) );
+      box[0]++;
+      box[1]++;
+      valBoxes.pushBack( mVal, box );
+      sz.push_back( ir );
+    }
+  }
+
+}
+
+void ObjectnessBING::predictBBoxSII( ValStructVec<float, Vec4i> &valBoxes, const std::vector<int> &sz )
+{
+  int numI = valBoxes.size();
+  for ( int i = 0; i < numI; i++ )
+  {
+    const float* svmIIw = _svmReW1f.ptr<float>( sz[i] );
+    valBoxes( i ) = valBoxes( i ) * svmIIw[0] + svmIIw[1];
+  }
+  //valBoxes.sort();
+  // Descending order. At the top there are the values with higher
+  // values, ie more likely to have objects in the their corresponding rectangles.
+  valBoxes.sort( true );
+}
+
+// Get potential bounding boxes, each of which is represented by a Vec4i for (minX, minY, maxX, maxY).
+// The trained model should be prepared before calling this function: loadTrainedModel() or trainStageI() + trainStageII().
+// Use numDet to control the final number of proposed bounding boxes, and number of per size (scale and aspect ratio)
+void ObjectnessBING::getObjBndBoxes( Mat &img3u, ValStructVec<float, Vec4i> &valBoxes, int numDetPerSize )
+{
+  //CV_Assert_(filtersLoaded() , ("SVM filters should be initialized before getting object proposals\n"));
+  vecI sz;
+  predictBBoxSI( img3u, valBoxes, sz, numDetPerSize, false );
+  predictBBoxSII( valBoxes, sz );
+  return;
+}
+
+void ObjectnessBING::nonMaxSup( Mat &matchCost1f, ValStructVec<float, Point> &matchCost, int NSS, int maxPoint, bool fast )
+{
+  const int _h = matchCost1f.rows, _w = matchCost1f.cols;
+  Mat isMax1u = Mat::ones( _h, _w, CV_8U ), costSmooth1f;
+  ValStructVec<float, Point> valPnt;
+  matchCost.reserve( _h * _w );
+  valPnt.reserve( _h * _w );
+  if( fast )
+  {
+    blur( matchCost1f, costSmooth1f, Size( 3, 3 ) );
+    for ( int r = 0; r < _h; r++ )
+    {
+      const float* d = matchCost1f.ptr<float>( r );
+      const float* ds = costSmooth1f.ptr<float>( r );
+      for ( int c = 0; c < _w; c++ )
+        if( d[c] >= ds[c] )
+          valPnt.pushBack( d[c], Point( c, r ) );
+    }
+  }
+  else
+  {
+    for ( int r = 0; r < _h; r++ )
+    {
+      const float* d = matchCost1f.ptr<float>( r );
+      for ( int c = 0; c < _w; c++ )
+        valPnt.pushBack( d[c], Point( c, r ) );
+    }
+  }
+
+  valPnt.sort();
+  for ( int i = 0; i < valPnt.size(); i++ )
+  {
+    Point &pnt = valPnt[i];
+    if( isMax1u.at<BYTE>( pnt ) )
+    {
+      matchCost.pushBack( valPnt( i ), pnt );
+      for ( int dy = -NSS; dy <= NSS; dy++ )
+        for ( int dx = -NSS; dx <= NSS; dx++ )
+        {
+          Point neighbor = pnt + Point( dx, dy );
+          if( !CHK_IND( neighbor ) )
+            continue;
+          isMax1u.at<BYTE>( neighbor ) = false;
+        }
+    }
+    if( matchCost.size() >= maxPoint )
+      return;
+  }
+}
+
+void ObjectnessBING::gradientMag( Mat &imgBGR3u, Mat &mag1u )
+{
+  switch ( _Clr )
+  {
+    case MAXBGR:
+      gradientRGB( imgBGR3u, mag1u );
+      break;
+    case G:
+      gradientGray( imgBGR3u, mag1u );
+      break;
+    case HSV:
+      gradientHSV( imgBGR3u, mag1u );
+      break;
+    default:
+      printf( "Error: not recognized color space\n" );
+  }
+}
+
+void ObjectnessBING::gradientRGB( Mat &bgr3u, Mat &mag1u )
+{
+  const int H = bgr3u.rows, W = bgr3u.cols;
+  Mat Ix( H, W, CV_32S ), Iy( H, W, CV_32S );
+
+  // Left/right most column Ix
+  for ( int y = 0; y < H; y++ )
+  {
+    Ix.at<int>( y, 0 ) = bgrMaxDist( bgr3u.at<Vec3b>( y, 1 ), bgr3u.at<Vec3b>( y, 0 ) ) * 2;
+    Ix.at<int>( y, W - 1 ) = bgrMaxDist( bgr3u.at<Vec3b>( y, W - 1 ), bgr3u.at<Vec3b>( y, W - 2 ) ) * 2;
+  }
+
+  // Top/bottom most column Iy
+  for ( int x = 0; x < W; x++ )
+  {
+    Iy.at<int>( 0, x ) = bgrMaxDist( bgr3u.at<Vec3b>( 1, x ), bgr3u.at<Vec3b>( 0, x ) ) * 2;
+    Iy.at<int>( H - 1, x ) = bgrMaxDist( bgr3u.at<Vec3b>( H - 1, x ), bgr3u.at<Vec3b>( H - 2, x ) ) * 2;
+  }
+
+  // Find the gradient for inner regions
+  for ( int y = 0; y < H; y++ )
+  {
+    const Vec3b *dataP = bgr3u.ptr<Vec3b>( y );
+    for ( int x = 2; x < W; x++ )
+      Ix.at<int>( y, x - 1 ) = bgrMaxDist( dataP[x - 2], dataP[x] );  //  bgr3u.at<Vec3b>(y, x+1), bgr3u.at<Vec3b>(y, x-1));
+  }
+  for ( int y = 1; y < H - 1; y++ )
+  {
+    const Vec3b *tP = bgr3u.ptr<Vec3b>( y - 1 );
+    const Vec3b *bP = bgr3u.ptr<Vec3b>( y + 1 );
+    for ( int x = 0; x < W; x++ )
+      Iy.at<int>( y, x ) = bgrMaxDist( tP[x], bP[x] );
+  }
+  gradientXY( Ix, Iy, mag1u );
+}
+
+void ObjectnessBING::gradientGray( Mat &bgr3u, Mat &mag1u )
+{
+  Mat g1u;
+  cvtColor( bgr3u, g1u, COLOR_BGR2GRAY );
+  const int H = g1u.rows, W = g1u.cols;
+  Mat Ix( H, W, CV_32S ), Iy( H, W, CV_32S );
+
+  // Left/right most column Ix
+  for ( int y = 0; y < H; y++ )
+  {
+    Ix.at<int>( y, 0 ) = abs( g1u.at<BYTE>( y, 1 ) - g1u.at<BYTE>( y, 0 ) ) * 2;
+    Ix.at<int>( y, W - 1 ) = abs( g1u.at<BYTE>( y, W - 1 ) - g1u.at<BYTE>( y, W - 2 ) ) * 2;
+  }
+
+  // Top/bottom most column Iy
+  for ( int x = 0; x < W; x++ )
+  {
+    Iy.at<int>( 0, x ) = abs( g1u.at<BYTE>( 1, x ) - g1u.at<BYTE>( 0, x ) ) * 2;
+    Iy.at<int>( H - 1, x ) = abs( g1u.at<BYTE>( H - 1, x ) - g1u.at<BYTE>( H - 2, x ) ) * 2;
+  }
+
+  // Find the gradient for inner regions
+  for ( int y = 0; y < H; y++ )
+    for ( int x = 1; x < W - 1; x++ )
+      Ix.at<int>( y, x ) = abs( g1u.at<BYTE>( y, x + 1 ) - g1u.at<BYTE>( y, x - 1 ) );
+  for ( int y = 1; y < H - 1; y++ )
+    for ( int x = 0; x < W; x++ )
+      Iy.at<int>( y, x ) = abs( g1u.at<BYTE>( y + 1, x ) - g1u.at<BYTE>( y - 1, x ) );
+
+  gradientXY( Ix, Iy, mag1u );
+}
+
+void ObjectnessBING::gradientHSV( Mat &bgr3u, Mat &mag1u )
+{
+  Mat hsv3u;
+  cvtColor( bgr3u, hsv3u, COLOR_BGR2HSV );
+  const int H = hsv3u.rows, W = hsv3u.cols;
+  Mat Ix( H, W, CV_32S ), Iy( H, W, CV_32S );
+
+  // Left/right most column Ix
+  for ( int y = 0; y < H; y++ )
+  {
+    Ix.at<int>( y, 0 ) = vecDist3b( hsv3u.at<Vec3b>( y, 1 ), hsv3u.at<Vec3b>( y, 0 ) );
+    Ix.at<int>( y, W - 1 ) = vecDist3b( hsv3u.at<Vec3b>( y, W - 1 ), hsv3u.at<Vec3b>( y, W - 2 ) );
+  }
+
+  // Top/bottom most column Iy
+  for ( int x = 0; x < W; x++ )
+  {
+    Iy.at<int>( 0, x ) = vecDist3b( hsv3u.at<Vec3b>( 1, x ), hsv3u.at<Vec3b>( 0, x ) );
+    Iy.at<int>( H - 1, x ) = vecDist3b( hsv3u.at<Vec3b>( H - 1, x ), hsv3u.at<Vec3b>( H - 2, x ) );
+  }
+
+  // Find the gradient for inner regions
+  for ( int y = 0; y < H; y++ )
+    for ( int x = 1; x < W - 1; x++ )
+      Ix.at<int>( y, x ) = vecDist3b( hsv3u.at<Vec3b>( y, x + 1 ), hsv3u.at<Vec3b>( y, x - 1 ) ) / 2;
+  for ( int y = 1; y < H - 1; y++ )
+    for ( int x = 0; x < W; x++ )
+      Iy.at<int>( y, x ) = vecDist3b( hsv3u.at<Vec3b>( y + 1, x ), hsv3u.at<Vec3b>( y - 1, x ) ) / 2;
+
+  gradientXY( Ix, Iy, mag1u );
+}
+
+void ObjectnessBING::gradientXY( Mat &x1i, Mat &y1i, Mat &mag1u )
+{
+  const int H = x1i.rows, W = x1i.cols;
+  mag1u.create( H, W, CV_8U );
+  for ( int r = 0; r < H; r++ )
+  {
+    const int *x = x1i.ptr<int>( r ), *y = y1i.ptr<int>( r );
+    BYTE* m = mag1u.ptr<BYTE>( r );
+    for ( int c = 0; c < W; c++ )
+      m[c] = (BYTE) min( x[c] + y[c], 255 );   //((int)sqrt(sqr(x[c]) + sqr(y[c])), 255);
+  }
+}
+
+void ObjectnessBING::getObjBndBoxesForSingleImage( Mat img, ValStructVec<float, Vec4i> &finalBoxes, int numDetPerSize )
+{
+  ValStructVec<float, Vec4i> boxes;
+  finalBoxes.reserve( 10000 );
+
+  int scales[3] =
+  { 1, 3, 5 };
+  for ( int clr = MAXBGR; clr <= G; clr++ )
+  {
+    setColorSpace( clr );
+    loadTrainedModel();
+    CmTimer tm( "Predict" );
+    tm.Start();
+
+    getObjBndBoxes( img, boxes, numDetPerSize );
+    finalBoxes.append( boxes, scales[clr] );
+
+    tm.Stop();
+    printf( "Average time for predicting an image (%s) is %gs\n", _clrName[_Clr], tm.TimeInSeconds() );
+  }
+
+  //Write on file the total number and the list of rectangles returned by objectess, one for each row.
+
+  CmFile::MkDir( _bbResDir );
+  CStr fName = _bbResDir + "bb";
+  std::vector<Vec4i> sortedBB = finalBoxes.getSortedStructVal();
+  std::ofstream ofs;
+  ofs.open( _S( fName + ".txt" ), std::ofstream::out );
+  std::stringstream dim;
+  dim << sortedBB.size();
+  ofs << dim.str() << "\n";
+  for ( size_t k = 0; k < sortedBB.size(); k++ )
+  {
+    std::stringstream str;
+    str << sortedBB[k][0] << " " << sortedBB[k][1] << " " << sortedBB[k][2] << " " << sortedBB[k][3] << "\n";
+    ofs << str.str();
+  }
+  ofs.close();
+}
+
+struct MatchPathSeparator
+{
+  bool operator()( char ch ) const
+  {
+    return ch == '/';
+  }
+};
+
+std::string inline basename( std::string const& pathname )
+{
+  return std::string( std::find_if( pathname.rbegin(), pathname.rend(), MatchPathSeparator() ).base(), pathname.end() );
+}
+
+std::string inline removeExtension( std::string const& filename )
+{
+  std::string::const_reverse_iterator pivot = std::find( filename.rbegin(), filename.rend(), '.' );
+  return pivot == filename.rend() ? filename : std::string( filename.begin(), pivot.base() - 1 );
+}
+
+// Read matrix from binary file
+bool ObjectnessBING::matRead( const std::string& filename, Mat& _M )
+{
+  String filenamePlusExt( filename.c_str() );
+  filenamePlusExt += ".yml.gz";
+  FileStorage fs2( filenamePlusExt, FileStorage::READ );
+  Mat M;
+  fs2[String( removeExtension( basename( filename ) ).c_str() )] >> M;
+
+  M.copyTo( _M );
+  return true;
+}
+std::vector<float> ObjectnessBING::getobjectnessValues()
+{
+  return objectnessValues;
+}
+
+void ObjectnessBING::read()
+{
+
+}
+
+void ObjectnessBING::write() const
+{
+
+}
+
+bool ObjectnessBING::computeSaliencyImpl( const InputArray image, OutputArray objectnessBoundingBox )
+{
+  ValStructVec<float, Vec4i> finalBoxes;
+  getObjBndBoxesForSingleImage( image.getMat(), finalBoxes, 250 );
+
+  // List of rectangles returned by objectess function in descending order.
+  // At the top there are the rectangles with higher values, ie more
+  // likely to have objects in them.
+  std::vector<Vec4i> sortedBB = finalBoxes.getSortedStructVal();
+  Mat( sortedBB ).copyTo( objectnessBoundingBox );
+
+  // List of the rectangles' objectness value
+  unsigned long int valIdxesSize = (unsigned long int) finalBoxes.getvalIdxes().size();
+  objectnessValues.resize( valIdxesSize );
+  for ( uint i = 0; i < valIdxesSize; i++ )
+    objectnessValues[i] = finalBoxes.getvalIdxes()[i].first;
+
+  return true;
+}
+
+template<typename VT, typename ST>
+void ObjectnessBING::ValStructVec<VT, ST>::append( const ValStructVec<VT, ST> &newVals, int startV )
+{
+  int newValsSize = newVals.size();
+  for ( int i = 0; i < newValsSize; i++ )
+    pushBack( (float) ( ( i + 300 ) * startV ), newVals[i] );
+}
+
+template<typename VT, typename ST>
+void ObjectnessBING::ValStructVec<VT, ST>::sort( bool descendOrder /* = true */)
+{
+  if( descendOrder )
+    std::sort( valIdxes.begin(), valIdxes.end(), std::greater<std::pair<VT, int> >() );
+  else
+    std::sort( valIdxes.begin(), valIdxes.end(), std::less<std::pair<VT, int> >() );
+}
+
+template<typename VT, typename ST>
+const std::vector<ST>& ObjectnessBING::ValStructVec<VT, ST>::getSortedStructVal()
+{
+  sortedStructVals.resize( sz );
+  for ( int i = 0; i < sz; i++ )
+    sortedStructVals[i] = structVals[valIdxes[i].second];
+  return sortedStructVals;
+}
+
+template<typename VT, typename ST>
+std::vector<std::pair<VT, int> > ObjectnessBING::ValStructVec<VT, ST>::getvalIdxes()
+{
+  return valIdxes;
+}
+
+template<typename VT, typename ST>
+ObjectnessBING::ValStructVec<VT, ST>::ValStructVec()
+{
+  clear();
+}
+
+template<typename VT, typename ST>
+int ObjectnessBING::ValStructVec<VT, ST>::size() const
+{
+  return sz;
+}
+
+template<typename VT, typename ST>
+void ObjectnessBING::ValStructVec<VT, ST>::clear()
+{
+  sz = 0;
+  structVals.clear();
+  valIdxes.clear();
+}
+
+template<typename VT, typename ST>
+void ObjectnessBING::ValStructVec<VT, ST>::reserve( int resSz )
+{
+  clear();
+  structVals.reserve( resSz );
+  valIdxes.reserve( resSz );
+}
+
+template<typename VT, typename ST>
+void ObjectnessBING::ValStructVec<VT, ST>::pushBack( const VT& val, const ST& structVal )
+{
+  valIdxes.push_back( std::make_pair( val, sz ) );
+  structVals.push_back( structVal );
+  sz++;
+}
+
+template<typename VT, typename ST>
+const VT& ObjectnessBING::ValStructVec<VT, ST>::operator ()( int i ) const
+{
+  return valIdxes[i].first;
+}  // Should be called after sort
+
+template<typename VT, typename ST>
+const ST& ObjectnessBING::ValStructVec<VT, ST>::operator []( int i ) const
+{
+  return structVals[valIdxes[i].second];
+}  // Should be called after sort
+
+template<typename VT, typename ST>
+VT& ObjectnessBING::ValStructVec<VT, ST>::operator ()( int i )
+{
+  return valIdxes[i].first;
+}  // Should be called after sort
+
+template<typename VT, typename ST>
+ST& ObjectnessBING::ValStructVec<VT, ST>::operator []( int i )
+{
+  return structVals[valIdxes[i].second];
+}
+
+} /* namespace saliency */
+}/* namespace cv */

modules/saliency/src/motionSaliency.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+/**
+ * Motion Saliency
+ */
+
+} /* namespace saliency */
+}/* namespace cv */

modules/saliency/src/motionSaliencyBinWangApr2014.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+//TODO delete highgui include
+//#include <opencv2/highgui.hpp>
+
+#define thetaA_VAL 200
+#define thetaL_VAL 250
+
+namespace cv
+{
+namespace saliency
+{
+
+void MotionSaliencyBinWangApr2014::setImagesize( int W, int H )
+{
+  imageWidth = W;
+  imageHeight = H;
+}
+
+MotionSaliencyBinWangApr2014::MotionSaliencyBinWangApr2014()
+{
+  N_DS = 2;  // Number of template to be downsampled and used in lowResolutionDetection function
+  K = 3;  // Number of background model template
+  N = 4;   // NxN is the size of the block for downsampling in the lowlowResolutionDetection
+  alpha = (float) 0.01;  // Learning rate
+  L0 = 1000;  // Upper-bound values for C0 (efficacy of the first template (matrices) of backgroundModel
+  L1 = 800;  // Upper-bound values for C1 (efficacy of the second template (matrices) of backgroundModel
+  thetaL = thetaL_VAL;  // T0, T1 swap threshold
+  thetaA = thetaA_VAL;
+  gamma = 3;
+  neighborhoodCheck = true;
+
+  className = "BinWangApr2014";
+}
+
+bool MotionSaliencyBinWangApr2014::init()
+{
+  Size imgSize( imageWidth, imageHeight );
+  epslonPixelsValue = Mat( imgSize.height, imgSize.width, CV_32F, Scalar( 20 ) );
+  // Median of range [18, 80] advised in reference paper.
+  // Since data is even, the median is estimated using two values ​​that occupy
+  // the position (n / 2) and ((n / 2) +1) (choose their arithmetic mean).
+
+  potentialBackground = Mat( imgSize.height, imgSize.width, CV_32FC2, Scalar( NAN, 0 ) );
+
+  backgroundModel.resize( K + 1 );
+
+  for ( int i = 0; i < K + 1; i++ )
+  {
+    Mat* tmpm = new Mat;
+    tmpm->create( imgSize.height, imgSize.width, CV_32FC2 );
+    tmpm->setTo( Scalar( NAN, 0 ) );
+    Ptr<Mat> tmp = Ptr<Mat>( tmpm );
+    backgroundModel[i] = tmp;
+  }
+
+  return true;
+
+}
+
+MotionSaliencyBinWangApr2014::~MotionSaliencyBinWangApr2014()
+{
+
+}
+
+// classification (and adaptation) functions
+bool MotionSaliencyBinWangApr2014::fullResolutionDetection( const Mat& image2, Mat& highResBFMask )
+{
+  Mat image = image2.clone();
+
+  float currentPixelValue;
+  float currentEpslonValue;
+  bool backgFlag = false;
+
+  // Initially, all pixels are considered as foreground and then we evaluate with the background model
+  highResBFMask.create( image.rows, image.cols, CV_32F );
+  highResBFMask.setTo( 1 );
+
+  uchar* pImage;
+  float* pEpslon;
+  float* pMask;
+
+  // Scan all pixels of image
+  for ( int i = 0; i < image.rows; i++ )
+  {
+
+    pImage = image.ptr<uchar>( i );
+    pEpslon = epslonPixelsValue.ptr<float>( i );
+    pMask = highResBFMask.ptr<float>( i );
+    for ( int j = 0; j < image.cols; j++ )
+    {
+
+      backgFlag = false;
+      currentPixelValue = pImage[j];
+      currentEpslonValue = pEpslon[j];
+
+      int counter = 0;
+      for ( size_t z = 0; z < backgroundModel.size(); z++ )
+      {
+
+        counter += (int) backgroundModel[z]->ptr<Vec2f>( i )[j][1];
+      }
+
+      if( counter != 0 )  //if at least the first template is activated / initialized
+      {
+
+        // scan background model vector
+        for ( size_t z = 0; z < backgroundModel.size(); z++ )
+        {
+          float* currentB;
+          float* currentC;
+          currentB = & ( backgroundModel[z]->ptr<Vec2f>( i )[j][0] );
+          currentC = & ( backgroundModel[z]->ptr<Vec2f>( i )[j][1] );
+
+          //continue;
+          if( ( *currentC ) > 0 )  //The current template is active
+          {
+            // If there is a match with a current background template
+            if( abs( currentPixelValue - ( *currentB ) ) < currentEpslonValue && !backgFlag )
+            {
+              // The correspondence pixel in the  BF mask is set as background ( 0 value)
+              pMask[j] = 0;
+              if( ( *currentC < L0 && z == 0 ) || ( *currentC < L1 && z == 1 ) || ( z > 1 ) )
+              {
+                *currentC += 1;  // increment the efficacy of this template
+              }
+
+              *currentB = ( ( 1 - alpha ) * ( *currentB ) ) + ( alpha * currentPixelValue );  // Update the template value
+              backgFlag = true;
+            }
+            else
+            {
+              *currentC -= 1;  // decrement the efficacy of this template
+            }
+
+          }
+
+        }  // end "for" cicle of template vector
+
+      }
+      else
+      {
+        pMask[j] = 1;  //if the model of the current pixel is not yet initialized, we mark the pixels as foreground
+      }
+
+    }
+  }  // end "for" cicle of all image's pixels
+
+  return true;
+}
+
+bool MotionSaliencyBinWangApr2014::lowResolutionDetection( const Mat& image, Mat& lowResBFMask )
+{
+  std::vector<Mat> mv;
+  split( *backgroundModel[0], mv );
+
+  //if at least the first template is activated / initialized for all pixels
+  if( countNonZero( mv[1] ) > ( mv[1].cols * mv[1].rows ) / 2 )
+  {
+    float currentPixelValue;
+    float currentEpslonValue;
+    float currentB;
+    float currentC;
+
+    // Create a mask to select ROI in the original Image and Backgound model and at the same time compute the mean
+
+    Rect roi( Point( 0, 0 ), Size( N, N ) );
+    Scalar imageROImean;
+    Scalar backGModelROImean;
+    Mat currentModel;
+
+    // Initially, all pixels are considered as foreground and then we evaluate with the background model
+    lowResBFMask.create( image.rows, image.cols, CV_32F );
+    lowResBFMask.setTo( 1 );
+
+    // Scan all the ROI of original matrices
+    for ( int i = 0; i < ceil( (float) image.rows / N ); i++ )
+    {
+      if( ( roi.y + ( N - 1 ) ) <= ( image.rows - 1 ) )
+      {
+        // Reset original ROI dimension
+        roi = Rect( Point( roi.x, roi.y ), Size( N, N ) );
+      }
+
+      for ( int j = 0; j < ceil( (float) image.cols / N ); j++ )
+      {
+        // Compute the mean of image's block and epslonMatrix's block based on ROI
+        Mat roiImage = image( roi );
+        Mat roiEpslon = epslonPixelsValue( roi );
+        currentPixelValue = (float) mean( roiImage ).val[0];
+        currentEpslonValue = (float) mean( roiEpslon ).val[0];
+
+        // scan background model vector
+        for ( int z = 0; z < N_DS; z++ )
+        {
+          // Select the current template 2 channel matrix, select ROI and compute the mean for each channel separately
+          Mat roiTemplate = ( * ( backgroundModel[z] ) )( roi );
+          Scalar templateMean = mean( roiTemplate );
+          currentB = (float) templateMean[0];
+          currentC = (float) templateMean[1];
+
+          if( ( currentC ) > 0 )  //The current template is active
+          {
+            // If there is a match with a current background template
+            if( abs( currentPixelValue - ( currentB ) ) < currentEpslonValue )
+            {
+              // The correspondence pixel in the  BF mask is set as background ( 0 value)
+              rectangle( lowResBFMask, roi, Scalar( 0 ), FILLED );
+              break;
+            }
+          }
+        }
+        // Shift the ROI from left to right follow the block dimension
+        roi = roi + Point( N, 0 );
+        if( ( roi.x + ( roi.width - 1 ) ) > ( image.cols - 1 ) && ( roi.y + ( N - 1 ) ) <= ( image.rows - 1 ) )
+        {
+          roi = Rect( Point( roi.x, roi.y ), Size( abs( ( image.cols - 1 ) - roi.x ) + 1, N ) );
+        }
+        else if( ( roi.x + ( roi.width - 1 ) ) > ( image.cols - 1 ) && ( roi.y + ( N - 1 ) ) > ( image.rows - 1 ) )
+        {
+          roi = Rect( Point( roi.x, roi.y ), Size( abs( ( image.cols - 1 ) - roi.x ) + 1, abs( ( image.rows - 1 ) - roi.y ) + 1 ) );
+        }
+      }
+      //Shift the ROI from up to down follow the block dimension, also bringing it back to beginning of row
+      roi.x = 0;
+      roi.y += N;
+      if( ( roi.y + ( roi.height - 1 ) ) > ( image.rows - 1 ) )
+      {
+        roi = Rect( Point( roi.x, roi.y ), Size( N, abs( ( image.rows - 1 ) - roi.y ) + 1 ) );
+      }
+
+    }
+    return true;
+  }
+  else
+  {
+    lowResBFMask.create( image.rows, image.cols, CV_32F );
+    lowResBFMask.setTo( 1 );
+    return false;
+  }
+
+}
+
+bool inline pairCompare( std::pair<float, float> t, std::pair<float, float> t_plusOne )
+{
+
+  return ( t.second > t_plusOne.second );
+
+}
+
+// Background model maintenance functions
+bool MotionSaliencyBinWangApr2014::templateOrdering()
+{
+  std::vector<std::pair<float, float> > pixelTemplates( backgroundModel.size() );
+
+  Vec2f* bgModel_0P;
+  Vec2f* bgModel_1P;
+
+// Scan all pixels of image
+  for ( int i = 0; i < backgroundModel[0]->rows; i++ )
+  {
+    bgModel_0P = backgroundModel[0]->ptr<Vec2f>( i );
+    bgModel_1P = backgroundModel[1]->ptr<Vec2f>( i );
+    for ( int j = 0; j < backgroundModel[0]->cols; j++ )
+    {
+      // scan background model vector from T1 to Tk
+      for ( size_t z = 1; z < backgroundModel.size(); z++ )
+      {
+        Vec2f bgModel_zP = backgroundModel[z]->ptr<Vec2f>( i )[j];
+        // Fill vector of pairs
+        pixelTemplates[z - 1].first = bgModel_zP[0];  // Current B (background value)
+        pixelTemplates[z - 1].second = bgModel_zP[1];  // Current C (efficacy value)
+      }
+
+      //SORT template from T1 to Tk
+      std::sort( pixelTemplates.begin(), pixelTemplates.end(), pairCompare );
+
+      //REFILL CURRENT MODEL ( T1...Tk)
+      for ( size_t zz = 1; zz < backgroundModel.size(); zz++ )
+      {
+        backgroundModel[zz]->ptr<Vec2f>( i )[j][0] = pixelTemplates[zz - 1].first;  // Replace previous B with new ordered B value
+        backgroundModel[zz]->ptr<Vec2f>( i )[j][1] = pixelTemplates[zz - 1].second;  // Replace previous C with new ordered C value
+      }
+
+      // SORT Template T0 and T1
+      if( bgModel_1P[j][1] > thetaL && bgModel_0P[j][1] < thetaL )
+      {
+
+        // swap B value of T0 with B value of T1 (for current model)
+        swap( bgModel_0P[j][0], bgModel_1P[j][0] );
+
+        // set new C0 value for current model)
+        swap( bgModel_0P[j][1], bgModel_1P[j][1] );
+        bgModel_0P[j][1] = (float) gamma * thetaL;
+
+      }
+
+    }
+  }
+
+  return true;
+}
+bool MotionSaliencyBinWangApr2014::templateReplacement( const Mat& finalBFMask, const Mat& image )
+{
+  std::vector<Mat> temp;
+  split( *backgroundModel[0], temp );
+
+//if at least the first template is activated / initialized for all pixels
+  if( countNonZero( temp[1] ) <= ( temp[1].cols * temp[1].rows ) / 2 )
+  {
+    thetaA = 50;
+    thetaL = 150;
+    neighborhoodCheck = false;
+
+  }
+  else
+  {
+    thetaA = thetaA_VAL;
+    thetaL = thetaL_VAL;
+    neighborhoodCheck = true;
+  }
+
+  int roiSize = 3;  // FIXED ROI SIZE, not change until you first appropriately adjust the following controls in the EVALUATION section!
+  int countNonZeroElements = 0;
+  std::vector<Mat> mv;
+  Mat replicateCurrentBAMat( roiSize, roiSize, CV_32F );
+  Mat backgroundModelROI( roiSize, roiSize, CV_32F );
+  Mat diffResult( roiSize, roiSize, CV_32F );
+
+// Scan all pixels of finalBFMask and all pixels of others models (the dimension are the same)
+  const float* finalBFMaskP;
+  Vec2f* pbgP;
+  const uchar* imageP;
+  float* epslonP;
+  for ( int i = 0; i < finalBFMask.rows; i++ )
+  {
+    finalBFMaskP = finalBFMask.ptr<float>( i );
+    pbgP = potentialBackground.ptr<Vec2f>( i );
+    imageP = image.ptr<uchar>( i );
+    epslonP = epslonPixelsValue.ptr<float>( i );
+    for ( int j = 0; j < finalBFMask.cols; j++ )
+    {
+      /////////////////// MAINTENANCE of potentialBackground model ///////////////////
+      if( finalBFMaskP[j] == 1 )  // i.e. the corresponding frame pixel has been market as foreground
+      {
+        /* For the pixels with CA= 0, if the current frame pixel has been classified as foreground, its value
+         * will be loaded into BA and CA will be set to 1*/
+        if( pbgP[j][1] == 0 )
+        {
+          pbgP[j][0] = (float) imageP[j];
+          pbgP[j][1] = 1;
+        }
+
+        /*the distance between this pixel value and BA is calculated, and if this distance is smaller than
+         the decision threshold epslon, then CA is increased by 1, otherwise is decreased by 1*/
+        else if( abs( (float) imageP[j] - pbgP[j][0] ) < epslonP[j] )
+        {
+          pbgP[j][1] += 1;
+        }
+        else
+        {
+          pbgP[j][1] -= 1;
+        }
+        /*}*/  /////////////////// END of potentialBackground model MAINTENANCE///////////////////
+        /////////////////// EVALUATION of potentialBackground values ///////////////////
+        if( pbgP[j][1] > thetaA )
+        {
+          if( neighborhoodCheck )
+          {
+            // replicate currentBA value
+            replicateCurrentBAMat.setTo( pbgP[j][0] );
+
+            for ( size_t z = 0; z < backgroundModel.size(); z++ )
+            {
+              // Neighborhood of current pixel in the current background model template.
+              // The ROI is centered in the pixel coordinates
+
+              if( i > 0 && j > 0 && i < ( backgroundModel[z]->rows - 1 ) && j < ( backgroundModel[z]->cols - 1 ) )
+              {
+                split( *backgroundModel[z], mv );
+                backgroundModelROI = mv[0]( Rect( j - (int) floor( roiSize / 2 ), i - (int) floor( roiSize / 2 ), roiSize, roiSize ) );
+              }
+              else if( i == 0 && j == 0 )  // upper leftt
+              {
+                split( *backgroundModel[z], mv );
+                backgroundModelROI = mv[0]( Rect( j, i, (int) ceil( roiSize / 2 ), (int) ceil( roiSize / 2 ) ) );
+              }
+              else if( j == 0 && i > 0 && i < ( backgroundModel[z]->rows - 1 ) )  // middle left
+              {
+                split( *backgroundModel[z], mv );
+                backgroundModelROI = mv[0]( Rect( j, i - (int) floor( roiSize / 2 ), (int) ceil( roiSize / 2 ), roiSize ) );
+              }
+              else if( i == ( backgroundModel[z]->rows - 1 ) && j == 0 )  //down left
+              {
+                split( *backgroundModel[z], mv );
+                backgroundModelROI = mv[0]( Rect( j, i - (int) floor( roiSize / 2 ), (int) ceil( roiSize / 2 ), (int) ceil( roiSize / 2 ) ) );
+              }
+              else if( i == 0 && j > 0 && j < ( backgroundModel[z]->cols - 1 ) )  // upper - middle
+              {
+                split( *backgroundModel[z], mv );
+                backgroundModelROI = mv[0]( Rect( ( j - (int) floor( roiSize / 2 ) ), i, roiSize, (int) ceil( roiSize / 2 ) ) );
+              }
+              else if( i == ( backgroundModel[z]->rows - 1 ) && j > 0 && j < ( backgroundModel[z]->cols - 1 ) )  //down middle
+              {
+                split( *backgroundModel[z], mv );
+                backgroundModelROI = mv[0](
+                    Rect( j - (int) floor( roiSize / 2 ), i - (int) floor( roiSize / 2 ), roiSize, (int) ceil( roiSize / 2 ) ) );
+              }
+              else if( i == 0 && j == ( backgroundModel[z]->cols - 1 ) )  // upper right
+              {
+                split( *backgroundModel[z], mv );
+                backgroundModelROI = mv[0]( Rect( j - (int) floor( roiSize / 2 ), i, (int) ceil( roiSize / 2 ), (int) ceil( roiSize / 2 ) ) );
+              }
+              else if( j == ( backgroundModel[z]->cols - 1 ) && i > 0 && i < ( backgroundModel[z]->rows - 1 ) )  // middle - right
+              {
+                split( *backgroundModel[z], mv );
+                backgroundModelROI = mv[0](
+                    Rect( j - (int) floor( roiSize / 2 ), i - (int) floor( roiSize / 2 ), (int) ceil( roiSize / 2 ), roiSize ) );
+              }
+              else if( i == ( backgroundModel[z]->rows - 1 ) && j == ( backgroundModel[z]->cols - 1 ) )  // down right
+              {
+                split( *backgroundModel[z], mv );
+                backgroundModelROI = mv[0](
+                    Rect( j - (int) floor( roiSize / 2 ), i - (int) floor( roiSize / 2 ), (int) ceil( roiSize / 2 ), (int) ceil( roiSize / 2 ) ) );
+              }
+
+              /* Check if the value of current pixel BA in potentialBackground model is already contained in at least one of its neighbors'
+               * background model
+               */
+              resize( replicateCurrentBAMat, replicateCurrentBAMat, Size( backgroundModelROI.cols, backgroundModelROI.rows ), 0, 0, INTER_LINEAR );
+              resize( diffResult, diffResult, Size( backgroundModelROI.cols, backgroundModelROI.rows ), 0, 0, INTER_LINEAR );
+
+              absdiff( replicateCurrentBAMat, backgroundModelROI, diffResult );
+              threshold( diffResult, diffResult, epslonP[j], 255, THRESH_BINARY_INV );
+              countNonZeroElements = countNonZero( diffResult );
+
+              if( countNonZeroElements > 0 )
+              {
+                /////////////////// REPLACEMENT of backgroundModel template ///////////////////
+                //replace TA with current TK
+                backgroundModel[backgroundModel.size() - 1]->at<Vec2f>( i, j ) = potentialBackground.at<Vec2f>( i, j );
+                potentialBackground.at<Vec2f>( i, j )[0] = NAN;
+                potentialBackground.at<Vec2f>( i, j )[1] = 0;
+
+                break;
+              }
+            }  // end for backgroundModel size
+          }
+          else
+          {
+            backgroundModel[backgroundModel.size() - 1]->at<Vec2f>( i, j ) = potentialBackground.at<Vec2f>( i, j );
+            potentialBackground.at<Vec2f>( i, j )[0] = NAN;
+            potentialBackground.at<Vec2f>( i, j )[1] = 0;
+          }
+        }  // close if of EVALUATION
+      }  // end of  if( finalBFMask.at<uchar>( i, j ) == 1 )  // i.e. the corresponding frame pixel has been market as foreground
+
+    }  // end of second for
+  }  // end of first for
+
+  return true;
+}
+
+bool MotionSaliencyBinWangApr2014::computeSaliencyImpl( const InputArray image, OutputArray saliencyMap )
+{
+  Mat highResBFMask;
+  Mat lowResBFMask;
+  Mat not_lowResBFMask;
+  Mat noisePixelsMask;
+
+  fullResolutionDetection( image.getMat(), highResBFMask );
+  lowResolutionDetection( image.getMat(), lowResBFMask );
+
+
+// Compute the final background-foreground mask. One pixel is marked as foreground if and only if it is
+// foreground in both masks (full and low)
+  bitwise_and( highResBFMask, lowResBFMask, saliencyMap );
+
+  templateOrdering();
+  templateReplacement( saliencyMap.getMat(), image.getMat() );
+  templateOrdering();
+
+  return true;
+}
+
+}  // namespace saliency
+}  // namespace cv

modules/saliency/src/objectness.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+/**
+ * Objectness
+ */
+
+} /* namespace saliency*/
+}/* namespace cv */

modules/saliency/src/precomp.hpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#ifndef __OPENCV_PRECOMP_H__
+#define __OPENCV_PRECOMP_H__
+
+#include "opencv2/saliency.hpp"
+#include "opencv2/core/utility.hpp"
+#include "opencv2/core/private.hpp"
+
+#endif

modules/saliency/src/saliency.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+Saliency::~Saliency()
+{
+
+}
+
+Ptr<Saliency> Saliency::create( const String& saliencyType )
+{
+
+  return Algorithm::create<Saliency>( "SALIENCY." + saliencyType );
+
+}
+
+bool Saliency::computeSaliency( const InputArray image, OutputArray saliencyMap )
+{
+  if( image.empty() )
+    return false;
+
+  return computeSaliencyImpl( image, saliencyMap );
+}
+
+String Saliency::getClassName() const
+{
+  return className;
+}
+
+} /* namespace saliency */
+} /* namespace cv */

modules/saliency/src/saliency_init.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+#include "opencv2/saliency.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+CV_INIT_ALGORITHM( StaticSaliencySpectralResidual, "SALIENCY.SPECTRAL_RESIDUAL",
+                   obj.info()->addParam( obj, "resImWidth", obj.resImWidth); obj.info()->addParam( obj, "resImHeight", obj.resImHeight) );
+
+
+CV_INIT_ALGORITHM(
+    ObjectnessBING, "SALIENCY.BING",
+    obj.info()->addParam(obj, "_base", obj._base); obj.info()->addParam(obj, "_NSS", obj._NSS); obj.info()->addParam(obj, "_W", obj._W) );
+
+CV_INIT_ALGORITHM( MotionSaliencyBinWangApr2014, "SALIENCY.BinWangApr2014",
+                   obj.info()->addParam( obj, "imageWidth", obj.imageWidth); obj.info()->addParam( obj, "imageHeight", obj.imageHeight) );
+
+bool initModule_saliency( void )
+{
+  bool all = true;
+  all &= !StaticSaliencySpectralResidual_info_auto.name().empty();
+  //all &= !MotionSaliencySuBSENSE_info_auto.name().empty();
+  all &= !MotionSaliencyBinWangApr2014_info_auto.name().empty();
+  all &= !ObjectnessBING_info_auto.name().empty();
+
+  return all;
+}
+
+}  // namespace saliency
+}  // namespace cv

modules/saliency/src/staticSaliency.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+/**
+ * StaticSaliency
+ */
+
+bool StaticSaliency::computeBinaryMap( const Mat& saliencyMap, Mat& BinaryMap )
+{
+
+  Mat labels = Mat::zeros( saliencyMap.rows * saliencyMap.cols, 1, 1 );
+  Mat samples = Mat_<float>( saliencyMap.rows * saliencyMap.cols, 1 );
+  Mat centers;
+  TermCriteria terminationCriteria;
+  terminationCriteria.epsilon = 0.2;
+  terminationCriteria.maxCount = 1000;
+  terminationCriteria.type = TermCriteria::COUNT + TermCriteria::EPS;
+
+  int elemCounter = 0;
+  for ( int i = 0; i < saliencyMap.rows; i++ )
+  {
+    for ( int j = 0; j < saliencyMap.cols; j++ )
+    {
+      samples.at<float>( elemCounter, 0 ) = saliencyMap.at<float>( i, j );
+      elemCounter++;
+    }
+  }
+
+  kmeans( samples, 5, labels, terminationCriteria, 5, KMEANS_RANDOM_CENTERS, centers );
+
+  Mat outputMat = Mat_<float>( saliencyMap.size() );
+  int intCounter = 0;
+  for ( int x = 0; x < saliencyMap.rows; x++ )
+  {
+    for ( int y = 0; y < saliencyMap.cols; y++ )
+    {
+      outputMat.at<float>( x, y ) = centers.at<float>( labels.at<int>( intCounter, 0 ), 0 );
+      intCounter++;
+    }
+
+  }
+
+  //Convert
+  outputMat = outputMat * 255;
+  outputMat.convertTo( outputMat, CV_8U );
+
+  // adaptative thresholding using Otsu's method, to make saliency map binary
+  threshold( outputMat, BinaryMap, 0, 255, THRESH_BINARY | THRESH_OTSU );
+
+  return true;
+
+}
+
+}/* namespace saliency */
+}/* namespace cv */

modules/saliency/src/staticSaliencySpectralResidual.cpp

+/*M///////////////////////////////////////////////////////////////////////////////////////
+ //
+ //  IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
+ //
+ //  By downloading, copying, installing or using the software you agree to this license.
+ //  If you do not agree to this license, do not download, install,
+ //  copy or use the software.
+ //
+ //
+ //                           License Agreement
+ //                For Open Source Computer Vision Library
+ //
+ // Copyright (C) 2014, OpenCV Foundation, all rights reserved.
+ // Third party copyrights are property of their respective owners.
+ //
+ // Redistribution and use in source and binary forms, with or without modification,
+ // are permitted provided that the following conditions are met:
+ //
+ //   * Redistribution's of source code must retain the above copyright notice,
+ //     this list of conditions and the following disclaimer.
+ //
+ //   * Redistribution's in binary form must reproduce the above copyright notice,
+ //     this list of conditions and the following disclaimer in the documentation
+ //     and/or other materials provided with the distribution.
+ //
+ //   * The name of the copyright holders may not be used to endorse or promote products
+ //     derived from this software without specific prior written permission.
+ //
+ // This software is provided by the copyright holders and contributors "as is" and
+ // any express or implied warranties, including, but not limited to, the implied
+ // warranties of merchantability and fitness for a particular purpose are disclaimed.
+ // In no event shall the Intel Corporation or contributors be liable for any direct,
+ // indirect, incidental, special, exemplary, or consequential damages
+ // (including, but not limited to, procurement of substitute goods or services;
+ // loss of use, data, or profits; or business interruption) however caused
+ // and on any theory of liability, whether in contract, strict liability,
+ // or tort (including negligence or otherwise) arising in any way out of
+ // the use of this software, even if advised of the possibility of such damage.
+ //
+ //M*/
+
+#include "precomp.hpp"
+
+namespace cv
+{
+namespace saliency
+{
+
+/**
+ * SaliencySpectralResidual
+ */
+
+
+StaticSaliencySpectralResidual::StaticSaliencySpectralResidual()
+{
+  className = "SPECTRAL_RESIDUAL";
+  resImWidth = 64;
+  resImHeight = 64;
+}
+
+StaticSaliencySpectralResidual::~StaticSaliencySpectralResidual()
+{
+
+}
+
+void StaticSaliencySpectralResidual::read( const cv::FileNode& /*fn*/)
+{
+  //params.read( fn );
+}
+
+void StaticSaliencySpectralResidual::write( cv::FileStorage& /*fs*/) const
+{
+  //params.write( fs );
+}
+
+bool StaticSaliencySpectralResidual::computeSaliencyImpl( const InputArray image, OutputArray saliencyMap )
+{
+  Mat grayTemp, grayDown;
+  std::vector<Mat> mv;
+  Size resizedImageSize( resImWidth, resImHeight );
+
+  Mat realImage( resizedImageSize, CV_64F );
+  Mat imaginaryImage( resizedImageSize, CV_64F );
+  imaginaryImage.setTo( 0 );
+  Mat combinedImage( resizedImageSize, CV_64FC2 );
+  Mat imageDFT;
+  Mat logAmplitude;
+  Mat angle( resizedImageSize, CV_64F );
+  Mat magnitude( resizedImageSize, CV_64F );
+  Mat logAmplitude_blur, imageGR;
+
+  if( image.channels() == 3 )
+  {
+    cvtColor( image, imageGR, COLOR_BGR2GRAY );
+    resize( imageGR, grayDown, resizedImageSize, 0, 0, INTER_LINEAR );
+  }
+  else
+  {
+    resize( image, grayDown, resizedImageSize, 0, 0, INTER_LINEAR );
+  }
+
+  grayDown.convertTo( realImage, CV_64F );
+
+  mv.push_back( realImage );
+  mv.push_back( imaginaryImage );
+  merge( mv, combinedImage );
+  dft( combinedImage, imageDFT );
+  split( imageDFT, mv );
+
+  //-- Get magnitude and phase of frequency spectrum --//
+  cartToPolar( mv.at( 0 ), mv.at( 1 ), magnitude, angle, false );
+  log( magnitude, logAmplitude );
+  //-- Blur log amplitude with averaging filter --//
+  blur( logAmplitude, logAmplitude_blur, Size( 3, 3 ), Point( -1, -1 ), BORDER_DEFAULT );
+
+  exp( logAmplitude - logAmplitude_blur, magnitude );
+  //-- Back to cartesian frequency domain --//
+  polarToCart( magnitude, angle, mv.at( 0 ), mv.at( 1 ), false );
+  merge( mv, imageDFT );
+  dft( imageDFT, combinedImage, DFT_INVERSE );
+  split( combinedImage, mv );
+
+  cartToPolar( mv.at( 0 ), mv.at( 1 ), magnitude, angle, false );
+  GaussianBlur( magnitude, magnitude, Size( 5, 5 ), 8, 0, BORDER_DEFAULT );
+  magnitude = magnitude.mul( magnitude );
+
+  double minVal, maxVal;
+  minMaxLoc( magnitude, &minVal, &maxVal );
+
+  magnitude = magnitude / maxVal;
+  magnitude.convertTo( magnitude, CV_32F );
+
+  resize( magnitude, saliencyMap, image.size(), 0, 0, INTER_LINEAR );
+
+#ifdef SALIENCY_DEBUG
+  // visualize saliency map
+  imshow( "Saliency Map Interna", saliencyMap );
+#endif
+
+  return true;
+
+}
+
+} /* namespace saliency */
+}/* namespace cv */