Added tutorial. Modified somples, deleted test printing from binary_descriptor

modules/line_descriptor/doc/line_descriptor.rst

@@ -127,15 +127,20 @@ Each :math:`BD_j` can be obtained using the standard deviation vector :math:`S_j
 
 Once the LBD has been obtained, it must be converted into a binary form. For such purpose, we consider 32 possible pairs of BD inside it; each couple of BD is compared bit by bit and comparison generates an 8 bit string. Concatenating 32 comparison strings, we get the 256-bit final binary representation of a single LBD.
 
-
-Related Pages
--------------
-
-* `BinaryDescriptor Class <binary_descriptor.html>`_
-* `Matching with binary descriptors <matching.html>`_
-* `Drawing Function of Keylines and Matches <drawing_functions.html>`_
-
 References
 ----------
 
 .. [LBD] Zhang, Lilian, and Reinhard Koch. *An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency*, Journal of Visual Communication and Image Representation 24.7 (2013): 794-805.
+
+
+Summary
+-------
+		
+.. toctree::
+	:maxdepth: 2
+
+	binary_descriptor
+	matching
+	drawing_functions
+	tutorial
+

modules/line_descriptor/doc/tutorial.rst

-Tracking API
-============
+Line Features Tutorial
+======================
 
-.. highlight:: cpp
+In this tutorial it will be shown how to:
 
+* use the `BinaryDescriptor <binary_descriptor.html>`_ interface to extract lines and store them in *KeyLine* objects
+* use the same interface to compute descriptors for every extracted line
+* use the `BynaryDescriptorMatcher <matching.html#binarydescriptormatcher-class>`_ to determine matches among descriptors obtained from different images
 
-Long-term optical tracking API
-------------------------------
-Long-term optical tracking is one of most important issue for many computer vision applications in real world scenario.
-The development in this area is very fragmented and this API is an unique interface useful for plug several algorithms and compare them.
 
+Lines extraction and descriptors computation
+--------------------------------------------
 
-This algorithms start from a bounding box of the target and with their internal representation they avoid the drift during the tracking.
-These long-term trackers are able to evaluate online the quality of the location of the target in the new frame, without ground truth.
+In the following snippet of code, it is shown how to detect lines from an image. The LSD extractor is initialized with *LSD_REFINE_ADV* option; remaining parameters are left to their default values. A mask of ones is used in order to accept all extracted lines, which, at the end, are displayed using random colors for octave 0.
 
-There are three main components: the TrackerSampler, the TrackerFeatureSet and the TrackerModel. The first component is the object that computes the patches over the frame based on the last target location.
-The TrackerFeatureSet is the class that manages the Features, is possible plug many kind of these (HAAR, HOG, LBP, Feature2D, etc).
-The last component is the internal representation of the target, it is the appearence model. It stores all state candidates and compute the trajectory (the most likely target states). The class TrackerTargetState represents a possible state of the target.
-The TrackerSampler and the TrackerFeatureSet are the visual representation of the target, instead the TrackerModel is the statistical model.
+.. code-block:: cpp
 
+	#include <opencv2/line_descriptor.hpp>
 
-UML design:
------------
+	#include "opencv2/core/utility.hpp"
+	#include "opencv2/core/private.hpp"
+	#include <opencv2/imgproc.hpp>
+	#include <opencv2/features2d.hpp>
+	#include <opencv2/highgui.hpp>
 
+	#include <iostream>
 
-**Tracker diagram**
+	using namespace cv;
+	using namespace std;
 
-.. image:: pics/Trackerline.png
-   :width: 80%
-   :alt: Tracker diagram
-   :align: center
+	static const char* keys =
+	{ "{@image_path | | Image path }" };
 
+	static void help()
+	{
+	  cout << "\nThis example shows the functionalities of lines extraction " << "furnished by BinaryDescriptor class\n"
+		   << "Please, run this sample using a command in the form\n" << "./example_line_descriptor_lines_extraction <path_to_input_image>" << endl;
+	}
 
+	int main( int argc, char** argv )
+	{
+	  /* get parameters from comand line */
+	  CommandLineParser parser( argc, argv, keys );
+	  String image_path = parser.get<String>( 0 );
 
+	  if( image_path.empty() )
+	  {
+		help();
+		return -1;
+	  }
+
+	  /* load image */
+	  cv::Mat imageMat = imread( image_path, 1 );
+	  if( imageMat.data == NULL )
+	  {
+		std::cout << "Error, image could not be loaded. Please, check its path" << std::endl;
+	  }
+
+	  /* create a ramdom binary mask */
+	  cv::Mat mask = Mat::ones( imageMat.size(), CV_8UC1 );
+
+	  /* create a pointer to a BinaryDescriptor object with deafult parameters */
+	  Ptr<BinaryDescriptor> bd = BinaryDescriptor::createBinaryDescriptor();
+
+	  /* create a structure to store extracted lines */
+	  vector<KeyLine> lines;
+
+	  /* extract lines */
+	  bd->detect( imageMat, lines, mask );
+
+	  /* draw lines extracted from octave 0 */
+	  cv::Mat output = imageMat.clone();
+	  if( output.channels() == 1 )
+		cvtColor( output, output, COLOR_GRAY2BGR );
+	  for ( size_t i = 0; i < lines.size(); i++ )
+	  {
+		KeyLine kl = lines[i];
+		if( kl.octave == 0)
+		{
+		  /* get a random color */
+		  int R = ( rand() % (int) ( 255 + 1 ) );
+		  int G = ( rand() % (int) ( 255 + 1 ) );
+		  int B = ( rand() % (int) ( 255 + 1 ) );
+
+		  /* get extremes of line */
+		  Point pt1 = Point( kl.startPointX, kl.startPointY );
+		  Point pt2 = Point( kl.endPointX, kl.endPointY );
+
+		  /* draw line */
+		  line( output, pt1, pt2, Scalar( B, G, R ), 5 );
+		}
+
+	  }
+
+	  /* show lines on image */
+	  imshow( "Lines", output );
+	  waitKey();
+	}
+
+..
+
+This is the result obtained for famous cameraman image:
+
+.. image:: pics/cameraman_lines2.png
+    :width: 512px
+    :align: center
+    :height: 512px
+    :alt: alternate text
+
+
+Once keylines have been detected, it is possible to compute their descriptors as shown in the following:
+
+.. code-block:: cpp
+
+	#include <opencv2/line_descriptor.hpp>
+
+	#include "opencv2/core/utility.hpp"
+	#include "opencv2/core/private.hpp"
+	#include <opencv2/imgproc.hpp>
+	#include <opencv2/features2d.hpp>
+	#include <opencv2/highgui.hpp>
+
+	#include <iostream>
+
+	using namespace cv;
+
+	static const char* keys =
+	{ "{@image_path | | Image path }" };
+
+	static void help()
+	{
+	  std::cout << "\nThis example shows the functionalities of lines extraction " << "and descriptors computation furnished by BinaryDescriptor class\n"
+		        << "Please, run this sample using a command in the form\n" << "./example_line_descriptor_compute_descriptors <path_to_input_image>"
+		        << std::endl;
+	}
+
+	int main( int argc, char** argv )
+	{
+	  /* get parameters from command line */
+	  CommandLineParser parser( argc, argv, keys );
+	  String image_path = parser.get<String>( 0 );
+
+	  if( image_path.empty() )
+	  {
+		help();
+		return -1;
+	  }
+
+	  /* load image */
+	  cv::Mat imageMat = imread( image_path, 1 );
+	  if( imageMat.data == NULL )
+	  {
+		std::cout << "Error, image could not be loaded. Please, check its path" << std::endl;
+	  }
+
+	  /* create a binary mask */
+	  cv::Mat mask = Mat::ones( imageMat.size(), CV_8UC1 );
+
+	  /* create a pointer to a BinaryDescriptor object with default parameters */
+	  Ptr<BinaryDescriptor> bd = BinaryDescriptor::createBinaryDescriptor();
+
+	  /* compute lines */
+	  std::vector<KeyLine> keylines;
+	  bd->detect( imageMat, keylines, mask );
+
+	  /* compute descriptors */
+	  cv::Mat descriptors;
+	  bd->compute( imageMat, keylines, descriptors );
+
+	}
+
+..
+
+
+Matching among descriptors
+--------------------------
+
+If we have extracted descriptors from two different images, it is possible to search for matches among them. One way of doing it is matching exactly a descriptor to each input query descriptor, choosing the one at closest distance:
+
+.. code-block:: cpp
+
+	#include <opencv2/line_descriptor.hpp>
+
+	#include "opencv2/core/utility.hpp"
+	#include "opencv2/core/private.hpp"
+	#include <opencv2/imgproc.hpp>
+	#include <opencv2/features2d.hpp>
+	#include <opencv2/highgui.hpp>
+
+	#include <iostream>
+
+	using namespace cv;
+
+	static const char* keys =
+	{ "{@image_path1 | | Image path 1 }"
+		"{@image_path2 | | Image path 2 }" };
+
+	static void help()
+	{
+	  std::cout << "\nThis example shows the functionalities of lines extraction " << "and descriptors computation furnished by BinaryDescriptor class\n"
+		        << "Please, run this sample using a command in the form\n" << "./example_line_descriptor_compute_descriptors <path_to_input_image 1>"
+		        << "<path_to_input_image 2>" << std::endl;
+
+	}
+
+	int main( int argc, char** argv )
+	{
+	  /* get parameters from comand line */
+	  CommandLineParser parser( argc, argv, keys );
+	  String image_path1 = parser.get<String>( 0 );
+	  String image_path2 = parser.get<String>( 1 );
+
+	  if( image_path1.empty() || image_path2.empty() )
+	  {
+		help();
+		return -1;
+	  }
+
+	  /* load image */
+	  cv::Mat imageMat1 = imread( image_path1, 1 );
+	  cv::Mat imageMat2 = imread( image_path2, 1 );
+
+	  waitKey();
+	  if( imageMat1.data == NULL || imageMat2.data == NULL )
+	  {
+		std::cout << "Error, images could not be loaded. Please, check their path" << std::endl;
+	  }
+
+	  /* create binary masks */
+	  cv::Mat mask1 = Mat::ones( imageMat1.size(), CV_8UC1 );
+	  cv::Mat mask2 = Mat::ones( imageMat2.size(), CV_8UC1 );
+
+	  /* create a pointer to a BinaryDescriptor object with default parameters */
+	  Ptr<BinaryDescriptor> bd = BinaryDescriptor::createBinaryDescriptor();
+
+	  /* compute lines */
+	  std::vector<KeyLine> keylines1, keylines2;
+	  bd->detect( imageMat1, keylines1, mask1 );
+	  bd->detect( imageMat2, keylines2, mask2 );
+
+	  /* compute descriptors */
+	  cv::Mat descr1, descr2;
+	  bd->compute( imageMat1, keylines1, descr1 );
+	  bd->compute( imageMat2, keylines2, descr2 );
+
+	  /* create a BinaryDescriptorMatcher object */
+	  Ptr<BinaryDescriptorMatcher> bdm = BinaryDescriptorMatcher::createBinaryDescriptorMatcher();
+
+	  /* require match */
+	  std::vector<DMatch> matches;
+	  bdm->match( descr1, descr2, matches );
+
+	  /* plot matches */
+	  cv::Mat outImg;
+	  std::vector<char> mask( matches.size(), 1 );
+	  drawLineMatches( imageMat1, keylines1, imageMat2, keylines2, matches, outImg, Scalar::all( -1 ), Scalar::all( -1 ), mask,
+		               DrawLinesMatchesFlags::DEFAULT );
+
+	  imshow( "Matches", outImg );
+	  waitKey();
+	}
+
+..
+
+
+
+Sometimes, we could be interested in searching for the closest *k* descriptors, given an input one. This requires to modify slightly previous code:
+
+.. code-block:: cpp
+
+	  /* prepare a structure to host matches */
+	  std::vector<std::vector<DMatch> > matches;
+
+	  /* require knn match */
+	  bdm->knnMatch( descr1, descr2, matches, 6 );
+..
+
+In the above example, the closest 6 descriptors are returned for every query. In some cases, we could have a search radius and look for all descriptors distant at the most *r* from input query. Previous code must me modified:
+
+.. code-block:: cpp
+
+
+	/* prepare a structure to host matches */
+	std::vector<std::vector<DMatch> > matches;
+
+	/* compute matches */
+	bdm->radiusMatch( queries, matches, 30 );
+
+..
+
+Here's an example om matching among descriptors extratced from original cameraman image and its downsampled (and blurred) version:
+
+.. image:: pics/matching2.png
+    :width: 765px
+    :align: center
+    :height: 540px
+    :alt: alternate text
+
+Querying internal database
+--------------------------
+
+The `BynaryDescriptorMatcher <matching.html#binarydescriptormatcher-class>`_ class, owns an internal database that can be populated with descriptors extracted from different images and queried using one of the modalities described in previous section. 
+Population of internal dataset can be done using the *add* function; such function doesn't directly add new data to database, but it just stores it them locally. The real update happens when function *train* is invoked or when any querying function is executed, since each of them invokes *train* before querying. 
+When queried, internal database not only returns required descriptors, but, for every returned match, it is able to say which image matched descriptor was extracted from.
+An example of internal dataset usage is described in the following code; after adding locally new descriptors, a radius search is invoked. This provokes local data to be transferred to dataset, which, in turn, is then queried.
+
+.. code-block:: cpp
+
+	#include <opencv2/line_descriptor.hpp>
+
+	#include "opencv2/core/utility.hpp"
+	#include "opencv2/core/private.hpp"
+	#include <opencv2/imgproc.hpp>
+	#include <opencv2/features2d.hpp>
+	#include <opencv2/highgui.hpp>
+
+	#include <iostream>
+	#include <vector>
+
+	using namespace cv;
+
+	static const std::string images[] =
+	{ "cameraman.jpg", "church.jpg", "church2.png", "einstein.jpg", "stuff.jpg" };
+
+	static const char* keys =
+	{ "{@image_path | | Image path }" };
+
+	static void help()
+	{
+	  std::cout << "\nThis example shows the functionalities of radius matching " << "Please, run this sample using a command in the form\n"
+		        << "./example_line_descriptor_radius_matching <path_to_input_images>/" << std::endl;
+	}
+
+	int main( int argc, char** argv )
+	{
+	  /* get parameters from comand line */
+	  CommandLineParser parser( argc, argv, keys );
+	  String pathToImages = parser.get<String>( 0 );
+
+	  /* create structures for hosting KeyLines and descriptors */
+	  int num_elements = sizeof ( images ) / sizeof ( images[0] );
+	  std::vector<Mat> descriptorsMat;
+	  std::vector<std::vector<KeyLine> > linesMat;
+
+	  /*create a pointer to a BinaryDescriptor object */
+	  Ptr<BinaryDescriptor> bd = BinaryDescriptor::createBinaryDescriptor();
+
+	  /* compute lines and descriptors */
+	  for ( int i = 0; i < num_elements; i++ )
+	  {
+		/* get path to image */
+		std::stringstream image_path;
+		image_path << pathToImages << images[i];
+
+		/* load image */
+		Mat loadedImage = imread( image_path.str().c_str(), 1 );
+		if( loadedImage.data == NULL )
+		{
+		  std::cout << "Could not load images." << std::endl;
+		  help();
+		  exit( -1 );
+		}
+
+		/* compute lines and descriptors */
+		std::vector<KeyLine> lines;
+		Mat computedDescr;
+		bd->detect( loadedImage, lines );
+		bd->compute( loadedImage, lines, computedDescr );
+
+		descriptorsMat.push_back( computedDescr );
+		linesMat.push_back( lines );
+
+	  }
+
+	  /* compose a queries matrix */
+	  Mat queries;
+	  for ( size_t j = 0; j < descriptorsMat.size(); j++ )
+	  {
+		if( descriptorsMat[j].rows >= 5 )
+		  queries.push_back( descriptorsMat[j].rowRange( 0, 5 ) );
+
+		else if( descriptorsMat[j].rows > 0 && descriptorsMat[j].rows < 5 )
+		  queries.push_back( descriptorsMat[j] );
+	  }
+
+	  std::cout << "It has been generated a matrix of " << queries.rows << " descriptors" << std::endl;
+
+	  /* create a BinaryDescriptorMatcher object */
+	  Ptr<BinaryDescriptorMatcher> bdm = BinaryDescriptorMatcher::createBinaryDescriptorMatcher();
+
+	  /* populate matcher */
+	  bdm->add( descriptorsMat );
+
+	  /* compute matches */
+	  std::vector<std::vector<DMatch> > matches;
+	  bdm->radiusMatch( queries, matches, 30 );
+
+	  /* print matches */
+	  for ( size_t q = 0; q < matches.size(); q++ )
+	  {
+		for ( size_t m = 0; m < matches[q].size(); m++ )
+		{
+		  DMatch dm = matches[q][m];
+		  std::cout << "Descriptor: " << q << " Image: " << dm.imgIdx << " Distance: " << dm.distance << std::endl;
+		}
+	  }
+	}
+..

modules/line_descriptor/samples/knn_matching.cpp

@@ -189,15 +189,5 @@ int main( int argc, char** argv )
   /* require knn match */
   bdm->knnMatch( descr1, descr2, matches, 6 );
 
-  /* visualize matches and Hamming distances */
-  for ( size_t v = 0; v < matches.size(); v++ )
-  {
-    for ( size_t m = 0; m < matches[v].size(); m++ )
-    {
-      DMatch dm = matches[v][m];
-      std::cout << dm.queryIdx << " " << dm.trainIdx << " " << dm.distance << std::endl;
-    }
-  }
-
 }
 

modules/line_descriptor/src/binary_descriptor.cpp

@@ -699,17 +699,6 @@ int BinaryDescriptor::OctaveKeyLines( ScaleLines &keyLines )
 
   }
 
-  Mat appoggio = octaveImages[0].clone();
-  cvtColor(appoggio, appoggio, COLOR_GRAY2BGR);
-  for(size_t t = 0; t<extractedLines[0].size(); t++)
-  {
-    Vec4i v = extractedLines[0][t];
-    line(appoggio, Point(v[0], v[1]), Point(v[2], v[3]), Scalar(255, 0, 0), 3);
-    std::cout<< std::endl << prec[t] << " " << w_idth[t] /*<< " " << nfa[t]*/ << std::endl;
-    imshow("M", appoggio);
-    //waitKey();
-  }
-
   /* prepare a vector to store octave information associated to extracted lines */
   std::vector<OctaveLine> octaveLines( numOfFinalLine );
 

modules/tracking/doc/tracking.rst

@@ -77,7 +77,7 @@ Tracker classes:
 ----------------
 
 .. toctree::
-    :maxdepth: 2
+	:maxdepth: 2
 
     tracker_algorithms
     common_interfaces_tracker

modules/tracking/doc/tracking.rst~

+Tracking API
+============
+
+.. highlight:: cpp
+
+
+Long-term optical tracking API
+------------------------------
+Long-term optical tracking is one of most important issue for many computer vision applications in real world scenario.
+The development in this area is very fragmented and this API is an unique interface useful for plug several algorithms and compare them.
+This work is partially based on [AAM]_ and [AMVOT]_.
+
+This algorithms start from a bounding box of the target and with their internal representation they avoid the drift during the tracking.
+These long-term trackers are able to evaluate online the quality of the location of the target in the new frame, without ground truth.
+
+There are three main components: the TrackerSampler, the TrackerFeatureSet and the TrackerModel. The first component is the object that computes the patches over the frame based on the last target location.
+The TrackerFeatureSet is the class that manages the Features, is possible plug many kind of these (HAAR, HOG, LBP, Feature2D, etc).
+The last component is the internal representation of the target, it is the appearence model. It stores all state candidates and compute the trajectory (the most likely target states). The class TrackerTargetState represents a possible state of the target.
+The TrackerSampler and the TrackerFeatureSet are the visual representation of the target, instead the TrackerModel is the statistical model.
+
+A recent benchmark between these algorithms can be found in [OOT]_.
+
+UML design:
+-----------
+
+**General diagram**
+
+.. image:: pics/package.png
+   :width: 50%
+   :alt: General diagram
+   :align: center
+
+**Tracker diagram**
+
+.. image:: pics/Tracker.png
+   :width: 80%
+   :alt: Tracker diagram
+   :align: center
+
+**TrackerSampler diagram**
+
+.. image:: pics/TrackerSampler.png
+   :width: 100%
+   :alt: TrackerSampler diagram
+   :align: center
+
+**TrackerFeatureSet diagram**
+
+.. image:: pics/TrackerFeature.png
+   :width: 100%
+   :alt: TrackerFeatureSet diagram
+   :align: center
+
+**TrackerModel diagram**
+
+.. image:: pics/TrackerModel.png
+   :width: 100%
+   :alt: TrackerModel diagram
+   :align: center
+
+To see how API works, try tracker demo:
+https://github.com/lenlen/opencv/blob/tracking_api/samples/cpp/tracker.cpp
+
+
+.. note:: This Tracking API has been designed with PlantUML. If you modify this API please change UML files under modules/tracking/misc/
+
+The following reference was used in the API
+
+.. [AAM] S Salti, A Cavallaro, L Di Stefano, Adaptive Appearance Modeling for Video Tracking: Survey and Evaluation, IEEE Transactions on Image Processing, Vol. 21, Issue 10, October 2012, pp. 4334-4348
+
+.. [AMVOT] X Li, W Hu, C Shen, Z Zhang, A Dick, A van den Hengel, A Survey of Appearance Models in Visual Object Tracking, ACM Transactions on Intelligent Systems and Technology (TIST), 2013
+
+.. [OOT] Yi Wu and Jongwoo Lim and Ming-Hsuan Yang, Online Object Tracking: A Benchmark, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013
+
+
+Tracker classes:
+----------------
+
+		.. toctree::
+			:maxdepth: 2
+
+    tracker_algorithms
+    common_interfaces_tracker
+    common_interfaces_tracker_sampler
+    common_interfaces_tracker_feature_set
+    common_interfaces_tracker_model