Merge pull request #1257 from kurnianggoro:facelandmark

GSOC17 - Facemark API (#1257)

* Initial commit of facemark API

Initial structure of the facemark API and AAM header

* make training function as virtual

* Add: dataset parser

* Bug fix: clear the container before add points

* Add: AAM training - procrustes analysis

* Add AAM model

* Added training function for AAM

* Building bot fixes: remove training overload, explicit cast to float for atof

* + add dependency: imgcodecs

* Build bot fixes: add imgproc.hpp and type casting

* Building bot fix: type casting

* fixing the AAM training to match with Matlab version

fewer model parameters, change the image warp method, change the feature extraction method

* add: AAM fitting

added several functionalities for fitting

* fix warings

* Add: transformation for the initial fitting

* add sample file for aam implementation

* fix warning

* Add LFB Header

* loadTrainingData: Throw an error message if file not exist

* add: LBF prepare training data

* add: data augmentation

* change to double

* add: getMeanShape

* shuffling the dataset and parameters initialization

* add: initial structure of LBF class

* add: getDeltaShapes

Difference between the current shape and the desired shape

* add: random forest training

* generate lbf features

* global regression

* save training data

* fix the parameter initialization

* set the default parameters

* add: initial version of lbf sample

* update the current shape

* compute error

* add: prediction function

* fix some warnings

* fitting function

the result is mis-aligned, shuould be double checked

* add: fitting in the demo

* add dependencies

* Add: tutorial

* add: load model

* fixing training

* use user defined face detector

* Documents, tests, and samples

* Allow custom parameters

* Cleaning up

* Custom parameters for default detector, training, and get custom data

* AAM scales

* minor fixes , update the opencv_extra files

* change path to lbp cascade

* face: avoid memory leaks

* utilize the filestorage for the model, fixing some minor issues

* remove the liblinear dependency

* fix the aam test, avoiding to write any files

* use RNG and changes the test files

modules/face/include/opencv2/face.hpp

@@ -40,7 +40,7 @@ the use of this software, even if advised of the possibility of such damage.
 #define __OPENCV_FACE_HPP__
 
 /**
-@defgroup face Face Recognition
+@defgroup face Face Analysis
 
 - @ref face_changelog
 - @ref tutorial_face_main
@@ -374,5 +374,7 @@ protected:
 }}
 
 #include "opencv2/face/facerec.hpp"
-
+#include "opencv2/face/facemark.hpp"
+#include "opencv2/face/facemarkLBF.hpp"
+#include "opencv2/face/facemarkAAM.hpp"
 #endif

modules/face/include/opencv2/face/facemarkAAM.hpp

+/*
+By downloading, copying, installing or using the software you agree to this
+license. If you do not agree to this license, do not download, install,
+copy or use the software.
+                          License Agreement
+               For Open Source Computer Vision Library
+                       (3-clause BSD License)
+Copyright (C) 2013, OpenCV Foundation, all rights reserved.
+Third party copyrights are property of their respective owners.
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+  * Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+  * Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+  * Neither the names of the copyright holders nor the names of the contributors
+    may be used to endorse or promote products derived from this software
+    without specific prior written permission.
+This software is provided by the copyright holders and contributors "as is" and
+any express or implied warranties, including, but not limited to, the implied
+warranties of merchantability and fitness for a particular purpose are
+disclaimed. In no event shall copyright holders or contributors be liable for
+any direct, indirect, incidental, special, exemplary, or consequential damages
+(including, but not limited to, procurement of substitute goods or services;
+loss of use, data, or profits; or business interruption) however caused
+and on any theory of liability, whether in contract, strict liability,
+or tort (including negligence or otherwise) arising in any way out of
+the use of this software, even if advised of the possibility of such damage.
+
+This file was part of GSoC Project: Facemark API for OpenCV
+Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
+Student: Laksono Kurnianggoro
+Mentor: Delia Passalacqua
+*/
+
+#ifndef __OPENCV_FACEMARK_AAM_HPP__
+#define __OPENCV_FACEMARK_AAM_HPP__
+
+#include "opencv2/face/facemark.hpp"
+namespace cv {
+namespace face {
+
+//! @addtogroup face
+//! @{
+
+    class CV_EXPORTS_W FacemarkAAM : public Facemark
+    {
+    public:
+        struct CV_EXPORTS Params
+        {
+            /**
+            * \brief Constructor
+            */
+            Params();
+
+            /**
+            * \brief Read parameters from file, currently unused
+            */
+            void read(const FileNode& /*fn*/);
+
+            /**
+            * \brief Read parameters from file, currently unused
+            */
+            void write(FileStorage& /*fs*/) const;
+
+            std::string model_filename;
+            int m;
+            int n;
+            int n_iter;
+            bool verbose;
+            bool save_model;
+            int max_m, max_n, texture_max_m;
+            std::vector<float>scales;
+        };
+
+        /**
+        * \brief Optional parameter for fitting process.
+        */
+        struct CV_EXPORTS Config
+        {
+            Config( Mat rot = Mat::eye(2,2,CV_32F),
+                    Point2f trans = Point2f(0.0,0.0),
+                    float scaling = 1.0,
+                    int scale_id=0
+            );
+
+            Mat R;
+            Point2f t;
+            float scale;
+            int model_scale_idx;
+
+        };
+
+        /**
+        * \brief Data container for the facemark::getData function
+        */
+        struct CV_EXPORTS Data
+        {
+            std::vector<Point2f> s0;
+        };
+
+        /**
+        * \brief The model of AAM Algorithm
+        */
+        struct CV_EXPORTS Model
+        {
+            int npts; //!<  unused delete
+            int max_n; //!<  unused delete
+            std::vector<float>scales;
+            //!<  defines the scales considered to build the model
+
+            /*warping*/
+            std::vector<Vec3i> triangles;
+            //!<  each element contains 3 values, represent index of facemarks that construct one triangle (obtained using delaunay triangulation)
+
+            struct Texture{
+                int max_m; //!<  unused delete
+                Rect resolution;
+                //!<  resolution of the current scale
+                Mat A;
+                //!<  gray values from all face region in the dataset, projected in PCA space
+                Mat A0;
+                //!<  average of gray values from all face region in the dataset
+                Mat AA;
+                //!<  gray values from all erorded face region in the dataset, projected in PCA space
+                Mat AA0;
+                //!<  average of gray values from all erorded face region in the dataset
+
+                std::vector<std::vector<Point> > textureIdx;
+                //!<  index for warping of each delaunay triangle region constructed by 3 facemarks
+                std::vector<Point2f> base_shape;
+                //!<  basic shape, normalized to be fit in an image with current detection resolution
+                std::vector<int> ind1;
+                //!<  index of pixels for mapping process to obtains the grays values of face region
+                std::vector<int> ind2;
+                //!<  index of pixels for mapping process to obtains the grays values of eroded face region
+            };
+            std::vector<Texture> textures;
+            //!<  a container to holds the texture data for each scale of fitting
+
+            /*shape*/
+            std::vector<Point2f> s0;
+            //!<  the basic shape obtained from training dataset
+            Mat S,Q;
+            //!<  the encoded shapes from training data
+
+        };
+
+        //!<  initializer
+        static Ptr<FacemarkAAM> create(const FacemarkAAM::Params &parameters = FacemarkAAM::Params() );
+        virtual ~FacemarkAAM() {}
+
+    }; /* AAM */
+
+//! @}
+
+} /* namespace face */
+} /* namespace cv */
+#endif

modules/face/include/opencv2/face/facemarkLBF.hpp

+/*
+By downloading, copying, installing or using the software you agree to this
+license. If you do not agree to this license, do not download, install,
+copy or use the software.
+                          License Agreement
+               For Open Source Computer Vision Library
+                       (3-clause BSD License)
+Copyright (C) 2013, OpenCV Foundation, all rights reserved.
+Third party copyrights are property of their respective owners.
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+  * Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+  * Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+  * Neither the names of the copyright holders nor the names of the contributors
+    may be used to endorse or promote products derived from this software
+    without specific prior written permission.
+This software is provided by the copyright holders and contributors "as is" and
+any express or implied warranties, including, but not limited to, the implied
+warranties of merchantability and fitness for a particular purpose are
+disclaimed. In no event shall copyright holders or contributors be liable for
+any direct, indirect, incidental, special, exemplary, or consequential damages
+(including, but not limited to, procurement of substitute goods or services;
+loss of use, data, or profits; or business interruption) however caused
+and on any theory of liability, whether in contract, strict liability,
+or tort (including negligence or otherwise) arising in any way out of
+the use of this software, even if advised of the possibility of such damage.
+
+This file was part of GSoC Project: Facemark API for OpenCV
+Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
+Student: Laksono Kurnianggoro
+Mentor: Delia Passalacqua
+*/
+
+#ifndef __OPENCV_FACEMARK_LBF_HPP__
+#define __OPENCV_FACEMARK_LBF_HPP__
+
+#include "opencv2/face/facemark.hpp"
+
+namespace cv {
+namespace face {
+
+//! @addtogroup face
+//! @{
+
+    class CV_EXPORTS_W FacemarkLBF : public Facemark
+    {
+    public:
+        struct CV_EXPORTS Params
+        {
+            /**
+            * \brief Constructor
+            */
+            Params();
+
+            double shape_offset;
+            //!<  offset for the loaded face landmark points
+            String cascade_face;
+            //!<  filename of the face detector model
+            bool verbose;
+            //!< show the training print-out
+
+            int n_landmarks;
+            //!<  number of landmark points
+            int initShape_n;
+            //!<  multiplier for augment the training data
+
+            int stages_n;
+            //!<  number of refinement stages
+            int tree_n;
+            //!<  number of tree in the model for each landmark point refinement
+            int tree_depth;
+            //!<  the depth of decision tree, defines the size of feature
+            double bagging_overlap;
+            //!<  overlap ratio for training the LBF feature
+
+            std::string model_filename;
+            //!<  filename where the trained model will be saved
+            bool save_model; //!< flag to save the trained model or not
+            unsigned int seed; //!< seed for shuffling the training data
+
+            std::vector<int> feats_m;
+            std::vector<double> radius_m;
+            std::vector<int> pupils[2];
+            //!<  index of facemark points on pupils of left and right eye
+
+            Rect detectROI;
+
+            void read(const FileNode& /*fn*/);
+            void write(FileStorage& /*fs*/) const;
+
+        };
+
+        class BBox {
+        public:
+            BBox();
+            ~BBox();
+            BBox(double x, double y, double w, double h);
+
+            cv::Mat project(const cv::Mat &shape) const;
+            cv::Mat reproject(const cv::Mat &shape) const;
+
+            double x, y;
+            double x_center, y_center;
+            double x_scale, y_scale;
+            double width, height;
+        };
+
+        static Ptr<FacemarkLBF> create(const FacemarkLBF::Params &parameters = FacemarkLBF::Params() );
+        virtual ~FacemarkLBF(){};
+    }; /* LBF */
+
+//! @}
+
+} /* namespace face */
+}/* namespace cv */
+
+#endif

modules/face/samples/facemark_demo_lbf.cpp

+/*
+By downloading, copying, installing or using the software you agree to this
+license. If you do not agree to this license, do not download, install,
+copy or use the software.
+                          License Agreement
+               For Open Source Computer Vision Library
+                       (3-clause BSD License)
+Copyright (C) 2013, OpenCV Foundation, all rights reserved.
+Third party copyrights are property of their respective owners.
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+  * Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+  * Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+  * Neither the names of the copyright holders nor the names of the contributors
+    may be used to endorse or promote products derived from this software
+    without specific prior written permission.
+This software is provided by the copyright holders and contributors "as is" and
+any express or implied warranties, including, but not limited to, the implied
+warranties of merchantability and fitness for a particular purpose are
+disclaimed. In no event shall copyright holders or contributors be liable for
+any direct, indirect, incidental, special, exemplary, or consequential damages
+(including, but not limited to, procurement of substitute goods or services;
+loss of use, data, or profits; or business interruption) however caused
+and on any theory of liability, whether in contract, strict liability,
+or tort (including negligence or otherwise) arising in any way out of
+the use of this software, even if advised of the possibility of such damage.
+
+This file was part of GSoC Project: Facemark API for OpenCV
+Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
+Student: Laksono Kurnianggoro
+Mentor: Delia Passalacqua
+*/
+
+/*----------------------------------------------
+ * Usage:
+ * facemark_demo_lbf <face_cascade_model> <saved_model_filename> <training_images> <annotation_files> [test_files]
+ *
+ * Example:
+ * facemark_demo_lbf ../face_cascade.xml ../LBF.model ../images_train.txt ../points_train.txt ../test.txt
+ *
+ * Notes:
+ * the user should provides the list of training images_train
+ * accompanied by their corresponding landmarks location in separated files.
+ * example of contents for images_train.txt:
+ * ../trainset/image_0001.png
+ * ../trainset/image_0002.png
+ * example of contents for points_train.txt:
+ * ../trainset/image_0001.pts
+ * ../trainset/image_0002.pts
+ * where the image_xxxx.pts contains the position of each face landmark.
+ * example of the contents:
+ *  version: 1
+ *  n_points:  68
+ *  {
+ *  115.167660 220.807529
+ *  116.164839 245.721357
+ *  120.208690 270.389841
+ *  ...
+ *  }
+ * example of the dataset is available at https://ibug.doc.ic.ac.uk/download/annotations/ibug.zip
+ *--------------------------------------------------*/
+
+ #include <stdio.h>
+ #include <fstream>
+ #include <sstream>
+ #include <iostream>
+ #include "opencv2/core.hpp"
+ #include "opencv2/highgui.hpp"
+ #include "opencv2/imgproc.hpp"
+ #include "opencv2/face.hpp"
+
+ using namespace std;
+ using namespace cv;
+ using namespace cv::face;
+
+  CascadeClassifier face_cascade;
+  bool myDetector( InputArray image, OutputArray roi, void * config=0 );
+  bool parseArguments(int argc, char** argv, CommandLineParser & , String & cascade,
+      String & model, String & images, String & annotations, String & testImages
+  );
+
+  int main(int argc, char** argv)
+  {
+      CommandLineParser parser(argc, argv,"");
+      String cascade_path,model_path,images_path, annotations_path, test_images_path;
+      if(!parseArguments(argc, argv, parser,cascade_path,model_path,images_path, annotations_path, test_images_path))
+         return -1;
+
+      /*create the facemark instance*/
+      FacemarkLBF::Params params;
+      params.model_filename = model_path;
+      params.cascade_face = cascade_path;
+      Ptr<Facemark> facemark = FacemarkLBF::create(params);
+
+      face_cascade.load(params.cascade_face.c_str());
+      facemark->setFaceDetector(myDetector);
+
+      /*Loads the dataset*/
+      std::vector<String> images_train;
+      std::vector<String> landmarks_train;
+      loadDatasetList(images_path,annotations_path,images_train,landmarks_train);
+
+      Mat image;
+      std::vector<Point2f> facial_points;
+      for(size_t i=0;i<images_train.size();i++){
+          printf("%i/%i :: %s\n", (int)(i+1), (int)images_train.size(),images_train[i].c_str());
+          image = imread(images_train[i].c_str());
+          loadFacePoints(landmarks_train[i],facial_points);
+          facemark->addTrainingSample(image, facial_points);
+      }
+
+      /*train the Algorithm*/
+      facemark->training();
+
+      /*test using some images*/
+      String testFiles(images_path), testPts(annotations_path);
+      if(!test_images_path.empty()){
+          testFiles = test_images_path;
+          testPts = test_images_path; //unused
+      }
+      std::vector<String> images;
+      std::vector<String> facePoints;
+      loadDatasetList(testFiles, testPts, images, facePoints);
+
+      std::vector<Rect> rects;
+      CascadeClassifier cc(params.cascade_face.c_str());
+      for(size_t i=0;i<images.size();i++){
+          std::vector<std::vector<Point2f> > landmarks;
+          cout<<images[i];
+          Mat img = imread(images[i]);
+          facemark->getFaces(img, rects);
+          facemark->fit(img, rects, landmarks);
+
+          for(size_t j=0;j<rects.size();j++){
+              drawFacemarks(img, landmarks[j], Scalar(0,0,255));
+              rectangle(img, rects[j], Scalar(255,0,255));
+          }
+
+          if(rects.size()>0){
+              cout<<endl;
+              imshow("result", img);
+              waitKey(0);
+          }else{
+              cout<<"face not found"<<endl;
+          }
+      }
+
+  }
+
+  bool myDetector( InputArray image, OutputArray roi, void * config ){
+      Mat gray;
+      std::vector<Rect> & faces = *(std::vector<Rect>*) roi.getObj();
+      faces.clear();
+
+      if(config!=0){
+          //do nothing
+      }
+
+      if(image.channels()>1){
+          cvtColor(image,gray,CV_BGR2GRAY);
+      }else{
+          gray = image.getMat().clone();
+      }
+      equalizeHist( gray, gray );
+
+      face_cascade.detectMultiScale( gray, faces, 1.4, 2, CV_HAAR_SCALE_IMAGE, Size(30, 30) );
+
+      return true;
+  }
+
+  bool parseArguments(int argc, char** argv, CommandLineParser & parser,
+      String & cascade,
+      String & model,
+      String & images,
+      String & annotations,
+      String & test_images
+  ){
+     const String keys =
+         "{ @c cascade         |      | (required) path to the face cascade xml file fo the face detector }"
+         "{ @i images          |      | (required) path of a text file contains the list of paths to all training images}"
+         "{ @a annotations     |      | (required) Path of a text file contains the list of paths to all annotations files}"
+         "{ @m model           |      | (required) path to save the trained model }"
+         "{ t test-images      |      | Path of a text file contains the list of paths to the test images}"
+         "{ help h usage ?     |      | facemark_demo_lbf -cascade -images -annotations -model [-t] \n"
+          " example: facemark_demo_lbf ../face_cascade.xml ../images_train.txt ../points_train.txt ../lbf.model}"
+     ;
+     parser = CommandLineParser(argc, argv,keys);
+     parser.about("hello");
+
+     if (parser.has("help")){
+         parser.printMessage();
+         return false;
+     }
+
+     cascade = String(parser.get<String>("cascade"));
+     model = String(parser.get<string>("model"));
+     images = String(parser.get<string>("images"));
+     annotations = String(parser.get<string>("annotations"));
+     test_images = String(parser.get<string>("t"));
+
+     cout<<"cascade : "<<cascade.c_str()<<endl;
+     cout<<"model : "<<model.c_str()<<endl;
+     cout<<"images : "<<images.c_str()<<endl;
+     cout<<"annotations : "<<annotations.c_str()<<endl;
+
+     if(cascade.empty() || model.empty() || images.empty() || annotations.empty()){
+         std::cerr << "one or more required arguments are not found" << '\n';
+
+         parser.printMessage();
+         return false;
+     }
+
+     return true;
+  }

modules/face/samples/facemark_lbf_fitting.cpp

+/*
+By downloading, copying, installing or using the software you agree to this
+license. If you do not agree to this license, do not download, install,
+copy or use the software.
+                          License Agreement
+               For Open Source Computer Vision Library
+                       (3-clause BSD License)
+Copyright (C) 2013, OpenCV Foundation, all rights reserved.
+Third party copyrights are property of their respective owners.
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+  * Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+  * Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+  * Neither the names of the copyright holders nor the names of the contributors
+    may be used to endorse or promote products derived from this software
+    without specific prior written permission.
+This software is provided by the copyright holders and contributors "as is" and
+any express or implied warranties, including, but not limited to, the implied
+warranties of merchantability and fitness for a particular purpose are
+disclaimed. In no event shall copyright holders or contributors be liable for
+any direct, indirect, incidental, special, exemplary, or consequential damages
+(including, but not limited to, procurement of substitute goods or services;
+loss of use, data, or profits; or business interruption) however caused
+and on any theory of liability, whether in contract, strict liability,
+or tort (including negligence or otherwise) arising in any way out of
+the use of this software, even if advised of the possibility of such damage.
+
+This file was part of GSoC Project: Facemark API for OpenCV
+Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
+Student: Laksono Kurnianggoro
+Mentor: Delia Passalacqua
+*/
+
+/*----------------------------------------------
+ * Usage:
+ * facemark_lbf_fitting <face_cascade_model> <lbf_model> <video_name>
+ *
+ * example:
+ * facemark_lbf_fitting ../face_cascade.xml ../LBF.model ../video.mp4
+ *
+ * note: do not forget to provide the LBF_MODEL and DETECTOR_MODEL
+ * the model are available at opencv_contrib/modules/face/data/
+ *--------------------------------------------------*/
+
+#include <stdio.h>
+#include <ctime>
+ #include <iostream>
+#include "opencv2/core.hpp"
+#include "opencv2/highgui.hpp"
+#include "opencv2/imgproc.hpp"
+#include "opencv2/face.hpp"
+
+using namespace std;
+using namespace cv;
+using namespace cv::face;
+
+CascadeClassifier face_cascade;
+bool myDetector( InputArray image, OutputArray ROIs, void * config = 0);
+bool parseArguments(int argc, char** argv, CommandLineParser & parser,
+    String & cascade, String & model,String & video);
+
+int main(int argc, char** argv ){
+    CommandLineParser parser(argc, argv,"");
+    String cascade_path,model_path,images_path, video_path;
+    if(!parseArguments(argc, argv, parser,cascade_path,model_path,video_path))
+       return -1;
+
+    face_cascade.load(cascade_path);
+
+    FacemarkLBF::Params params;
+    params.model_filename = model_path;
+    params.cascade_face = cascade_path;
+
+    Ptr<Facemark> facemark = FacemarkLBF::create(params);
+    facemark->setFaceDetector(myDetector);
+    facemark->loadModel(params.model_filename.c_str());
+
+    VideoCapture capture(video_path);
+    Mat frame;
+
+    if( !capture.isOpened() ){
+        printf("Error when reading vide\n");
+        return 0;
+    }
+
+    Mat img;
+    String text;
+    char buff[255];
+    double fittime;
+    int nfaces;
+    std::vector<Rect> rects,rects_scaled;
+    std::vector<std::vector<Point2f> > landmarks;
+    CascadeClassifier cc(params.cascade_face.c_str());
+    namedWindow( "w", 1);
+    for( ; ; )
+    {
+        capture >> frame;
+        if(frame.empty())
+            break;
+
+        double __time__ = (double)getTickCount();
+
+        float scale = (float)(400.0/frame.cols);
+        resize(frame, img, Size((int)(frame.cols*scale), (int)(frame.rows*scale)));
+
+        facemark->getFaces(img, rects);
+        rects_scaled.clear();
+
+        for(int j=0;j<(int)rects.size();j++){
+            rects_scaled.push_back(Rect(
+                (int)(rects[j].x/scale),
+                (int)(rects[j].y/scale),
+                (int)(rects[j].width/scale),
+                (int)(rects[j].height/scale)));
+        }
+        rects = rects_scaled;
+        fittime=0;
+        nfaces = (int)rects.size();
+        if(rects.size()>0){
+            double newtime = (double)getTickCount();
+
+            facemark->fit(frame, rects, landmarks);
+
+
+            fittime = ((getTickCount() - newtime)/getTickFrequency());
+            for(int j=0;j<(int)rects.size();j++){
+                landmarks[j] = Mat(Mat(landmarks[j]));
+                drawFacemarks(frame, landmarks[j], Scalar(0,0,255));
+            }
+        }
+
+
+        double fps = (getTickFrequency()/(getTickCount() - __time__));
+        sprintf(buff, "faces: %i %03.2f fps, fit:%03.0f ms",nfaces,fps,fittime*1000);
+        text = buff;
+        putText(frame, text, Point(20,40), FONT_HERSHEY_PLAIN , 2.0,Scalar::all(255), 2, 8);
+
+        imshow("w", frame);
+        waitKey(1); // waits to display frame
+    }
+    waitKey(0); // key press to close window
+}
+
+bool myDetector( InputArray image, OutputArray ROIs, void * config ){
+    Mat gray;
+    std::vector<Rect> & faces = *(std::vector<Rect>*) ROIs.getObj();
+    faces.clear();
+
+    if(config!=0){
+        //do nothing
+    }
+
+    if(image.channels()>1){
+        cvtColor(image.getMat(),gray,CV_BGR2GRAY);
+    }else{
+        gray = image.getMat().clone();
+    }
+    equalizeHist( gray, gray );
+
+    face_cascade.detectMultiScale( gray, faces, 1.4, 2, CV_HAAR_SCALE_IMAGE, Size(30, 30) );
+    return true;
+}
+
+bool parseArguments(int argc, char** argv, CommandLineParser & parser,
+    String & cascade,
+    String & model,
+    String & video
+){
+   const String keys =
+       "{ @c cascade         |      | (required) path to the cascade model file for the face detector }"
+       "{ @m model           |      | (required) path to the trained model }"
+       "{ @v video           |      | (required) path input video}"
+       "{ help h usage ?     |      | facemark_lbf_fitting -cascade -model -video [-t]\n"
+            " example: facemark_lbf_fitting ../face_cascade.xml ../LBF.model ../video.mp4}"
+   ;
+   parser = CommandLineParser(argc, argv,keys);
+   parser.about("hello");
+
+   if (parser.has("help")){
+       parser.printMessage();
+       return false;
+   }
+
+   cascade = String(parser.get<String>("cascade"));
+   model = String(parser.get<string>("model"));
+   video = String(parser.get<string>("video"));
+
+
+   if(cascade.empty() || model.empty() || video.empty() ){
+       std::cerr << "one or more required arguments are not found" << '\n';
+       cout<<"cascade : "<<cascade.c_str()<<endl;
+       cout<<"model : "<<model.c_str()<<endl;
+       cout<<"video : "<<video.c_str()<<endl;
+       parser.printMessage();
+       return false;
+   }
+
+   return true;
+}

modules/face/src/facemark.cpp

+/*
+By downloading, copying, installing or using the software you agree to this
+license. If you do not agree to this license, do not download, install,
+copy or use the software.
+                          License Agreement
+               For Open Source Computer Vision Library
+                       (3-clause BSD License)
+Copyright (C) 2013, OpenCV Foundation, all rights reserved.
+Third party copyrights are property of their respective owners.
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+  * Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+  * Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+  * Neither the names of the copyright holders nor the names of the contributors
+    may be used to endorse or promote products derived from this software
+    without specific prior written permission.
+This software is provided by the copyright holders and contributors "as is" and
+any express or implied warranties, including, but not limited to, the implied
+warranties of merchantability and fitness for a particular purpose are
+disclaimed. In no event shall copyright holders or contributors be liable for
+any direct, indirect, incidental, special, exemplary, or consequential damages
+(including, but not limited to, procurement of substitute goods or services;
+loss of use, data, or profits; or business interruption) however caused
+and on any theory of liability, whether in contract, strict liability,
+or tort (including negligence or otherwise) arising in any way out of
+the use of this software, even if advised of the possibility of such damage.
+
+This file was part of GSoC Project: Facemark API for OpenCV
+Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
+Student: Laksono Kurnianggoro
+Mentor: Delia Passalacqua
+*/
+
+#include "opencv2/face.hpp"
+#include "opencv2/core.hpp"
+#include "precomp.hpp"
+
+/*dataset parser*/
+#include <fstream>
+#include <sstream>
+#include <string>
+#include <stdlib.h>     /* atoi */
+
+namespace cv {
+namespace face {
+    CParams::CParams(String s, double sf, int minN, Size minSz, Size maxSz){
+        cascade = s;
+        scaleFactor = sf;
+        minNeighbors = minN;
+        minSize = minSz;
+        maxSize = maxSz;
+    }
+
+    bool getFaces(InputArray image, OutputArray faces, void * parameters){
+        Mat gray;
+        std::vector<Rect> roi;
+
+        if(parameters!=0){
+            CParams * params = (CParams *)parameters;
+            cvtColor( image.getMat(), gray, CV_BGR2GRAY );
+            equalizeHist( gray, gray );
+
+            CascadeClassifier face_cascade;
+            if( !face_cascade.load( params->cascade ) ){ printf("--(!)Error loading face_cascade\n"); return false; };
+            face_cascade.detectMultiScale( gray, roi, params->scaleFactor, params->minNeighbors, 0|CV_HAAR_SCALE_IMAGE, params->minSize, params->maxSize);
+
+            Mat(roi).copyTo(faces);
+            return true;
+        }else{
+            return false;
+        }
+
+    }
+
+    bool loadDatasetList(String imageList, String groundTruth, std::vector<String> & images, std::vector<String> & landmarks){
+        std::string line;
+
+        /*clear the output containers*/
+        images.clear();
+        landmarks.clear();
+
+        /*open the files*/
+        std::ifstream infile;
+        infile.open(imageList.c_str(), std::ios::in);
+        std::ifstream ss_gt;
+        ss_gt.open(groundTruth.c_str(), std::ios::in);
+        if ((!infile) || !(ss_gt)) {
+           printf("No valid input file was given, please check the given filename.\n");
+           return false;
+        }
+
+         /*load the images path*/
+        while (getline (infile, line)){
+            images.push_back(line);
+        }
+
+        /*load the points*/
+        while (getline (ss_gt, line)){
+            landmarks.push_back(line);
+        }
+
+        return true;
+    }
+
+    bool loadTrainingData(String filename, std::vector<String> & images, OutputArray _facePoints, char delim, float offset){
+        std::string line;
+        std::string item;
+        std::vector<Point2f> pts;
+        std::vector<float> raw;
+
+        std::vector<std::vector<Point2f> > & facePoints =
+            *(std::vector<std::vector<Point2f> >*) _facePoints.getObj();
+
+        std::ifstream infile;
+        infile.open(filename.c_str(), std::ios::in);
+        if (!infile) {
+           std::string error_message = "No valid input file was given, please check the given filename.";
+           CV_Error(CV_StsBadArg, error_message);
+        }
+
+        /*clear the output containers*/
+        images.clear();
+        facePoints.clear();
+
+        /*the main loading process*/
+        while (getline (infile, line)){
+            std::istringstream ss(line); // string stream for the current line
+
+            /*pop the image path*/
+            getline (ss, item, delim);
+            images.push_back(item);
+
+            /*load all numbers*/
+            raw.clear();
+            while (getline (ss, item, delim)){
+                raw.push_back((float)atof(item.c_str()));
+            }
+
+            /*convert to opencv points*/
+            pts.clear();
+            for(unsigned i = 0;i< raw.size();i+=2){
+                pts.push_back(Point2f(raw[i]+offset,raw[i+1]+offset));
+            }
+            facePoints.push_back(pts);
+        } // main loading process
+
+        return true;
+    }
+
+    bool loadTrainingData(String imageList, String groundTruth, std::vector<String> & images, OutputArray _facePoints, float offset){
+        std::string line;
+        std::vector<Point2f> facePts;
+
+        std::vector<std::vector<Point2f> > & facePoints =
+                *(std::vector<std::vector<Point2f> >*) _facePoints.getObj();
+
+        /*clear the output containers*/
+        images.clear();
+        facePoints.clear();
+
+        /*load the images path*/
+        std::ifstream infile;
+        infile.open(imageList.c_str(), std::ios::in);
+        if (!infile) {
+           std::string error_message = "No valid input file was given, please check the given filename.";
+           CV_Error(CV_StsBadArg, error_message);
+        }
+
+        while (getline (infile, line)){
+            images.push_back(line);
+        }
+
+        /*load the points*/
+        std::ifstream ss_gt(groundTruth.c_str());
+        while (getline (ss_gt, line)){
+            facePts.clear();
+            loadFacePoints(line, facePts, offset);
+            facePoints.push_back(facePts);
+        }
+
+        return true;
+    }
+
+    bool loadFacePoints(String filename, OutputArray points, float offset){
+        std::vector<Point2f> & pts = *(std::vector<Point2f> *)points.getObj();
+
+        std::string line, item;
+        std::ifstream infile(filename.c_str());
+
+        /*pop the version*/
+        std::getline(infile, line);
+        CV_Assert(line.compare(0,7,"version")==0);
+
+        /*pop the number of points*/
+        std::getline(infile, line);
+        CV_Assert(line.compare(0,8,"n_points")==0);
+
+        /*get the number of points*/
+        std::string item_npts;
+        int npts;
+
+        std::istringstream linestream(line);
+        linestream>>item_npts>>npts;
+
+        /*pop out '{' character*/
+        std::getline(infile, line);
+
+        /*main process*/
+        int cnt = 0;
+        std::string x, y;
+        pts.clear();
+        while (std::getline(infile, line) && cnt<npts )
+        {
+            cnt+=1;
+
+            std::istringstream ss(line);
+            ss>>x>>y;
+            pts.push_back(Point2f((float)atof(x.c_str())+offset,(float)atof(y.c_str())+offset));
+
+        }
+
+        return true;
+    }
+
+    void drawFacemarks(InputOutputArray image, InputArray points, Scalar color){
+        Mat img = image.getMat();
+        std::vector<Point2f> pts = *(std::vector<Point2f>*)points.getObj();
+        for(size_t i=0;i<pts.size();i++){
+            circle(img, pts[i],3, color,-1);
+        }
+    } //drawPoints
+
+} /* namespace face */
+} /* namespace cv */

modules/face/test/test_facemark.cpp

+/*
+By downloading, copying, installing or using the software you agree to this
+license. If you do not agree to this license, do not download, install,
+copy or use the software.
+                          License Agreement
+               For Open Source Computer Vision Library
+                       (3-clause BSD License)
+Copyright (C) 2013, OpenCV Foundation, all rights reserved.
+Third party copyrights are property of their respective owners.
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+  * Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+  * Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+  * Neither the names of the copyright holders nor the names of the contributors
+    may be used to endorse or promote products derived from this software
+    without specific prior written permission.
+This software is provided by the copyright holders and contributors "as is" and
+any express or implied warranties, including, but not limited to, the implied
+warranties of merchantability and fitness for a particular purpose are
+disclaimed. In no event shall copyright holders or contributors be liable for
+any direct, indirect, incidental, special, exemplary, or consequential damages
+(including, but not limited to, procurement of substitute goods or services;
+loss of use, data, or profits; or business interruption) however caused
+and on any theory of liability, whether in contract, strict liability,
+or tort (including negligence or otherwise) arising in any way out of
+the use of this software, even if advised of the possibility of such damage.
+
+This file was part of GSoC Project: Facemark API for OpenCV
+Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
+Student: Laksono Kurnianggoro
+Mentor: Delia Passalacqua
+*/
+
+#include "test_precomp.hpp"
+#include "opencv2/imgcodecs.hpp"
+#include "opencv2/face.hpp"
+#include <vector>
+#include <string>
+using namespace std;
+using namespace cv;
+using namespace cv::face;
+
+TEST(CV_Face_Facemark, test_utilities) {
+    string image_file = cvtest::findDataFile("face/david1.jpg", true);
+    string annotation_file = cvtest::findDataFile("face/david1.pts", true);
+    string cascade_filename =
+        cvtest::findDataFile("cascadeandhog/cascades/lbpcascade_frontalface.xml", true);
+
+    std::vector<Point2f> facial_points;
+    EXPECT_NO_THROW(loadFacePoints(annotation_file,facial_points));
+
+    Mat img = imread(image_file);
+    EXPECT_NO_THROW(drawFacemarks(img, facial_points, Scalar(0,0,255)));
+
+    CParams params(cascade_filename);
+    std::vector<Rect> faces;
+    EXPECT_TRUE(getFaces(img, faces, &params));
+}

modules/face/test/test_facemark_aam.cpp

+/*
+By downloading, copying, installing or using the software you agree to this
+license. If you do not agree to this license, do not download, install,
+copy or use the software.
+                          License Agreement
+               For Open Source Computer Vision Library
+                       (3-clause BSD License)
+Copyright (C) 2013, OpenCV Foundation, all rights reserved.
+Third party copyrights are property of their respective owners.
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+  * Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+  * Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+  * Neither the names of the copyright holders nor the names of the contributors
+    may be used to endorse or promote products derived from this software
+    without specific prior written permission.
+This software is provided by the copyright holders and contributors "as is" and
+any express or implied warranties, including, but not limited to, the implied
+warranties of merchantability and fitness for a particular purpose are
+disclaimed. In no event shall copyright holders or contributors be liable for
+any direct, indirect, incidental, special, exemplary, or consequential damages
+(including, but not limited to, procurement of substitute goods or services;
+loss of use, data, or profits; or business interruption) however caused
+and on any theory of liability, whether in contract, strict liability,
+or tort (including negligence or otherwise) arising in any way out of
+the use of this software, even if advised of the possibility of such damage.
+
+This file was part of GSoC Project: Facemark API for OpenCV
+Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
+Student: Laksono Kurnianggoro
+Mentor: Delia Passalacqua
+*/
+
+/*Usage:
+ download the opencv_extra from https://github.com/opencv/opencv_extra
+ and then execute the following commands:
+ export OPENCV_TEST_DATA_PATH=/home/opencv/opencv_extra/testdata
+ <build_folder>/bin/opencv_test_face
+*/
+
+#include "test_precomp.hpp"
+#include "opencv2/imgcodecs.hpp"
+#include "opencv2/face.hpp"
+#include <vector>
+#include <string>
+using namespace std;
+using namespace cv;
+using namespace cv::face;
+
+CascadeClassifier face_detector;
+static bool customDetector( InputArray image, OutputArray ROIs, void * config = 0 ){
+    Mat gray;
+    std::vector<Rect> & faces = *(std::vector<Rect>*) ROIs.getObj();
+    faces.clear();
+
+    if(config!=0){
+        //do nothing
+    }
+
+    if(image.channels()>1){
+        cvtColor(image.getMat(),gray,CV_BGR2GRAY);
+    }else{
+        gray = image.getMat().clone();
+    }
+    equalizeHist( gray, gray );
+
+    face_detector.detectMultiScale( gray, faces, 1.4, 2, CV_HAAR_SCALE_IMAGE, Size(30, 30) );
+    return true;
+}
+
+TEST(CV_Face_FacemarkAAM, can_create_default) {
+    FacemarkAAM::Params params;
+
+    Ptr<Facemark> facemark;
+    EXPECT_NO_THROW(facemark = FacemarkAAM::create(params));
+    EXPECT_FALSE(facemark.empty());
+}
+
+TEST(CV_Face_FacemarkAAM, can_set_custom_detector) {
+    string cascade_filename =
+        cvtest::findDataFile("cascadeandhog/cascades/lbpcascade_frontalface.xml", true);
+
+    EXPECT_TRUE(face_detector.load(cascade_filename));
+
+    Ptr<Facemark> facemark = FacemarkAAM::create();
+    EXPECT_TRUE(facemark->setFaceDetector(customDetector));
+}
+
+TEST(CV_Face_FacemarkAAM, test_workflow) {
+
+    string i1 = cvtest::findDataFile("face/david1.jpg", true);
+    string p1 = cvtest::findDataFile("face/david1.pts", true);
+    string i2 = cvtest::findDataFile("face/david2.jpg", true);
+    string p2 = cvtest::findDataFile("face/david2.pts", true);
+
+    std::vector<string> images_train;
+    images_train.push_back(i1);
+    images_train.push_back(i2);
+
+    std::vector<String> points_train;
+    points_train.push_back(p1);
+    points_train.push_back(p2);
+
+    string cascade_filename =
+        cvtest::findDataFile("cascadeandhog/cascades/lbpcascade_frontalface.xml", true);
+    FacemarkAAM::Params params;
+    params.n = 1;
+    params.m = 1;
+    params.verbose = false;
+    params.save_model = false;
+    Ptr<Facemark> facemark = FacemarkAAM::create(params);
+
+    Mat image;
+    std::vector<Point2f> landmarks;
+    for(size_t i=0;i<images_train.size();i++){
+        image = imread(images_train[i].c_str());
+        EXPECT_TRUE(loadFacePoints(points_train[i].c_str(),landmarks));
+        EXPECT_TRUE(landmarks.size()>0);
+        EXPECT_TRUE(facemark->addTrainingSample(image, landmarks));
+    }
+
+    EXPECT_NO_THROW(facemark->training());
+
+    /*------------ Fitting Part ---------------*/
+    facemark->setFaceDetector(customDetector);
+    string image_filename = cvtest::findDataFile("face/david1.jpg", true);
+    image = imread(image_filename.c_str());
+    EXPECT_TRUE(!image.empty());
+
+    std::vector<Rect> rects;
+    std::vector<std::vector<Point2f> > facial_points;
+
+    EXPECT_TRUE(facemark->getFaces(image, rects));
+    EXPECT_TRUE(rects.size()>0);
+    EXPECT_TRUE(facemark->fit(image, rects, facial_points));
+    EXPECT_TRUE(facial_points[0].size()>0);
+
+    /*------------ Test getData ---------------*/
+    FacemarkAAM::Data data;
+    EXPECT_TRUE(facemark->getData(&data));
+    EXPECT_TRUE(data.s0.size()>0);
+}

modules/face/test/test_facemark_lbf.cpp

+/*
+By downloading, copying, installing or using the software you agree to this
+license. If you do not agree to this license, do not download, install,
+copy or use the software.
+                          License Agreement
+               For Open Source Computer Vision Library
+                       (3-clause BSD License)
+Copyright (C) 2013, OpenCV Foundation, all rights reserved.
+Third party copyrights are property of their respective owners.
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+  * Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+  * Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+  * Neither the names of the copyright holders nor the names of the contributors
+    may be used to endorse or promote products derived from this software
+    without specific prior written permission.
+This software is provided by the copyright holders and contributors "as is" and
+any express or implied warranties, including, but not limited to, the implied
+warranties of merchantability and fitness for a particular purpose are
+disclaimed. In no event shall copyright holders or contributors be liable for
+any direct, indirect, incidental, special, exemplary, or consequential damages
+(including, but not limited to, procurement of substitute goods or services;
+loss of use, data, or profits; or business interruption) however caused
+and on any theory of liability, whether in contract, strict liability,
+or tort (including negligence or otherwise) arising in any way out of
+the use of this software, even if advised of the possibility of such damage.
+
+This file was part of GSoC Project: Facemark API for OpenCV
+Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
+Student: Laksono Kurnianggoro
+Mentor: Delia Passalacqua
+*/
+
+/*Usage:
+ download the opencv_extra from https://github.com/opencv/opencv_extra
+ and then execute the following commands:
+ export OPENCV_TEST_DATA_PATH=/home/opencv/opencv_extra/testdata
+ <build_folder>/bin/opencv_test_face
+*/
+
+#include "test_precomp.hpp"
+#include "opencv2/imgcodecs.hpp"
+#include "opencv2/face.hpp"
+#include <vector>
+#include <string>
+using namespace std;
+using namespace cv;
+using namespace cv::face;
+
+CascadeClassifier cascade_detector;
+static bool myCustomDetector( InputArray image, OutputArray ROIs, void * config = 0 ){
+    Mat gray;
+    std::vector<Rect> & faces = *(std::vector<Rect>*) ROIs.getObj();
+    faces.clear();
+
+    if(config!=0){
+        //do nothing
+    }
+
+    if(image.channels()>1){
+        cvtColor(image.getMat(),gray,CV_BGR2GRAY);
+    }else{
+        gray = image.getMat().clone();
+    }
+    equalizeHist( gray, gray );
+
+    cascade_detector.detectMultiScale( gray, faces, 1.4, 2, CV_HAAR_SCALE_IMAGE, Size(30, 30) );
+    return true;
+}
+
+TEST(CV_Face_FacemarkLBF, can_create_default) {
+    FacemarkLBF::Params params;
+    params.n_landmarks = 68;
+
+    Ptr<Facemark> facemark;
+    EXPECT_NO_THROW(facemark = FacemarkLBF::create(params));
+    EXPECT_FALSE(facemark.empty());
+}
+
+TEST(CV_Face_FacemarkLBF, can_set_custom_detector) {
+    string cascade_filename =
+        cvtest::findDataFile("cascadeandhog/cascades/lbpcascade_frontalface.xml", true);
+
+    EXPECT_TRUE(cascade_detector.load(cascade_filename));
+
+    Ptr<Facemark> facemark = FacemarkLBF::create();
+    EXPECT_TRUE(facemark->setFaceDetector(myCustomDetector));
+}
+
+TEST(CV_Face_FacemarkLBF, test_workflow) {
+
+    string i1 = cvtest::findDataFile("face/david1.jpg", true);
+    string p1 = cvtest::findDataFile("face/david1.pts", true);
+    string i2 = cvtest::findDataFile("face/david2.jpg", true);
+    string p2 = cvtest::findDataFile("face/david2.pts", true);
+
+    std::vector<string> images_train;
+    images_train.push_back(i1);
+    images_train.push_back(i2);
+
+    std::vector<String> points_train;
+    points_train.push_back(p1);
+    points_train.push_back(p2);
+
+    string cascade_filename =
+        cvtest::findDataFile("cascadeandhog/cascades/lbpcascade_frontalface.xml", true);
+    FacemarkLBF::Params params;
+    params.cascade_face = cascade_filename;
+    params.verbose = false;
+    params.save_model = false;
+
+    Ptr<Facemark> facemark = FacemarkLBF::create(params);
+
+    Mat image;
+    std::vector<Point2f> landmarks;
+    for(size_t i=0;i<images_train.size();i++){
+        image = imread(images_train[i].c_str());
+        EXPECT_TRUE(loadFacePoints(points_train[i].c_str(),landmarks));
+        EXPECT_TRUE(landmarks.size()>0);
+        EXPECT_TRUE(facemark->addTrainingSample(image, landmarks));
+    }
+
+    EXPECT_NO_THROW(facemark->training());
+
+    /*------------ Fitting Part ---------------*/
+    cascade_detector.load(cascade_filename);
+    facemark->setFaceDetector(myCustomDetector);
+
+    string image_filename = cvtest::findDataFile("face/david1.jpg", true);
+    image = imread(image_filename.c_str());
+    EXPECT_TRUE(!image.empty());
+
+    std::vector<Rect> rects;
+    std::vector<std::vector<Point2f> > facial_points;
+
+    EXPECT_TRUE(facemark->getFaces(image, rects));
+    EXPECT_TRUE(rects.size()>0);
+    EXPECT_TRUE(facemark->fit(image, rects, facial_points));
+    EXPECT_TRUE(facial_points[0].size()>0);
+}
+
+TEST(CV_Face_FacemarkLBF, get_data) {
+    Ptr<Facemark> facemark = FacemarkLBF::create();
+    EXPECT_TRUE(facemark->getData());
+}

modules/face/tutorials/facemark_aam/facemark_aam.markdown

+Using the FacemarkAAM {#tutorial_facemark_aam}
+==========================================================
+
+Goals
+----
+
+In this tutorial you will learn how to:
+- creating the instance of FacemarkAAM
+- training the AAM model
+- Fitting using FacemarkAAM
+
+Preparation
+--------
+
+Before you continue with this tutorial, you should download the dataset of facial landmarks detection.
+We suggest you to download the LFPW dataset which can be retrieved at <https://ibug.doc.ic.ac.uk/download/annotations/lfpw.zip>.
+
+Make sure that the annotation format is supported by the API, the contents in annotation file should look like the following snippet:
+@code
+version: 1
+n_points:  68
+{
+212.716603 499.771793
+230.232816 566.290071
+...
+}
+@endcode
+
+The next thing to do is to make 2 text files containing the list of image files and annotation files respectively. Make sure that the order or image and annotation in both files are matched. Furthermore, it is advised to use absolute path instead of relative path.
+Example to make the file list in Linux machine
+@code
+ls $PWD/trainset/*.jpg > images_train.txt
+ls $PWD/trainset/*.pts > annotation_train.txt
+@endcode
+
+example of content in the images_train.txt
+@code
+/home/user/lfpw/trainset/100032540_1.jpg
+/home/user/lfpw/trainset/100040721_1.jpg
+/home/user/lfpw/trainset/100040721_2.jpg
+/home/user/lfpw/trainset/1002681492_1.jpg
+@endcode
+
+example of content in the annotation_train.txt
+@code
+/home/user/lfpw/trainset/100032540_1.pts
+/home/user/lfpw/trainset/100040721_1.pts
+/home/user/lfpw/trainset/100040721_2.pts
+/home/user/lfpw/trainset/1002681492_1.pts
+@endcode
+
+Optionally, you can create the similar files for the testset.
+
+In this tutorial, the pre-trained model will not be provided due to its large file size (~500MB). By following this tutorial, you will be able to train obtain your own trained model within few minutes.
+
+Working with the AAM Algorithm
+--------
+
+The full working code is available in the face/samples/facemark_demo_aam.cpp file. In this tutorial, the explanation of some important parts are covered.
+
+-# <B>Creating the instance of AAM algorithm</B>
+
+   @snippet face/samples/facemark_demo_aam.cpp instance_creation
+   Firstly, an instance of parameter for the AAM algorithm is created. In this case, we will modify the default list of the scaling factor. By default, the scaling factor used is 1.0 (no scaling). Here we add two more scaling factor which will make the instance trains two more model at scale 2 and 4 (2 time smaller and 4 time smaller, faster faster fitting time). However, you should make sure that this scaling factor is not too big since it will make the image scaled into a very small one. Thus it will lost all of its important information for the landmark detection purpose.
+
+   Alternatively, you can override the default scaling in similar way to this example:
+   @code
+   std::vector<float>scales;
+   scales.push_back(1.5);
+   scales.push_back(2.4);
+
+   FacemarkAAM::Params params;
+   params.scales = scales;
+   @endcode
+
+-# <B>Loading the dataset</B>
+
+   @snippet face/samples/facemark_demo_aam.cpp load_dataset
+   List of the dataset are loaded into the program. We will put the samples from dataset one by one in the next step.
+
+-# <B>Adding the samples to the trainer</B>
+
+   @snippet face/samples/facemark_demo_aam.cpp add_samples
+   The image from the dataset list are loaded one by one as well as its corresponding annotation data. Then the pair of sample is added to the trainer.
+
+-# <B>Training process</B>
+
+   @snippet face/samples/facemark_demo_aam.cpp training
+   The training process is called using a single line of code. Make sure that all the required training samples are already added to the trainer.
+
+-# <B>Preparation for fitting</B>
+
+   First of all, you need to load the list of test files.
+   @snippet face/samples/facemark_demo_aam.cpp load_test_images
+
+   Since the AAM needs initialization parameters (rotation, translation, and scaling), you need to declare the required variable to store these information which will be obtained using a custom function. Since the implementation of getInitialFitting() function in this example is not optimal, you can create your own function.
+
+   The initialization is obtained by comparing the base shape of the trained model with the current face image. In this case, the rotation is obtained by comparing the angle of line formed by two eyes in the input face image with the same line in the base shape. Meanwhile, the scaling is obtained by comparing the length of line between eyes in the input image compared to the base shape.
+
+-# <B>Fitting process</B>
+
+   The fitting process is started by detecting the face in a given image.
+   @snippet face/samples/facemark_demo_aam.cpp detect_face
+
+   If at least one face is found, then the next step is computing the initialization parameters. In this case, since the getInitialFitting() function is not optimal, it may not find pair of eyes from a given face. Therefore, we will filter out the face without initialization parameters and in this case, each element in the `conf` vector represent the initialization parameter for each filtered face.
+   @snippet face/samples/facemark_demo_aam.cpp get_initialization
+
+   For the fitting parameter stored in the `conf` vector, the last parameter represent the ID of scaling factor that will be used in the fitting process. In this example the fitting will use the biggest scaling factor (4) which is expected to have the fastest computation time compared to the other scales. If the ID if bigger than the available trained scale in the model, the the model with the biggest scale ID is used.
+
+   The fitting process is quite simple, you just need to put the corresponding image, vector of `cv::Rect` representing the ROIs of all faces in the given image, container of the landmark points represented by `landmarks` variable, and the configuration variables.
+   @snippet face/samples/facemark_demo_aam.cpp fitting_process
+
+   After the fitting process is finished, you can visualize the result using the `drawFacemarks` function.

modules/face/tutorials/facemark_tutorial.markdown

+Tutorial on Facial Landmark Detector API {#tutorial_table_of_content_facemark}
+==========================================================
+
+The facial landmark detector API is useful to detect facial landmarks from an input image.
+
+-   @subpage tutorial_facemark_add_algorithm
+
+    *Compatibility:* \> OpenCV 3.0
+
+    *Author:* Laksono Kurnianggoro
+
+    Adding a new algorithm in to the API.
+
+
+-   @subpage tutorial_facemark_usage
+
+    *Compatibility:* \> OpenCV 3.0
+
+    *Author:* Laksono Kurnianggoro
+
+    Tutorial on how to use the API.
+
+-   @subpage tutorial_facemark_aam
+
+    *Compatibility:* \> OpenCV 3.0
+
+    *Author:* Laksono Kurnianggoro
+
+    Tutorial on how to use the FacemarkAAM algorithm.