Skip to content

O
opencv_contrib
Commit bccbec79 authored by sukhad-app's avatar sukhad-app Committed by Alexander Alekhin
Browse files
Options

Merge pull request #1199 from sukhad-app:face_alignment 

Face alignment (#1199)

* This commit will add a new functionality of one millisecond face_alignment to OpenCV.
Face alignment is a computer vision technology for identifying the geometric structure of human faces in digital images.
Given the location and size of a face, it automatically determines the shape
 of the face components such as eyes and nose.
Added following functions :
 1) Application to train a face landmark detector.
 2) Application to detect face landmarks using a trained model.
 3) Application to swap faces using face landmark detection
 4) Application to detect landmarks in a video.
Merged the code with a global facemark API.
Added Doxygen Documentation for the Class created.
Added tutorials for the samples added.
Added visualisations depicting error rate and training time.

Made desired changes

fix

fix

fix

fix

fix

fix

fix

fix

fix

* face: drop duplicated file

-face_alignmentImpl.hpp
+face_alignmentimpl.hpp

* face: minor refactoring

- replace license headers
- fix usage of "precomp.hpp"
parent 004ac553
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 2434 additions and 74 deletions
modules/face/CMakeLists.txt
View file @ bccbec79
set(the_description "Face recognition etc")
ocv_define_module(face opencv_core opencv_imgproc opencv_objdetect WRAP python)
ocv_define_module(face opencv_core
opencv_imgproc
opencv_objdetect
opencv_tracking # estimateRigidTransform()
opencv_photo # samples
WRAP python
)
# NOTE: objdetect module is needed for one of the samples
set(__commit_hash "8afa57abc8229d611c4937165d20e2a2d9fc5a12")
set(__file_hash "7505c44ca4eb54b4ab1e4777cb96ac05")
ocv_download(
FILENAME face_landmark_model.dat
HASH ${__file_hash}
URL
"${OPENCV_FACE_ALIGNMENT_URL}"
"$ENV{OPENCV_FACE_ALIGNMENT_URL}"
"https://raw.githubusercontent.com/opencv/opencv_3rdparty/${__commit_hash}/"
DESTINATION_DIR "${CMAKE_BINARY_DIR}/${OPENCV_TEST_DATA_INSTALL_PATH}/cv/face/"
ID "data"
RELATIVE_URL
STATUS res
)
if(NOT res)
message(WARNING "Face: Can't get model file for face alignment.")
endif()
modules/face/include/opencv2/face.hpp
View file @ bccbec79
......@@ -377,4 +377,6 @@ protected:
#include "opencv2/face/facemark.hpp"
#include "opencv2/face/facemarkLBF.hpp"
#include "opencv2/face/facemarkAAM.hpp"
#endif
#include "opencv2/face/face_alignment.hpp"
#endif // __OPENCV_FACE_HPP__
modules/face/include/opencv2/face/face_alignment.hpp 0 → 100644
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#ifndef __OPENCV_FACE_ALIGNMENT_HPP__
#define __OPENCV_FACE_ALIGNMENT_HPP__
#include "facemark.hpp"
namespace cv{
namespace face{
class CV_EXPORTS_W FacemarkKazemi : public Algorithm
{
public:
struct CV_EXPORTS Params
{
/**
* \brief Constructor
*/
Params();
/// cascade_depth This stores the deapth of cascade used for training.
unsigned long cascade_depth;
/// tree_depth This stores the max height of the regression tree built.
unsigned long tree_depth;
/// num_trees_per_cascade_level This stores number of trees fit per cascade level.
unsigned long num_trees_per_cascade_level;
/// learning_rate stores the learning rate in gradient boosting, also reffered as shrinkage.
float learning_rate;
/// oversampling_amount stores number of initialisations used to create training samples.
unsigned long oversampling_amount;
/// num_test_coordinates stores number of test coordinates.
unsigned long num_test_coordinates;
/// lambda stores a value to calculate probability of closeness of two coordinates.
float lambda;
/// num_test_splits stores number of random test splits generated.
unsigned long num_test_splits;
/// configfile stores the name of the file containing the values of training parameters
String configfile;
};
static Ptr<FacemarkKazemi> create(const FacemarkKazemi::Params &parameters = FacemarkKazemi::Params());
virtual ~FacemarkKazemi();
/// @brief training the facemark model, input are the file names of image list and landmark annotation
virtual void training(String imageList, String groundTruth)=0;
/** @brief This function is used to train the model using gradient boosting to get a cascade of regressors
*which can then be used to predict shape.
*@param images A vector of type cv::Mat which stores the images which are used in training samples.
*@param landmarks A vector of vectors of type cv::Point2f which stores the landmarks detected in a particular image.
*@param scale A size of type cv::Size to which all images and landmarks have to be scaled to.
*@param configfile A variable of type std::string which stores the name of the file storing parameters for training the model.
*@param modelFilename A variable of type std::string which stores the name of the trained model file that has to be saved.
*@returns A boolean value. The function returns true if the model is trained properly or false if it is not trained.
*/
virtual bool training(std::vector<Mat>& images, std::vector< std::vector<Point2f> >& landmarks,std::string configfile,Size scale,std::string modelFilename = "face_landmarks.dat")=0;
/** @brief This function is used to load the trained model..
*@param filename A variable of type cv::String which stores the name of the file in which trained model is stored.
*/
virtual void loadModel(String filename)=0;
/** @brief This functions retrieves a centered and scaled face shape, according to the bounding rectangle.
*@param image A variable of type cv::InputArray which stores the image whose landmarks have to be found
*@param faces A variable of type cv::InputArray which stores the bounding boxes of faces found in a given image.
*@param landmarks A variable of type cv::InputOutputArray which stores the landmarks of all the faces found in the image
*/
virtual bool fit( InputArray image, InputArray faces, InputOutputArray landmarks )=0;//!< from many ROIs
/// set the custom face detector
virtual bool setFaceDetector(bool(*f)(InputArray , OutputArray, void*), void* userData)=0;
/// get faces using the custom detector
virtual bool getFaces(InputArray image, OutputArray faces)=0;
};
}} // namespace
#endif
modules/face/include/opencv2/face/facemark.hpp
View file @ bccbec79
......@@ -3,7 +3,7 @@
// of this distribution and at http://opencv.org/license.html.
/*
This file contains results of GSoC Project: Facemark API for OpenCV
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
......@@ -73,6 +73,8 @@ cv::imshow("detection", frame);
*/
CV_EXPORTS bool getFaces(InputArray image, OutputArray faces, CParams* params);
CV_EXPORTS_W bool getFacesHAAR(InputArray image, OutputArray faces, const String& face_cascade_name);
/** @brief A utility to load list of paths to training image and annotation file.
@param imageList The specified file contains paths to the training images.
@param annotationList The specified file contains paths to the training annotations.
......@@ -163,6 +165,25 @@ CV_EXPORTS_W bool loadTrainingData( String imageList, String groundTruth,
OutputArray facePoints,
float offset = 0.0f);
/** @brief This function extracts the data for training from .txt files which contains the corresponding image name and landmarks.
*The first file in each file should give the path of the image whose
*landmarks are being described in the file. Then in the subsequent
*lines there should be coordinates of the landmarks in the image
*i.e each line should be of the form x,y
*where x represents the x coordinate of the landmark and y represents
*the y coordinate of the landmark.
*
*For reference you can see the files as provided in the
*<a href="http://www.ifp.illinois.edu/~vuongle2/helen/">HELEN dataset</a>
*
* @param filename A vector of type cv::String containing name of the .txt files.
* @param trainlandmarks A vector of type cv::Point2f that would store shape or landmarks of all images.
* @param trainimages A vector of type cv::String which stores the name of images whose landmarks are tracked
* @returns A boolean value. It returns true when it reads the data successfully and false otherwise
*/
CV_EXPORTS_W bool loadTrainingData(std::vector<String> filename,std::vector< std::vector<Point2f> >
&trainlandmarks,std::vector<String> & trainimages);
/** @brief A utility to load facial landmark information from a given file.
@param filename The filename of file contains the facial landmarks data.
......
modules/face/include/opencv2/face/facemarkAAM.hpp
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
/*
This file contains results of GSoC Project: Facemark API for OpenCV
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
......@@ -55,7 +80,7 @@ public:
struct CV_EXPORTS Config
{
Config( Mat rot = Mat::eye(2,2,CV_32F),
Point2f trans = Point2f(0.0f,0.0f),
Point2f trans = Point2f(0.0f, 0.0f),
float scaling = 1.0f,
int scale_id=0
);
......
modules/face/include/opencv2/face/facemarkLBF.hpp
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
/*
This file contains results of GSoC Project: Facemark API for OpenCV
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
......
modules/face/samples/facemark_lbf_fitting.cpp
View file @ bccbec79
/*
This file contains results of GSoC Project: Facemark API for OpenCV
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
......@@ -18,7 +47,7 @@ Mentor: Delia Passalacqua
#include <stdio.h>
#include <ctime>
#include <iostream>
#include <iostream>
#include "opencv2/core.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
......@@ -137,34 +166,34 @@ bool parseArguments(int argc, char** argv, CommandLineParser & parser,
String & model,
String & video
){
const String keys =
"{ @c cascade | | (required) path to the cascade model file for the face detector }"
"{ @m model | | (required) path to the trained model }"
"{ @v video | | (required) path input video}"
"{ help h usage ? | | facemark_lbf_fitting -cascade -model -video [-t]\n"
" example: facemark_lbf_fitting ../face_cascade.xml ../LBF.model ../video.mp4}"
;
parser = CommandLineParser(argc, argv,keys);
parser.about("hello");
if (parser.has("help")){
parser.printMessage();
return false;
}
cascade = String(parser.get<String>("cascade"));
model = String(parser.get<string>("model"));
video = String(parser.get<string>("video"));
if(cascade.empty() || model.empty() || video.empty() ){
std::cerr << "one or more required arguments are not found" << '\n';
cout<<"cascade : "<<cascade.c_str()<<endl;
cout<<"model : "<<model.c_str()<<endl;
cout<<"video : "<<video.c_str()<<endl;
parser.printMessage();
return false;
}
return true;
const String keys =
"{ @c cascade | | (required) path to the cascade model file for the face detector }"
"{ @m model | | (required) path to the trained model }"
"{ @v video | | (required) path input video}"
"{ help h usage ? | | facemark_lbf_fitting -cascade -model -video [-t]\n"
" example: facemark_lbf_fitting ../face_cascade.xml ../LBF.model ../video.mp4}"
;
parser = CommandLineParser(argc, argv,keys);
parser.about("hello");
if (parser.has("help")){
parser.printMessage();
return false;
}
cascade = String(parser.get<String>("cascade"));
model = String(parser.get<string>("model"));
video = String(parser.get<string>("video"));
if(cascade.empty() || model.empty() || video.empty() ){
std::cerr << "one or more required arguments are not found" << '\n';
cout<<"cascade : "<<cascade.c_str()<<endl;
cout<<"model : "<<model.c_str()<<endl;
cout<<"video : "<<video.c_str()<<endl;
parser.printMessage();
return false;
}
return true;
}
modules/face/samples/sampleDetectLandmarks.cpp 0 → 100644
View file @ bccbec79
#include "opencv2/face.hpp"
#include "opencv2/videoio.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/objdetect.hpp"
#include "opencv2/imgproc.hpp"
#include <iostream>
#include <vector>
#include <string>
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
int main(int argc,char** argv){
//Give the path to the directory containing all the files containing data
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ model_filename f | | (required) path to binary file storing the trained model which is to be loaded [example - /data/file.dat]}"
"{ image i | | (required) path to image in which face landmarks have to be detected.[example - /data/image.jpg] }"
"{ face_cascade c | | Path to the face cascade xml file which you want to use as a detector}"
);
// Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
string filename(parser.get<string>("model_filename"));
if (filename.empty()){
parser.printMessage();
cerr << "The name of the model file to be loaded for detecting landmarks is not found" << endl;
return -1;
}
string image(parser.get<string>("image"));
if (image.empty()){
parser.printMessage();
cerr << "The name of the image file in which landmarks have to be detected is not found" << endl;
return -1;
}
string cascade_name(parser.get<string>("face_cascade"));
if (cascade_name.empty()){
parser.printMessage();
cerr << "The name of the cascade classifier to be loaded to detect faces is not found" << endl;
return -1;
}
Mat img = imread(image);
//pass the face cascade xml file which you want to pass as a detector
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
Ptr<FacemarkKazemi> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
facemark->loadModel(filename);
cout<<"Loaded model"<<endl;
vector<Rect> faces;
resize(img,img,Size(460,460));
facemark->getFaces(img,faces);
vector< vector<Point2f> > shapes;
if(facemark->fit(img,faces,shapes))
{
for( size_t i = 0; i < faces.size(); i++ )
{
cv::rectangle(img,faces[i],Scalar( 255, 0, 0 ));
}
for(unsigned long i=0;i<faces.size();i++){
for(unsigned long k=0;k<shapes[i].size();k++)
cv::circle(img,shapes[i][k],5,cv::Scalar(0,0,255),FILLED);
}
namedWindow("Detected_shape");
imshow("Detected_shape",img);
waitKey(0);
}
return 0;
}
\ No newline at end of file
modules/face/samples/sampleDetectLandmarksvideo.cpp 0 → 100644
View file @ bccbec79
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/videoio.hpp"
#include "opencv2/objdetect.hpp"
#include <iostream>
#include <vector>
#include <string>
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
int main(int argc,char** argv){
//Give the path to the directory containing all the files containing data
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ model_filename f | | (required) path to binary file storing the trained model which is to be loaded [example - /data/file.dat]}"
"{ video v | | (required) path to video in which face landmarks have to be detected.[example - /data/video.avi] }"
"{ face_cascade c | | Path to the face cascade xml file which you want to use as a detector}"
);
// Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
string filename(parser.get<string>("model_filename"));
if (filename.empty()){
parser.printMessage();
cerr << "The name of the model file to be loaded for detecting landmarks is not found" << endl;
return -1;
}
string video(parser.get<string>("video"));
if (video.empty()){
parser.printMessage();
cerr << "The name of the video file in which landmarks have to be detected is not found" << endl;
return -1;
}
string cascade_name(parser.get<string>("face_cascade"));
if (cascade_name.empty()){
parser.printMessage();
cerr << "The name of the cascade classifier to be loaded to detect faces is not found" << endl;
return -1;
}
VideoCapture cap(video);
if(!cap.isOpened()){
cerr<<"Video cannot be loaded. Give correct path"<<endl;
return -1;
}
//pass the face cascade xml file which you want to pass as a detector
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
Ptr<FacemarkKazemi> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
facemark->loadModel(filename);
cout<<"Loaded model"<<endl;
//vector to store the faces detected in the image
vector<Rect> faces;
vector< vector<Point2f> > shapes;
Mat img;
while(1){
faces.clear();
shapes.clear();
cap>>img;
//Detect faces in the current image
resize(img,img,Size(600,600));
facemark->getFaces(img,faces);
if(faces.size()==0){
cout<<"No faces found in this frame"<<endl;
}
else{
for( size_t i = 0; i < faces.size(); i++ )
{
cv::rectangle(img,faces[i],Scalar( 255, 0, 0 ));
}
//vector to store the landmarks of all the faces in the image
if(facemark->fit(img,faces,shapes))
{
for(unsigned long i=0;i<faces.size();i++){
for(unsigned long k=0;k<shapes[i].size();k++)
cv::circle(img,shapes[i][k],3,cv::Scalar(0,0,255),FILLED);
}
}
}
namedWindow("Detected_shape");
imshow("Detected_shape",img);
if(waitKey(1) >= 0) break;
}
return 0;
}
\ No newline at end of file
modules/face/samples/sample_config_file.xml 0 → 100644
View file @ bccbec79
<?xml version="1.0"?>
<!-- cascade_depth stores the depth of cascade of regressors used for training.
tree_depth stores the depth of trees created as weak learners during gradient boosting.
num_trees_per_cascade_level stores number of trees required per cascade level.
learning_rate stores the learning rate for gradient boosting.This is required to prevent overfitting using shrinkage.
oversampling_amount stores the oversampling amount for the samples.
num_test_coordinates stores number of test coordinates to be generated as samples to decide for making the split.
lambda stores the value used for calculating the probabilty which helps to select closer pixels for making the split.
num_test_splits stores the number of test splits to be generated before making the best split.
-->
<opencv_storage>
<cascade_depth>15</cascade_depth>
<tree_depth>4</tree_depth>
<num_trees_per_cascade_level>500</num_trees_per_cascade_level>
<learning_rate>1.0000000149011612e-01</learning_rate>
<oversampling_amount>20</oversampling_amount>
<num_test_coordinates>400</num_test_coordinates>
<lambda>1.0000000149011612e-01</lambda>
<num_test_splits>20</num_test_splits>
</opencv_storage>
modules/face/samples/sample_face_swapping.cpp 0 → 100644
View file @ bccbec79
#include "opencv2/face.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/objdetect.hpp"
#include "opencv2/photo.hpp" // seamlessClone()
#include <iostream>
using namespace cv;
using namespace cv::face;
using namespace std;
static bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
void divideIntoTriangles(Rect rect, vector<Point2f> &points, vector< vector<int> > &delaunayTri);
void warpTriangle(Mat &img1, Mat &img2, vector<Point2f> &triangle1, vector<Point2f> &triangle2);
//Divide the face into triangles for warping
void divideIntoTriangles(Rect rect, vector<Point2f> &points, vector< vector<int> > &Tri){
// Create an instance of Subdiv2D
Subdiv2D subdiv(rect);
// Insert points into subdiv
for( vector<Point2f>::iterator it = points.begin(); it != points.end(); it++)
subdiv.insert(*it);
vector<Vec6f> triangleList;
subdiv.getTriangleList(triangleList);
vector<Point2f> pt(3);
vector<int> ind(3);
for( size_t i = 0; i < triangleList.size(); i++ )
{
Vec6f triangle = triangleList[i];
pt[0] = Point2f(triangle[0], triangle[1]);
pt[1] = Point2f(triangle[2], triangle[3]);
pt[2] = Point2f(triangle[4], triangle[5]);
if ( rect.contains(pt[0]) && rect.contains(pt[1]) && rect.contains(pt[2])){
for(int j = 0; j < 3; j++)
for(size_t k = 0; k < points.size(); k++)
if(abs(pt[j].x - points[k].x) < 1.0 && abs(pt[j].y - points[k].y) < 1)
ind[j] =(int) k;
Tri.push_back(ind);
}
}
}
void warpTriangle(Mat &img1, Mat &img2, vector<Point2f> &triangle1, vector<Point2f> &triangle2)
{
Rect rectangle1 = boundingRect(triangle1);
Rect rectangle2 = boundingRect(triangle2);
// Offset points by left top corner of the respective rectangles
vector<Point2f> triangle1Rect, triangle2Rect;
vector<Point> triangle2RectInt;
for(int i = 0; i < 3; i++)
{
triangle1Rect.push_back( Point2f( triangle1[i].x - rectangle1.x, triangle1[i].y - rectangle1.y) );
triangle2Rect.push_back( Point2f( triangle2[i].x - rectangle2.x, triangle2[i].y - rectangle2.y) );
triangle2RectInt.push_back( Point((int)(triangle2[i].x - rectangle2.x),(int) (triangle2[i].y - rectangle2.y))); // for fillConvexPoly
}
// Get mask by filling triangle
Mat mask = Mat::zeros(rectangle2.height, rectangle2.width, CV_32FC3);
fillConvexPoly(mask, triangle2RectInt, Scalar(1.0, 1.0, 1.0), 16, 0);
// Apply warpImage to small rectangular patches
Mat img1Rect;
img1(rectangle1).copyTo(img1Rect);
Mat img2Rect = Mat::zeros(rectangle2.height, rectangle2.width, img1Rect.type());
Mat warp_mat = getAffineTransform(triangle1Rect, triangle2Rect);
warpAffine( img1Rect, img2Rect, warp_mat, img2Rect.size(), INTER_LINEAR, BORDER_REFLECT_101);
multiply(img2Rect,mask, img2Rect);
multiply(img2(rectangle2), Scalar(1.0,1.0,1.0) - mask, img2(rectangle2));
img2(rectangle2) = img2(rectangle2) + img2Rect;
}
int main( int argc, char** argv)
{
//Give the path to the directory containing all the files containing data
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ image1 i1 | | (required) path to the first image file in which you want to apply swapping }"
"{ image2 i2 | | (required) path to the second image file in which you want to apply face swapping }"
"{ model m | | (required) path to the file containing model to be loaded for face landmark detection}"
"{ face_cascade f | | Path to the face cascade xml file which you want to use as a detector}"
);
// Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
Mat img1=imread(parser.get<string>("image1"));
Mat img2=imread(parser.get<string>("image2"));
if (img1.empty()||img2.empty()){
if(img1.empty()){
parser.printMessage();
cerr << parser.get<string>("image1")<<" not found" << endl;
return -1;
}
if (img2.empty()){
parser.printMessage();
cerr << parser.get<string>("image2")<<" not found" << endl;
return -1;
}
}
string modelfile_name(parser.get<string>("model"));
if (modelfile_name.empty()){
parser.printMessage();
cerr << "Model file name not found." << endl;
return -1;
}
string cascade_name(parser.get<string>("face_cascade"));
if (cascade_name.empty()){
parser.printMessage();
cerr << "The name of the cascade classifier to be loaded to detect faces is not found" << endl;
return -1;
}
//create a pointer to call the base class
//pass the face cascade xml file which you want to pass as a detector
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
Ptr<FacemarkKazemi> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
facemark->loadModel(modelfile_name);
cout<<"Loaded model"<<endl;
//vector to store the faces detected in the image
vector<Rect> faces1,faces2;
vector< vector<Point2f> > shape1,shape2;
//Detect faces in the current image
float ratio1 = (float)img1.cols/(float)img1.rows;
float ratio2 = (float)img2.cols/(float)img2.rows;
resize(img1,img1,Size((int)(640*ratio1),(int)(640*ratio1)));
resize(img2,img2,Size((int)(640*ratio2),(int)(640*ratio2)));
Mat img1Warped = img2.clone();
facemark->getFaces(img1,faces1);
facemark->getFaces(img2,faces2);
//Initialise the shape of the faces
facemark->fit(img1,faces1,shape1);
facemark->fit(img2,faces2,shape2);
unsigned long numswaps = (unsigned long)min((unsigned long)shape1.size(),(unsigned long)shape2.size());
for(unsigned long z=0;z<numswaps;z++){
vector<Point2f> points1 = shape1[z];
vector<Point2f> points2 = shape2[z];
img1.convertTo(img1, CV_32F);
img1Warped.convertTo(img1Warped, CV_32F);
// Find convex hull
vector<Point2f> boundary_image1;
vector<Point2f> boundary_image2;
vector<int> index;
convexHull(Mat(points2),index, false, false);
for(size_t i = 0; i < index.size(); i++)
{
boundary_image1.push_back(points1[index[i]]);
boundary_image2.push_back(points2[index[i]]);
}
// Triangulation for points on the convex hull
vector< vector<int> > triangles;
Rect rect(0, 0, img1Warped.cols, img1Warped.rows);
divideIntoTriangles(rect, boundary_image2, triangles);
// Apply affine transformation to Delaunay triangles
for(size_t i = 0; i < triangles.size(); i++)
{
vector<Point2f> triangle1, triangle2;
// Get points for img1, img2 corresponding to the triangles
for(int j = 0; j < 3; j++)
{
triangle1.push_back(boundary_image1[triangles[i][j]]);
triangle2.push_back(boundary_image2[triangles[i][j]]);
}
warpTriangle(img1, img1Warped, triangle1, triangle2);
}
// Calculate mask
vector<Point> hull;
for(size_t i = 0; i < boundary_image2.size(); i++)
{
Point pt((int)boundary_image2[i].x,(int)boundary_image2[i].y);
hull.push_back(pt);
}
Mat mask = Mat::zeros(img2.rows, img2.cols, img2.depth());
fillConvexPoly(mask,&hull[0],(int)hull.size(), Scalar(255,255,255));
// Clone seamlessly.
Rect r = boundingRect(boundary_image2);
Point center = (r.tl() + r.br()) / 2;
Mat output;
img1Warped.convertTo(img1Warped, CV_8UC3);
seamlessClone(img1Warped,img2, mask, center, output, NORMAL_CLONE);
imshow("Face_Swapped", output);
waitKey(0);
destroyAllWindows();
}
return 0;
}
\ No newline at end of file
modules/face/samples/sample_train_landmark_detector.cpp 0 → 100644
View file @ bccbec79
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/objdetect.hpp"
#include "opencv2/imgproc.hpp"
#include <iostream>
#include <vector>
#include <string>
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
int main(int argc,char** argv){
//Give the path to the directory containing all the files containing data
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ annotations a |. | (required) path to annotations txt file [example - /data/annotations.txt] }"
"{ config c | | (required) path to configuration xml file containing parameters for training.[ example - /data/config.xml] }"
"{ model m | | (required) path to configuration xml file containing parameters for training.[ example - /data/model.dat] }"
"{ width w | 460 | The width which you want all images to get to scale the annotations. large images are slow to process [default = 460] }"
"{ height h | 460 | The height which you want all images to get to scale the annotations. large images are slow to process [default = 460] }"
"{ face_cascade f | | Path to the face cascade xml file which you want to use as a detector}"
);
//Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
string directory(parser.get<string>("annotations"));
//default initialisation
Size scale(460,460);
scale = Size(parser.get<int>("width"),parser.get<int>("height"));
if (directory.empty()){
parser.printMessage();
cerr << "The name of the directory from which annotations have to be found is empty" << endl;
return -1;
}
string configfile_name(parser.get<string>("config"));
if (configfile_name.empty()){
parser.printMessage();
cerr << "No configuration file name found which contains the parameters for training" << endl;
return -1;
}
string modelfile_name(parser.get<string>("model"));
if (modelfile_name.empty()){
parser.printMessage();
cerr << "No name for the model_file found in which the trained model has to be saved" << endl;
return -1;
}
string cascade_name(parser.get<string>("face_cascade"));
if (cascade_name.empty()){
parser.printMessage();
cerr << "The name of the cascade classifier to be loaded to detect faces is not found" << endl;
return -1;
}
//create a vector to store names of files in which annotations
//and image names are found
/*The format of the file containing annotations should be of following format
/data/abc/abc.jpg
123.45,345.65
321.67,543.89
The above format is similar to HELEN dataset which is used for training model
*/
vector<String> filenames;
//reading the files from the given directory
glob(directory + "*.txt",filenames);
//create a pointer to call the base class
//pass the face cascade xml file which you want to pass as a detector
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<FacemarkKazemi> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
//create a vector to store image names
vector<String> imagenames;
//create object to get landmarks
vector< vector<Point2f> > trainlandmarks,Trainlandmarks;
//gets landmarks and corresponding image names in both the vectors
//vector to store images
vector<Mat> trainimages;
loadTrainingData(filenames,trainlandmarks,imagenames);
for(unsigned long i=0;i<300;i++){
string imgname = imagenames[i].substr(0, imagenames[i].size()-1);
string img = directory + string(imgname) + ".jpg";
Mat src = imread(img);
if(src.empty()){
cerr<<string("Image "+img+" not found\n.")<<endl;
continue;
}
trainimages.push_back(src);
Trainlandmarks.push_back(trainlandmarks[i]);
}
cout<<"Got data"<<endl;
facemark->training(trainimages,Trainlandmarks,configfile_name,scale,modelfile_name);
cout<<"Training complete"<<endl;
return 0;
}
\ No newline at end of file
modules/face/samples/sample_train_landmark_detector2.cpp 0 → 100644
View file @ bccbec79
/*----------------------------------------------
* the user should provides the list of training images_train
* accompanied by their corresponding landmarks location in separated files.
* example of contents for images.txt:
* ../trainset/image_0001.png
* ../trainset/image_0002.png
* example of contents for annotation.txt:
* ../trainset/image_0001.pts
* ../trainset/image_0002.pts
* where the image_xxxx.pts contains the position of each face landmark.
* example of the contents:
* version: 1
* n_points: 68
* {
* 115.167660 220.807529
* 116.164839 245.721357
* 120.208690 270.389841
* ...
* }
* example of the dataset is available at https://ibug.doc.ic.ac.uk/resources/facial-point-annotations/
*--------------------------------------------------*/
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/objdetect.hpp"
#include <iostream>
#include <vector>
#include <string>
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
int main(int argc,char** argv){
//Give the path to the directory containing all the files containing data
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ images i | | (required) path to images txt file [example - /data/images.txt] }"
"{ annotations a |. | (required) path to annotations txt file [example - /data/annotations.txt] }"
"{ config c | | (required) path to configuration xml file containing parameters for training.[example - /data/config.xml] }"
"{ model m | | (required) path to file containing trained model for face landmark detection[example - /data/model.dat] }"
"{ width w | 460 | The width which you want all images to get to scale the annotations. large images are slow to process [default = 460] }"
"{ height h | 460 | The height which you want all images to get to scale the annotations. large images are slow to process [default = 460] }"
"{ face_cascade f | | Path to the face cascade xml file which you want to use as a detector}"
);
// Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
string annotations(parser.get<string>("annotations"));
string imagesList(parser.get<string>("images"));
//default initialisation
Size scale(460,460);
scale = Size(parser.get<int>("width"),parser.get<int>("height"));
if (annotations.empty()){
parser.printMessage();
cerr << "Name for annotations file not found. Aborting...." << endl;
return -1;
}
if (imagesList.empty()){
parser.printMessage();
cerr << "Name for file containing image list not found. Aborting....." << endl;
return -1;
}
string configfile_name(parser.get<string>("config"));
if (configfile_name.empty()){
parser.printMessage();
cerr << "No configuration file name found which contains the parameters for training" << endl;
return -1;
}
string modelfile_name(parser.get<string>("model"));
if (modelfile_name.empty()){
parser.printMessage();
cerr << "No name for the model_file found in which the trained model has to be saved" << endl;
return -1;
}
string cascade_name(parser.get<string>("face_cascade"));
if (cascade_name.empty()){
parser.printMessage();
cerr << "The name of the cascade classifier to be loaded to detect faces is not found" << endl;
return -1;
}
//create a pointer to call the base class
//pass the face cascade xml file which you want to pass as a detector
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<FacemarkKazemi> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
std::vector<String> images;
std::vector<std::vector<Point2f> > facePoints;
loadTrainingData(imagesList, annotations, images, facePoints, 0.0);
//gets landmarks and corresponding image names in both the vectors
vector<Mat> Trainimages;
std::vector<std::vector<Point2f> > Trainlandmarks;
//vector to store images
Mat src;
for(unsigned long i=0;i<images.size();i++){
src = imread(images[i]);
if(src.empty()){
cout<<images[i]<<endl;
cerr<<string("Image not found\n.Aborting...")<<endl;
continue;
}
Trainimages.push_back(src);
Trainlandmarks.push_back(facePoints[i]);
}
cout<<"Got data"<<endl;
facemark->training(Trainimages,Trainlandmarks,configfile_name,scale,modelfile_name);
cout<<"Training complete"<<endl;
return 0;
}
\ No newline at end of file
modules/face/samples/samplewriteconfigfile.cpp 0 → 100644
View file @ bccbec79
#include "opencv2/core.hpp"
#include <iostream>
#include <string>
using namespace cv;
using namespace std;
int main(int argc,const char ** argv){
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ filename f |. | (required) path to file which you want to create as config file [example - /data/config.xml] }"
"{ cascade_depth cd | 10 | (required) This stores the depth of cascade of regressors used for training.}"
"{ tree_depth td | 4 | (required) This stores the depth of trees created as weak learners during gradient boosting.}"
"{ num_trees_per_cascade_level| 500 | (required) This stores number of trees required per cascade level.}"
"{ learning_rate | 0.1 | (required) This stores the learning rate for gradient boosting.}"
"{ oversampling_amount | 20 | (required) This stores the oversampling amount for the samples.}"
"{ num_test_coordinates | 400 | (required) This stores number of test coordinates required for making the split.}"
"{ lambda | 0.1 | (required) This stores the value used for calculating the probabilty.}"
"{ num_test_splits | 20 | (required) This stores the number of test splits to be generated before making the best split.}"
);
// Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
//These variables have been initialised as defined in the research paper "One millisecond face alignment" CVPR 2014
int cascade_depth = 15;
int tree_depth = 4;
int num_trees_per_cascade_level = 500;
float learning_rate = float(0.1);
int oversampling_amount = 20;
int num_test_coordinates = 400;
float lambda = float(0.1);
int num_test_splits = 20;
cascade_depth = parser.get<int>("cascade_depth");
tree_depth = parser.get<int>("tree_depth");
num_trees_per_cascade_level = parser.get<int>("num_trees_per_cascade_level");
learning_rate = parser.get<float>("learning_rate");
oversampling_amount = parser.get<int>("oversampling_amount");
num_test_coordinates = parser.get<int>("num_test_coordinates");
lambda = parser.get<float>("lambda");
num_test_splits = parser.get<int>("num_test_splits");
string filename(parser.get<string>("filename"));
FileStorage fs(filename, FileStorage::WRITE);
if (!fs.isOpened())
{
cerr << "Failed to open " << filename << endl;
parser.printMessage();
return -1;
}
fs << "cascade_depth" << cascade_depth;
fs << "tree_depth"<< tree_depth;
fs << "num_trees_per_cascade_level" << num_trees_per_cascade_level;
fs << "learning_rate" << learning_rate;
fs << "oversampling_amount" << oversampling_amount;
fs << "num_test_coordinates" << num_test_coordinates;
fs << "lambda" << lambda ;
fs << "num_test_splits"<< num_test_splits;
fs.release();
cout << "Write Done." << endl;
return 0;
}
modules/face/src/face_alignment.cpp 0 → 100644
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#include "precomp.hpp"
#include "face_alignmentimpl.hpp"
#include <vector>
using namespace std;
namespace cv{
namespace face{
FacemarkKazemi::~FacemarkKazemi(){}
FacemarkKazemiImpl:: ~FacemarkKazemiImpl(){}
unsigned long FacemarkKazemiImpl::left(unsigned long index){
return 2*index+1;
}
unsigned long FacemarkKazemiImpl::right(unsigned long index){
return 2*index+2;
}
bool FacemarkKazemiImpl::setFaceDetector(FN_FaceDetector f, void* userData){
faceDetector = f;
faceDetectorData = userData;
//printf("face detector is configured\n");
return true;
}
bool FacemarkKazemiImpl::getFaces(InputArray image, OutputArray faces)
{
CV_Assert(faceDetector);
return faceDetector(image, faces, faceDetectorData);
}
FacemarkKazemiImpl::FacemarkKazemiImpl(const FacemarkKazemi::Params& parameters) :
faceDetector(NULL),
faceDetectorData(NULL)
{
minmeanx=8000.0;
maxmeanx=0.0;
minmeany=8000.0;
maxmeany=0.0;
isModelLoaded =false;
params = parameters;
}
FacemarkKazemi::Params::Params(){
//These variables are used for training data
//These are initialised as described in the research paper
//referenced above
cascade_depth = 15;
tree_depth = 5;
num_trees_per_cascade_level = 500;
learning_rate = float(0.1);
oversampling_amount = 20;
num_test_coordinates = 500;
lambda = float(0.1);
num_test_splits = 20;
}
bool FacemarkKazemiImpl::convertToActual(Rect r,Mat &warp){
Point2f srcTri[3],dstTri[3];
srcTri[0]=Point2f(0,0);
srcTri[1]=Point2f(1,0);
srcTri[2]=Point2f(0,1);
dstTri[0]=Point2f((float)r.x,(float)r.y);
dstTri[1]=Point2f((float)r.x+r.width,(float)r.y);
dstTri[2]=Point2f((float)r.x,(float)r.y+(float)1.3*r.height);
warp=getAffineTransform(srcTri,dstTri);
return true;
}
bool FacemarkKazemiImpl::convertToUnit(Rect r,Mat &warp){
Point2f srcTri[3],dstTri[3];
dstTri[0]=Point2f(0,0);
dstTri[1]=Point2f(1,0);
dstTri[2]=Point2f(0,1);
srcTri[0]=Point2f((float)r.x,(float)r.y);
srcTri[1]=Point2f((float)r.x+r.width,(float)r.y);
srcTri[2]=Point2f((float)r.x,(float)r.y+(float)1.3*r.height);
warp=getAffineTransform(srcTri,dstTri);
return true;
}
bool FacemarkKazemiImpl::setMeanExtreme(){
if(meanshape.empty()){
String error_message = "Model not loaded properly.No mean shape found.Aborting...";
CV_ErrorNoReturn(Error::StsBadArg, error_message);
return false;
}
for(size_t i=0;i<meanshape.size();i++){
if(meanshape[i].x>maxmeanx)
maxmeanx = meanshape[i].x;
if(meanshape[i].x<minmeanx)
minmeanx = meanshape[i].x;
if(meanshape[i].y>maxmeany)
maxmeany = meanshape[i].y;
if(meanshape[i].y<minmeany)
minmeany = meanshape[i].y;
}
return true;
}
bool FacemarkKazemiImpl::calcMeanShape (vector< vector<Point2f> >& trainlandmarks,vector<Mat>& trainimages,std::vector<Rect>& faces){
//clear the loaded meanshape
if(trainimages.empty()||trainlandmarks.size()!=trainimages.size()) {
// throw error if no data (or simply return -1?)
CV_ErrorNoReturn(Error::StsBadArg, "Number of images is not equal to corresponding landmarks. Aborting...");
}
meanshape.clear();
vector<Mat> finalimages;
vector< vector<Point2f> > finallandmarks;
float xmean[200] = {0.0};
//array to store mean of y coordinates
float ymean[200] = {0.0};
size_t k=0;
//loop to calculate mean
Mat warp_mat,src,C,D;
vector<Rect> facesp;
Rect face;
for(size_t i = 0;i < trainimages.size();i++){
src = trainimages[i].clone();
//get bounding rectangle of image for reference
//function from facemark class
facesp.clear();
if(!getFaces(src,facesp)){
continue;
}
if(facesp.size()>1||facesp.empty())
continue;
face = facesp[0];
convertToUnit(face,warp_mat);
//loop to bring points to a common reference and adding
for(k=0;k<trainlandmarks[i].size();k++){
Point2f pt=trainlandmarks[i][k];
C = (Mat_<double>(3,1) << pt.x, pt.y, 1);
D = warp_mat*C;
pt.x = float(D.at<double>(0,0));
pt.y = float(D.at<double>(1,0));
trainlandmarks[i][k] = pt;
xmean[k] = xmean[k]+pt.x;
ymean[k] = ymean[k]+pt.y;
}
finalimages.push_back(trainimages[i]);
finallandmarks.push_back(trainlandmarks[i]);
faces.push_back(face);
}
//dividing by size to get mean and initialize meanshape
for(size_t i=0;i<k;i++){
xmean[i]=xmean[i]/finalimages.size();
ymean[i]=ymean[i]/finalimages.size();
if(xmean[i]>maxmeanx)
maxmeanx = xmean[i];
if(xmean[i]<minmeanx)
minmeanx = xmean[i];
if(ymean[i]>maxmeany)
maxmeany = ymean[i];
if(ymean[i]<minmeany)
minmeany = ymean[i];
meanshape.push_back(Point2f(xmean[i],ymean[i]));
}
trainimages.clear();
trainlandmarks.clear();
trainimages = finalimages;
trainlandmarks = finallandmarks;
finalimages.clear();
finallandmarks.clear();
return true;
}
bool FacemarkKazemiImpl::scaleData( vector< vector<Point2f> > & trainlandmarks,
vector<Mat> & trainimages ,Size s)
{
if(trainimages.empty()||trainimages.size()!=trainlandmarks.size()){
// throw error if no data (or simply return -1?)
CV_ErrorNoReturn(Error::StsBadArg, "The data is not loaded properly by train function. Aborting...");
}
float scalex,scaley;
//scale all images and their landmarks according to input size
for(size_t i=0;i< trainimages.size();i++){
//calculating scale for x and y axis
scalex=float(s.width)/float(trainimages[i].cols);
scaley=float(s.height)/float(trainimages[i].rows);
resize(trainimages[i],trainimages[i],s);
for (vector<Point2f>::iterator it = trainlandmarks[i].begin(); it != trainlandmarks[i].end(); it++) {
Point2f pt = (*it);
pt.x = pt.x*scalex;
pt.y = pt.y*scaley;
(*it) = pt;
}
}
return true;
}
Ptr<FacemarkKazemi> FacemarkKazemi::create(const FacemarkKazemi::Params &parameters){
return Ptr<FacemarkKazemiImpl>(new FacemarkKazemiImpl(parameters));
}
}//cv
}//face
\ No newline at end of file
modules/face/src/face_alignmentimpl.hpp 0 → 100644
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#ifndef __OPENCV_FACE_ALIGNMENTIMPL_HPP__
#define __OPENCV_FACE_ALIGNMENTIMPL_HPP__
#include "opencv2/face.hpp"
#include <string>
#include <sstream>
#include <vector>
#include <fstream>
#include <queue>
#include <algorithm>
#include <ctime>
using namespace std;
namespace cv{
namespace face{
/**@brief structure determining split in regression tree
*/
struct splitr{
//!index1 Index of the first coordinates among the test coordinates for deciding split.
uint64_t index1;
//! index2 index of the second coordinate among the test coordinates for deciding split.
uint64_t index2;
//! thresh threshold for deciding the split.
float thresh;
};
/** @brief represents a node of the regression tree*/
struct node_info{
//First pixel coordinate of split
long index1;
//Second pixel coordinate .split
long index2;
long depth;
long node_no;
};
/** @brief regression tree structure. Each leaf node is a vector storing residual shape.
* The tree is represented as vector of leaves.
*/
struct tree_node{
splitr split;
std::vector<Point2f> leaf;
};
struct regtree{
std::vector<tree_node> nodes;
};
/** @brief Represents a training sample
*It contains current shape, difference between actual shape
*and current shape. It also stores the image whose shape is being
*detected.
*/
struct training_sample{
//! shapeResiduals vector which stores the residual shape remaining to be corrected.
std::vector<Point2f> shapeResiduals;
//! current_shape vector containing current estimate of the shape
std::vector<Point2f> current_shape;
//! actual_shape vector containing the actual shape of the face or the ground truth.
std::vector<Point2f> actual_shape;
//! image A mat object which stores the image.
Mat image ;
//! pixel_intensities vector containing pixel intensities of the coordinates chosen for testing
std::vector<int> pixel_intensities;
//! pixel_coordinates vector containing pixel coordinates used for testing
std::vector<Point2f> pixel_coordinates;
//! bound Rectangle enclosing the face found in the image for training
Rect bound;
};
class FacemarkKazemiImpl : public FacemarkKazemi{
public:
FacemarkKazemiImpl(const FacemarkKazemi::Params& parameters);
void loadModel(String fs);
bool setFaceDetector(FN_FaceDetector f, void* userdata);
bool getFaces(InputArray image, OutputArray faces);
bool fit(InputArray image, InputArray faces, InputOutputArray landmarks );
void training(String imageList, String groundTruth);
bool training(vector<Mat>& images, vector< vector<Point2f> >& landmarks,string filename,Size scale,string modelFilename);
// Destructor for the class.
virtual ~FacemarkKazemiImpl();
protected:
FacemarkKazemi::Params params;
float minmeanx;
float maxmeanx;
float minmeany;
float maxmeany;
bool isModelLoaded;
/* meanshape This is a vector which stores the mean shape of all the images used in training*/
std::vector<Point2f> meanshape;
std::vector< std::vector<regtree> > loaded_forests;
std::vector< std::vector<Point2f> > loaded_pixel_coordinates;
FN_FaceDetector faceDetector;
void* faceDetectorData;
bool findNearestLandmarks(std::vector< std::vector<int> >& nearest);
/*Extract left node of the current node in the regression tree*/
unsigned long left(unsigned long index);
// Extract the right node of the current node in the regression tree
unsigned long right(unsigned long index);
// This function randomly generates test splits to get the best split.
splitr getTestSplits(std::vector<Point2f> pixel_coordinates,int seed);
// This function writes a split node to the XML file storing the trained model
void writeSplit(std::ofstream& os,const splitr split);
// This function writes a leaf node to the binary file storing the trained model
void writeLeaf(std::ofstream& os, const std::vector<Point2f> &leaf);
// This function writes a tree to the binary file containing the model
void writeTree(std::ofstream &f,regtree tree);
// This function saves the pixel coordinates to a binary file
void writePixels(std::ofstream& f,int index);
// This function saves model to the binary file
bool saveModel(String filename);
// This funcrion reads pixel coordinates from the model file
void readPixels(std::ifstream& is,uint64_t index);
//This function reads the split node of the tree from binary file
void readSplit(std::ifstream& is, splitr &vec);
//This function reads a leaf node of the tree.
void readLeaf(std::ifstream& is, std::vector<Point2f> &leaf);
/* This function generates pixel intensities of the randomly generated test coordinates used to decide the split.
*/
bool getPixelIntensities(Mat img,std::vector<Point2f> pixel_coordinates_,std::vector<int>& pixel_intensities_,Rect face);
//This function initialises the training parameters.
bool setTrainingParameters(String filename);
//This function finds a warp matrix that warp the pixels from the normalised space to the actual space
bool convertToActual(Rect r,Mat &warp);
//This function finds a warp matrix that warps the pixels from the actual space to normaluised space
bool convertToUnit(Rect r,Mat &warp);
/** @brief This function calculates mean shape while training.
* This function is only called when new training data is supplied by the train function.
*@param trainlandmarks A vector of type cv::Point2f which stores the landmarks of corresponding images.
*@param trainimages A vector of type cv::Mat which stores the images which serve as training data.
*@param faces A vector of type cv::Rect which stores the bounding recatngle of each training image
*@returns A boolean value. It returns true if mean shape is found successfully else returns false.
*/
bool calcMeanShape(std::vector< std::vector<Point2f> > & trainlandmarks,std::vector<Mat>& trainimages,std::vector<Rect>& faces);
/** @brief This functions scales the annotations to a common size which is considered same for all images.
* @param trainlandmarks A vector of type cv::Point2f stores the landmarks of the corresponding training images.
* @param trainimages A vector of type cv::Mat which stores the images which are to be scaled.
* @param s A variable of type cv::Size stores the common size to which all the images are scaled.
* @returns A boolean value. It returns true when data is scaled properly else returns false.
*/
bool scaleData(std::vector< std::vector<Point2f> >& trainlandmarks,
std::vector<Mat>& trainimages , Size s=Size(460,460) );
// This function gets the landmarks in the meanshape nearest to the pixel coordinates.
unsigned long getNearestLandmark (Point2f pixels );
// This function gets the relative position of the test pixel coordinates relative to the current shape.
bool getRelativePixels(std::vector<Point2f> sample,std::vector<Point2f>& pixel_coordinates , std::vector<int> nearest_landmark = std::vector<int>());
// This function partitions samples according to the split
unsigned long divideSamples (splitr split,std::vector<training_sample>& samples,unsigned long start,unsigned long end);
// This function fits a regression tree according to the shape residuals calculated to give weak learners for GBT algorithm.
bool buildRegtree(regtree &tree,std::vector<training_sample>& samples,std::vector<Point2f> pixel_coordinates);
// This function greedily decides the best split among the test splits generated.
bool getBestSplit(std::vector<Point2f> pixel_coordinates, std::vector<training_sample>& samples,unsigned long start ,
unsigned long end,splitr& split,std::vector< std::vector<Point2f> >& sum,long node_no);
// This function randomly generates test coordinates for each level of cascade.
void getTestCoordinates ();
// This function implements gradient boosting by fitting regression trees
std::vector<regtree> gradientBoosting(std::vector<training_sample>& samples,std::vector<Point2f> pixel_coordinates);
// This function creates training sample by randomly assigning a current shape from set of shapes available.
void createLeafNode(regtree& tree,long node_no,std::vector<Point2f> assign);
// This function creates a split node in the regression tree.
void createSplitNode(regtree& tree, splitr split,long node_no);
// This function prepares the training samples
bool createTrainingSamples(std::vector<training_sample> &samples,std::vector<Mat> images,std::vector< std::vector<Point2f> > landmarks,
std::vector<Rect> rectangle);
//This function generates a split
bool generateSplit(std::queue<node_info>& curr,std::vector<Point2f> pixel_coordinates, std::vector<training_sample>& samples,
splitr &split , std::vector< std::vector<Point2f> >& sum);
bool setMeanExtreme();
//friend class getRelShape;
friend class getRelPixels;
};
}//face
}//cv
#endif
\ No newline at end of file
modules/face/src/facemark.cpp
View file @ bccbec79
......@@ -3,16 +3,14 @@
// of this distribution and at http://opencv.org/license.html.
/*
This file contains results of GSoC Project: Facemark API for OpenCV
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
#include "precomp.hpp"
#include "opencv2/face.hpp"
#include "opencv2/core.hpp"
/*dataset parser*/
#include <fstream>
......@@ -204,6 +202,59 @@ bool loadFacePoints(String filename, OutputArray points, float offset){
return true;
}
bool getFacesHAAR(InputArray image, OutputArray faces, const String& face_cascade_name)
{
Mat gray;
vector<Rect> roi;
CascadeClassifier face_cascade;
CV_Assert(face_cascade.load(face_cascade_name) && "Can't loading face_cascade");
cvtColor(image.getMat(), gray, COLOR_BGR2GRAY);
equalizeHist(gray, gray);
face_cascade.detectMultiScale(gray, roi, 1.1, 2, 0|CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(roi).copyTo(faces);
return true;
}
bool loadTrainingData(vector<String> filename,vector< vector<Point2f> >
& trainlandmarks,vector<String> & trainimages)
{
string img;
vector<Point2f> temp;
string s;
string tok;
vector<string> coordinates;
ifstream f1;
for(unsigned long j=0;j<filename.size();j++){
f1.open(filename[j].c_str(),ios::in);
if(!f1.is_open()){
cout<<filename[j]<<endl;
CV_ErrorNoReturn(Error::StsError, "File can't be opened for reading!");
return false;
}
//get the path of the image whose landmarks have to be detected
getline(f1,img);
//push the image paths in the vector
trainimages.push_back(img);
img.clear();
while(getline(f1,s)){
Point2f pt;
stringstream ss(s); // Turn the string into a stream.
while(getline(ss, tok,',')) {
coordinates.push_back(tok);
tok.clear();
}
pt.x = (float)atof(coordinates[0].c_str());
pt.y = (float)atof(coordinates[1].c_str());
coordinates.clear();
temp.push_back(pt);
}
trainlandmarks.push_back(temp);
temp.clear();
f1.close();
}
return true;
}
void drawFacemarks(InputOutputArray image, InputArray points, Scalar color){
Mat img = image.getMat();
vector<Point2f> pts = points.getMat();
......
modules/face/src/facemarkAAM.cpp
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
/*
This file contains results of GSoC Project: Facemark API for OpenCV
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
......
modules/face/src/facemarkLBF.cpp
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
/*
This file contains results of GSoC Project: Facemark API for OpenCV
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
......
modules/face/src/getlandmarks.cpp 0 → 100644
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#include "precomp.hpp"
#include "face_alignmentimpl.hpp"
#include <fstream>
#include <ctime>
using namespace std;
namespace cv{
namespace face{
bool FacemarkKazemiImpl :: findNearestLandmarks( vector< vector<int> >& nearest){
if(meanshape.empty()||loaded_pixel_coordinates.empty()){
String error_message = "Model not loaded properly.Aborting...";
CV_Error(Error::StsBadArg, error_message);
return false;
}
nearest.resize(loaded_pixel_coordinates.size());
for(unsigned long i=0 ; i< loaded_pixel_coordinates.size(); i++){
for(unsigned long j = 0;j<loaded_pixel_coordinates[i].size();j++){
nearest[i].push_back(getNearestLandmark(loaded_pixel_coordinates[i][j]));
}
}
return true;
}
void FacemarkKazemiImpl :: readSplit(ifstream& is, splitr &vec)
{
is.read((char*)&vec, sizeof(splitr));
}
void FacemarkKazemiImpl :: readLeaf(ifstream& is, vector<Point2f> &leaf)
{
uint64_t size;
is.read((char*)&size, sizeof(size));
leaf.resize((size_t)size);
is.read((char*)&leaf[0], leaf.size() * sizeof(Point2f));
}
void FacemarkKazemiImpl :: readPixels(ifstream& is,uint64_t index)
{
is.read((char*)&loaded_pixel_coordinates[(unsigned long)index][0], loaded_pixel_coordinates[(unsigned long)index].size() * sizeof(Point2f));
}
void FacemarkKazemiImpl :: loadModel(String filename){
if(filename.empty()){
String error_message = "No filename found.Aborting....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
ifstream f(filename.c_str(),ios::binary);
if(!f.is_open()){
String error_message = "No file with given name found.Aborting....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
uint64_t len;
f.read((char*)&len, sizeof(len));
char* temp = new char[(size_t)len+1];
f.read(temp, len);
temp[len] = '\0';
string s(temp);
delete [] temp;
if(s.compare("cascade_depth")!=0){
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
uint64_t cascade_size;
f.read((char*)&cascade_size,sizeof(cascade_size));
loaded_forests.resize((unsigned long)cascade_size);
f.read((char*)&len, sizeof(len));
temp = new char[(unsigned long)len+1];
f.read(temp, len);
temp[len] = '\0';
s = string(temp);
delete [] temp;
if(s.compare("pixel_coordinates")!=0){
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
loaded_pixel_coordinates.resize((unsigned long)cascade_size);
uint64_t num_pixels;
f.read((char*)&num_pixels,sizeof(num_pixels));
for(unsigned long i=0 ; i < cascade_size ; i++){
loaded_pixel_coordinates[i].resize((unsigned long)num_pixels);
readPixels(f,i);
}
f.read((char*)&len, sizeof(len));
temp = new char[(unsigned long)len+1];
f.read(temp, len);
temp[len] = '\0';
s = string(temp);
delete [] temp;
if(s.compare("mean_shape")!=0){
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
uint64_t mean_shape_size;
f.read((char*)&mean_shape_size,sizeof(mean_shape_size));
meanshape.resize((unsigned long)mean_shape_size);
f.read((char*)&meanshape[0], meanshape.size() * sizeof(Point2f));
if(!setMeanExtreme())
exit(0);
f.read((char*)&len, sizeof(len));
temp = new char[(unsigned long)len+1];
f.read(temp, len);
temp[len] = '\0';
s = string(temp);
delete [] temp;
if(s.compare("num_trees")!=0){
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
uint64_t num_trees;
f.read((char*)&num_trees,sizeof(num_trees));
for(unsigned long i=0;i<cascade_size;i++){
for(unsigned long j=0;j<num_trees;j++){
regtree tree;
f.read((char*)&len, sizeof(len));
char* temp2 = new char[(unsigned long)len+1];
f.read(temp2, len);
temp2[len] = '\0';
s =string(temp2);
delete [] temp2;
if(s.compare("num_nodes")!=0){
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
uint64_t num_nodes;
f.read((char*)&num_nodes,sizeof(num_nodes));
tree.nodes.resize((unsigned long)num_nodes+1);
for(unsigned long k=0; k < num_nodes ; k++){
f.read((char*)&len, sizeof(len));
char* temp3 = new char[(unsigned long)len+1];
f.read(temp3, len);
temp3[len] = '\0';
s =string(temp3);
delete [] temp3;
tree_node node;
if(s.compare("split")==0){
splitr split;
readSplit(f,split);
node.split = split;
node.leaf.clear();
}
else if(s.compare("leaf")==0){
vector<Point2f> leaf;
readLeaf(f,leaf);
node.leaf = leaf;
}
else{
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
tree.nodes[k]=node;
}
loaded_forests[i].push_back(tree);
}
}
f.close();
isModelLoaded = true;
}
bool FacemarkKazemiImpl::fit(InputArray img, InputArray roi, InputOutputArray landmarks){
if(!isModelLoaded){
String error_message = "No model loaded. Aborting....";
CV_Error(Error::StsBadArg, error_message);
return false;
}
Mat image = img.getMat();
std::vector<Rect> & faces = *(std::vector<Rect>*)roi.getObj();
std::vector<std::vector<Point2f> > & shapes = *(std::vector<std::vector<Point2f> >*) landmarks.getObj();
shapes.resize(faces.size());
if(image.empty()){
String error_message = "No image found.Aborting..";
CV_Error(Error::StsBadArg, error_message);
return false;
}
if(faces.empty()){
String error_message = "No faces found.Aborting..";
CV_Error(Error::StsBadArg, error_message);
return false;
}
if(meanshape.empty()||loaded_forests.empty()||loaded_pixel_coordinates.empty()){
String error_message = "Model not loaded properly.Aborting...";
CV_Error(Error::StsBadArg, error_message);
return false;
}
if(loaded_forests.size()==0){
String error_message = "Model not loaded properly.Aboerting...";
CV_Error(Error::StsBadArg, error_message);
return false;
}
if(loaded_pixel_coordinates.size()==0){
String error_message = "Model not loaded properly.Aboerting...";
CV_Error(Error::StsBadArg, error_message);
return false;
}
vector< vector<int> > nearest_landmarks;
findNearestLandmarks(nearest_landmarks);
tree_node curr_node;
vector<Point2f> pixel_relative;
vector<int> pixel_intensity;
Mat warp_mat;
for(size_t e=0;e<faces.size();e++){
shapes[e]=meanshape;
convertToActual(faces[e],warp_mat);
for(size_t i=0;i<loaded_forests.size();i++){
pixel_intensity.clear();
pixel_relative = loaded_pixel_coordinates[i];
getRelativePixels(shapes[e],pixel_relative,nearest_landmarks[i]);
getPixelIntensities(image,pixel_relative,pixel_intensity,faces[e]);
for(size_t j=0;j<loaded_forests[i].size();j++){
regtree tree = loaded_forests[i][j];
curr_node = tree.nodes[0];
unsigned long curr_node_index = 0;
while(curr_node.leaf.size()==0)
{
if ((float)pixel_intensity[(unsigned long)curr_node.split.index1] - (float)pixel_intensity[(unsigned long)curr_node.split.index2] > curr_node.split.thresh)
{
curr_node_index=left(curr_node_index);
} else
curr_node_index=right(curr_node_index);
curr_node = tree.nodes[curr_node_index];
}
for(size_t p=0;p<curr_node.leaf.size();p++){
shapes[e][p]=shapes[e][p] + curr_node.leaf[p];
}
}
}
for(unsigned long j=0;j<shapes[e].size();j++){
Mat C = (Mat_<double>(3,1) << shapes[e][j].x, shapes[e][j].y, 1);
Mat D = warp_mat*C;
shapes[e][j].x=float(D.at<double>(0,0));
shapes[e][j].y=float(D.at<double>(1,0));
}
}
return true;
}
}//cv
}//face
\ No newline at end of file
modules/face/src/regtree.cpp 0 → 100644
View file @ bccbec79
This diff is collapsed.
modules/face/src/trainFacemark.cpp 0 → 100644
View file @ bccbec79
This diff is collapsed.
modules/face/test/test_face_align.cpp 0 → 100644
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#include "test_precomp.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/face.hpp"
#include "opencv2/objdetect.hpp"
#include <vector>
#include <string>
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector( InputArray image, OutputArray ROIs, CascadeClassifier* face_cascade)
{
Mat gray;
std::vector<Rect> faces;
if(image.channels()>1){
cvtColor(image.getMat(),gray,COLOR_BGR2GRAY);
}
else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
face_cascade->detectMultiScale( gray, faces, 1.1, 3, 0, Size(30, 30) );
Mat(faces).copyTo(ROIs);
return true;
}
TEST(CV_Face_FacemarkKazemi, can_create_default) {
string cascade_name = cvtest::findDataFile("face/lbpcascade_frontalface_improved.xml", true);
string configfile_name = cvtest::findDataFile("face/config.xml", true);
CascadeClassifier face_cascade;
EXPECT_TRUE(face_cascade.load(cascade_name));
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<FacemarkKazemi> facemark;
EXPECT_NO_THROW(facemark = FacemarkKazemi::create(params));
EXPECT_TRUE(facemark->setFaceDetector((cv::face::FN_FaceDetector)myDetector, &face_cascade));
EXPECT_FALSE(facemark.empty());
}
TEST(CV_Face_FacemarkKazemi, can_loadTrainingData) {
string filename = cvtest::findDataFile("face/lbpcascade_frontalface_improved.xml", true);
string configfile_name = cvtest::findDataFile("face/config.xml", true);
CascadeClassifier face_cascade;
EXPECT_TRUE(face_cascade.load(filename));
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<FacemarkKazemi> facemark;
EXPECT_NO_THROW(facemark = FacemarkKazemi::create(params));
EXPECT_TRUE(facemark->setFaceDetector((cv::face::FN_FaceDetector)myDetector, &face_cascade));
vector<String> filenames;
filename = cvtest::findDataFile("face/1.txt", true);
filenames.push_back(filename);
filename = cvtest::findDataFile("face/2.txt", true);
filenames.push_back(filename);
vector<String> imagenames;
vector< vector<Point2f> > trainlandmarks,Trainlandmarks;
vector<Rect> rectangles;
//Test getData function
EXPECT_NO_THROW(loadTrainingData(filenames,trainlandmarks,imagenames));
vector<Mat> trainimages;
for(unsigned long i=0;i<imagenames.size();i++){
string img = cvtest::findDataFile(imagenames[i], true);
Mat src = imread(img);
EXPECT_TRUE(!src.empty());
trainimages.push_back(src);
Trainlandmarks.push_back(trainlandmarks[i]);
}
string modelfilename = "face_landmark_model.dat";
Size scale = Size(460,460);
EXPECT_TRUE(facemark->training(trainimages,Trainlandmarks,configfile_name,scale,modelfilename));
}
TEST(CV_Face_FacemarkKazemi, can_detect_landmarks) {
string cascade_name = cvtest::findDataFile("face/lbpcascade_frontalface_improved.xml", true);
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
Ptr<FacemarkKazemi> facemark;
EXPECT_NO_THROW(facemark = FacemarkKazemi::create(params));
EXPECT_TRUE(facemark->setFaceDetector((cv::face::FN_FaceDetector)myDetector, &face_cascade));
string imgname = cvtest::findDataFile("face/detect.jpg");
string modelfilename = cvtest::findDataFile("face/face_landmark_model.dat",true);
Mat img = imread(imgname);
EXPECT_TRUE(!img.empty());
EXPECT_FALSE(facemark.empty());
EXPECT_NO_THROW(facemark->loadModel(modelfilename));
vector<Rect> faces;
//Detect faces in the current image
EXPECT_TRUE(facemark->getFaces(img,faces));
//vector to store the landmarks of all the faces in the image
vector< vector<Point2f> > shapes;
EXPECT_NO_THROW(facemark->fit(img,faces,shapes));
shapes.clear();
}
modules/face/test/test_facemark.cpp
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
/*
This file contains results of GSoC Project: Facemark API for OpenCV
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
......
modules/face/test/test_facemark_aam.cpp
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
/*
This file contains results of GSoC Project: Facemark API for OpenCV
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
......
modules/face/test/test_facemark_lbf.cpp
View file @ bccbec79
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
/*
This file contains results of GSoC Project: Facemark API for OpenCV
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
......
modules/face/tutorials/face_landmark/face_landmark_detection.markdown 0 → 100644
View file @ bccbec79
Face landmark detection in an image {#tutorial_face_landmark_detection_in_an_image}
===================================
![](images/facereg.jpg)
This application lets you detect landmarks of detected faces in an image. You can detect landmarks of all the faces found in an image
and use them further in various applications like face swapping, face averaging etc.
This functionality is now available in OpenCV.
```
// Command to be typed for running the sample
./sampleDetectLandmarks -file=trained_model.dat -face_cascade=lbpcascadefrontalface.xml -image=/path_to_image/image.jpg
```
### Description of command parameters {tutorial_face_training_parameters}
> * **model_filename** f : (REQUIRED) A path to binary file storing the trained model which is to be loaded [example - /data/file.dat]
> * **image** i : (REQUIRED) A path to image in which face landmarks have to be detected.[example - /data/image.jpg]
> * **face_cascade** c : (REQUIRED) A path to the face cascade xml file which you want to use as a face detector.
Understanding code
------------------
![](images/d.png)
This tutorial will explain the sample code for face landmark detection. Jumping directly to the code :
``` c++
CascadeClassifier face_cascade;
bool myDetector( InputArray image, OutputArray ROIs );
bool myDetector( InputArray image, OutputArray ROIs ){
Mat gray;
std::vector<Rect> faces;
if(image.channels()>1){
cvtColor(image.getMat(),gray,CV_BGR2GRAY);
}
else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
Mat(faces).copyTo(ROIs);
return true;
}
```
The facemark API provides the functionality to the user to use their own face detector to be used in training.The above code creartes a sample face detector. The above function would be passed to a function pointer in the facemark API.
``` c++
Mat img = imread(image);
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<Facemark> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector(myDetector);
```
The above code creates a pointer of the face landmark detection class. The face detector created above has to be passed
as function pointer to the facemark pointer created for detecting faces while landmark detection. The above code also loads the image
in which landmarks have to be detected.
``` c++
facemark->loadModel(filename);
cout<<"Loaded model"<<endl;
vector<Rect> faces;
resize(img,img,Size(460,460));
facemark->getFaces(img,faces);
vector< vector<Point2f> > shapes;
``` c++
The above code loads a trained model for face landmark detection and creates a vector to store the detected faces. It then resizes the image to a smaller size as processing speed is faster with small images. It then creates a vector of vector to store shapes for each face detected.
``` c++
if(facemark->fit(img,faces,shapes))
{
for( size_t i = 0; i < faces.size(); i++ )
{
cv::rectangle(img,faces[i],Scalar( 255, 0, 0 ));
}
for(unsigned long i=0;i<faces.size();i++){
for(unsigned long k=0;k<shapes[i].size();k++)
cv::circle(img,shapes[i][k],5,cv::Scalar(0,0,255),FILLED);
}
namedWindow("Detected_shape");
imshow("Detected_shape",img);
waitKey(0);
}
```
The above code then calls the function fit to get shapes of all detected faces in the image
and then draws the rectangles bounding the faces and marks the desired landmarks.
### Detection Results
![](ab.jpg)
![](ab-1.jpg)
\ No newline at end of file
modules/face/tutorials/face_landmark/face_landmark_trainer.markdown 0 → 100644
View file @ bccbec79
![](images/2.jpg)
Training face landmark detector{#tutorial_face_training_face_landmark_detector}
==============================
This application helps to train your own face landmark detector. You can train your own face landmark detection by just providing the paths for
directory containing the images and files containing their corresponding face landmarks. As this landmark detector was originally trained on
[HELEN dataset](http://www.ifp.illinois.edu/~vuongle2/helen/), the training follows the format of data provided in HELEN dataset.
The dataset consists of .txt files whose first line contains the image name which then follows the annotations.
The format of the file containing annotations should be of following format :
> /directory/images/abc.jpg
> 123.45,345.65
> 321.67,543.89
> .... , ....
> .... , ....
The above format is similar to HELEN dataset which is used for training the model.
```
// Command to be typed for running the sample
./sample_train_landmark_detector -annotations=/home/sukhad/Downloads/code/trainset/ -config=config.xml -face_cascade=lbpcascadefrontalface.xml -model=trained_model.dat -width=460 -height=460
```
### Description of command parameters
> * **annotations** a : (REQUIRED) Path to annotations txt file [example - /data/annotations.txt]
> * **config** c : (REQUIRED) Path to configuration xml file containing parameters for training.[ example - /data/config.xml]
> * **model** m : (REQUIRED) Path to configuration xml file containing parameters for training.[ example - /data/model.dat]
> * **width** w : (OPTIONAL) The width which you want all images to get to scale the annotations. Large images are slow to process [default = 460]
> * **height** h : (OPTIONAL) The height which you want all images to get to scale the annotations. Large images are slow to process [default = 460]
> * **face_cascade** f (REQUIRED) Path to the face cascade xml file which you want to use as a detector.
### Description of training parameters
The configuration file described above which is used while training contains the training parameters which are required for training.
**The description of parameters is as follows :**
1. **Cascade depth :** This stores the depth of cascade of regressors used for training.
2. **Tree depth :** This stores the depth of trees created as weak learners during gradient boosting.
3. **Number of trees per cascade level :** This stores number of trees required per cascade level.
4. **Learning rate :** This stores the learning rate for gradient boosting.This is required to prevent overfitting using shrinkage.
5. **Oversampling amount :** This stores the oversampling amount for the samples.
6. **Number of test coordinates :** This stores number of test coordinates to be generated as samples to decide for making the split.
7. **Lambda :** This stores the value used for calculating the probabilty which helps to select closer pixels for making the split.
8. **Number of test splits :** This stores the number of test splits to be generated before making the best split.
To get more detailed description about the training parameters you can refer to the [Research paper](https://pdfs.semanticscholar.org/d78b/6a5b0dcaa81b1faea5fb0000045a62513567.pdf).
### Understanding code
![](images/3.jpg)
Jumping directly to the code :
``` c++
CascadeClassifier face_cascade;
bool myDetector( InputArray image, OutputArray ROIs );
bool myDetector( InputArray image, OutputArray ROIs ){
Mat gray;
std::vector<Rect> faces;
if(image.channels()>1){
cvtColor(image.getMat(),gray,CV_BGR2GRAY);
}
else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
Mat(faces).copyTo(ROIs);
return true;
}
```
The facemark API provides the functionality to the user to use their own face detector to be used in training.The above code creartes a sample face detector. The above function would be passed to a function pointer in the facemark API.
``` c++
vector<String> filenames;
glob(directory,filenames);
```
The above code creates a vector filenames for storing the names of the .txt files.
It gets the filenames of the files in the directory.
``` c++
Mat img = imread(image);
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<Facemark> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector(myDetector);
```
The above code creates a pointer of the face landmark detection class. The face detector created above has to be passed
as function pointer to the facemark pointer created for detecting faces while training the model.
``` c++
vector<String> imagenames;
vector< vector<Point2f> > trainlandmarks,Trainlandmarks;
vector<Mat> trainimages;
loadTrainingData(filenames,trainlandmarks,imagenames);
for(unsigned long i=0;i<300;i++){
string imgname = imagenames[i].substr(0, imagenames[i].size()-1);
string img = directory + string(imgname) + ".jpg";
Mat src = imread(img);
if(src.empty()){
cerr<<string("Image "+img+" not found\n.")<<endl;
continue;
}
trainimages.push_back(src);
Trainlandmarks.push_back(trainlandmarks[i]);
}
```
The above code creates std::vectors to store the images and their corresponding landmarks.
The above code calls a function loadTrainingData to load the landmarks and the images into their respective vectors.
If the dataset you downloaded is of the following format :
```
version: 1
n_points: 68
{
115.167660 220.807529
116.164839 245.721357
120.208690 270.389841
...
}
This is the example of the dataset available at https://ibug.doc.ic.ac.uk/resources/facial-point-annotations/
```
Then skip the above code for loading training data and use the following code. This sample is provided as sampleTrainLandmarkDetector2.cpp
in the face module in opencv contrib.
``` c++
std::vector<String> images;
std::vector<std::vector<Point2f> > facePoints;
loadTrainingData(imagesList, annotations, images, facePoints, 0.0);
```
In the above code imagelist and annotations are the file of following format :
```
example of contents for images.txt:
../trainset/image_0001.png
../trainset/image_0002.png
example of contents for annotation.txt:
../trainset/image_0001.pts
../trainset/image_0002.pts
```
These symbolize the names of images and their corresponding annotations.
The above code scales images and landmarks as training on images of smaller size takes less time.
This is because processing larger images requires more time. After scaling data it calculates mean
shape of the data which is used as initial shape while training.
Finally call the following function to perform training :
``` c++
facemark->training(Trainimages,Trainlandmarks,configfile_name,scale,modelfile_name);
```
In the above function scale is passed to scale all images and the corresponding landmarks so that the size of all
images can be reduced as it takes greater time to process large images.
This call to the train function trains the model and stores the trained model file with the given
filename specified.As the training starts successfully you will see something like this :
![](images/train1.png)
**The error rate on trained images depends on the number of images used for training used as follows :**
![](images/train.png)
**The error rate on test images depends on the number of images used for training used as follows :**
![](images/test.png)
\ No newline at end of file
modules/face/tutorials/face_landmark/face_landmark_video.markdown 0 → 100644
View file @ bccbec79
Face landmark detection in a video{#tutorial_face_landmark_detection_in_video}
===================================
This application lets you detect landmarks of detected faces in a video.This application first detects faces in a current video frame
and then finds their facial landmarks. You just have to pass the video as input.
```
// Command to be typed for running the sample
./sampleDetectLandmarks -file=trained_model.dat -face_cascade=lbpcascadefrontalface.xml -video=/path_to_video/video.avi
```
Description of command parameters
---------------------------------
> * **model_filename** f : (REQUIRED) A path to binary file storing the trained model which is to be loaded [example - /data/file.dat]
> * **video** v : (REQUIRED) A path to video in which face landmarks have to be detected.[example - /data/video.avi]
> * **face_cascade** c : (REQUIRED) A path to the face cascade xml file which you want to use as a face detector.
### Understanding code
This tutorial will explain the sample code for face landmark detection. Jumping directly to the code :
``` c++
CascadeClassifier face_cascade;
bool myDetector( InputArray image, OutputArray ROIs );
bool myDetector( InputArray image, OutputArray ROIs ){
Mat gray;
std::vector<Rect> faces;
if(image.channels()>1){
cvtColor(image.getMat(),gray,CV_BGR2GRAY);
}
else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
Mat(faces).copyTo(ROIs);
return true;
}
```
The facemark API provides the functionality to the user to use their own face detector to be used in face landmark detection.The above code creartes a sample face detector. The above function would be passed to a function pointer in the facemark API.
``` c++
VideoCapture cap(video);
if(!cap.isOpened()){
cerr<<"Video cannot be loaded. Give correct path"<<endl;
return -1;
}
```
The above code creates a video capture object and then loads the video.
If the video is not loaded properly it prompts the user else the code proceeds.
``` c++
Mat img = imread(image);
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<Facemark> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector(myDetector);
```
The above code creates a pointer of the face landmark detection class. The face detector created above has to be passed
as function pointer to the facemark pointer created for detecting faces.
``` c++
vector<Rect> faces;
vector< vector<Point2f> > shapes;
Mat img;
```
The above code creates a vector to store the detected faces and a vector of vector to store shapes for each
face detected in the current frame.
``` c++
while(1){
faces.clear();
shapes.clear();
cap>>img;
resize(img,img,Size(600,600));
facemark->getFaces(img,faces);
if(faces.size()==0){
cout<<"No faces found in this frame"<<endl;
}
else{
for( size_t i = 0; i < faces.size(); i++ )
{
cv::rectangle(img,faces[i],Scalar( 255, 0, 0 ));
}
if(facemark->fit(img,faces,shapes))
{
for(unsigned long i=0;i<faces.size();i++){
for(unsigned long k=0;k<shapes[i].size();k++)
cv::circle(img,shapes[i][k],3,cv::Scalar(0,0,255),FILLED);
}
}
}
namedWindow("Detected_shape");
imshow("Detected_shape",img);
if(waitKey(1) >= 0) break;
}
```
The above code then reads each frame and detects faces and the landmarks corresponding to each shape detected.
It then displays the current frame.
After running the above code you will get results something like this
Sample video:
@htmlonly
<iframe width="560" height="315" src="https://www.youtube.com/embed/ZtaV07T90D8" frameborder="0" allowfullscreen></iframe>
@endhtmlonly
\ No newline at end of file
modules/face/tutorials/face_landmark/sample_face_swapping.markdown 0 → 100644
View file @ bccbec79
Face swapping using face landmark detection{#tutorial_face_swapping_face_landmark_detection}
===========================================
This application lets you swap a face in one image with another face in other image. The application first detects faces in both images and finds its landmarks. Then it swaps the face in first image with in another image. You just have to give paths to the images run the application to swap the two faces.
```
// Command to be typed for running the sample
./sample_face_swapping -file=trained_model.dat -face_cascade=lbpcascadefrontalface.xml -image1=/path_to_image/image1.jpg -image2=/path_to_image/image2.jpg
```
### Description of command parameters
> * **image1** i1 (REQUIRED) Path to the first image file in which you want to apply swapping.
> * **image2** i2 (REQUIRED) Path to the second image file in which you want to apply face swapping.
> * **model** m (REQUIRED) Path to the file containing model to be loaded for face landmark detection.
> * **face_cascade** f (REQUIRED) Path to the face cascade xml file which you want to use as a face detector.
### Understanding the code
This tutorial will explain the sample code for face swapping using OpenCV. Jumping directly to the code :
``` c++
CascadeClassifier face_cascade;
bool myDetector( InputArray image, OutputArray ROIs );
bool myDetector( InputArray image, OutputArray ROIs ){
Mat gray;
std::vector<Rect> faces;
if(image.channels()>1){
cvtColor(image.getMat(),gray,CV_BGR2GRAY);
}
else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
Mat(faces).copyTo(ROIs);
return true;
}
```
The facemark API provides the functionality to the user to use their own face detector to be used in face landmark detection.The above code creartes a sample face detector. The above function would be passed to a function pointer in the facemark API.
``` c++
Mat img = imread(image);
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<Facemark> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector(myDetector);
```
The above code creates a pointer of the face landmark detection class. The face detector created above has to be passed
as function pointer to the facemark pointer created for detecting faces while training the model.
``` c++
vector<Rect> faces1,faces2;
vector< vector<Point2f> > shape1,shape2;
float ratio1 = (float)img1.cols/(float)img1.rows;
float ratio2 = (float)img2.cols/(float)img2.rows;
resize(img1,img1,Size(640*ratio1,640*ratio1));
resize(img2,img2,Size(640*ratio2,640*ratio2));
Mat img1Warped = img2.clone();
facemark->getFaces(img1,faces1);
facemark->getFaces(img2,faces2);
facemark->fit(img1,faces1,shape1);
facemark->fit(img2,faces2,shape2);
```
The above code creates vectors to store the detected faces and a vector of vector to store shapes for each
face detected in both the images.It then detects landmarks of each face detected in both the images.the images are resized
as it is easier to process small images. The images are resized according their actual ratio.
``` c++
vector<Point2f> boundary_image1;
vector<Point2f> boundary_image2;
vector<int> index;
convexHull(Mat(points2),index, false, false);
for(size_t i = 0; i < index.size(); i++)
{
boundary_image1.push_back(points1[index[i]]);
boundary_image2.push_back(points2[index[i]]);
}
```
The above code then finds convex hull to find the boundary points of the face in the image which has to be swapped.
``` c++
vector< vector<int> > triangles;
Rect rect(0, 0, img1Warped.cols, img1Warped.rows);
divideIntoTriangles(rect, boundary_image2, triangles);
for(size_t i = 0; i < triangles.size(); i++)
{
vector<Point2f> triangle1, triangle2;
for(int j = 0; j < 3; j++)
{
triangle1.push_back(boundary_image1[triangles[i][j]]);
triangle2.push_back(boundary_image2[triangles[i][j]]);
}
warpTriangle(img1, img1Warped, triangle1, triangle2);
}
```
Now as we need to warp one face over the other and we need to find affine transform.
Now as the function in OpenCV to find affine transform requires three set of points to calculate
the affine matrix. Also we just need to warp the face instead of the surrounding regions. Hence
we divide the face into triangles so that each triiangle can be easily warped onto the other image.
The function divideIntoTriangles divides the detected faces into triangles.
The function warpTriangle then warps each triangle of one image to other image to swap the faces.
``` c++
vector<Point> hull;
for(size_t i = 0; i < boundary_image2.size(); i++)
{
Point pt((int)boundary_image2[i].x,(int)boundary_image2[i].y);
hull.push_back(pt);
}
Mat mask = Mat::zeros(img2.rows, img2.cols, img2.depth());
fillConvexPoly(mask,&hull[0],(int)hull.size(), Scalar(255,255,255));
Rect r = boundingRect(boundary_image2);
Point center = (r.tl() + r.br()) / 2;
Mat output;
img1Warped.convertTo(img1Warped, CV_8UC3);
seamlessClone(img1Warped,img2, mask, center, output, NORMAL_CLONE);
imshow("Face_Swapped", output);
```
Even after warping the results somehow look unnatural. Hence to improve the results we apply seamless cloning
to get the desired results as required.
### Results
Consider two images to be used for face swapping as follows :
First image
-----------
![](images/227943776_1.jpg)
Second image
------------
![](images/230501201_1.jpg)
Results after swapping
----------------------
![](images/face_swapped.jpg)
\ No newline at end of file
modules/face/tutorials/face_recognition/face_tutorial.markdown 0 → 100644
View file @ bccbec79
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment