Skip to content

O
opencv_contrib
Commit 4c2f5135 authored by kushashwa ravi shrimali's avatar kushashwa ravi shrimali
Browse files
Options

BRISQUE commit 

Minor edits

minor edits

double* to vector<data>

Updates: BRISQUE and Python

Added BRISQUE and Python Samples

Minor edits

Final edits: BRISQUE

Final edits: BRISQUE

Copyright notice added

deleted .vscode

Warnings rectified. Added CV_Assert

Final Commit: BRISQUE

Removed whitespaces. Corrected array initializations and override warnings

Removed whitespaces

Resolved array initialization

Override warning

Mac build resolved

Added bibliography and corrected Mac/Android/iOS warnings

Rollback changes

Added BRISQUE bib

Solves Mac/iOS/Android warnings

Removed trailing whitespaces

Updated sample code for BRISQUE C++

Fixes for Win32, Win64

Warnings removed

trailing whitespaces

Trailing whitespace removed
parent 476167ed
Hide whitespace changes
Inline Side-by-side
Showing with 325 additions and 172 deletions
modules/quality/README.md
View file @ 4c2f5135
...@@ -16,6 +16,8 @@ Implementation of various image quality analysis (IQA) algorithms ...@@ -16,6 +16,8 @@ Implementation of various image quality analysis (IQA) algorithms
http://www4.comp.polyu.edu.hk/~cslzhang/IQA/GMSD/GMSD.htm http://www4.comp.polyu.edu.hk/~cslzhang/IQA/GMSD/GMSD.htm
In general, the GMSD algorithm should yield the best result for full-reference IQA. In general, the GMSD algorithm should yield the best result for full-reference IQA.
- **Blind/Referenceless Image Spatial Quality Evaluation (BRISQUE)**
http://live.ece.utexas.edu/research/Quality/nrqa.htm
Interface/Usage Interface/Usage
----------------------------------------- -----------------------------------------
...@@ -31,23 +33,75 @@ to convert input images to grayscale images prior to processing. ...@@ -31,23 +33,75 @@ to convert input images to grayscale images prior to processing.
SSIM and GMSD were originally tested by their respective researchers on grayscale uint8 images, SSIM and GMSD were originally tested by their respective researchers on grayscale uint8 images,
but this implementation will compute the values for each channel if the user desires to do so. but this implementation will compute the values for each channel if the user desires to do so.
BRISQUE is a NR-IQA algorithm (No-Reference) which doesn't require a reference image.
Quick Start/Usage Quick Start/Usage
----------------------------------------- -----------------------------------------
**C++ Implementations**
#include <opencv2/quality.hpp> **For Full Reference IQA Algorithms (MSE, PSNR, SSIM, GMSD)**
```cpp
#include <opencv2/quality.hpp>
cv::Mat img1, img2; /* your cv::Mat images */ cv::Mat img1, img2; /* your cv::Mat images */
std::vector<cv::Mat> quality_maps; /* output quality map(s) (optional) */ std::vector<cv::Mat> quality_maps; /* output quality map(s) (optional) */
/* compute MSE via static method */
/* compute MSE via static method */
cv::Scalar result_static = quality::QualityMSE::compute(img1, img2, quality_maps); /* or cv::noArray() if not interested in output quality maps */ cv::Scalar result_static = quality::QualityMSE::compute(img1, img2, quality_maps); /* or cv::noArray() if not interested in output quality maps */
/* alternatively, compute MSE via instance */ /* alternatively, compute MSE via instance */
cv::Ptr<quality::QualityBase> ptr = quality::QualityMSE::create(img1); cv::Ptr<quality::QualityBase> ptr = quality::QualityMSE::create(img1);
cv::Scalar result = ptr->compute( img2 ); /* compute MSE, compare img1 vs img2 */ cv::Scalar result = ptr->compute( img2 ); /* compute MSE, compare img1 vs img2 */
ptr->getQualityMaps(quality_maps); /* optionally, access output quality maps */ ptr->getQualityMaps(quality_maps); /* optionally, access output quality maps */
```
**For No Reference IQA Algorithm (BRISQUE)**
```cpp
#include <opencv2/quality.hpp>
cv::Mat img;
// check testdata subdirectory
cv::String model_path = "testdata/brisque_allmodel.dat"; // path to the trained model
cv::String range_path = "testdata/brisque_allrange.dat"; // path to range file
/* compute BRISQUE quality score via static method */
cv::Scalar result_static = quality::QualityBRISQUE::compute(img,
model_path, range_path);
/* alternatively, compute BRISQUE via instance */
cv::Ptr<quality::QualityBase> ptr = quality::QualityBRISQUE::create(model_path, range_path);
cv::Scalar result = ptr->compute(img); /* computes BRISQUE score for img */
```
**Python Implementations**
**For Full Reference IQA Algorithms (MSE, PSNR, SSIM, GSMD)**
```python
import cv2
# read images
img1 = cv2.imread(img1, 1) # specify img1
img2 = cv2.imread(img2_path, 1) # specify img2_path
# compute MSE score and quality maps via static method
result_static, quality_maps = cv2.quality.QualityMSE_compute(img1, img2)
# compute MSE score and quality maps via Instance
obj = cv2.quality.QualityMSE_create(img1)
result = obj.compute(img2)
quality_maps = obj.getQualityMaps()
```
**For No Reference IQA Algorithm (BRISQUE)**
```python
import cv2
# read image
img = cv2.imread(img_path, 1) # mention img_path
# make a list of image to be passed
img_list = [img]
# compute brisque quality score via static method
score = cv2.quality.QualityBRISQUE_compute(img_list, model_path,
range_path) # specify model_path and range_path
# compute brisque quality score via instance
# specify model_path and range_path
obj = cv2.quality.QualityBRISQUE_create(model_path, range_path)
score = obj.compute(img_list)
```
Library Design Library Design
----------------------------------------- -----------------------------------------
...@@ -64,5 +118,4 @@ Each implemented algorithm shall: ...@@ -64,5 +118,4 @@ Each implemented algorithm shall:
To Do To Do
----------------------------------------- -----------------------------------------
- Document the output quality maps for each algorithm - Document the output quality maps for each algorithm
- Implement at least one no-reference IQA algorithm
- Investigate precision loss with cv::Filter2D + UMat + CV_32F + OCL for GMSD - Investigate precision loss with cv::Filter2D + UMat + CV_32F + OCL for GMSD
\ No newline at end of file
modules/quality/doc/quality.bib 0 → 100644
View file @ 4c2f5135
@article{Mittal2,
title={No-Reference Image Quality Assessment in the Spatial Domain},
author={A. {Mittal} and A. K. {Moorthy} and A. C. {Bovik}},
journal={IEEE Transactions on Image Processing},
volume={21},
number={12},
pages={4695-4708},
year={2012},
ISSN={1057-7149},
doi={10.1109/TIP.2012.2214050},
}
@misc{Mittal2_software,
title={BRISQUE Software Release},
author={A. {Mittal} and A. K. {Moorthy} and A. C. {Bovik}},
howpublished={\url{http://live.ece.utexas.edu/research/quality/BRISQUE_release.zip}},
year={2011},
}
\ No newline at end of file
modules/quality/include/opencv2/quality/qualitybrisque.hpp
View file @ 4c2f5135
...@@ -12,59 +12,65 @@ namespace cv ...@@ -12,59 +12,65 @@ namespace cv
namespace quality namespace quality
{ {
/** @brief Custom deleter for QualityBRISQUE internal data */
struct _QualityBRISQUEDeleter
{
void operator()(void*) const;
};
/** /**
@brief TODO: Brief description and reference to original BRISQUE paper/implementation @brief BRISQUE (Blind/Referenceless Image Spatial Quality Evaluator) is a type of No Reference
Image Quality Assessment. It measures score based on extracting Natural Scene Satistics (https://en.wikipedia.org/wiki/Scene_statistics)
and calculating feature vectors. The current implementation uses trained model on TID 2008 Database (http://www.ponomarenko.info/tid2008.htm).
@cite Mittal2 for original paper and @cite Mittal2_software for original implementation
*/ */
class CV_EXPORTS_W QualityBRISQUE : public QualityBase { class CV_EXPORTS_W QualityBRISQUE : public QualityBase {
public: public:
/** @brief Computes XXX for reference images supplied in class constructor and provided comparison images /** @brief Computes BRISQUE quality score for input images
@param imgs Images for which to compute quality @param imgs Images for which to compute quality (should be passed as a vector<Mat> in C++ and list of images in Python)
@returns TODO: describe the resulting cv::Scalar @returns Score (averaged over individual scores of all images) ranging from 0 to 100
(0 denotes the best quality and 100 denotes the poorest quality). The format of the score is: {score, 0., 0., 0.}
*/ */
CV_WRAP cv::Scalar compute( InputArrayOfArrays imgs ) CV_OVERRIDE; CV_WRAP cv::Scalar compute( InputArrayOfArrays imgs ) CV_OVERRIDE;
/** /**
@brief Create an object which calculates quality @brief Create an object which calculates quality
@param model_file_path cv::String which contains a path to the BRISQUE model data. If empty, attempts to load from ${OPENCV_DIR}/testdata/contrib/quality/brisque_allmodel.dat
@param range_file_path cv::String which contains a path to the BRISQUE range data. If empty, attempts to load from ${OPENCV_DIR}/testdata/contrib/quality/brisque_allrange.dat
*/ */
CV_WRAP static Ptr<QualityBRISQUE> create(cv::String model, cv::String range); CV_WRAP static Ptr<QualityBRISQUE> create( const cv::String& model_file_path = "", const cv::String& range_file_path = "" );
// CV_WRAP static cv::Scalar compute_single( const Mat& cmpImg, const cv::String& model_file_path, const cv::String& range_file_path );
/** /**
@brief static method for computing quality @brief static method for computing quality
@param model cv::String containing BRISQUE calculation model @param imgs image(s) for which to compute quality (passed as vector<Mat> in C++ and as list of images in Python)
@param range cv::String containing BRISQUE calculation range @param model_file_path cv::String which contains a path to the BRISQUE model data. If empty, attempts to load from ${OPENCV_DIR}/testdata/contrib/quality/brisque_allmodel.dat
@param imgs image(s) for which to compute quality @param range_file_path cv::String which contains a path to the BRISQUE range data. If empty, attempts to load from ${OPENCV_DIR}/testdata/contrib/quality/brisque_allrange.dat
@param qualityMaps output quality map(s), or cv::noArray() TODO: remove this parameter if algorithm doesn't generate output quality maps @returns cv::Scalar result of format {std::double score, 0., 0., 0.}. Score ranges from 0 to 100 (100 means worst and 0 means best)
@returns TODO: describe the resulting cv::Scalar
*/ */
CV_WRAP static cv::Scalar compute( const cv::String& model, const cv::String& range, InputArrayOfArrays imgs, OutputArrayOfArrays qualityMaps ); CV_WRAP static cv::Scalar compute( InputArrayOfArrays imgs, const cv::String& model_file_path, const cv::String& range_file_path );
/** @brief return the model used for computation */
CV_WRAP const cv::String& getModel() const { return _model; }
/** @brief sets the model used for computation */
CV_WRAP void setModel( cv::String val ) { this->_model = std::move(val); }
/** @brief return the range used for computation */ /* brief destructor */
CV_WRAP const cv::String& getRange() const { return _range; } ~QualityBRISQUE() CV_OVERRIDE;
/** @brief sets the range used for computation */
CV_WRAP void setRange(cv::String val) { this->_range = std::move(val); }
protected: protected:
/** /** @brief Internal constructor */
@brief Constructor QualityBRISQUE( const cv::String& model_file_path, const cv::String& range_file_path );
*/
QualityBRISQUE( cv::String model, cv::String range ); // type-erased svmmodel
void* _svm_model = nullptr;
/** @brief BRISQUE model string */ static constexpr const std::size_t _SVM_RANGE_SIZE = 36U;
cv::String _model; using _svm_range_type = float;
/** @brief BRISQUE range string */ std::array<_svm_range_type, _SVM_RANGE_SIZE>
cv::String _range; _svm_range_min = {}
, _svm_range_max = {}
;
}; // QualityBRISQUE }; // QualityBRISQUE
} // quality } // quality
} // cv } // cv
#endif #endif
\ No newline at end of file
modules/quality/src/qualitybrisque.cpp
View file @ 4c2f5135
...@@ -2,46 +2,85 @@ ...@@ -2,46 +2,85 @@
// It is subject to the license terms in the LICENSE file found in the top-level directory // It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html. // of this distribution and at http://opencv.org/license.html.
/*
Copyright (c) 2011 The University of Texas at Austin
All rights reserved.
Permission is hereby granted, without written agreement and without license or royalty fees, to use, copy,
modify, and distribute this code (the source files) and its documentation for
any purpose, provided that the copyright notice in its entirety appear in all copies of this code, and the
original source of this code, Laboratory for Image and Video Engineering (LIVE, http://live.ece.utexas.edu)
and Center for Perceptual Systems (CPS, http://www.cps.utexas.edu) at the University of Texas at Austin (UT Austin,
http://www.utexas.edu), is acknowledged in any publication that reports research using this code. The research
is to be cited in the bibliography as:
1) A. Mittal, A. K. Moorthy and A. C. Bovik, "BRISQUE Software Release",
URL: http://live.ece.utexas.edu/research/quality/BRISQUE_release.zip, 2011
2) A. Mittal, A. K. Moorthy and A. C. Bovik, "No Reference Image Quality Assessment in the Spatial Domain"
submitted
IN NO EVENT SHALL THE UNIVERSITY OF TEXAS AT AUSTIN BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,
OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF THIS DATABASE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF TEXAS
AT AUSTIN HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
THE UNIVERSITY OF TEXAS AT AUSTIN SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE DATABASE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS,
AND THE UNIVERSITY OF TEXAS AT AUSTIN HAS NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
*/
/* Original Paper: @cite Mittal2 and Original Implementation: @cite Mittal2_software */
#include "precomp.hpp" #include "precomp.hpp"
#include <fstream>
#include "opencv2/imgproc.hpp" #include "opencv2/imgproc.hpp"
#include "opencv2/quality/qualitybrisque.hpp" #include "opencv2/quality/qualitybrisque.hpp"
#include "opencv2/quality/quality_utils.hpp" #include "opencv2/quality/quality_utils.hpp"
#include "opencv2/quality/libsvm/svm.hpp" // libsvm #include "opencv2/quality/libsvm/svm.hpp" // libsvm
#include <iostream>
namespace namespace
{ {
using namespace cv; using namespace cv;
using namespace cv::quality; using namespace cv::quality;
// type of mat we're working with internally; use cv::Mat for debugging // type of mat we're working with internally
using brisque_mat_type = Mat; // Win32+UMat: performance is 15-20X worse than Mat
// Win32+UMat+OCL: performance is 200-300X worse than Mat, plus accuracy errors
// Linux+UMat: 15X worse performance than Linux+Mat
using brisque_mat_type = cv::Mat;
// the type of quality map we'll generate (if brisque generates one) // brisque intermediate calculation type
using _quality_map_type = brisque_mat_type; // Linux+Mat: CV_64F is 3X slower than CV_32F
// Win32+Mat: CV_64F is 2X slower than CV_32F
static constexpr const int BRISQUE_CALC_MAT_TYPE = CV_32F;
// brisque intermediate matrix element type. float if BRISQUE_CALC_MAT_TYPE == CV_32F, double if BRISQUE_CALC_MAT_TYPE == CV_64F
using brique_calc_element_type = float;
template<class T> class Image { template<class T> class Image {
private: private:
Mat imgP; brisque_mat_type imgP;
public: public:
Image(Mat img = 0) { Image(brisque_mat_type img) {
imgP = img.clone(); imgP = img.clone();
} }
~Image() { ~Image() {
imgP = 0; imgP = 0;
} }
Mat equate(Mat img) { brisque_mat_type equate(brisque_mat_type img) {
img = imgP.clone(); img = imgP.clone();
return img; return img;
} }
inline T* operator[](const int rowIndx) { inline T* operator[](const int rowIndx) {
return (T*)(imgP.data + rowIndx * imgP.step); // return (T*)(imgP.getMat(ACCESS_READ).data + rowIndx * imgP.step); // UMat version
return (T*)(imgP.data + rowIndx * imgP.step); // Mat version
} }
}; };
typedef Image<double> BwImage; typedef Image<brique_calc_element_type> BwImage;
// function to compute best fit parameters from AGGDfit // function to compute best fit parameters from AGGDfit
cv::Mat AGGDfit(cv::Mat structdis, double& lsigma_best, double& rsigma_best, double& gamma_best) brisque_mat_type AGGDfit(brisque_mat_type structdis, double& lsigma_best, double& rsigma_best, double& gamma_best)
{ {
// create a copy of an image using BwImage constructor (brisque.h - more info) // create a copy of an image using BwImage constructor (brisque.h - more info)
BwImage ImArr(structdis); BwImage ImArr(structdis);
...@@ -78,8 +117,8 @@ namespace ...@@ -78,8 +117,8 @@ namespace
double prevgamma = 0; double prevgamma = 0;
double prevdiff = 1e10; double prevdiff = 1e10;
float sampling = 0.001; double sampling = 0.001;
for (float gam = 0.2; gam < 10; gam += sampling) //possible to coarsen sampling to quicken the code, with some loss of accuracy for (double gam = 0.2; gam < 10; gam += sampling) //possible to coarsen sampling to quicken the code, with some loss of accuracy
{ {
double r_gam = tgamma(2 / gam)*tgamma(2 / gam) / (tgamma(1 / gam)*tgamma(3 / gam)); double r_gam = tgamma(2 / gam)*tgamma(2 / gam) / (tgamma(1 / gam)*tgamma(3 / gam));
double diff = abs(r_gam - rhatnorm); double diff = abs(r_gam - rhatnorm);
...@@ -92,44 +131,40 @@ namespace ...@@ -92,44 +131,40 @@ namespace
return structdis.clone(); return structdis.clone();
} }
void ComputeBrisqueFeature(cv::Mat& orig, std::vector<double>& featurevector) void ComputeBrisqueFeature(brisque_mat_type& orig, std::vector<double>& featurevector)
{ {
CV_Assert(orig.channels() == 1);
Mat orig_bw_int(orig.size(), CV_64F, 1); auto orig_bw = orig;
// convert to grayscale
cv::cvtColor(orig, orig_bw_int, cv::COLOR_BGR2GRAY);
// create a copy of original image
Mat orig_bw(orig_bw_int.size(), CV_64FC1, 1);
orig_bw_int.convertTo(orig_bw, 1.0 / 255);
orig_bw_int.release();
// orig_bw now contains the grayscale image normalized to the range 0,1 // orig_bw now contains the grayscale image normalized to the range 0,1
int scalenum = 2; // number of times to scale the image int scalenum = 2; // number of times to scale the image
for (int itr_scale = 1; itr_scale <= scalenum; itr_scale++) for (int itr_scale = 1; itr_scale <= scalenum; itr_scale++)
{ {
// resize image // resize image
cv::Size dst_size(orig_bw.cols / cv::pow((double)2, itr_scale - 1), orig_bw.rows / pow((double)2, itr_scale - 1)); cv::Size dst_size( int( orig_bw.cols / cv::pow((double)2, itr_scale - 1) ), int( orig_bw.rows / pow((double)2, itr_scale - 1)));
cv::Mat imdist_scaled; brisque_mat_type imdist_scaled;
cv::resize(orig_bw, imdist_scaled, dst_size, 0, 0, cv::INTER_CUBIC); // INTER_CUBIC cv::resize(orig_bw, imdist_scaled, dst_size, 0, 0, cv::INTER_CUBIC); // INTER_CUBIC
imdist_scaled.convertTo(imdist_scaled, CV_64FC1, 1.0 / 255.0);
// calculating MSCN coefficients // calculating MSCN coefficients
// compute mu (local mean) // compute mu (local mean)
cv::Mat mu(imdist_scaled.size(), CV_64FC1, 1); brisque_mat_type mu;// (imdist_scaled.size(), CV_64FC1, 1);
cv::GaussianBlur(imdist_scaled, mu, cv::Size(7, 7), 1.166); cv::GaussianBlur(imdist_scaled, mu, cv::Size(7, 7), 1.16666, 0., cv::BORDER_REPLICATE );
cv::Mat mu_sq; brisque_mat_type mu_sq;
cv::pow(mu, double(2.0), mu_sq); cv::pow(mu, double(2.0), mu_sq);
//compute sigma (local sigma) //compute sigma (local sigma)
cv::Mat sigma(imdist_scaled.size(), CV_64FC1, 1); brisque_mat_type sigma;// (imdist_scaled.size(), CV_64FC1, 1);
cv::multiply(imdist_scaled, imdist_scaled, sigma); cv::multiply(imdist_scaled, imdist_scaled, sigma);
cv::GaussianBlur(sigma, sigma, cv::Size(7, 7), 1.166);
cv::GaussianBlur(sigma, sigma, cv::Size(7, 7), 1.16666, 0., cv::BORDER_REPLICATE );
cv::subtract(sigma, mu_sq, sigma); cv::subtract(sigma, mu_sq, sigma);
cv::pow(sigma, double(0.5), sigma); cv::pow(sigma, double(0.5), sigma);
cv::add(sigma, Scalar(1.0 / 255), sigma); // to avoid DivideByZero Error cv::add(sigma, Scalar(1.0 / 255), sigma); // to avoid DivideByZero Error
cv::Mat structdis(imdist_scaled.size(), CV_64FC1, 1); brisque_mat_type structdis;// (imdist_scaled.size(), CV_64FC1, 1);
cv::subtract(imdist_scaled, mu, structdis); cv::subtract(imdist_scaled, mu, structdis);
cv::divide(structdis, sigma, structdis); // structdis is MSCN image cv::divide(structdis, sigma, structdis); // structdis is MSCN image
...@@ -149,8 +184,8 @@ namespace ...@@ -149,8 +184,8 @@ namespace
// select the shifting index from the 2D array // select the shifting index from the 2D array
int* reqshift = shifts[itr_shift - 1]; int* reqshift = shifts[itr_shift - 1];
// declare shifted_structdis as pairwise image // declare, create shifted_structdis as pairwise image
cv::Mat shifted_structdis(imdist_scaled.size(), CV_64F, 1); brisque_mat_type shifted_structdis(imdist_scaled.size(), BRISQUE_CALC_MAT_TYPE); //(imdist_scaled.size(), CV_64FC1, 1);
// create copies of the images using BwImage constructor // create copies of the images using BwImage constructor
// utility constructor for better subscript access (for pixels) // utility constructor for better subscript access (for pixels)
...@@ -179,7 +214,7 @@ namespace ...@@ -179,7 +214,7 @@ namespace
// calculate the products of the pairs // calculate the products of the pairs
cv::multiply(structdis, shifted_structdis, shifted_structdis); cv::multiply(structdis, shifted_structdis, shifted_structdis);
// fit the pairwise product to AGGD // fit the pairwise product to AGGD
shifted_structdis = AGGDfit(shifted_structdis, lsigma_best, rsigma_best, gamma_best); shifted_structdis = AGGDfit(shifted_structdis, lsigma_best, rsigma_best, gamma_best);
double constant = sqrt(tgamma(1 / gamma_best)) / sqrt(tgamma(3 / gamma_best)); double constant = sqrt(tgamma(1 / gamma_best)) / sqrt(tgamma(3 / gamma_best));
...@@ -194,39 +229,22 @@ namespace ...@@ -194,39 +229,22 @@ namespace
} }
} }
// float computescore(String imagename) { double computescore(const svm_model* model, const float* range_min, const float* range_max, brisque_mat_type& orig) {
double computescore( const cv::String& model_, const cv::String& range_, cv::Mat& orig ) {
// pre-loaded vectors from allrange file
float min_[36] = { 0.336999 ,0.019667 ,0.230000 ,-0.125959 ,0.000167 ,0.000616 ,0.231000 ,-0.125873 ,0.000165 ,0.000600 ,0.241000 ,-0.128814 ,0.000179 ,0.000386 ,0.243000 ,-0.133080 ,0.000182 ,0.000421 ,0.436998 ,0.016929 ,0.247000 ,-0.200231 ,0.000104 ,0.000834 ,0.257000 ,-0.200017 ,0.000112 ,0.000876 ,0.257000 ,-0.155072 ,0.000112 ,0.000356 ,0.258000 ,-0.154374 ,0.000117 ,0.000351 };
float max_[36] = { 9.999411, 0.807472, 1.644021, 0.202917, 0.712384, 0.468672, 1.644021, 0.169548, 0.713132, 0.467896, 1.553016, 0.101368, 0.687324, 0.533087, 1.554016, 0.101000, 0.689177, 0.533133, 3.639918, 0.800955, 1.096995, 0.175286, 0.755547, 0.399270, 1.095995, 0.155928, 0.751488, 0.402398, 1.041992, 0.093209, 0.623516, 0.532925, 1.042992, 0.093714, 0.621958, 0.534484 };
double qualityscore; double qualityscore;
int i; int i;
struct svm_model* model; // create svm model object
// cv::Mat orig = cv::imread(imagename, 1); // read image (color mode)
std::vector<double> brisqueFeatures; // feature vector initialization std::vector<double> brisqueFeatures; // feature vector initialization
ComputeBrisqueFeature(orig, brisqueFeatures); // compute brisque features ComputeBrisqueFeature(orig, brisqueFeatures); // compute brisque features
// use the pre-trained allmodel file
// TODO: Use model and range parameters provided to this function instead
String modelfile = "allmodel";
if ((model = svm_load_model(modelfile.c_str())) == 0) {
fprintf(stderr, "can't open model file allmodel\n");
exit(1);
}
// float min_[37];
// float max_[37];
//
struct svm_node x[37]; struct svm_node x[37];
// rescale the brisqueFeatures vector from -1 to 1
// rescale the brisqueFeatures vector from -1 to 1
// also convert vector to svm node array object // also convert vector to svm node array object
for (i = 0; i < 36; ++i) { for (i = 0; i < 36; ++i) {
float min = min_[i]; const float
float max = max_[i]; min = range_min[i]
, max = range_max[i]
;
x[i].value = -1 + (2.0 / (max - min) * (brisqueFeatures[i] - min)); x[i].value = -1 + (2.0 / (max - min) * (brisqueFeatures[i] - min));
x[i].index = i + 1; x[i].index = i + 1;
...@@ -235,57 +253,35 @@ namespace ...@@ -235,57 +253,35 @@ namespace
int nr_class = svm_get_nr_class(model); int nr_class = svm_get_nr_class(model);
double *prob_estimates = (double *)malloc(nr_class * sizeof(double));
std::vector<double> prob_estimates = std::vector<double>(nr_class);
// predict quality score using libsvm class // predict quality score using libsvm class
qualityscore = svm_predict_probability(model, x, prob_estimates); qualityscore = svm_predict_probability(model, x, prob_estimates.data());
free(prob_estimates);
svm_free_and_destroy_model(&model);
return qualityscore; return qualityscore;
} }
// computes score and quality map for single frame // computes score for a single frame
std::pair<cv::Scalar, _quality_map_type> compute( const cv::String& model, const cv::String& range, brisque_mat_type& img ) cv::Scalar compute(const svm_model* model, const float* range_min, const float* range_max, brisque_mat_type& img)
{ {
std::pair<cv::Scalar, _quality_map_type> result; auto result = cv::Scalar{ 0. };
result[0] = computescore(model, range_min, range_max, img);
// TODO: calculate score and quality map for the input image, which is a single frame
// place score in result.first, and quality map in result.second
result.first = cv::Scalar{ 0. };
result.first[0] = computescore( model, range, img);
// is there a quality map that can be generated?
return result; return result;
} }
// computes score and quality maps for multiple frames cv::Scalar compute(const svm_model* model, const float* range_min, const float* range_max, std::vector<brisque_mat_type>& imgs)
cv::Scalar compute( const cv::String& model, const cv::String& range, std::vector<brisque_mat_type>& imgs, OutputArrayOfArrays qualityMaps )
{ {
CV_Assert(imgs.size() > 0); CV_Assert(imgs.size() > 0);
cv::Scalar result = {}; cv::Scalar result = {};
std::vector<_quality_map_type> quality_maps = {};
const auto sz = imgs.size();
// future optimization: convert model and range to libsvm types here instead of every time we compute for a frame const auto sz = imgs.size();
// ideally, these would be created in the constructor so they are only created once per instantiation
for (unsigned i = 0; i < sz; ++i) for (unsigned i = 0; i < sz; ++i)
{ {
auto cmp = compute( model, range, imgs[i] ); auto cmp = compute(model, range_min, range_max, imgs[i]);
cv::add(result, cmp.first, result); cv::add(result, cmp, result);
if ( qualityMaps.needed() )
quality_maps.emplace_back(std::move(cmp.second));
}
if (qualityMaps.needed())
{
auto qMaps = InputArray(quality_maps);
qualityMaps.create(qMaps.size(), qMaps.type());
qualityMaps.assign(quality_maps);
} }
if (sz > 1) if (sz > 1)
...@@ -296,29 +292,103 @@ namespace ...@@ -296,29 +292,103 @@ namespace
} }
// static // static
Ptr<QualityBRISQUE> QualityBRISQUE::create(cv::String model, cv::String range) Ptr<QualityBRISQUE> QualityBRISQUE::create(const cv::String& model_file_path, const cv::String& range_file_path)
{ {
return Ptr<QualityBRISQUE>(new QualityBRISQUE( std::move(model), std::move(range))); return Ptr<QualityBRISQUE>(new QualityBRISQUE(model_file_path, range_file_path));
} }
// static cv::Scalar QualityBRISQUE::compute(InputArrayOfArrays imgs, const cv::String& model_file_path, const cv::String& range_file_path)
cv::Scalar QualityBRISQUE::compute( const cv::String& model, const cv::String& range, InputArrayOfArrays imgs, OutputArrayOfArrays qualityMaps)
{ {
auto vec = quality_utils::expand_mats<brisque_mat_type>(imgs);// convert inputarrayofarrays to vector of brisque_mat_type auto obj = create(model_file_path, range_file_path);
return ::compute(model, range, vec, qualityMaps); return obj->compute(imgs);
} }
// QualityBRISQUE() constructor // QualityBRISQUE() constructor
QualityBRISQUE::QualityBRISQUE( cv::String model, cv::String range ) QualityBRISQUE::QualityBRISQUE(const cv::String& model_file_path, const cv::String& range_file_path)
: _model(std::move(model))
, _range(std::move(range))
{ {
// would be nice to convert the model/range strings to libsvm models // construct data file path from OPENCV_DIR env var and quality subdir
// and store in a unique_ptr<void> with a custom deleter in the qualitybrisque object so we don't expose libsvm headers const auto get_data_path = [](const cv::String& fname)
{
cv::String path{ std::getenv("OPENCV_DIR") };
return path.empty()
? path // empty
: path + "/testdata/contrib/quality/" + fname
;
};
const auto
modelpath = model_file_path.empty() ? get_data_path("brisque_allmodel.dat") : model_file_path
, rangepath = range_file_path.empty() ? get_data_path("brisque_allrange.dat") : range_file_path
;
if (modelpath.empty())
CV_Error(cv::Error::StsObjectNotFound, "BRISQUE model data not found");
if (rangepath.empty())
CV_Error(cv::Error::StsObjectNotFound, "BRISQUE range data not found");
// load svm data
this->_svm_model = svm_load_model(modelpath.c_str());
if (!this->_svm_model)
CV_Error(cv::Error::StsParseError, "Error loading BRISQUE model file");
// load range data
// based on original brisque impl
//check if file exists
char buff[100];
FILE* range_file = fopen(rangepath.c_str(), "r");
if (range_file == NULL)
CV_Error(cv::Error::StsParseError, "Error loading BRISQUE range file");
//assume standard file format for this program
CV_Assert(fgets(buff, 100, range_file) != NULL);
CV_Assert(fgets(buff, 100, range_file) != NULL);
//now we can fill the array
for (std::size_t i = 0; i < _SVM_RANGE_SIZE; ++i) {
float a, b, c;
CV_Assert(fscanf(range_file, "%f %f %f", &a, &b, &c) == 3);
this->_svm_range_min[i] = (_svm_range_type)b;
this->_svm_range_max[i] = (_svm_range_type)c;
}
fclose(range_file);
} }
cv::Scalar QualityBRISQUE::compute( InputArrayOfArrays imgs ) cv::Scalar QualityBRISQUE::compute(InputArrayOfArrays imgs)
{ {
auto vec = quality_utils::expand_mats<brisque_mat_type>(imgs);// convert inputarrayofarrays to vector of brisque_mat_type auto vec = quality_utils::expand_mats<brisque_mat_type>(imgs);// convert inputarrayofarrays to vector of brisque_mat_type
return ::compute( this->getModel(), this->getRange(), vec, this->_qualityMaps);
// convert all mats to single channel/bgr2gray as needed, scale to 0-1
for (auto& mat : vec)
{
switch (mat.channels())
{
case 1:
break;
case 3:
cv::cvtColor(mat, mat, cv::COLOR_RGB2GRAY, 1);
break;
case 4:
cv::cvtColor(mat, mat, cv::COLOR_RGBA2GRAY, 1);
break;
default:
CV_Error(cv::Error::StsNotImplemented, "Unknown/unsupported channel count");
};//switch
// scale to 0-1 range
mat.convertTo(mat, BRISQUE_CALC_MAT_TYPE, 1. / 255.);
}
// const brisque_svm_data* data_ptr = static_cast<const brisque_svm_data*>(this->_svm_data.get());
return ::compute( (const svm_model*)this->_svm_model, this->_svm_range_min.data(), this->_svm_range_max.data(), vec);
}
QualityBRISQUE::~QualityBRISQUE()
{
if (this->_svm_model != nullptr)
{
svm_model* ptr = (svm_model*)this->_svm_model;
svm_free_and_destroy_model(&ptr);
this->_svm_model = nullptr;
}
} }
\ No newline at end of file
modules/quality/test/test_brisque.cpp
View file @ 4c2f5135
...@@ -12,49 +12,40 @@ namespace quality_test ...@@ -12,49 +12,40 @@ namespace quality_test
{ {
// brisque per channel // brisque per channel
// computed test file values via original brisque impl, libsvm 318, opencv 2.x // computed test file values via original brisque c++ impl, libsvm 318, opencv 2.x
const cv::Scalar const cv::Scalar
BRISQUE_EXPECTED_1 = { 31.155 } // testfile_1a BRISQUE_EXPECTED_1 = { 31.154966299963547 } // testfile_1a
, BRISQUE_EXPECTED_2 = { 15.4114 } // testfile_2a // , BRISQUE_EXPECTED_2 = { 15.411353283158718 } // testfile_2a with original c++ impl
, BRISQUE_EXPECTED_2 = { 15.600739064304520 } // testfile 2a; delta is due to differences between opencv 2.x and opencv 4.x cvtColor, RGB2GRAY
; ;
inline cv::String readfile(const cv::String& path)
{
std::ifstream is{ path };
std::stringstream buffer;
buffer << is.rdbuf();
return buffer.str();
}
// location of BRISQUE model and range file
// place these files in ${OPENCV_TEST_DATA_PATH}/quality/, or the tests will be skipped
inline cv::String readbrisquemodel() { return readfile(cvtest::findDataFile("brisque_allmodel.dat", false)); }
inline cv::String readbrisquerange() { return readfile(cvtest::findDataFile("brisque_allrange.dat", false)); }
// instantiates a brisque object for testing // instantiates a brisque object for testing
inline cv::Ptr<quality::QualityBRISQUE> create_brisque() inline cv::Ptr<quality::QualityBRISQUE> create_brisque()
{ {
return quality::QualityBRISQUE::create( // location of BRISQUE model and range file
readbrisquemodel() // place these files in ${OPENCV_TEST_DATA_PATH}/quality/, or the tests will be skipped
, readbrisquerange() const auto model = cvtest::findDataFile("brisque_allmodel.dat", false);
); const auto range = cvtest::findDataFile("brisque_allrange.dat", false);
return quality::QualityBRISQUE::create(model, range);
} }
// static method // static method
TEST(TEST_CASE_NAME, static_ ) TEST(TEST_CASE_NAME, static_ )
{ {
std::vector<cv::Mat> qMats = {};
quality_expect_near( quality_expect_near(
quality::QualityBRISQUE::compute( readbrisquemodel(), readbrisquerange(), get_testfile_1a(), qMats), BRISQUE_EXPECTED_1 quality::QualityBRISQUE::compute(
get_testfile_1a()
, cvtest::findDataFile("brisque_allmodel.dat", false)
, cvtest::findDataFile("brisque_allrange.dat", false)
)
, BRISQUE_EXPECTED_1
); );
EXPECT_EQ(qMats.size(), 1U);
} }
// single channel, instance method, with and without opencl // single channel, instance method, with and without opencl
TEST(TEST_CASE_NAME, single_channel ) TEST(TEST_CASE_NAME, single_channel )
{ {
auto fn = []() { quality_test(create_brisque(), get_testfile_1a(), BRISQUE_EXPECTED_1); }; auto fn = []() { quality_test(create_brisque(), get_testfile_1a(), BRISQUE_EXPECTED_1, 0, true ); };
OCL_OFF( fn() ); OCL_OFF( fn() );
OCL_ON( fn() ); OCL_ON( fn() );
} }
...@@ -62,7 +53,7 @@ TEST(TEST_CASE_NAME, single_channel ) ...@@ -62,7 +53,7 @@ TEST(TEST_CASE_NAME, single_channel )
// multi-channel // multi-channel
TEST(TEST_CASE_NAME, multi_channel) TEST(TEST_CASE_NAME, multi_channel)
{ {
quality_test(create_brisque(), get_testfile_2a(), BRISQUE_EXPECTED_2); quality_test(create_brisque(), get_testfile_2a(), BRISQUE_EXPECTED_2, 0, true);
} }
// multi-frame test // multi-frame test
...@@ -73,7 +64,16 @@ TEST(TEST_CASE_NAME, multi_frame) ...@@ -73,7 +64,16 @@ TEST(TEST_CASE_NAME, multi_frame)
cv::add(BRISQUE_EXPECTED_1, BRISQUE_EXPECTED_2, expected); cv::add(BRISQUE_EXPECTED_1, BRISQUE_EXPECTED_2, expected);
expected /= 2.; expected /= 2.;
quality_test(create_brisque(), get_testfile_1a2a(), expected, 2 ); quality_test(create_brisque(), get_testfile_1a2a(), expected, 0, true );
}
// check brisque model/range persistence
TEST(TEST_CASE_NAME, model_persistence )
{
auto ptr = create_brisque();
auto fn = [&ptr]() { quality_test(ptr, get_testfile_1a(), BRISQUE_EXPECTED_1, 0, true); };
fn();
fn(); // model/range should persist with brisque ptr through multiple invocations
} }
// internal a/b test // internal a/b test
...@@ -81,8 +81,9 @@ TEST(TEST_CASE_NAME, multi_frame) ...@@ -81,8 +81,9 @@ TEST(TEST_CASE_NAME, multi_frame)
TEST(TEST_CASE_NAME, performance) TEST(TEST_CASE_NAME, performance)
{ {
auto ref = get_testfile_1a(); auto ref = get_testfile_1a();
auto alg = create_brisque();
quality_performance_test("BRISQUE", [&]() { cv::quality::QualityBRISQUE::compute(ref, cv::noArray()); }); quality_performance_test("BRISQUE", [&]() { alg->compute(ref); });
} }
*/ */
} }
......
modules/quality/test/test_precomp.hpp
View file @ 4c2f5135
...@@ -62,7 +62,7 @@ inline void quality_expect_near( const cv::Scalar& a, const cv::Scalar& b, doubl ...@@ -62,7 +62,7 @@ inline void quality_expect_near( const cv::Scalar& a, const cv::Scalar& b, doubl
// execute quality test for a pair of images // execute quality test for a pair of images
template <typename TMat> template <typename TMat>
inline void quality_test(cv::Ptr<quality::QualityBase> ptr, const TMat& cmp, const Scalar& expected, const std::size_t quality_maps_expected = 1) inline void quality_test(cv::Ptr<quality::QualityBase> ptr, const TMat& cmp, const Scalar& expected, const std::size_t quality_maps_expected = 1, const bool empty_expected = false )
{ {
std::vector<cv::Mat> qMats = {}; std::vector<cv::Mat> qMats = {};
ptr->getQualityMaps(qMats); ptr->getQualityMaps(qMats);
...@@ -70,8 +70,13 @@ inline void quality_test(cv::Ptr<quality::QualityBase> ptr, const TMat& cmp, con ...@@ -70,8 +70,13 @@ inline void quality_test(cv::Ptr<quality::QualityBase> ptr, const TMat& cmp, con
quality_expect_near( expected, ptr->compute(cmp)); quality_expect_near( expected, ptr->compute(cmp));
EXPECT_FALSE(ptr->empty()); if (empty_expected)
EXPECT_TRUE(ptr->empty());
else
EXPECT_FALSE(ptr->empty());
ptr->getQualityMaps(qMats); ptr->getQualityMaps(qMats);
EXPECT_EQ( qMats.size(), quality_maps_expected); EXPECT_EQ( qMats.size(), quality_maps_expected);
for (auto& qm : qMats) for (auto& qm : qMats)
{ {
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment