Commit d068e274 authored by tribta's avatar tribta

Tutorial Morph Lines Detection

parent 18bc4db7
Extract horizontal and vertical lines by using morphological operations {#tutorial_moprh_lines_detection}
Extract horizontal and vertical lines by using morphological operations {#tutorial_morph_lines_detection}
=============
@prev_tutorial{tutorial_hitOrMiss}
@next_tutorial{tutorial_pyramids}
Goal
----
In this tutorial you will learn how to:
- Apply two very common morphology operators (i.e. Dilation and Erosion), with the creation of custom kernels, in order to extract straight lines on the horizontal and vertical axes. For this purpose, you will use the following OpenCV functions:
- @ref cv::erode
- @ref cv::dilate
- @ref cv::getStructuringElement
- **erode()**
- **dilate()**
- **getStructuringElement()**
in an example where your goal will be to extract the music notes from a music sheet.
......@@ -48,39 +51,144 @@ A structuring element can have many common shapes, such as lines, diamonds, disk
Code
----
This tutorial code's is shown lines below. You can also download it from [here](https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/ImgProc/Morphology_3.cpp).
@include samples/cpp/tutorial_code/ImgProc/Morphology_3.cpp
This tutorial code's is shown lines below.
@add_toggle_cpp
You can also download it from [here](https://raw.githubusercontent.com/opencv/opencv/master/samples/cpp/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.cpp).
@include samples/cpp/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.cpp
@end_toggle
@add_toggle_java
You can also download it from [here](https://raw.githubusercontent.com/opencv/opencv/master/samples/java/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.java).
@include samples/java/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.java
@end_toggle
@add_toggle_python
You can also download it from [here](https://raw.githubusercontent.com/opencv/opencv/master/samples/python/tutorial_code/imgProc/morph_lines_detection/morph_lines_detection.py).
@include samples/python/tutorial_code/imgProc/morph_lines_detection/morph_lines_detection.py
@end_toggle
Explanation / Result
--------------------
-# Load the source image and check if it is loaded without any problem, then show it:
@snippet samples/cpp/tutorial_code/ImgProc/Morphology_3.cpp load_image
![](images/src.png)
Get image from [here](https://raw.githubusercontent.com/opencv/opencv/master/doc/tutorials/imgproc/morph_lines_detection/images/src.png) .
#### Load Image
@add_toggle_cpp
@snippet samples/cpp/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.cpp load_image
@end_toggle
@add_toggle_java
@snippet samples/java/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.java load_image
@end_toggle
@add_toggle_python
@snippet samples/python/tutorial_code/imgProc/morph_lines_detection/morph_lines_detection.py load_image
@end_toggle
![](images/src.png)
#### Grayscale
@add_toggle_cpp
@snippet samples/cpp/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.cpp gray
@end_toggle
@add_toggle_java
@snippet samples/java/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.java gray
@end_toggle
@add_toggle_python
@snippet samples/python/tutorial_code/imgProc/morph_lines_detection/morph_lines_detection.py gray
@end_toggle
![](images/gray.png)
#### Grayscale to Binary image
@add_toggle_cpp
@snippet samples/cpp/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.cpp bin
@end_toggle
@add_toggle_java
@snippet samples/java/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.java bin
@end_toggle
@add_toggle_python
@snippet samples/python/tutorial_code/imgProc/morph_lines_detection/morph_lines_detection.py bin
@end_toggle
![](images/binary.png)
#### Output images
Now we are ready to apply morphological operations in order to extract the horizontal and vertical lines and as a consequence to separate the the music notes from the music sheet, but first let's initialize the output images that we will use for that reason:
@add_toggle_cpp
@snippet samples/cpp/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.cpp init
@end_toggle
@add_toggle_java
@snippet samples/java/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.java init
@end_toggle
@add_toggle_python
@snippet samples/python/tutorial_code/imgProc/morph_lines_detection/morph_lines_detection.py init
@end_toggle
#### Structure elements
As we specified in the theory in order to extract the object that we desire, we need to create the corresponding structure element. Since we want to extract the horizontal lines, a corresponding structure element for that purpose will have the following shape:
![](images/linear_horiz.png)
and in the source code this is represented by the following code snippet:
@add_toggle_cpp
@snippet samples/cpp/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.cpp horiz
@end_toggle
@add_toggle_java
@snippet samples/java/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.java horiz
@end_toggle
@add_toggle_python
@snippet samples/python/tutorial_code/imgProc/morph_lines_detection/morph_lines_detection.py horiz
@end_toggle
![](images/horiz.png)
The same applies for the vertical lines, with the corresponding structure element:
![](images/linear_vert.png)
and again this is represented as follows:
@add_toggle_cpp
@snippet samples/cpp/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.cpp vert
@end_toggle
@add_toggle_java
@snippet samples/java/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.java vert
@end_toggle
@add_toggle_python
@snippet samples/python/tutorial_code/imgProc/morph_lines_detection/morph_lines_detection.py vert
@end_toggle
![](images/vert.png)
-# Then transform image to grayscale if it not already:
@snippet samples/cpp/tutorial_code/ImgProc/Morphology_3.cpp gray
![](images/gray.png)
#### Refine edges / Result
-# Afterwards transform grayscale image to binary. Notice the ~ symbol which indicates that we use the inverse (i.e. bitwise_not) version of it:
@snippet samples/cpp/tutorial_code/ImgProc/Morphology_3.cpp bin
![](images/binary.png)
As you can see we are almost there. However, at that point you will notice that the edges of the notes are a bit rough. For that reason we need to refine the edges in order to obtain a smoother result:
-# Now we are ready to apply morphological operations in order to extract the horizontal and vertical lines and as a consequence to separate the the music notes from the music sheet, but first let's initialize the output images that we will use for that reason:
@snippet samples/cpp/tutorial_code/ImgProc/Morphology_3.cpp init
@add_toggle_cpp
@snippet samples/cpp/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.cpp smooth
@end_toggle
-# As we specified in the theory in order to extract the object that we desire, we need to create the corresponding structure element. Since here we want to extract the horizontal lines, a corresponding structure element for that purpose will have the following shape:
![](images/linear_horiz.png)
and in the source code this is represented by the following code snippet:
@snippet samples/cpp/tutorial_code/ImgProc/Morphology_3.cpp horiz
![](images/horiz.png)
@add_toggle_java
@snippet samples/java/tutorial_code/ImgProc/morph_lines_detection/Morphology_3.java smooth
@end_toggle
-# The same applies for the vertical lines, with the corresponding structure element:
![](images/linear_vert.png)
and again this is represented as follows:
@snippet samples/cpp/tutorial_code/ImgProc/Morphology_3.cpp vert
![](images/vert.png)
@add_toggle_python
@snippet samples/python/tutorial_code/imgProc/morph_lines_detection/morph_lines_detection.py smooth
@end_toggle
-# As you can see we are almost there. However, at that point you will notice that the edges of the notes are a bit rough. For that reason we need to refine the edges in order to obtain a smoother result:
@snippet samples/cpp/tutorial_code/ImgProc/Morphology_3.cpp smooth
![](images/smooth.png)
![](images/smooth.png)
......@@ -39,7 +39,9 @@ In this section you will learn about the image processing (manipulation) functio
Learn how to find patterns in binary images using the Hit-or-Miss operation
- @subpage tutorial_moprh_lines_detection
- @subpage tutorial_morph_lines_detection
*Languages:* C++, Java, Python
*Compatibility:* \> OpenCV 2.0
......
......@@ -4,28 +4,32 @@
* @author OpenCV team
*/
#include <iostream>
#include <opencv2/opencv.hpp>
void show_wait_destroy(const char* winname, cv::Mat img);
using namespace std;
using namespace cv;
int main(int, char** argv)
{
//! [load_image]
//! [load_image]
// Load the image
Mat src = imread(argv[1]);
// Check if image is loaded fine
if(!src.data)
cerr << "Problem loading image!!!" << endl;
if(src.empty()){
printf(" Error opening image\n");
printf(" Program Arguments: [image_path]\n");
return -1;
}
// Show source image
imshow("src", src);
//! [load_image]
//! [load_image]
//! [gray]
// Transform source image to gray if it is not
//! [gray]
// Transform source image to gray if it is not already
Mat gray;
if (src.channels() == 3)
......@@ -38,58 +42,58 @@ int main(int, char** argv)
}
// Show gray image
imshow("gray", gray);
//! [gray]
show_wait_destroy("gray", gray);
//! [gray]
//! [bin]
//! [bin]
// Apply adaptiveThreshold at the bitwise_not of gray, notice the ~ symbol
Mat bw;
adaptiveThreshold(~gray, bw, 255, CV_ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 15, -2);
// Show binary image
imshow("binary", bw);
//! [bin]
show_wait_destroy("binary", bw);
//! [bin]
//! [init]
//! [init]
// Create the images that will use to extract the horizontal and vertical lines
Mat horizontal = bw.clone();
Mat vertical = bw.clone();
//! [init]
//! [init]
//! [horiz]
//! [horiz]
// Specify size on horizontal axis
int horizontalsize = horizontal.cols / 30;
int horizontal_size = horizontal.cols / 30;
// Create structure element for extracting horizontal lines through morphology operations
Mat horizontalStructure = getStructuringElement(MORPH_RECT, Size(horizontalsize,1));
Mat horizontalStructure = getStructuringElement(MORPH_RECT, Size(horizontal_size, 1));
// Apply morphology operations
erode(horizontal, horizontal, horizontalStructure, Point(-1, -1));
dilate(horizontal, horizontal, horizontalStructure, Point(-1, -1));
// Show extracted horizontal lines
imshow("horizontal", horizontal);
//! [horiz]
show_wait_destroy("horizontal", horizontal);
//! [horiz]
//! [vert]
//! [vert]
// Specify size on vertical axis
int verticalsize = vertical.rows / 30;
int vertical_size = vertical.rows / 30;
// Create structure element for extracting vertical lines through morphology operations
Mat verticalStructure = getStructuringElement(MORPH_RECT, Size( 1,verticalsize));
Mat verticalStructure = getStructuringElement(MORPH_RECT, Size(1, vertical_size));
// Apply morphology operations
erode(vertical, vertical, verticalStructure, Point(-1, -1));
dilate(vertical, vertical, verticalStructure, Point(-1, -1));
// Show extracted vertical lines
imshow("vertical", vertical);
//! [vert]
show_wait_destroy("vertical", vertical);
//! [vert]
//! [smooth]
//! [smooth]
// Inverse vertical image
bitwise_not(vertical, vertical);
imshow("vertical_bit", vertical);
show_wait_destroy("vertical_bit", vertical);
// Extract edges and smooth image according to the logic
// 1. extract edges
......@@ -101,12 +105,12 @@ int main(int, char** argv)
// Step 1
Mat edges;
adaptiveThreshold(vertical, edges, 255, CV_ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 3, -2);
imshow("edges", edges);
show_wait_destroy("edges", edges);
// Step 2
Mat kernel = Mat::ones(2, 2, CV_8UC1);
dilate(edges, edges, kernel);
imshow("dilate", edges);
show_wait_destroy("dilate", edges);
// Step 3
Mat smooth;
......@@ -119,9 +123,15 @@ int main(int, char** argv)
smooth.copyTo(vertical, edges);
// Show final result
imshow("smooth", vertical);
//! [smooth]
show_wait_destroy("smooth - final", vertical);
//! [smooth]
waitKey(0);
return 0;
}
\ No newline at end of file
}
void show_wait_destroy(const char* winname, cv::Mat img) {
imshow(winname, img);
moveWindow(winname, 500, 0);
waitKey(0);
destroyWindow(winname);
}
/**
* @file Morphology_3.java
* @brief Use morphology transformations for extracting horizontal and vertical lines sample code
*/
import org.opencv.core.*;
import org.opencv.highgui.HighGui;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.imgproc.Imgproc;
class Morphology_3Run {
public void run(String[] args) {
//! [load_image]
// Check number of arguments
if (args.length == 0){
System.out.println("Not enough parameters!");
System.out.println("Program Arguments: [image_path]");
System.exit(-1);
}
// Load the image
Mat src = Imgcodecs.imread(args[0]);
// Check if image is loaded fine
if( src.empty() ) {
System.out.println("Error opening image: " + args[0]);
System.exit(-1);
}
// Show source image
HighGui.imshow("src", src);
//! [load_image]
//! [gray]
// Transform source image to gray if it is not already
Mat gray = new Mat();
if (src.channels() == 3)
{
Imgproc.cvtColor(src, gray, Imgproc.COLOR_BGR2GRAY);
}
else
{
gray = src;
}
// Show gray image
showWaitDestroy("gray" , gray);
//! [gray]
//! [bin]
// Apply adaptiveThreshold at the bitwise_not of gray
Mat bw = new Mat();
Core.bitwise_not(gray, gray);
Imgproc.adaptiveThreshold(gray, bw, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 15, -2);
// Show binary image
showWaitDestroy("binary" , bw);
//! [bin]
//! [init]
// Create the images that will use to extract the horizontal and vertical lines
Mat horizontal = bw.clone();
Mat vertical = bw.clone();
//! [init]
//! [horiz]
// Specify size on horizontal axis
int horizontal_size = horizontal.cols() / 30;
// Create structure element for extracting horizontal lines through morphology operations
Mat horizontalStructure = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(horizontal_size,1));
// Apply morphology operations
Imgproc.erode(horizontal, horizontal, horizontalStructure);
Imgproc.dilate(horizontal, horizontal, horizontalStructure);
// Show extracted horizontal lines
showWaitDestroy("horizontal" , horizontal);
//! [horiz]
//! [vert]
// Specify size on vertical axis
int vertical_size = vertical.rows() / 30;
// Create structure element for extracting vertical lines through morphology operations
Mat verticalStructure = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size( 1,vertical_size));
// Apply morphology operations
Imgproc.erode(vertical, vertical, verticalStructure);
Imgproc.dilate(vertical, vertical, verticalStructure);
// Show extracted vertical lines
showWaitDestroy("vertical", vertical);
//! [vert]
//! [smooth]
// Inverse vertical image
Core.bitwise_not(vertical, vertical);
showWaitDestroy("vertical_bit" , vertical);
// Extract edges and smooth image according to the logic
// 1. extract edges
// 2. dilate(edges)
// 3. src.copyTo(smooth)
// 4. blur smooth img
// 5. smooth.copyTo(src, edges)
// Step 1
Mat edges = new Mat();
Imgproc.adaptiveThreshold(vertical, edges, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 3, -2);
showWaitDestroy("edges", edges);
// Step 2
Mat kernel = Mat.ones(2, 2, CvType.CV_8UC1);
Imgproc.dilate(edges, edges, kernel);
showWaitDestroy("dilate", edges);
// Step 3
Mat smooth = new Mat();
vertical.copyTo(smooth);
// Step 4
Imgproc.blur(smooth, smooth, new Size(2, 2));
// Step 5
smooth.copyTo(vertical, edges);
// Show final result
showWaitDestroy("smooth - final", vertical);
//! [smooth]
System.exit(0);
}
private void showWaitDestroy(String winname, Mat img) {
HighGui.imshow(winname, img);
HighGui.moveWindow(winname, 500, 0);
HighGui.waitKey(0);
HighGui.destroyWindow(winname);
}
}
public class Morphology_3 {
public static void main(String[] args) {
// Load the native library.
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
new Morphology_3Run().run(args);
}
}
"""
@file morph_lines_detection.py
@brief Use morphology transformations for extracting horizontal and vertical lines sample code
"""
import numpy as np
import sys
import cv2
def show_wait_destroy(winname, img):
cv2.imshow(winname, img)
cv2.moveWindow(winname, 500, 0)
cv2.waitKey(0)
cv2.destroyWindow(winname)
def main(argv):
# [load_image]
# Check number of arguments
if len(argv) < 1:
print ('Not enough parameters')
print ('Usage:\nmorph_lines_detection.py < path_to_image >')
return -1
# Load the image
src = cv2.imread(argv[0], cv2.IMREAD_COLOR)
# Check if image is loaded fine
if src is None:
print ('Error opening image: ' + argv[0])
return -1
# Show source image
cv2.imshow("src", src)
# [load_image]
# [gray]
# Transform source image to gray if it is not already
if len(src.shape) != 2:
gray = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY)
else:
gray = src
# Show gray image
show_wait_destroy("gray", gray)
# [gray]
# [bin]
# Apply adaptiveThreshold at the bitwise_not of gray, notice the ~ symbol
gray = cv2.bitwise_not(gray)
bw = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, \
cv2.THRESH_BINARY, 15, -2)
# Show binary image
show_wait_destroy("binary", bw)
# [bin]
# [init]
# Create the images that will use to extract the horizontal and vertical lines
horizontal = np.copy(bw)
vertical = np.copy(bw)
# [init]
# [horiz]
# Specify size on horizontal axis
cols = horizontal.shape[1]
horizontal_size = cols / 30
# Create structure element for extracting horizontal lines through morphology operations
horizontalStructure = cv2.getStructuringElement(cv2.MORPH_RECT, (horizontal_size, 1))
# Apply morphology operations
horizontal = cv2.erode(horizontal, horizontalStructure)
horizontal = cv2.dilate(horizontal, horizontalStructure)
# Show extracted horizontal lines
show_wait_destroy("horizontal", horizontal)
# [horiz]
# [vert]
# Specify size on vertical axis
rows = vertical.shape[0]
verticalsize = rows / 30
# Create structure element for extracting vertical lines through morphology operations
verticalStructure = cv2.getStructuringElement(cv2.MORPH_RECT, (1, verticalsize))
# Apply morphology operations
vertical = cv2.erode(vertical, verticalStructure)
vertical = cv2.dilate(vertical, verticalStructure)
# Show extracted vertical lines
show_wait_destroy("vertical", vertical)
# [vert]
# [smooth]
# Inverse vertical image
vertical = cv2.bitwise_not(vertical)
show_wait_destroy("vertical_bit", vertical)
'''
Extract edges and smooth image according to the logic
1. extract edges
2. dilate(edges)
3. src.copyTo(smooth)
4. blur smooth img
5. smooth.copyTo(src, edges)
'''
# Step 1
edges = cv2.adaptiveThreshold(vertical, 255, cv2.ADAPTIVE_THRESH_MEAN_C, \
cv2.THRESH_BINARY, 3, -2)
show_wait_destroy("edges", edges)
# Step 2
kernel = np.ones((2, 2), np.uint8)
edges = cv2.dilate(edges, kernel)
show_wait_destroy("dilate", edges)
# Step 3
smooth = np.copy(vertical)
# Step 4
smooth = cv2.blur(smooth, (2, 2))
# Step 5
(rows, cols) = np.where(edges != 0)
vertical[rows, cols] = smooth[rows, cols]
# Show final result
show_wait_destroy("smooth - final", vertical)
# [smooth]
return 0
if __name__ == "__main__":
main(sys.argv[1:])
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment