Commit 584c2e66 authored by Henrik Kjellander's avatar Henrik Kjellander

Add all wiki pages as MarkDown documents + README.md

This is based on today's content at
https://code.google.com/p/libyuv/w/list

BUG=chromium:587193

Review URL: https://codereview.chromium.org/1703653002 .
parent f9496cfd
**libyuv** is an open source project that includes YUV scaling and conversion functionality.
* Scale YUV to prepare content for compression, with point, bilinear or box filter.
* Convert to YUV from webcam formats.
* Convert from YUV to formats for rendering/effects.
* Rotate by 90/180/270 degrees to adjust for mobile devices in portrait mode.
* Optimized for SSE2/SSSE3/AVX2 on x86/x64.
* Optimized for Neon on Arm.
* Optimized for DSP R2 on Mips.
### Development
See [Getting started] [1] for instructions on how to get started developing.
You can also browse the [docs directory] [2] for more documentation.
[1]: docs/getting_started.md
[2]: docs/
# Introduction
For test purposes, environment variables can be set to control libyuv behavior. These should only be used for testing, to narrow down bugs or to test performance.
# CPU
By default the cpu is detected and the most advanced form of SIMD is used. But you can disable instruction sets selectively, or completely, falling back on C code. Set the variable to 1 to disable the specified instruction set.
LIBYUV_DISABLE_ASM
LIBYUV_DISABLE_X86
LIBYUV_DISABLE_SSE2
LIBYUV_DISABLE_SSSE3
LIBYUV_DISABLE_SSE41
LIBYUV_DISABLE_SSE42
LIBYUV_DISABLE_AVX
LIBYUV_DISABLE_AVX2
LIBYUV_DISABLE_AVX3
LIBYUV_DISABLE_ERMS
LIBYUV_DISABLE_FMA3
LIBYUV_DISABLE_MIPS_DSPR2
LIBYUV_DISABLE_NEON
# Test Width/Height/Repeat
The unittests default to a small image (32x18) to run fast. This can be set by environment variable to test a specific resolutions.
You can also repeat the test a specified number of iterations, allowing benchmarking and profiling.
set LIBYUV_WIDTH=1280
set LIBYUV_HEIGHT=720
set LIBYUV_REPEAT=999
set LIBYUV_FLAGS=-1
set LIBYUV_CPU_INFO=-1
# Introduction
This document discusses the current state of filtering in libyuv. An emphasis on maximum performance while avoiding memory exceptions, and minimal amount of code/complexity. See future work at end.
# LibYuv Filter Subsampling
There are 2 challenges with subsampling
1. centering of samples, which involves clamping on edges
2. clipping a source region
Centering depends on scale factor and filter mode.
# Down Sampling
If scaling down, the stepping rate is always src_width / dst_width.
dx = src_width / dst_width;
e.g. If scaling from 1280x720 to 640x360, the step thru the source will be 2.0, stepping over 2 pixels of source for each pixel of destination.
Centering, depends on filter mode.
*Point* downsampling takes the middle pixel.
x = dx >> 1;
For odd scale factors (e.g. 3x down) this is exactly the middle. For even scale factors, this rounds up and takes the pixel to the right of center. e.g. scale of 4x down will take pixel 2.
**Bilinear** filter, uses the 2x2 pixels in the middle.
x = dx / 2 - 0.5;
For odd scale factors (e.g. 3x down) this is exactly the middle, and point sampling is used.
For even scale factors, this evenly filters the middle 2x2 pixels. e.g. 4x down will filter pixels 1,2 at 50% in both directions.
**Box** filter averages the entire box so sampling starts at 0.
x = 0;
For a scale factor of 2x down, this is equivalent to bilinear.
# Up Sampling
**Point** upsampling use stepping rate of src_width / dst_width and a starting coordinate of 0.
x = 0;
dx = src_width / dst_width;
e.g. If scaling from 640x360 to 1280x720 the step thru the source will be 0.0, stepping half a pixel of source for each pixel of destination. Each pixel is replicated by the scale factor.
**Bilinear** filter stretches such that the first pixel of source maps to the first pixel of destination, and the last pixel of source maps to the last pixel of destination.
x = 0;
dx = (src_width - 1) / (dst_width - 1);
This method is not technically correct, and will likely change in the future.
* It is inconsistent with the bilinear down sampler. The same method could be used for down sampling, and then it would be more reversible, but that would prevent specialized 2x down sampling.
* Although centered, the image is slightly magnified.
* The filtering was changed in early 2013 - previously it used:
x = 0;
dx = (src_width - 1) / (dst_width - 1);
Which is the correct scale factor, but shifted the image left, and extruded the last pixel. The reason for the change was to remove the extruding code from the low level row functions, allowing 3 functions to sshare the same row functions - ARGBScale, I420Scale, and ARGBInterpolate. Then the one function was ported to many cpu variations: SSE2, SSSE3, AVX2, Neon and 'Any' version for any number of pixels and alignment. The function is also specialized for 0,25,50,75%.
The above goes still has the potential to read the last pixel 100% and last pixel + 1 0%, which may cause a memory exception. So the left pixel goes to a fraction less than the last pixel, but filters in the minimum amount of it, and the maximum of the last pixel.
dx = FixedDiv((src_width << 16) - 0x00010001, (dst << 16) - 0x00010000);
**Box** filter for upsampling switches over to Bilinear.
# Scale snippet:
#define CENTERSTART(dx, s) (dx < 0) ? -((-dx >> 1) + s) : ((dx >> 1) + s)
#define FIXEDDIV1(src, dst) FixedDiv((src << 16) - 0x00010001, \
(dst << 16) - 0x00010000);
// Compute slope values for stepping.
void ScaleSlope(int src_width, int src_height,
int dst_width, int dst_height,
FilterMode filtering,
int* x, int* y, int* dx, int* dy) {
assert(x != NULL);
assert(y != NULL);
assert(dx != NULL);
assert(dy != NULL);
assert(src_width != 0);
assert(src_height != 0);
assert(dst_width > 0);
assert(dst_height > 0);
if (filtering == kFilterBox) {
// Scale step for point sampling duplicates all pixels equally.
*dx = FixedDiv(Abs(src_width), dst_width);
*dy = FixedDiv(src_height, dst_height);
*x = 0;
*y = 0;
} else if (filtering == kFilterBilinear) {
// Scale step for bilinear sampling renders last pixel once for upsample.
if (dst_width <= Abs(src_width)) {
*dx = FixedDiv(Abs(src_width), dst_width);
*x = CENTERSTART(*dx, -32768);
} else if (dst_width > 1) {
*dx = FIXEDDIV1(Abs(src_width), dst_width);
*x = 0;
}
if (dst_height <= src_height) {
*dy = FixedDiv(src_height, dst_height);
*y = CENTERSTART(*dy, -32768); // 32768 = -0.5 to center bilinear.
} else if (dst_height > 1) {
*dy = FIXEDDIV1(src_height, dst_height);
*y = 0;
}
} else if (filtering == kFilterLinear) {
// Scale step for bilinear sampling renders last pixel once for upsample.
if (dst_width <= Abs(src_width)) {
*dx = FixedDiv(Abs(src_width), dst_width);
*x = CENTERSTART(*dx, -32768);
} else if (dst_width > 1) {
*dx = FIXEDDIV1(Abs(src_width), dst_width);
*x = 0;
}
*dy = FixedDiv(src_height, dst_height);
*y = *dy >> 1;
} else {
// Scale step for point sampling duplicates all pixels equally.
*dx = FixedDiv(Abs(src_width), dst_width);
*dy = FixedDiv(src_height, dst_height);
*x = CENTERSTART(*dx, 0);
*y = CENTERSTART(*dy, 0);
}
// Negative src_width means horizontally mirror.
if (src_width < 0) {
*x += (dst_width - 1) * *dx;
*dx = -*dx;
src_width = -src_width;
}
}
# Future Work
Point sampling should ideally be the same as bilinear, but pixel by pixel, round to nearest neighbor. But as is, it is reversible and exactly matches ffmpeg at all scale factors, both up and down. The scale factor is
dx = src_width / dst_width;
The step value is centered for down sample:
x = dx / 2;
Or starts at 0 for upsample.
x = 0;
Bilinear filtering is currently correct for down sampling, but not for upsampling.
Upsampling is stretching the first and last pixel of source, to the first and last pixel of destination.
dx = (src_width - 1) / (dst_width - 1);<br>
x = 0;
It should be stretching such that the first pixel is centered in the middle of the scale factor, to match the pixel that would be sampled for down sampling by the same amount. And same on last pixel.
dx = src_width / dst_width;<br>
x = dx / 2 - 0.5;
This would start at -0.5 and go to last pixel + 0.5, sampling 50% from last pixel + 1.
Then clamping would be needed. On GPUs there are numerous ways to clamp.
1. Clamp the coordinate to the edge of the texture, duplicating the first and last pixel.
2. Blend with a constant color, such as transparent black. Typically best for fonts.
3. Mirror the UV coordinate, which is similar to clamping. Good for continuous tone images.
4. Wrap the coordinate, for texture tiling.
5. Allow the coordinate to index beyond the image, which may be the correct data if sampling a subimage.
6. Extrapolate the edge based on the previous pixel. pixel -0.5 is computed from slope of pixel 0 and 1.
Some of these are computational, even for a GPU, which is one reason textures are sometimes limited to power of 2 sizes.
We do care about the clipping case, where allowing coordinates to become negative and index pixels before the image is the correct data. But normally for simple scaling, we want to clamp to the edge pixel. For example, if bilinear scaling from 3x3 to 30x30, we’d essentially want 10 pixels of each of the original 3 pixels. But we want the original pixels to land in the middle of each 10 pixels, at offsets 5, 15 and 25. There would be filtering between 5 and 15 between the original pixels 0 and 1. And filtering between 15 and 25 from original pixels 1 and 2. The first 5 pixels are clamped to pixel 0 and the last 5 pixels are clamped to pixel 2.
The easiest way to implement this is copy the original 3 pixels to a buffer, and duplicate the first and last pixels. 0,1,2 becomes 0, 0,1,2, 2. Then implement a filtering without clamping. We call this source extruding. Its only necessary on up sampling, since down sampler will always have valid surrounding pixels.
Extruding is practical when the image is already copied to a temporary buffer. It could be done to the original image, as long as the original memory is restored, but valgrind and/or memory protection would disallow this, so it requires a memcpy to a temporary buffer, which may hurt performance. The memcpy has a performance advantage, from a cache point of view, that can actually make this technique faster, depending on hardware characteristics.
Vertical extrusion can be done with a memcpy of the first/last row, or clamping a pointer.
The other way to implement clamping is handle the edges with a memset. e.g. Read first source pixel and memset the first 5 pixels. Filter pixels 0,1,2 to 5 to 25. Read last pixel and memset the last 5 pixels. Blur is implemented with this method like this, which has 3 loops per row - left, middle and right.
Box filter is only used for 2x down sample or more. Its based on integer sized boxes. Technically it should be filtered edges, but thats substantially slower (roughly 100x), and at that point you may as well do a cubic filter which is more correct.
Box filter currently sums rows into a row buffer. It does this with
Mirroring will use the same slope as normal, but with a negative.
The starting coordinate needs to consider the scale factor and filter. e.g. box filter of 30x30 to 3x3 with mirroring would use -10 for step, but x = 20. width (30) - dx.
Step needs to be accurate, so it uses an integer divide. This is as much as 5% of the profile. An approximated divide is substantially faster, but the inaccuracy causes stepping beyond the original image boundaries. 3 general solutions:
1. copy image to buffer with padding. allows for small errors in stepping.
2. hash the divide, so common values are quickly found.
3. change api so caller provides the slope.
# Introduction
Formats (FOURCC) supported by libyuv are detailed here.
# Core Formats
There are 2 core formats supported by libyuv - I420 and ARGB. All YUV formats can be converted to/from I420. All RGB formats can be converted to/from ARGB.
Filtering functions such as scaling and planar functions work on I420 and/or ARGB.
# OSX Core Media Pixel Formats
This is how OSX formats map to libyuv
enum {
kCMPixelFormat_32ARGB = 32, FOURCC_BGRA
kCMPixelFormat_32BGRA = 'BGRA', FOURCC_ARGB
kCMPixelFormat_24RGB = 24, FOURCC_RAW
kCMPixelFormat_16BE555 = 16, Not supported.
kCMPixelFormat_16BE565 = 'B565', Not supported.
kCMPixelFormat_16LE555 = 'L555', FOURCC_RGBO
kCMPixelFormat_16LE565 = 'L565', FOURCC_RGBP
kCMPixelFormat_16LE5551 = '5551', FOURCC_RGBO
kCMPixelFormat_422YpCbCr8 = '2vuy', FOURCC_UYVY
kCMPixelFormat_422YpCbCr8_yuvs = 'yuvs', FOURCC_YUY2
kCMPixelFormat_444YpCbCr8 = 'v308', FOURCC_I444 ?
kCMPixelFormat_4444YpCbCrA8 = 'v408', Not supported.
kCMPixelFormat_422YpCbCr16 = 'v216', Not supported.
kCMPixelFormat_422YpCbCr10 = 'v210', FOURCC_V210 previously. Removed now.
kCMPixelFormat_444YpCbCr10 = 'v410', Not supported.
kCMPixelFormat_8IndexedGray_WhiteIsZero = 0x00000028, Not supported.
};
# FOURCC (Four Charactacter Code) List
The following is extracted from video_common.h as a complete list of formats supported by libyuv.
enum FourCC {
// 9 Primary YUV formats: 5 planar, 2 biplanar, 2 packed.
FOURCC_I420 = FOURCC('I', '4', '2', '0'),
FOURCC_I422 = FOURCC('I', '4', '2', '2'),
FOURCC_I444 = FOURCC('I', '4', '4', '4'),
FOURCC_I411 = FOURCC('I', '4', '1', '1'),
FOURCC_I400 = FOURCC('I', '4', '0', '0'),
FOURCC_NV21 = FOURCC('N', 'V', '2', '1'),
FOURCC_NV12 = FOURCC('N', 'V', '1', '2'),
FOURCC_YUY2 = FOURCC('Y', 'U', 'Y', '2'),
FOURCC_UYVY = FOURCC('U', 'Y', 'V', 'Y'),
// 2 Secondary YUV formats: row biplanar.
FOURCC_M420 = FOURCC('M', '4', '2', '0'),
FOURCC_Q420 = FOURCC('Q', '4', '2', '0'),
// 9 Primary RGB formats: 4 32 bpp, 2 24 bpp, 3 16 bpp.
FOURCC_ARGB = FOURCC('A', 'R', 'G', 'B'),
FOURCC_BGRA = FOURCC('B', 'G', 'R', 'A'),
FOURCC_ABGR = FOURCC('A', 'B', 'G', 'R'),
FOURCC_24BG = FOURCC('2', '4', 'B', 'G'),
FOURCC_RAW = FOURCC('r', 'a', 'w', ' '),
FOURCC_RGBA = FOURCC('R', 'G', 'B', 'A'),
FOURCC_RGBP = FOURCC('R', 'G', 'B', 'P'), // rgb565 LE.
FOURCC_RGBO = FOURCC('R', 'G', 'B', 'O'), // argb1555 LE.
FOURCC_R444 = FOURCC('R', '4', '4', '4'), // argb4444 LE.
// 4 Secondary RGB formats: 4 Bayer Patterns.
FOURCC_RGGB = FOURCC('R', 'G', 'G', 'B'),
FOURCC_BGGR = FOURCC('B', 'G', 'G', 'R'),
FOURCC_GRBG = FOURCC('G', 'R', 'B', 'G'),
FOURCC_GBRG = FOURCC('G', 'B', 'R', 'G'),
// 1 Primary Compressed YUV format.
FOURCC_MJPG = FOURCC('M', 'J', 'P', 'G'),
// 5 Auxiliary YUV variations: 3 with U and V planes are swapped, 1 Alias.
FOURCC_YV12 = FOURCC('Y', 'V', '1', '2'),
FOURCC_YV16 = FOURCC('Y', 'V', '1', '6'),
FOURCC_YV24 = FOURCC('Y', 'V', '2', '4'),
FOURCC_YU12 = FOURCC('Y', 'U', '1', '2'), // Linux version of I420.
FOURCC_J420 = FOURCC('J', '4', '2', '0'),
FOURCC_J400 = FOURCC('J', '4', '0', '0'),
// 14 Auxiliary aliases. CanonicalFourCC() maps these to canonical fourcc.
FOURCC_IYUV = FOURCC('I', 'Y', 'U', 'V'), // Alias for I420.
FOURCC_YU16 = FOURCC('Y', 'U', '1', '6'), // Alias for I422.
FOURCC_YU24 = FOURCC('Y', 'U', '2', '4'), // Alias for I444.
FOURCC_YUYV = FOURCC('Y', 'U', 'Y', 'V'), // Alias for YUY2.
FOURCC_YUVS = FOURCC('y', 'u', 'v', 's'), // Alias for YUY2 on Mac.
FOURCC_HDYC = FOURCC('H', 'D', 'Y', 'C'), // Alias for UYVY.
FOURCC_2VUY = FOURCC('2', 'v', 'u', 'y'), // Alias for UYVY on Mac.
FOURCC_JPEG = FOURCC('J', 'P', 'E', 'G'), // Alias for MJPG.
FOURCC_DMB1 = FOURCC('d', 'm', 'b', '1'), // Alias for MJPG on Mac.
FOURCC_BA81 = FOURCC('B', 'A', '8', '1'), // Alias for BGGR.
FOURCC_RGB3 = FOURCC('R', 'G', 'B', '3'), // Alias for RAW.
FOURCC_BGR3 = FOURCC('B', 'G', 'R', '3'), // Alias for 24BG.
FOURCC_CM32 = FOURCC(0, 0, 0, 32), // Alias for BGRA kCMPixelFormat_32ARGB
FOURCC_CM24 = FOURCC(0, 0, 0, 24), // Alias for RAW kCMPixelFormat_24RGB
FOURCC_L555 = FOURCC('L', '5', '5', '5'), // Alias for RGBO.
FOURCC_L565 = FOURCC('L', '5', '6', '5'), // Alias for RGBP.
FOURCC_5551 = FOURCC('5', '5', '5', '1'), // Alias for RGBO.
// 1 Auxiliary compressed YUV format set aside for capturer.
FOURCC_H264 = FOURCC('H', '2', '6', '4'),
# The ARGB FOURCC
There are 4 ARGB layouts - ARGB, BGRA, ABGR and RGBA. ARGB is most common by far, used for screen formats, and windows webcam drivers.
The fourcc describes the order of channels in a ***register***.
A fourcc provided by capturer, can be thought of string, e.g. "ARGB".
On little endian machines, as an int, this would have 'A' in the lowest byte. The FOURCC macro reverses the order:
#define FOURCC(a, b, c, d) (((uint32)(a)) | ((uint32)(b) << 8) | ((uint32)(c) << 16) | ((uint32)(d) << 24))
So the "ARGB" string, read as an uint32, is
FOURCC_ARGB = FOURCC('A', 'R', 'G', 'B')
If you were to read ARGB pixels as uint32's, the alpha would be in the high byte, and the blue in the lowest byte. In memory, these are stored little endian, so 'B' is first, then 'G', 'R' and 'A' last.
When calling conversion functions, the names match the FOURCC, so in this case it would be I420ToARGB().
All formats can be converted to/from ARGB.
Most 'planar_functions' work on ARGB (e.g. ARGBBlend).
Some are channel order agnostic (e.g. ARGBScale).
Some functions are symmetric (e.g. ARGBToBGRA is the same as BGRAToARGB, so its a macro).
ARGBBlend expects preattenuated ARGB. The R,G,B are premultiplied by alpha. Other functions don't care.
# Getting Started
How to get and build the libyuv code.
## Pre-requisites
You'll need to have depot tools installed: https://www.chromium.org/developers/how-tos/install-depot-tools
Refer to chromium instructions for each platform for other prerequisites.
## Getting the Code
Create a working directory, enter it, and run:
gclient config https://chromium.googlesource.com/libyuv/libyuv
gclient sync
Then you'll get a .gclient file like:
solutions = [
{ "name" : "libyuv",
"url" : "https://chromium.googlesource.com/libyuv/libyuv",
"deps_file" : "DEPS",
"managed" : True,
"custom_deps" : {
},
"safesync_url": "",
},
];
For iOS add `;target_os=['ios'];` to your OSX .gclient and run `GYP_DEFINES="OS=ios" gclient sync.`
Browse the Git reprository: https://chromium.googlesource.com/libyuv/libyuv/+/master
### Android
For Android add `;target_os=['android'];` to your Linux .gclient
solutions = [
{ "name" : "libyuv",
"url" : "https://chromium.googlesource.com/libyuv/libyuv",
"deps_file" : "DEPS",
"managed" : True,
"custom_deps" : {
},
"safesync_url": "",
},
]
target_os = ["android", "unix"]
Then run:
export GYP_DEFINES="OS=android"
gclient sync
For Windows the gclient sync must be done from an Administrator command prompt.
The sync will generate native build files for your environment using gyp (Windows: Visual Studio, OSX: XCode, Linux: make). This generation can also be forced manually: `gclient runhooks`
To get just the source (not buildable):
git clone https://chromium.googlesource.com/libyuv/libyuv
## Building the Library and Unittests
### Windows
set GYP_DEFINES=target_arch=ia32
call python gyp_libyuv -fninja -G msvs_version=2013
ninja -j7 -C out\Release
ninja -j7 -C out\Debug
set GYP_DEFINES=target_arch=x64
call python gyp_libyuv -fninja -G msvs_version=2013
ninja -C out\Debug_x64
ninja -C out\Release_x64
#### Building with clangcl
set GYP_DEFINES=clang=1 target_arch=ia32 libyuv_enable_svn=1
set LLVM_REPO_URL=svn://svn.chromium.org/llvm-project
call python tools\clang\scripts\update.py
call python gyp_libyuv -fninja libyuv_test.gyp
ninja -C out\Debug
ninja -C out\Release
### OSX
Clang 64 bit shown. Remove `clang=1` for GCC and change x64 to ia32 for 32 bit.
GYP_DEFINES="clang=1 target_arch=x64" ./gyp_libyuv
ninja -j7 -C out/Debug
ninja -j7 -C out/Release
GYP_DEFINES="clang=1 target_arch=ia32" ./gyp_libyuv
ninja -j7 -C out/Debug
ninja -j7 -C out/Release
### iOS
http://www.chromium.org/developers/how-tos/build-instructions-ios
Add to .gclient last line: `target_os=['ios'];`
armv7
GYP_DEFINES="OS=ios target_arch=armv7 target_subarch=arm32" GYP_CROSSCOMPILE=1 GYP_GENERATOR_FLAGS="output_dir=out_ios" ./gyp_libyuv
ninja -j7 -C out_ios/Debug-iphoneos libyuv_unittest
ninja -j7 -C out_ios/Release-iphoneos libyuv_unittest
arm64
GYP_DEFINES="OS=ios target_arch=arm64 target_subarch=arm64" GYP_CROSSCOMPILE=1 GYP_GENERATOR_FLAGS="output_dir=out_ios" ./gyp_libyuv
ninja -j7 -C out_ios/Debug-iphoneos libyuv_unittest
ninja -j7 -C out_ios/Release-iphoneos libyuv_unittest
both armv7 and arm64 (fat)
GYP_DEFINES="OS=ios target_arch=armv7 target_subarch=both" GYP_CROSSCOMPILE=1 GYP_GENERATOR_FLAGS="output_dir=out_ios" ./gyp_libyuv
ninja -j7 -C out_ios/Debug-iphoneos libyuv_unittest
ninja -j7 -C out_ios/Release-iphoneos libyuv_unittest
simulator
GYP_DEFINES="OS=ios target_arch=ia32 target_subarch=arm32" GYP_CROSSCOMPILE=1 GYP_GENERATOR_FLAGS="output_dir=out_sim" ./gyp_libyuv
ninja -j7 -C out_sim/Debug-iphonesimulator libyuv_unittest
ninja -j7 -C out_sim/Release-iphonesimulator libyuv_unittest
### Android
https://code.google.com/p/chromium/wiki/AndroidBuildInstructions
Add to .gclient last line: `target_os=['android'];`
armv7
GYP_DEFINES="OS=android" GYP_CROSSCOMPILE=1 ./gyp_libyuv
ninja -j7 -C out/Debug libyuv_unittest_apk
ninja -j7 -C out/Release libyuv_unittest_apk
arm64
GYP_DEFINES="OS=android target_arch=arm64 target_subarch=arm64" GYP_CROSSCOMPILE=1 ./gyp_libyuv
ninja -j7 -C out/Debug libyuv_unittest_apk
ninja -j7 -C out/Release libyuv_unittest_apk
ia32
GYP_DEFINES="OS=android target_arch=ia32" GYP_CROSSCOMPILE=1 ./gyp_libyuv
ninja -j7 -C out/Debug libyuv_unittest_apk
ninja -j7 -C out/Release libyuv_unittest_apk
GYP_DEFINES="OS=android target_arch=ia32 android_full_debug=1" GYP_CROSSCOMPILE=1 ./gyp_libyuv
ninja -j7 -C out/Debug libyuv_unittest_apk
mipsel
GYP_DEFINES="OS=android target_arch=mipsel" GYP_CROSSCOMPILE=1 ./gyp_libyuv
ninja -j7 -C out/Debug libyuv_unittest_apk
ninja -j7 -C out/Release libyuv_unittest_apk
arm64 disassembly:
./third_party/android_tools/ndk//toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-4android-objdump -d out/Release/obj/source/libyuv.row_neon64.o
Running tests:
util/android/test_runner.py gtest -s libyuv_unittest -t 7200 --verbose --release --gtest_filter=*
Running test as benchmark:
util/android/test_runner.py gtest -s libyuv_unittest -t 7200 --verbose --release --gtest_filter=* -a "--libyuv_width=1280 --libyuv_height=720 --libyuv_repeat=999 --libyuv_flags=-1"
Running test with C code:
util/android/test_runner.py gtest -s libyuv_unittest -t 7200 --verbose --release --gtest_filter=* -a "--libyuv_width=1280 --libyuv_height=720 --libyuv_repeat=999 --libyuv_flags=0 --libyuv_cpu_info=0"
#### Building with GN
call gn gen out/Release "--args=is_debug=false target_cpu=\"x86\""
call gn gen out/Debug "--args=is_debug=true target_cpu=\"x86\""
ninja -C out/Release
ninja -C out/Debug
### Linux
tools/clang/scripts/update.sh
GYP_DEFINES="target_arch=x64" ./gyp_libyuv
ninja -j7 -C out/Debug
ninja -j7 -C out/Release
GYP_DEFINES="target_arch=ia32" ./gyp_libyuv
ninja -j7 -C out/Debug
ninja -j7 -C out/Release
#### CentOS
On CentOS 32 bit the following work around allows a sync:
export GYP_DEFINES="host_arch=ia32"
gclient sync
### Windows Shared Library
Modify libyuv.gyp from 'static_library' to 'shared_library', and add 'LIBYUV_BUILDING_SHARED_LIBRARY' to 'defines'.
gclient runhooks
After this command follow the building the library instructions above.
If you get a compile error for atlthunk.lib on Windows, read http://www.chromium.org/developers/how-tos/build-instructions-windows
### Build targets
ninja -C out/Debug libyuv
ninja -C out/Debug libyuv_unittest
ninja -C out/Debug compare
ninja -C out/Debug convert
ninja -C out/Debug psnr
ninja -C out/Debug cpuid
## Building the Library with make
### Linux
make -j7 V=1 -f linux.mk
make -j7 V=1 -f linux.mk clean
make -j7 V=1 -f linux.mk CXX=clang++
## Building the Library with cmake
Install cmake: http://www.cmake.org/
Default debug build:
mkdir out
cd out
cmake ..
cmake --build .
Release build/install
mkdir out
cd out
cmake -DCMAKE_INSTALL_PREFIX="/usr/lib" -DCMAKE_BUILD_TYPE="Release" ..
cmake --build . --config Release
sudo cmake --build . --target install --config Release
### Windows 8 Phone
Pre-requisite:
* Install Visual Studio 2012 and Arm to your environment.<br>
Then:
call "c:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\x86_arm\vcvarsx86_arm.bat"
or with Visual Studio 2013:
call "c:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\x86_arm\vcvarsx86_arm.bat"
nmake /f winarm.mk clean
nmake /f winarm.mk
### Windows Shared Library
Modify libyuv.gyp from 'static_library' to 'shared_library', and add 'LIBYUV_BUILDING_SHARED_LIBRARY' to 'defines'. Then run this.
gclient runhooks
After this command follow the building the library instructions above.
If you get a compile error for atlthunk.lib on Windows, read http://www.chromium.org/developers/how-tos/build-instructions-windows
### 64 bit Windows
set GYP_DEFINES=target_arch=x64
gclient runhooks V=1
### ARM Linux
export GYP_DEFINES="target_arch=arm"
export CROSSTOOL=`<path>`/arm-none-linux-gnueabi
export CXX=$CROSSTOOL-g++
export CC=$CROSSTOOL-gcc
export AR=$CROSSTOOL-ar
export AS=$CROSSTOOL-as
export RANLIB=$CROSSTOOL-ranlib
gclient runhooks
## Running Unittests
### Windows
out\Release\libyuv_unittest.exe --gtest_catch_exceptions=0 --gtest_filter="*"
### OSX
out/Release/libyuv_unittest --gtest_filter="*"
### Linux
out/Release/libyuv_unittest --gtest_filter="*"
Replace --gtest_filter="*" with specific unittest to run. May include wildcards. e.g.
out/Release/libyuv_unittest --gtest_filter=libyuvTest.I420ToARGB_Opt
## CPU Emulator tools
### Intel SDE (Software Development Emulator)
Pre-requisite: Install IntelSDE for Windows: http://software.intel.com/en-us/articles/intel-software-development-emulator
Then run:
c:\intelsde\sde -hsw -- out\release\libyuv_unittest.exe --gtest_filter=*
## Memory tools
### Running Dr Memory memcheck for Windows
Pre-requisite: Install Dr Memory for Windows and add it to your path: http://www.drmemory.org/docs/page_install_windows.html
set GYP_DEFINES=build_for_tool=drmemory target_arch=ia32
call python gyp_libyuv -fninja -G msvs_version=2013
ninja -C out\Debug
drmemory out\Debug\libyuv_unittest.exe --gtest_catch_exceptions=0 --gtest_filter=*
### Running UBSan
See Chromium instructions for sanitizers: https://www.chromium.org/developers/testing/undefinedbehaviorsanitizer
Sanitizers available: TSan, MSan, ASan, UBSan, LSan
GYP_DEFINES='ubsan=1' gclient runhooks
ninja -C out/Release
### Running Valgrind memcheck
Memory errors and race conditions can be found by running tests under special memory tools. [Valgrind] [1] is an instrumentation framework for building dynamic analysis tools. Various tests and profilers are built upon it to find memory handling errors and memory leaks, for instance.
[1]: http://valgrind.org
solutions = [
{ "name" : "libyuv",
"url" : "https://chromium.googlesource.com/libyuv/libyuv",
"deps_file" : "DEPS",
"managed" : True,
"custom_deps" : {
"libyuv/chromium/src/third_party/valgrind": "https://chromium.googlesource.com/chromium/deps/valgrind/binaries",
},
"safesync_url": "",
},
]
Then run:
GYP_DEFINES="clang=0 target_arch=x64 build_for_tool=memcheck" python gyp_libyuv
ninja -C out/Debug
valgrind out/Debug/libyuv_unittest
For more information, see http://www.chromium.org/developers/how-tos/using-valgrind
### Running Thread Sanitizer (TSan)
GYP_DEFINES="clang=0 target_arch=x64 build_for_tool=tsan" python gyp_libyuv
ninja -C out/Debug
valgrind out/Debug/libyuv_unittest
For more info, see http://www.chromium.org/developers/how-tos/using-valgrind/threadsanitizer
### Running Address Sanitizer (ASan)
GYP_DEFINES="clang=0 target_arch=x64 build_for_tool=asan" python gyp_libyuv
ninja -C out/Debug
valgrind out/Debug/libyuv_unittest
For more info, see http://dev.chromium.org/developers/testing/addresssanitizer
## Benchmarking
The unittests can be used to benchmark.
### Windows
set LIBYUV_WIDTH=1280
set LIBYUV_HEIGHT=720
set LIBYUV_REPEAT=999
set LIBYUV_FLAGS=-1
out\Release\libyuv_unittest.exe --gtest_filter=*I420ToARGB_Opt
### Linux and Mac
LIBYUV_WIDTH=1280 LIBYUV_HEIGHT=720 LIBYUV_REPEAT=1000 out/Release/libyuv_unittest --gtest_filter=*I420ToARGB_Opt
libyuvTest.I420ToARGB_Opt (547 ms)
Indicates 0.547 ms/frame for 1280 x 720.
## Making a change
gclient sync
git checkout -b mycl -t origin/master
git pull
<edit files>
git add -u
git commit -m "my change"
git cl lint
git cl try
git cl upload -r a-reviewer@chomium.org -s
<once approved..>
git cl land
# Introduction
Rotation by multiplies of 90 degrees allows mobile devices to rotate webcams from landscape to portrait. The higher level functions ConvertToI420 and ConvertToARGB allow rotation of any format. Optimized functionality is supported for I420, ARGB, NV12 and NV21.
# ConvertToI420
int ConvertToI420(const uint8* src_frame, size_t src_size,
uint8* dst_y, int dst_stride_y,
uint8* dst_u, int dst_stride_u,
uint8* dst_v, int dst_stride_v,
int crop_x, int crop_y,
int src_width, int src_height,
int crop_width, int crop_height,
enum RotationMode rotation,
uint32 format);
This function crops, converts, and rotates. You should think of it in that order.
* Crops the original image, which is src_width x src_height, to crop_width x crop_height. At this point the image is still not rotated.
* Converts the cropped region to I420. Supports inverted source for src_height negative.
* Rotates by 90, 180 or 270 degrees.
The buffer the caller provides should account for rotation. Be especially important to get stride of the destination correct.
e.g.
640 x 480 NV12 captured<br>
Crop to 640 x 360<br>
Rotate by 90 degrees to 360 x 640.<br>
Caller passes stride of 360 for Y and 360 / 2 for U and V.<br>
Caller passes crop_width of 640, crop_height of 360.<br>
# ConvertToARGB
int ConvertToARGB(const uint8* src_frame, size_t src_size,
uint8* dst_argb, int dst_stride_argb,
int crop_x, int crop_y,
int src_width, int src_height,
int crop_width, int crop_height,
enum RotationMode rotation,
uint32 format);
Same as I420, but implementation is less optimized - reads columns and writes rows, 16 bytes at a time.
# I420Rotate
int I420Rotate(const uint8* src_y, int src_stride_y,
const uint8* src_u, int src_stride_u,
const uint8* src_v, int src_stride_v,
uint8* dst_y, int dst_stride_y,
uint8* dst_u, int dst_stride_u,
uint8* dst_v, int dst_stride_v,
int src_width, int src_height, enum RotationMode mode);
Destination is rotated, so pass dst_stride_y etc that consider rotation.<br>
Rotate by 180 can be done in place, but 90 and 270 can not.
Implementation (Neon/SSE2) uses 8 x 8 block transpose, so best efficiency is with sizes and pointers that are aligned to 8.
Cropping can be achieved by adjusting the src_y/u/v pointers and src_width, src_height.
Lower level plane functions are provided, allowing other planar formats to be rotated. (e.g. I444)
For other planar YUV formats (I444, I422, I411, I400, NV16, NV24), the planar functions are exposed and can be called directly
// Rotate a plane by 0, 90, 180, or 270.
int RotatePlane(const uint8* src, int src_stride,
uint8* dst, int dst_stride,
int src_width, int src_height, enum RotationMode mode);
# ARGBRotate
LIBYUV_API
int ARGBRotate(const uint8* src_argb, int src_stride_argb,
uint8* dst_argb, int dst_stride_argb,
int src_width, int src_height, enum RotationMode mode);
Same as I420, but implementation is less optimized - reads columns and writes rows.
Rotate by 90, or any angle, can be achieved using ARGBAffine.
# Mirror - Horizontal Flip
Mirror functions for horizontally flipping an image, which can be useful for 'self view' of a webcam.
int I420Mirror(const uint8* src_y, int src_stride_y,
const uint8* src_u, int src_stride_u,
const uint8* src_v, int src_stride_v,
uint8* dst_y, int dst_stride_y,
uint8* dst_u, int dst_stride_u,
uint8* dst_v, int dst_stride_v,
int width, int height);
int ARGBMirror(const uint8* src_argb, int src_stride_argb,
uint8* dst_argb, int dst_stride_argb,
int width, int height);
Mirror functionality can also be achieved with the I420Scale and ARGBScale functions by passing negative width and/or height.
# Invert - Vertical Flip
Inverting can be achieved with almost any libyuv function by passing a negative source height.
I420Mirror and ARGBMirror can also be used to rotate by 180 degrees by passing a negative height.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment