HDR笔记

发表于 | 分类于 | 评论数： | 阅读次数：
本文字数： 68k | 阅读时长 ≈ 1:02

Noun explanation

ITU：国际电信联盟 International Telecommunication Union
ITU-R：国际电信联盟无线电通信部门 ITU Radiocommunication Sector

CIE :国际照明协会 （英文：International Commission on Illumination ，法文：Commission internationale de l'éclairage ，采用法文缩写：CIE ）

SMPTE：电影电视工程师协会 Society of Motion Picture and Television Engineers

Hue：色调 色彩 色相
Chroma：色调饱和度 浓度
Luminance：亮度 明度

HDR Concept

HDR:High Dynamic Range

字面上是动态范围，一般指亮度上可以表达更大的亮度范围，呈现更大的亮度对比度。但是实际实际上HDR的技术和标准涉及色彩相关的一组属性的改善，可以带来更多的颜色、更大的亮度对比度、更高精度的量化。

OETF/EOTF: Optical-Electro/Electro-Optical Transfer Function 光电/电光转换函数

人对亮度的感知是非线性的，对暗部细节敏感，对亮部细节不敏感，利用这个特点设计了非线性的光电转换和电光转换的函数。这样的处理不仅可以节省带宽，也可以基本满足用户体验需求。
光电转换的时候做了特殊的非线性编码，为暗部细节分配更多的码率，亮部细节进行了压缩或者截断减少码率的分配。电光转换进行显示还原的时候，通过应用一个逆的非线性变化，还原出线性光。

Info

通过mediainfo 查看一个HDR视频信息

Video
ID 1
Format HEVC
Format/Info High Efficiency Video Coding
Format profile Main 10@L5@High
Codec ID hvc1
Codec ID/Info High Efficiency Video Coding
Duration 14 s 0 ms
Bit rate 123 kb/s
Width 3 840 pixels
Height 2 160 pixels
Display aspect ratio 16:9
Frame rate mode Constant
Frame rate 30.000 FPS
Color space YUV
Chroma subsampling 4:2:0
Bit depth 10 bits
Scan type Progressive
Bits/(PixelFrame)* 0.000
Stream size 211 KiB (98%)
Title Core Media Video
Encoded date UTC 2020-06-14 21:45:40
Tagged date UTC 2020-06-14 21:47:33
Color range Limited
Color primaries BT.2020
Transfer characteristics HLG
Matrix coefficients BT.2020 non-constant
Codec configuration box hvcC

下面几个参数属于元数据 ，可能没有 后文会讲到

Video
Mastering display color primaries Display P3 / R: x=0.677980 y=0.321980, G: x=0.245000 y=0.703000, B: x=0.137980 y=0.052000, White point: x=0.312680 y=0.328980
Mastering display luminance min: 0.0001 cd/m2, max: 1000 cd/m2
Maximum Content Light Level 1000 cd/m2
Maximum Frame-Average Light Level 400 cd/m2

重点关注 ：

  • Color range ：色彩范围
  • Color primaries ：色彩原色
  • Transfer characteristics ：传输特性
  • Matrix coefficients ：矩阵系数

元数据字段

  • Mastering display color primaries
  • Mastering display luminance
  • Maximum Content Light Level
  • Maximum Frame-Average Light Level

部分参数可以通过ffprobe查看对应的选项 ffprobe -h >> ffprobe.txt

Color range

色彩范围主要是两个：

  • Full range （PC range ）
  • Video range（limited range，tv range）

Full Range 就是我们所熟悉的 [0, 255]，而在 Limited Range 中，Y’ 的值被限制在 [16, 235]，Cb 和 Cr 的值被限制在 [16, 240] （针对8bit的）。

HDR一个重要属性就是量化精度。

SDR技术使用8bit进行颜色的表达，而HDR使用10bit/12bit进行颜色的表示，从而减少了8bit容易出现的人为条带效应。

ios 中的精度 & 色彩范围 ：

1
2
3
kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange = '420v', /* Bi-Planar Component Y'CbCr 8-bit 4:2:0, video-range (luma=[16,235] chroma=[16,240]).  baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */

kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange = 'x420', /* 2 plane YCbCr10 4:2:0, each 10 bits in the MSBs of 16bits, video-range (luma=[64,940] chroma=[64,960]) */
1
2
3
kCVPixelFormatType_420YpCbCr8BiPlanarFullRange  = '420f', /* Bi-Planar Component Y'CbCr 8-bit 4:2:0, full-range (luma=[0,255] chroma=[1,255]).  baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */ 

kCVPixelFormatType_420YpCbCr10BiPlanarFullRange  = 'xf20', /* 2 plane YCbCr10 4:2:0, each 10 bits in the MSBs of 16bits, full-range (Y range 0-1023) */

关于为什么要将YUV量化为tv range 16-235 ？

　　以下是维基百科摘抄的一段， 意思是tv range是为了解决滤波（模数转换）后的过冲现象，

Y′ values are conventionally shifted and scaled to the range [16, 235] (referred to as studio swing or “TV levels”) rather than using the full range of [0, 255] (referred to as full swing or “PC levels”). This practice was standardized in SMPTE-125M in order to accommodate signal overshoots (“ringing”) due to filtering. The value 235 accommodates a maximal black-to-white overshoot of 255 − 235 = 20, or 20 / (235 − 16) = 9.1%, which is slightly larger than the theoretical maximal overshoot (Gibbs phenomenon) of about 8.9% of the maximal step. The toe-room is smaller, allowing only 16 / 219 = 7.3% overshoot, which is less than the theoretical maximal overshoot of 8.9%. This is why 16 is added to Y′ and why the Y′ coefficients in the basic transform sum to 220 instead of 255.^[9]^ U and V values, which may be positive or negative, are summed with 128 to make them always positive, giving a studio range of 16–240 for U and V. (These ranges are important in video editing and production, since using the wrong range will result either in an image with “clipped” blacks and whites, or a low-contrast image.)

Color primaries

一般理解为色域，色域指可以显示的所有颜色的范围，常见的有Rec.709（全高清广播标准）、Rec.2020（4K/8K广播标准BT.2020）、Adobe RGB、P3等。

  1. bt709
  2. unknown
  3. reserved
  4. bt470m
  5. bt470bg
  6. smpte170m
  7. smpte240m
  8. film
  9. bt2020
  10. smpte428
  11. smpte431
  12. smpte432

下图显示了人眼能够感知的所有RGB值的范围。三角形表示色域：三角形越大，可以显示的颜色越多。

这张马蹄形的图上面可以看到HDR使用的色域是BT2020， SDR使用的是BT709，可以明显看到HDR的色域大于SDR。


Color Management for 3DCG | EIZO

理解颜色空间：

颜色空间 由 颜色模型 跟 色域 共同定义。
颜色模型的概念为：一种抽象数学模型，通过一组数字来描述颜色（例如RGB使用三元组、CMYK使用四元组）
例如Adobe RGB和sRGB都基于RGB颜色模型，但它们是两个不同的颜色空间，因为色域不一样

Color Transfer

描述光电转换过程的视频属性也叫颜色传输函数Color Transfer 就是上面表格里面的 Transfer characteristics

常见的hdr转换曲线为HLG和PQ，其中，smpte2084为PQ曲线（感知量化），arib-std-b67为HLG曲线（混合对数伽玛）

  1. bt709
  2. unknown
  3. reserved
  4. bt470m
  5. bt470bg
  6. smpte170m
  7. smpte240m
  8. linear
  9. log100
  10. log316
  11. iec61966-2-4
  12. bt1361e
  13. iec61966-2-1
  14. bt2020-10
  15. bt2020-12
  16. smpte2084
  17. smpte428
  18. arib-std-b67

传统的SDR视频使用的BT709的光电转换函数，对高亮部分进行了截断，可以表达的亮度动态范围有限，最大亮度只有100nit。而HDR视频，增加了高亮部分细节的表达，很大的扩展亮度的动态范围。

不同HDR的设计初衷不同，其中PQ的设计更接近人眼的特点，亮度表达更准确，可以表示高达10000nit的亮度。而HLG的设计考虑了老设备的兼容性，和传统bt709的传输函数有部分是重合的，天然的对老设备具有一定兼容性

Metadata

HDR元数据分为两种，静态元数据 和动态元数据 ；

使用PQ曲线的HDR10是采用静态元数据的，但是杜比公司提出来的杜比视界和三星的HDR10+，尽管使用了PQ曲线，但是他们使用的是动态元数据，HLG没有元数据。

其中 DolbyVision 等价于SMPTE ST 2094-10, HDR10+ 等价于 SMPTE ST 2094-40

静态元数据规定了整个片子像素级别最大亮度上限，在ST 2086中有标准化的定义。静态元数据的缺点是必须做全局的色调映射，没有足够的调节空间，兼容性不好。

动态元数据可以很好地解决这个问题。动态元数据主要有两个方面的作用：与静态元数据相比，它可以在每一个场景或者每一帧画面，给调色师一个发挥的空间，以展现更丰富的细节；另一个方面，通过动态元数据，在目标显示亮度上做色调映射，可以最大程度在目标显示器上呈现作者的创作意图。

  • 最大内容亮度（MaxCLL）：整个视频流中最亮像素的亮度。
  • 最大帧平均亮度（MaxFALL）：整个视频流中最亮帧的平均亮度

另外解释一下多出现的几个参数:progressive,SAR,DAR.

progressive,其实就是扫描方式,逐行扫描.另外的一种方式就是隔行扫描:interlaced.我们平时所谓的1080p,这个p就是progressive,表示的是1080尺寸的逐行扫描视频.
DAR - display aspect ratio就是视频播放时，我们看到的图像宽高的比例，缩放视频也要按这个比例来，否则会使图像看起来被压扁或者拉长了似的。

SAR - storage aspect ratio就是对图像采集时，横向采集与纵向采集构成的点阵，横向点数与纵向点数的比值。比如VGA图像640/480 = 4:3，D-1 PAL图像720/576 = 5:4

PAR - pixel aspect ratio大多数情况为1:1,就是一个正方形像素，否则为长方形像素这三者的关系PAR x SAR = DAR或者PAR = DAR/SAR

Tone Mapping

色调映射的目的是使高动态范围HDR图像能够适应低动态范围LDR显示器。

色调映射算法的目的在于将HDR图像的亮度进行压缩，进而映射到LDR显示设备的显示范围之内，同时，在映射的过程中要尽量保持原HDR图像的细节与颜色等重要信息。

所以色调映射算法需要具有两方面的性质：

  1. 能够将图像亮度进行压缩。
  2. 能够保持图像细节与颜色。

EOTF/OETF

HLG

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
float ARIB_B67_A = 0.17883277;
float ARIB_B67_B = 0.28466892;
float ARIB_B67_C = 0.55991073;
highp float arib_b67_inverse_oetf(highp float x)
{
    x = max(x, 0.0);
    if (x <= (1.0/2.0))
        x = (x * x) * (1.0 / 3.0);
    else
        x = (exp((x - ARIB_B67_C) / ARIB_B67_A) + ARIB_B67_B) / 12.0;
    return x;
}
highp float arib_b67_ootf(highp float x)
{
    return x < 0.0 ? x : pow(x, 1.2);
}
highp float arib_b67_eotf(highp float x)
{
    return arib_b67_ootf(arib_b67_inverse_oetf(x));
}
highp float arib_b67_oetf(highp float x)
{
    x = max(x, 0.0);
    if (x <= (1.0 / 12.0))
        x = sqrt(3.0 * x);
    else
        x = ARIB_B67_A * log(12.0 * x - ARIB_B67_B) + ARIB_B67_C;
    return x;
}

PQ

1
2
3
4
5
6
7
8
9
10
11
12
13
highp float ST2084_M1 = 0.1593017578125;
const float ST2084_M2 = 78.84375;
const float ST2084_C1 = 0.8359375;
const float ST2084_C2 = 18.8515625;
const float ST2084_C3 = 18.6875;
highp float FLT_MIN = 1.17549435082228750797e-38;
highp float st_2084_eotf(highp float x)
{
    highp float xpow = pow(x, float(1.0 / ST2084_M2));
    highp float num = max(xpow - ST2084_C1, 0.0);
    highp float den = max(ST2084_C2 - ST2084_C3 * xpow, FLT_MIN);
    return pow(num/den, 1.0 / ST2084_M1);
}

BT.709

1
2
3
4
5
6
7
8
9
10
11
const float REC709_ALPHA = 1.09929682680944;
const float REC709_BETA = 0.018053968510807;
highp float rec_709_oetf(highp float x)
{
    x = max(x, 0.0);
    if (x < REC709_BETA )
        x = x * 4.5;
    else
        x = REC709_ALPHA * pow(x, 0.45) - (REC709_ALPHA - 1.0);
    return x;
}

bt2020 -> bt709

1
2
3
1.6605, -0.5876, -0.0728
-0.1246,  1.1329, -0.0083
-0.0182, -0.1006,  1.1187

bt709 -> bt2020

1
2
3
0.6274, 0.3293, 0.0433
0.0691, 0.9195, 0.0114
0.0164, 0.0880, 0.8956

Matrix coefficients

YCbCr->RGB

  • Video Range
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// BT.601, which is the standard for SDTV.
static const GLfloat kColorConversion601[] = {
        1.164,  1.164, 1.164,
          0.0, -0.392, 2.017,
        1.596, -0.813,   0.0,
};

// BT.709, which is the standard for HDTV.
static const GLfloat kColorConversion709[] = {
        1.164,  1.164, 1.164,
          0.0, -0.213, 2.112,
        1.793, -0.533,   0.0,
};

// BT.2020 (which is the standard for UHDTV, ITU_R_2020 )
static const GLfloat kColorConversion2020[] = {
        1.1644f, 1.1644f, 1.1644f,
         0.0f, -0.1881, 2.1501,
        1.6853, -0.6529, 0.0f,
};
  • Full Range
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// BT.601 full range 
static const GLfloat kColorConversion601FullRange[] = {
1.0,    1.0,    1.0,
0.0,    -0.343, 1.765,
1.4,    -0.711, 0.0,
};

// BT.709 full range 
static const GLfloat kColorConversion709FullRange[] = {
1.0,    1.0,    1.0,
0.0, -0.187,  1.855,
1.574, -0.468,    0.0,
};

// BT.2020 full range 
static const GLfloat kColorConversion2020FullRange[] = {
1.0,    1.0,    1.0,
0.0, -0.1645,   1.8814,
1.4746, -0.5713,   0.0,
};

OpenGL

  • Video Range
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
precision mediump float;
 varying mediump vec2 textureCoordinate;
 
 uniform sampler2D luminanceTexture;
 uniform sampler2D chrominanceTexture;
 uniform highp mat3 colorConversionMatrix;

 void main()
 {
     mediump vec3 yuv;
     highp vec3 rgb;
   
     yuv.x = texture2D(luminanceTexture, textureCoordinate).r - (16.0/255.0);
     yuv.yz = texture2D(chrominanceTexture, textureCoordinate).ra - vec2(0.5, 0.5);
     rgb = colorConversionMatrix * yuv;
     gl_FragColor = vec4(rgb, 1.);
 }
  • Full Range
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
precision mediump float;
 varying mediump vec2 textureCoordinate;
 uniform sampler2D luminanceTexture;
 uniform sampler2D chrominanceTexture;
 uniform highp mat3 colorConversionMatrix;

 void main()
 {
     mediump vec3 yuv;
     highp vec3 rgb;
   
     yuv.x = texture2D(luminanceTexture, textureCoordinate).r;
     yuv.yz = texture2D(chrominanceTexture, textureCoordinate).ra - vec2(0.5, 0.5);
     rgb = colorConversionMatrix * yuv;
     gl_FragColor = vec4(rgb, 1.);
 }

RGB-YUV

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
// FULL RANGE
    static void bt2020_rgb2yuv10bit_PC(uint8_t R, uint8_t G, uint8_t B, uint16_t &Y, uint16_t &U, uint16_t &V)
    {
        Y =  0.2627 * R + 0.6780 * G + 0.0593 * B;
        U = -0.1396 * R - 0.3604 * G + 0.5000 * B + 128;
        V =  0.5000 * R - 0.4598 * G - 0.0402 * B + 128;
        Y = Y << 8;
        U = U << 8;
        V = V << 8;
    }
  
    // VIDEO RANGE
    static void bt2020_rgb2yuv10bit_TV(uint8_t R, uint8_t G, uint8_t B, uint16_t &Y, uint16_t &U, uint16_t &V)
    {
        Y =  0.2256 * R + 0.5823 * G + 0.0509 * B + 16;
        U = -0.1222 * R - 0.3154 * G + 0.4375 * B + 128;
        V =  0.4375 * R - 0.4023 * G - 0.0352 * B + 128;
    
        /// 8 分解为2 + 6
        /// <<2 ：对应 把 yuv 从 (luma=[16,235] chroma=[16,240]) 拉到 (luma=[64,940] chroma=[64,960])
        /// <<6 ：10bit 按字节为单位需要两个字节，因为按照大端模式存储（低地址到高地址的顺序存放数据的高位字节到低位字节）后面6个bit是padding 补0
  
        Y = Y << 8;
        U = U << 8;
        V = V << 8;
    }

    // FULL RANGE
    static void bt709_rgb2yuv10bit_PC(uint8_t R, uint8_t G, uint8_t B, uint16_t &Y, uint16_t &U, uint16_t &V)
    {
        Y =  0.2126 * R + 0.7152 * G + 0.0722 * B;
        U = -0.1146 * R - 0.3854 * G + 0.5000 * B + 128;
        V =  0.5000 * R - 0.4542 * G - 0.0458 * B + 128;
        Y = Y << 8;
        U = U << 8;
        V = V << 8;
    }

    // VIDEO RANGE
    static void bt709_rgb2yuv10bit_TV(uint8_t R, uint8_t G, uint8_t B, uint16_t &Y, uint16_t &U, uint16_t &V)
    {
        Y =  0.183 * R + 0.614 * G + 0.062 * B + 16;
        U = -0.101 * R - 0.339 * G + 0.439 * B + 128;
        V =  0.439 * R - 0.399 * G - 0.040 * B + 128;
        Y = Y << 8;
        U = U << 8;
        V = V << 8;
    }

    // FULL RANGE
    static void bt601_rgb2yuv10bit_PC(uint8_t R, uint8_t G, uint8_t B, uint16_t &Y, uint16_t &U, uint16_t &V)
    {
        Y =  0.299 * R + 0.587 * G + 0.114 * B;
        U = -0.169 * R - 0.331 * G + 0.500 * B + 128;
        V =  0.500 * R - 0.419 * G - 0.081 * B + 128;
        Y = Y << 8;
        U = U << 8;
        V = V << 8;
    }

    // VIDEO RANGE
    static void bt601_rgb2yuv10bit_TV(uint8_t R, uint8_t G, uint8_t B, uint16_t &Y, uint16_t &U, uint16_t &V)
    {
        Y =  0.257 * R + 0.504 * G + 0.098 * B + 16;
        U = -0.148 * R - 0.291 * G + 0.439 * B + 128;
        V =  0.439 * R - 0.368 * G - 0.071 * B + 128;
    
        Y = Y << 8;
        U = U << 8;
        V = V << 8;
    }

    // FULL RANGE
    static void bt601_rgb2yuv_PC(uint8_t R, uint8_t G, uint8_t B, uint8_t &Y, uint8_t &U, uint8_t &V)
    {
        Y =  0.299 * R + 0.587 * G + 0.114 * B;
        U = -0.169 * R - 0.331 * G + 0.500 * B + 128;
        V =  0.500 * R - 0.419 * G - 0.081 * B + 128;
    }

    // VIDEO RANGE
    static void bt601_rgb2yuv_TV(uint8_t R, uint8_t G, uint8_t B, uint8_t &Y, uint8_t &U, uint8_t &V)
    {
        Y =  0.257 * R + 0.504 * G + 0.098 * B + 16;
        U = -0.148 * R - 0.291 * G + 0.439 * B + 128;
        V =  0.439 * R - 0.368 * G - 0.071 * B + 128;
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
inline CFDictionaryRef HDRAttachmentsOfMedium(void)
    {
        const size_t attributes_size = 6;
        CFTypeRef keys[attributes_size] = {
            kCVImageBufferFieldCountKey,
            kCVImageBufferChromaLocationBottomFieldKey,
            kCVImageBufferChromaLocationTopFieldKey,
            kCVImageBufferTransferFunctionKey,
            kCVImageBufferColorPrimariesKey,
            kCVImageBufferYCbCrMatrixKey
        };
    
        CFTypeRef values[attributes_size] = {
            kCFBooleanTrue,
            kCVImageBufferChromaLocation_Left,
            kCVImageBufferChromaLocation_Left,
            kCVImageBufferTransferFunction_ITU_R_2100_HLG,
            kCVImageBufferColorPrimaries_ITU_R_2020,
            kCVImageBufferYCbCrMatrix_ITU_R_2020
        };
        return CreateCFDictionary(keys, values, attributes_size);
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
CVPixelBufferRef RGB2YCbCr10Bit(CVPixelBufferRef pixelBuffer, CFDictionaryRef dic)
    {
        CVPixelBufferLockBaseAddress(pixelBuffer, 0);
        uint8_t *baseAddress = (uint8_t *)CVPixelBufferGetBaseAddress(pixelBuffer);
        int w = (int) CVPixelBufferGetWidth(pixelBuffer);
        int h = (int) CVPixelBufferGetHeight(pixelBuffer);
        //int stride = (int) CVPixelBufferGetBytesPerRow(pixelBuffer) / 4;
    
        OSType pixelFormat = kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange;
        CVPixelBufferRef pixelBufferCopy = NULL;
        const size_t attributes_size = 5;
        CFTypeRef keys[attributes_size] = {
            kCVPixelBufferIOSurfacePropertiesKey,
            kCVPixelBufferExtendedPixelsBottomKey,
            kCVPixelBufferExtendedPixelsTopKey,
            kCVPixelBufferExtendedPixelsRightKey,
            kCVPixelBufferExtendedPixelsLeftKey
        };
        CFDictionaryRef io_surface_value = vtc::CreateCFDictionary(nullptr, nullptr, 0);
        CFTypeRef values[attributes_size] = {io_surface_value, kCFBooleanFalse, kCFBooleanFalse, kCFBooleanFalse, kCFBooleanFalse};
    
        CFDictionaryRef attributes = vtc::CreateCFDictionary(keys, values, attributes_size);
        CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault,
                                              w,
                                              h,
                                              pixelFormat,
                                              attributes,
                                              &pixelBufferCopy);
        if (status != kCVReturnSuccess) {
            std::cout << "YUVBufferCopyWithPixelBuffer :: failed" << std::endl;
            return nullptr;
        }
        if (attributes) {
            CFRelease(attributes);
            attributes = nullptr;
        }
    
        int plane_h1 = (int) CVPixelBufferGetHeightOfPlane(pixelBufferCopy, 0);
        int plane_h2 = (int) CVPixelBufferGetHeightOfPlane(pixelBufferCopy, 1);
    
        CVPixelBufferLockBaseAddress(pixelBufferCopy, 0);
        if (dic == nullptr || CFDictionaryGetCount(dic) <= 0) {
            dic = HDRAttachmentsOfMedium();
        }
        CVBufferSetAttachments(pixelBufferCopy, dic, kCVAttachmentMode_ShouldPropagate);
    
        size_t y_stride = CVPixelBufferGetBytesPerRowOfPlane(pixelBufferCopy, 0);
        //size_t uv_stride  = CVPixelBufferGetBytesPerRowOfPlane(pixelBufferCopy, 1);
    
        unsigned long y_bufferSize = w * h;
        unsigned long uv_bufferSize = w * h / 4;
        uint16_t *y_planeData = (uint16_t *) malloc(y_bufferSize * sizeof(uint16_t));
        uint16_t *u_planeData = (uint16_t *) malloc(uv_bufferSize * sizeof(uint16_t));
        uint16_t *v_planeData = (uint16_t *) malloc(uv_bufferSize * sizeof(uint16_t));
        uint16_t *y = (uint16_t *) CVPixelBufferGetBaseAddressOfPlane(pixelBufferCopy, 0);
        uint16_t *uv = (uint16_t *) CVPixelBufferGetBaseAddressOfPlane(pixelBufferCopy, 1);
        int u_offset = 0;
        int v_offset = 0;
        uint8_t R, G, B;
        uint16_t Y, U, V;
    
        for (int i = 0; i < h; i ++) {
            for (int j = 0; j < w; j ++) {
                int offset = i * w + j;
                B = baseAddress[offset * 4];
                G = baseAddress[offset * 4 + 1];
                R = baseAddress[offset * 4 + 2];
                bt2020_rgb2yuv10bit_TV(R, G, B, Y, U, V);
                y_planeData[offset] = Y;
                //隔行扫描 偶数行的偶数列取U 奇数行的偶数列取V
                if (j % 2 == 0) {
                    (i % 2 == 0) ? u_planeData[++u_offset] = U : v_planeData[++v_offset] = V;
                }
            }
        }
    
        for (int i = 0; i < plane_h1; i ++) {
            memcpy(y + i * y_stride / 2, y_planeData + i * w, 2 * w);
            if (i < plane_h2) {
                for (int j = 0 ; j < w ; j ++) {
                    //NV12 和 NV21 格式都属于 YUV420SP 类型。它也是先存储了 Y 分量，但接下来并不是再存储所有的 U 或者 V 分量，而是把 UV 分量交替连续存储。
                    //NV12 是 IOS 中有的模式，它的存储顺序是先存 Y 分量，再 UV 进行交替存储。
                    memcpy(uv + i * y_stride / 2 + 2*j, u_planeData + i * w/2 + j, 2);
                    memcpy(uv + i * y_stride / 2 + 2*j + 1, v_planeData + i * w/2 + j, 2);
                }
            }
        }
        free(y_planeData);
        free(u_planeData);
        free(v_planeData);
    
        CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
        CVPixelBufferUnlockBaseAddress(pixelBufferCopy, 0);
        return pixelBufferCopy;
    }

Formula

BT.601 BT.709 BT.2020
a 0.299 0.2126 0.2627
b 0.587 0.7152 0.6780
c 0.114 0.0722 0.0593
d 1.772 1.8556 1.8814
e 1.402 1.5748 1.4747 
1
2
3
Y  = a * R + b * G + c * B
Cb = (B - Y) / d
Cr = (R - Y) / e
1
2
3
R = Y + e * Cr
G = Y - (a * e / b) * Cr - (c * d / b) * Cb
B = Y + d * Cb
1
2
3
a+b+c = 1
e = 2 * (1 - a)
d = 2* (a + b)

Range Deduce

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[16/255,    16/255,    16/255, 1.0]
[235/255,   240/255,  240/255, 1.0]

x

[255/219,    0,          0,          0]
[0,          255/224,    0,          0]
[0,          0,          255/224,    0]
[-16/219,    -128/224,   -128/224,   1]

=

[0,    -0.5,    -0.5, 1.0]
[1,     0.5,     0.5, 1.0]

将 videorange 通过 齐次矩阵 转换为 fullrange

Other

大部分图像捕捉设备在保存图像时会自动加上伽马校正，也就是说图像中存储的是非线性空间中的颜色

非线性的RGB转换为YUV也是非线性

OpenGL 无法直接对 10bit YUV 进行处理，需要先转换为 8bit YUV

tone mapping 需要在线性RGB空间进行

References

搞清楚编程中YUV和RGB间的相互转换
YUV - RGB colorconversion
推导视频YUV转RGB矩阵
齐次坐标
# Gamma校正
# 我理解的伽马校正
Colour gamut conversion from Recommendation ITU-R BT.2020 to Recommendation ITU-R BT.709
Colour conversion from Recommendation ITU-R BT.709 to Recommendation ITU-R BT.2020
HDR转SDR实践之旅

相关文章