Hello.

I have a question about calculating formula to convert full/limited range.

According to "Figure 38. CP Data Path Channel A (Y) for HDMI Mode" of UG-180 Rev.B Page.112,

it seems that the formula is "Input" x "Gain" + "Offset".

But according to actual measurement, in the case of "Case.2" it seems that the formula is ("Input" + "Offset") x "Gain".

e.g.

16 x 1.16 + (-16) = 2.56

(16 + (-16)) x 2.56 = 0 <-- this is correct, I think.

Is my thought right?

Case.1

Input Full range

Output Limited range

Gain 0.86

offset 16

Case.2

Input Limited range

Output Full range

Gain 1.16

offset -16

Best regards.

Kikka

Hi,

Here is the answer from chip designer.

Formula is : data_out = (((data_in - clamp_level) * gain) + offset)

Let me show you the calculation for the "case-2" that was mentioned in your mail.

Input = 16-235 (i.e. limited range)

Output = 0-255 (i.e. full range)

So, blank level of input (which is ultimately the clamp level) = 16

Blank level of output (which is ultimately the offset to be added) = 0

The gain factor for limited to full range conversion = 1.164

Similarly, the attenuation factor for full to limited range conversion = 1/1.164 = 0.859 (as mentioned in your mail also).

Assume code of 200 (decimal) in limited range.

So, after clamp, the value is (200-16) = 184

After gain = 184*1.164 = 214.1 = 214

After offset = 214+0 = 214.

So, a value of 200 in limited range will transform to 214 in full range.

Rgds,

vito