when the ADV7511 is configured like described in Table 13 of the Hardware User Guide (RGB or YCbCr 4:4:4 (12 bits) DDR with Separate Syncs: Input ID = 5, right aligned (R0x48 = 0).
How is determined if the input data is RGB or YCbCr?
The ADV7511 is just a parallel to serial converter. It doesn't care what the color format is. Now if you want to use the 7511 color space converter then you need to know the incoming color space so you can set up the converter correctly. Also you need to know the out going color space so you can set up the AVI infoframe correctly so the sink knows what the TMDS color space is.
Also RGB will always be in a 4:4:4 format while YCbCr can be either 4:4:4 or 4:2:2. Note both 4:4:4 and 4:2:2 can be sent down the TMDS lines and the AVI infoframe informs the sink what format to expect.
You have to program that. It's covered in the programming guide in section 4.3.
Retrieving data ...