Please pardon my ignorance. CVBS is working, HDMI is not. I tried in two ways, the first one is using the driver from Cisco, that is known to be working (from the entity that provided me the edited driver to work with my embedded platform) with HDMI Port B, however I had to edit it for Port A, since that this is what I have in my platform.
I also tried using the evaluation board, connecting a HDMI cable to port A, waiting for the software to detect and configure the input, then dumped every register of the ADV and wrote it to the ADV in my custom board. That is not working as well, and the symptoms are the same.
1) TMDS locks properly every time.
2) HDCP checksum is correct, even if the part is a non keyed part. This only happens after running the Cisco driver for the first time, and forever then on. With a new board and by simply writing the evaluation board registers in the custom board ADV, I get a checksum error. (???).
3) DVI input is almost every time the detected input, even when the source is detecting a HDMI port. Sporadically, and not consistently (???), when using a 1080p 60fps source, I got the DVI bit changing to HDMI. This only happened with that resolution and I can't explain it.
IO, CP registers:
1) Vertical filter and DEregeneration filter were always false.
2) STDI for the two channels were always false as well.
I would really appreciate some suggestions on how to debug this further, I'm completely lost.