Please download and run AVES found at http://ez.analog.com/docs/DOC-1789. The program provides full control of the Blackfin on the eval. board and the ability to run scripts that are included with the install.
I just ran script "PortA,720p,1080i,1080p Any CSpaceIn In 24bit YCrCb422 from ADV761X thru HDMI...." for a UXGA60 input and don't see any image artifacts. This has a higher pixel rate then you specified in your format.
Can you attach a picture of the noise you are seeing?
The part has been tested up to 225MHz.