ADV8005 TTL Output Jitter

I'm using the TTL outputs on the ADV8005 to drive an SDI transmitter at 1080p and I'm seeing a lot of jitter on the PCLK output.  I've been using my computer background as a test image and I've been measuring almost 3ns of jitter using this background.  Interestingly, if I use an all white test image, or even better let the HDMI receiver free run, the jitter decreases significantly to around 500ps.  In both cases, the test image is also routed to the DVI and is stable with no issues.

I've checked my 1.8V power supply for switching noise and it doesn't appear to be the culprit to me.  I've attached a document with screenshots of both the PCLK jitter and the 1.8V ripple and switching noise.

Besides the 1.8V power supply, what could cause sever jitter on the TTL outputs while leaving the DVI output acceptable?