Thanks for the post. This appears to be a weak or noisy signal, as the jitter appears at the top of the waveform. It's pretty hard to tell without a schematic but you might calculate the expected signal amplitude with the input applied and the SNR/N of the system or converter. Also, 100MHz sampling is going to display a lot of bandwidth of the AD8336. Depending on the signal level and frequency, you'll likely see more response outside the 3dB BW of the AD8336, which is only 115MHz max, and less depending on factors listed in Table 1.
I think of jitter as a small time domain shift of the waveform, random or coherent in shift, and independent of signal level.
Forgot to offer an opinion as to what to do.
It all depends on what to you need. Your issue is basic SNR/R. I would do an error analysis and see if the performance under your input conditions can be improved. If not, you'll need to find more appropriate amplifiers (better sensitivity and/or lower noise), or somehow modify your requirements. The AD8336 is a very good part for general purpose applications but if it turns out you really need better SNR evaluate a really quiet low-noise preamp, or a VGA like the AD8331/2/4 which is the highest performance VGA of it's class we have.
Best of luck,
We use AD8336 because the input signal is positive and negative 5V, if AD8331 is not used, the input signal to be processed, in order to enter the AD8331？