1. The filter (page 41 of AD9789) mask is related to J.83, Annex C; the mask for the J.83, Annex B is not published in the standard and I am curious how they did this filter.

How many taps filter has and (if possible!) to get coefficients values.

2.On the page 42 AD9789, in the formula (1): fDAC = I x (P/Q) x 16 x fBAUD, how they get 16?

For Q1, I think you could use an analysis program such as Matlab to generate a root raised cosine filter with the J.83 spec as a guideline for performance requirements. There isn't anything special about the filter.

For Q2, the 16 is for the mandatory 16x interpolation filter that comes at the end of the digital data path. See the "Digital 16x Tunable Band-Pass Filter" section on page 44.