With reference to https://ez.analog.com/docs/DOC-16454
From the AD9528 data sheet:
M1 = 3,4,5
Also it appears from the diagrams Figs 27 & 30 that the x 2 multiplier is in series with the 5 bit divider. it appears (Table 50) that this can be enabled with any value of R1 divider, If this is true then additional dividers possible are:
0.5, 1.5, ... 15.5
Finally, regarding the requirement
Maximum frequency after the RF VCO divider (M1) is limited to 1000MHz
I don't find any such limitation in the data sheet. I do see (Table 8)
Output Frequency 1000MHz or 1250MHz otuputs 1 and 2 only
However output is after channel divider. is it possible that the channel divider will work with input < 1250MHz
X 2 multiplier is taken care in the block diagram , i.e.. 0.5 ( X2 /4 (R1 divider) ) & additional dividers as mentioned above are not possible.
Yes you are correct that is captured in Table 8 . If you see Figure 27 , M1 divider has options 3,4,5 Divider and it cannot be bypassed. So input to channel divider will be VCO frequency /3 max and /5 min.