I have a board design like the following. I do not use USB, so the USB pins are connected to the GND. I know in the manual shows that they need to be pulled low if not using.
My question is that what will happen if they are tied to the GND directly - will this short and damage the processor? If they are left open, would it be fine?
You should follow what is there in the datasheet:
Table 25. ADSP-CM40xF Designer Quick …
Table 25. ADSP-CM40xF Designer Quick Reference
It says to pull down, if unused.