LDOs are not designed to be used as voltage references. Typically, the output accuracy will not meet system-level specifications, especially across temperature. Voltage references are generally trimmed for accuracy and temperature coefficient.
LDO noise levels may also not meet system-level requirements. Power supply rejection may also be a deficiency of an LDO.
Additionally, many ADCs require the reference to both source and sink current. LDOs usually source current, but do not sink current. Some references, especially more recent ADI offerings, have integrated buffer amplifiers. In all cases, it is necessary to determine if an additional buffer is required.