I am using AD8302 in one of my application to detect the phase and gain.
what will be the typical/maximum error for gain and phase outputs ?
See datasheet graphs TPC4 through TPC 11 showing typical magnitude error. Turn a couple pages and you will see similar graphs for the phase error.
Error is only specified as typical. There is no max limit specified. You'll need to understand the statistical distribution of the error, comparing the sigma to your max error spec. -Bruce H.