AD8302 Phase Detector

Functionally it is an XOR, but it is done with an analog circuit, hence the language about the multiplier.  Raw phase detector output is essentially a square wave that gets filtered/averaged in the VPHS output stage.  

The error at 0 and 180 degrees is a function of the jitter/phase noise of the detector.  The phase detector can only resolve phase differences that are bigger than its own noise floor; in this case bigger than its own jitter/phase noise.You can imagine that as the edges of the two signals that are being compared by the phase detector (remember they are square waves now that they have gone through the limiting amp chain), get closer together in phase, the uncertainty increases.  Thus the large of amount of error at 0 & 180, where the signal are completely in-phase & out-of-phase with each other.  

Thank you for your reply. It is more clear now. Does the phase detector use XNOR gate? Because when phase different is 0 degree, the VPHS output will be 1.8V and 0V in case of 180 degree.

I wouldn't worry about what the actual phase detector is (XOR, XNOR, etc.) because there are inversions after the phase detector in the output stage.  If it helps to think of the whole thing as an XNOR, i.e. from INPA/INPB to VPHS output, then go ahead and do that.