The FullCapRep register should be within ~2-5% of the true battery capacity if a customer battery model is used. If your design has a bad current sensing layout, this will increase the error in the capacity. Also, if there is current entering or leaving your battery that is not tracked by the MAX17201, it will also increase this error.