I breadboarded a simple circuit to check the functionality of using the AD5206 in rheostat mode to simulate RTDs for a battery management system. I find that the resistor setting is incorrect around 1/10 of the time when the same instruction is given.
I believe few things are clearly missing here, and consequently results are not as expected.
a digipot is not more than resistors connected by switched, you can check some details here,
The main problem specially when measuring with a DMM is the current sink/source by the internal switches, this generates a error in teh instrument as the DMM makes the calculations based on the current injected... but if this current is different, the calculation are as well.
In addition, there are two things that makes things worst in your system, you are writing the device continuously, which means that the device is all time updating, and generating even worst specs in terms of current + your board is "floating"... is your GND connections good enough?
As an intermediate solution, could you connect one of the digipot terminals to VDD, and using the DMM in current mode, connect the DMM between the second digipot terminal and GND.
Is the value more constant?
What are the value displayed in the DMM?