There are two different thresholds that determine which mode the MAX38886 operates in. The first is when the FBS pin is at 0.56V. In this case, the part has the DC-DC regulator draws power from the SYS pin to charge the supercapacitor. The next threshold is when the FBS pin drops to 0.5V. In this case, the SYS pin is regulated by the capacitor.
To figure out what voltage the system will be at when it is powered by the capacitor, you need to divide the two thresholds. 0.5/0.56 is 0.893. This means your backup voltage will be 0.893 times the minimum system voltage.
For example, if your system is at 5V when the system is powering the load, it will need to be 0.893 times lower (4.465V) to meet the threshold of having the capacitor power the load.