As a general rule the smaller the error size (i.e .tolerance), the longer physical comparison will take.
The error size is a variable, because there are many unique reasons for performing a physical compare. There isn't any specific standard for comparison.
If checking between PCB design versions, probably setting to 4 mils tolerance would be fine. Typically in these cases, the designer is looking for the changes between layouts (which is typically significant - like adding a different component., re-routing traces, etc.).