Hi Spence,
Internal topology of 50ohm attenuator does not matter, if applied signal is with in frequency range of attenuator specs.
No doubt non-Zo impedance at input and output will add to attenuation value. To measure accurate attenuation, one can measure VSWR at input and output ports. Using VSWR you can calculate the power loss and add it to 20dB. This might be the actual loss shown by attenuator.