The clearance spec is how deep the flexplate tooth extends in to the starter motor pinion. The bare minimum need to be 0.040 inch. If the clearance is too tight, a shim will boost this clearance. As you stated, a shim did not aid so We'll think the clearance is abnormal—which we will think