Efficient production of coherent harmonic radiation from solid targets relies critically on the formation of smooth, short density scalelength plasmas. Recent experimental results (Dromey et al 2009 Nat. Phys. 5 146) suggest, however, that the target roughness on the scale of the emitted harmonic wavelength does not result in diffuse reflection-in apparent contradiction to the Rayleigh criterion for coherent reflection. In this paper we show, for the first time, using analytic theory and 2D PIC simulations, that the interaction of relativistically strong laser pulses with corrugated target surfaces results in a highly effective smoothing of the interaction surface and consequently the generation of highly collimated and temporally confined XUV pulses from rough targets, in excellent agreement with experimental observations.