We propose a novel framework for reconstructing homogenous, transparent, refractive height-ﬁelds from a single viewpoint. The height-ﬁeld is imaged against a known planar background, or sequence of backgrounds. Unlike existing approaches that do a point-by-point reconstruction – which is known to have intractable ambiguities – our method estimates and optimizes for the entire height-ﬁeld at the same time. The formulation supports shape recovery from measured distortions (deﬂections) or directly from the images themselves, including from a single image. We report results for a variety of refractive height-ﬁelds showing signiﬁcant improvement over prior art.