We propose a novel robust stereo algorithm for weakly-textured scenes. Unique correspondences existing between the silhouettes of corresponding image segments allow assigning accurate disparities to segment boundary points. This information as well as stereo from the weak texture inside segments, which is extracted using a regionconstrained window-based matching algorithm, are fused and disparities are interpolated inside segments while considering potentially occluded areas derived from the depthordering of segments. The algorithm is applied to a set of weakly-textured images and it is demonstrated that stereo from segment silhouettes often provides sufficient information to reconstruct disparities in weakly- and non-textured image areas. The algorithm is applied to several real stereo images and its performance is evaluated quantitatively using images from the 2006 Middlebury dataset.