Phonon transport is an essential property of thermoelectric materials. Although the phonon carries heat, which reduces the thermoelectric efficiency, it contributes positively to the Seebeck coefficient S through the phonon-drag effect, as typified by the high-purity semiconductors, which show fairly large S at cryogenic temperatures. Although such a large S is attractive in terms of Peltier cooling, a clear guiding principle for designing thermoelectric materials enriched by the phonon-drag effect remains to be established. Here we demonstrate that a correlated semiconductor, FeSb2, is a promising thermoelectric material featuring quasi-ballistic phonons dragging d electrons with large effective mass. By changing the sample size within the sub-millimetre order for high-purity single crystals, we succeed in substantially increasing S to as much as −27 mV K−1 at low temperatures. Our results exemplify a strategy for exploring phonon-drag-based thermoelectric materials, the performance of which can be maximized by combining heavy electrons with ballistic phonons.