[en] We study the rest-frame ultraviolet (UV) sizes of massive ($\sim <!--\; ???\; -->0.8\times <!--\; ??\; -->{10}^{11}{M}_{\odot <!--\; ???\; -->}$) galaxies at $3.4⩽<!--\; ???\; -->z<<!--\; <\; -->4.2$, selected from the FourStar Galaxy Evolution Survey, by fitting single Sérsic profiles to Hubble Space Telescope/WFC3/F160W images from the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey. Massive quiescent galaxies are very compact, with a median circularized half-light radius $r_{\mathrm{e}}=0.63\pm <!--\; ??\; -->0.18$ kpc. Removing $5/16$ (31%) sources with signs of active galactic nucleus activity does not change the result. Star-forming galaxies have $r_{\mathrm{e}}=2.0\pm <!--\; ??\; -->0.60$ kpc, $3.2\pm <!--\; ??\; -->1.3\times <!--\; ??\; -->$ larger than quiescent galaxies. Quiescent galaxies at $z\sim <!--\; ???\; -->4$ are on average $6.0\pm <!--\; ??\; -->1.7\times <!--\; ??\; -->$ smaller than at $z\sim <!--\; ???\; -->0$ and $1.9\pm <!--\; ??\; -->0.7\times <!--\; ??\; -->$ smaller than at $z\sim <!--\; ???\; -->2$. Star-forming galaxies of the same stellar mass are $2.4\pm <!--\; ??\; -->0.7\times <!--\; ??\; -->$ smaller than at $z\sim <!--\; ???\; -->0$. Overall, the size evolution at $0<<!--\; <\; -->z<<!--\; <\; -->4$ is well described by a power law, with $r_{\mathrm{e}}=5.08\pm <!--\; ??\; -->0.28{(1+z)}^{-<!--\; ???\; -->1.44\pm <!--\; ??\; -->0.08}$ kpc for quiescent galaxies and $r_{\mathrm{e}}=6.02\pm <!--\; ??\; -->0.28{(1+z)}^{-<!--\; ???\; -->0.72\pm <!--\; ??\; -->0.05}$ kpc for star-forming galaxies. Compact star-forming galaxies are rare in our sample: we find only $1/14$ (7%) with $r_{\mathrm{e}}/{(M/{10}^{11}{M}_{\odot <!--\; ???\; -->})}^{0.75}<<!--\; <\; -->1.5$, whereas $13/16$ (81%) of the quiescent galaxies are compact. The number density of compact quiescent galaxies at $z\sim <!--\; ???\; -->4$ is $1.8\pm <!--\; ??\; -->0.8\times <!--\; ??\; -->{10}^{-<!--\; ???\; -->5}{\mathrm{M}\mathrm{p}\mathrm{c}}^{-<!--\; ???\; -->3}$ and increases rapidly, by $><!--\; >\; -->5\times <!--\; ??\; -->$, between $2<<!--\; <\; -->z<<!--\; <\; -->4$. The paucity of compact star-forming galaxies at $z\sim <!--\; ???\; -->4$ and their large rest-frame UV median sizes suggest that the formation phase of compact cores is very short and/or highly dust obscured.