The chapter explores a paradox – the apparent impossibility of decentralization in a country where orography, a long history of political fragmentation, and a vulnerable central state authority would appear naturally to favor decentralized authority. Through a historical exploration of governance in different parts of Yemen since 1960, it reveals longstanding demands for decentralized governance, as well as the existence of a strong society capable of formulating and pressuring the central authority to implement decentralization reforms. It then analyzes efforts from 2013 to 2015 to produce a new constitutional order built on federal principles – first through a broad-based National Dialogue and, then, through a far less inclusive constitutional drafting process. This quite revolutionary project of founding a federal Yemen with six regions was eventually buried under Gulf Cooperation Council coalition bombs from March 2015 onward. The chapter concludes by exploring the circumstances that led to the rejection of a federalist solution and the eruption of civil war in Yemen, but notes that federalism, far from being an imported concept, has generated rich intra-Yemeni intellectual debates. A six-region federal Yemen might not be the way forward, but a “federalism of the provinces” could be a path for future reconstruction of Yemen.