Bone marrow failure (BMF) is a rare but life-threatening disorder that usually manifests as (pan)cytopenia. BMF can be caused by a variety of diseases, but inherited BMF (IBMF) syndromes are a clinically important cause, especially in children. IBMF syndromes are a heterogeneous group of genetic disorders characterized by BMF, physical abnormalities, and predisposition to malignancy. An accurate diagnosis is critical, as disease-specific management, surveillance, and genetic counselling are required for each patient. The major differential diagnoses of IBMF syndromes are acquired aplastic anemia (AA) and refractory cytopenia of childhood (RCC). These diseases have overlapping features, such as BM hypocellularity and/or dysplastic changes, which make the differential diagnosis challenging. RCC has been defined as a histomorphologically distinct entity. Therefore, understanding the BM histopathology of these diseases is essential for the differential diagnosis. However, the BM histopathological features have not been characterized in detail, as descriptions of BM histopathology are very limited due to the rarity of the diseases. This review provides a detailed description of the BM histopathology in cases of RCC, AA, and the four most common IBMF syndromes: Fanconi anemia (FA), dysketatosis congenita (DC), Diamond-Blackfan anemia (DBA), and Shwachman-Diamond syndrome (SDS). An overview, including the clinical features and diagnosis, is also provided.