The development of mouse embryos can be partially recapitulated by combining embryonic (ES), trophoblast (TS) and extra-embryonic endoderm (XEN) stem cells to generate ETX-embryos. Although ETX-embryos transcriptionally capture the mouse gastrula, their ability to recapitulate complex morphogenic events such as gastrulation is limited, possibly due to the limited potential of XEN cells. To address this, we generated ES cells transiently expressing transcription factor Gata4 that drives the extra-embryonic endoderm fate and combined them together with ES cells and TS cells to generate induced ETX-embryos (iETX-embryos). We show that iETX-embryos establish a robust anterior signalling centre that migrates unilaterally to break embryo symmetry. Furthermore, iETX-embryos gastrulate generating embryonic and extra-embryonic mesoderm, and definitive endoderm. Our findings reveal that replacement of XEN cells with ES cells transiently expressing Gata4 endows iETX-embryos with greater developmental potential, thus enabling the study of the establishment of anterior-posterior patterning and gastrulation in an in vitro system.