Teeth represent the hardest tissue in vertebrates and appear very early in their evolution as an ancestral character of the Eugnathostomata (true jawed vertebrates). In recent vertebrates, two strategies to form and mineralize the outermost functional layer have persisted. In cartilaginous fish, the enameloid is of ectomesenchymal origin with fluoroapatite as the mineral phase. All other groups form enamel of ectodermal origin using hydroxyapatite as the mineral phase. The high abundance of teeth in the fossil record is ideal to compare structure and composition of teeth from extinct groups with those of their recent successors to elucidate possible evolutionary changes. Here, we studied the chemical composition and the microstructure of the teeth of six extinct shark species, two species of extinct marine reptiles and two dinosaur species using high-resolution chemical and microscopic methods. Although many of the ultrastructural features of fossilized teeth are similar to recent ones (especially for sharks where the ultrastructure basically did not change over millions of years), we found surprising differences in chemical composition. The tooth mineral of all extinct sharks was fluoroapatite in both dentin and enameloid, in sharp contrast to recent sharks where fluoroapatite is only found in enameloid. Unlike extinct sharks, recent sharks use hydroxyapatite as mineral in dentin. Most notably and hitherto unknown, all dinosaur and extinct marine reptile teeth contained fluoroapatite as mineral in dentin and enamel. Our results indicate a drastic change in the tooth mineralization strategy especially for terrestrial vertebrates that must have set in after the cretaceous period. Possibly, this is related to hitherto unconsidered environmental changes that caused unfavourable conditions for the use of fluoroapatite as tooth mineral.