The rise of graphene marks the advent of two-dimensional atomic crystals, which have exhibited a cornucopia of intriguing properties, such as the integer and fractional quantum Hall effects, valley Hall effect, charge density waves, and superconductivity, to name a few. Yet, magnetism, a property of extreme importance in both science and technology, remains elusive. There is a paramount need for magnetic two-dimensional crystals. With the availability of many magnetic materials consisting of van der Waals coupled two-dimensional layers, it thus boils down to the question of how the magnetic order will evolve with reducing thickness. Here we investigate the effect of thickness on the magnetic ordering in nanothick ${\mathrm{V}}_5{\mathrm{S}}_8$. We uncover an anomalous Hall effect, by which the magnetic ordering in ${\mathrm{V}}_5{\mathrm{S}}_8$ down to 3.2 nm is probed. With decreasing thickness, a breakdown of antiferromagnetism is evident, followed by a spin-glass-like state. For thinnest samples, a weak ferromagnetic ordering emerges. The results not only show an interesting effect of reducing thickness on the magnetic ordering in a potential candidate for magnetic two-dimensional crystals, but demonstrate the anomalous Hall effect as a useful characterization tool for magnetic orderings in two-dimensional systems.

• Received 18 February 2017

DOI:https://doi.org/10.1103/PhysRevB.96.075402