${\mathrm{Ni}}_2{\mathrm{Mo}}_3{\mathrm{O}}_8$ is a pyroelectric honeycomb antiferromagnet exhibiting peculiar changes in its electric polarization at magnetic transitions. ${\mathrm{Ni}}_2{\mathrm{Mo}}_3{\mathrm{O}}_8$ stands out from isostructural magnetic compounds, showing an anomalously low magnetic transition temperature and unique magnetic anisotropy. We determine the magnetic structure of ${\mathrm{Ni}}_2{\mathrm{Mo}}_3{\mathrm{O}}_8$ utilizing high-resolution powder and single-crystal neutron diffraction. A noncollinear stripy antiferromagnetic order is found in the honeycomb planes. The magnetic space group is $P_{C}na{2}_1$. The in-plane magnetic connection is of the stripy type for both the $ab$-plane and $c$-axis spin components. This is a simpler connection than the one proposed previously. The ferromagnetic interlayer order of the $c$-axis spin components in our model is also distinct. The magnetic anisotropy of ${\mathrm{Ni}}_2{\mathrm{Mo}}_3{\mathrm{O}}_8$ is characterized by orientation-dependent magnetic susceptibility measurements on a single crystal, consistent with neutron diffraction analysis. The local magnetoelectric tensor analysis using our magnetic models provides insights into its magnetoelectric coupling and polarization. Thus, our results deliver essential information for understanding both the unusual magnetoelectric properties of ${\mathrm{Ni}}_2{\mathrm{Mo}}_3{\mathrm{O}}_8$ and the prospects for observing exotic nonreciprocal, Hall, and magnonic effects characteristic of this compound family.

• Received 17 February 2023
• Revised 16 May 2023
• Accepted 13 July 2023

DOI:https://doi.org/10.1103/PhysRevResearch.5.033099

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society