We propose a microscopic mechanism for the incommensurate phase in $\text{TiO}X$ compounds. The model includes the antiferromagnetic chains of Ti ions immersed in the phonon bath of the bilayer structure. Making use of the Cross-Fisher theory, we show that the geometrically frustrated character of the lattice is responsible for the structural instability which leads the chains to an incommensurate phase without an applied magnetic field. In the case of TiOCl, we show that our model is consistent with the measured phonon frequencies at $T=300\text{K}$ and the value of the incommensuration vector at the transition temperature. Moreover, we find that the dynamical structure factor shows a progressive softening of an incommensurate phonon near the zone boundary as the temperature decreases. This softening is accompanied by a broadening of the peak which gets asymmetrical as well when going toward the transition temperature. These features are in agreement with the experimental inelastic x-ray measurements.

• Received 18 March 2009

DOI:https://doi.org/10.1103/PhysRevB.79.134430