We investigate the effect of higher-order interactions induced by shape-dependent confinement in the modulational instability (MI) of a binary mixture of Bose-Einstein condensates. For this, we present and compute both analytically and numerically a system of coupled Gross-Pitaevskii equations with residual nonlinearity that rule the dynamics of the mixture. Using the linear stability approach, we obtain the instability criteria of the mixture and find that the MI can be excited in miscible condensates and altered in immiscible condensates due to the effect of residual nonlinearity. Direct numerical calculations are performed to support the analytical predictions, and a good agreement is found. The space-time evolution of the condensate density is displayed in both cases when the mixture is miscible and immiscible, showing the generation of bright solitons for modes predicted to be unstable.

• Received 22 November 2018
• Revised 23 January 2019

DOI:https://doi.org/10.1103/PhysRevE.99.032202