Several Rossby-type vortex solutions constructed for electromagnetic perturbations in magnetized plasma encounter the difficulty that the perturbed magnetic field and the parallel current are not continuous on the boundary between two regions. We find that fourth-order differential equations must be solved to remove this discontinuity. Special solutions for two types of boundary value problem for the fourth-order partial differential equations are presented. By applying these solutions to different nonlinear equations in magnetized plasma, the intrinsic electromagnetic solitary drift-Alfvén vortex (along with solitary Alfvén vortex) and the intrinsic electromagnetic solitary electron vortex (along with short-wavelength drift vortex) are constructed. While still keeping a localized dipole structure, these new vortices have more complicated radial structures in the inner and outer regions than the usual Rossby-wave vortex. The new type of vortex guarantees the continuity of the perturbed magnetic field δB⊥ and the parallel current j‖ on the boundary between inner and outer regions of the vortex. The allowed regions of propagation speeds for these vortices are analysed, and we find that the complementary relation between the vortex propagating speeds and the corresponding phase velocities of the linear modes no longer exists.