New layered oxides, Na₃M₂SbO₆ (M(II) = Mn, Fe) have been synthesized by solid state reactions under inert conditions. Rietveld refinements of the powder X-ray diffraction measurements confirmed the structures in the C2/m space group. The layered structure consists of honeycomb slabs with ordered M²⁺ (Mn or Fe) and Sb⁵⁺ cations, separated by Na⁺ ions in the interlayer octahedral sites. X-ray photoelectron spectroscopy measurements further substantiated the presence of Mn²⁺, Fe²⁺ and Sb⁵⁺ ions. A paramagnetic behavior has been exhibited by Na₃Mn₂SbO₆ in the range of 300–100 K with negative Weiss constant (θ = −144 K). Zero field cooled and field cooled values diverged below 50 K and without the indication of a long-range antiferromagnetic order down to 2 K. On the other hand, Na₃Fe₂SbO₆ has been found to display two different transitions in the magnetic susceptibility measurements. A distinct cusp appeared around T ∼ 120 K followed by the divergence between the zero field cooled and the field cooled values. The observation of antiferromagnetic ordering (T_N ∼ 7.5 K) suggested the magnetic frustration behavior arising out of the placement of Fe²⁺ ions in the triangular lattice of the honeycomb layers. Partial substitution of Mn²⁺ ions by Fe²⁺ and Co²⁺ ions has resulted in isostructural Na₃MnFeSbO₆ and Na₃MnCoSbO₆ under similar experimental conditions. Magnetic properties have been significantly modified by the coexistence of Mn²⁺ and Fe²⁺ ions in Na₃MnFeSbO₆. Co²⁺ substitution resulted in paramagnetic behavior with θ ∼ −8.4 K confirming thereby the competing antiferromagnetic interactions. The scope of the present work has been explored by carrying out the ion-exchange experiments of the sodium analogues using molten salt (LiNO₃) to synthesize new Li₃M₂SbO₆ (M = Mn, Fe) oxides.