Liquid crystal (LC) based geometric phase optical elements are widely used to effectively change the wavefront or propagation direction of light. Using photoalignment, the liquid crystal can be aligned according to the designed pattern, leading to components such as gratings, lenses or general wavefront shaping devices. The functionality and efficiency of the component is strongly influenced by how well the LC follows the imposed alignment pattern. Next to a considerable tilting of the LC at the air interface, we report on the observation of symmetry breaking in polymerized LC polarization gratings. By carefully analyzing the experimental and numerical data for gratings with different periods, we conclude that the non-negligible homeotropic anchoring strength at the air interface is responsible for the tilt angle and the symmetry breaking. The role of anchoring strength at the photoaligned and air interface and other parameters are investigated.