Fatigue crack in cast aluminum alloys subject to low-cycle fatigue is initiated mainly by the fracture of Si particles. To investigate the mechanism of Si particle fracture, we carried out the geometrical measurements and finite element analysis for a lot of Si particles in their actual geometrical configuration. The synchrotron X-ray CT was used for the geometrical measurement and the image-based finite element modeling. The fracture of Si particles were detected as a source of crack initiation from the CT images captured during the fatigue test. The large-scale voxel finite element model contains the outer surface, pores, Si and intermetallics. The number of elements was about 150 million. Two cycles of loading were calculated. The finite element analysis and its post-processing were performed on the supercomputers by the massively-parallel computing. The result of finite element analysis showed that the first principal stress was concentrated around the aligned or gathered Si particles, and the gradual increase by cyclic loading in the stress of the Si particles appeared near the outer surface and pores. Moreover, the result of geometrical measurements showed that the volume, shape index and adjacency index of each Si particle could be used to evaluate the likelihood of particle fracture. The probability of particle fracture was statistically evaluated with respect to those geometrical and mechanical parameters. The relationship between mechanical and geometrical parameters were also examined. As the result, the criteria of fractured Si particle was proposed by using the combination of geometrical and mechanical parameters.