While the advantages of nanomaterials are being increasingly recognized, their potential toxicity is drawing more and more attention and concern. In this study, we explore the toxicity mechanism of 20–30 nm rod-shaped hydroxyapatite (HA) nanoparticles in vitro and in vivo. The nanoparticles were prepared by precipitation and characterized by IR, XRD and TEM. Concentrations of 0 μg mL⁻¹, 10 μg mL⁻¹, 100 μg mL⁻¹, 1 mg mL⁻¹, and 10 mg mL⁻¹ were applied to the MC3T3-E1 cells for viability (MTT-test). Based on the characteristic differences of the two methods of cell death, the morphological features of the MC3T3-E1 cell line co-cultured with nano-hydroxyapatite (n-HA) (10 mg mL⁻¹) for 24 h were also observed by TEM. Furthermore, important serum biochemical markers and histopathological examinations were used to evaluate the potential toxicological effect of n-HA on the major organs of SD rats injected intraperitoneally with n-HA (33.3 mg kg⁻¹ body weight). In the results, we found cell growth inhibition and apoptosis in MC3T3-E1 cells co-cultured with n-HA. Moreover, apoptosis but not necrosis was illustrated in liver and renal tissue by using histopathology slices and serum biochemical markers. It suggests that apoptosis may be the possible mechanism of n-HA toxicity and provides a better understanding of the biocompatibility of nanomaterials applied in human bone repair.