This paper investigated the growth of spinel crystals in vanadium slag under different heat treatment conditions. The growth mechanism was clarified by SEM + EDS and statistical analyses based on the CSD theory. The results showed that all CSDs follow lognormal distributions. Long holding time and low holding temperature below 1300 °C benefit the growth of spinel crystals. We proposed that the growth mechanisms of the spinel crystals consist of two regimes: in the first regime, nucleation control with a decaying rate within the temperature range of 1400 to 1250 °C; in the second regime, surface control when the temperature is below 1250 °C. The mean crystal diameter changes from 16.6 to 26.2 μm with an increase in holding time at 1250 °C. The relationship between the mean diameter of the spinel crystals and the holding time in the present study can be best described by the equation D = 16.32t0.11.