The contrast of second-order nonlinear optical response in the dihydroazulene (DHA)-vinylheptafulvene (VHF) equilibrium has been investigated as a function of the nature of the substituent (R) on the phenyl ring by means of quantum chemistry calculations including electron correlation, frequency dispersion, and solvent effects. By considering the hyper-Rayleigh scattering (HRS) response, the contrast for R = H and R = CH3 between the DHA and VHF forms is larger than 5 while the contrast between the cis and transVHF forms is close to 1. Adding the NH2 donor group in para position of the phenyl leads to a substantial increase of the HRS first hyperpolarizability of the three forms, which is detrimental to the contrast. Then, in the case of the NO2 acceptor group, a contrast is recovered because the HRS first hyperpolarizability of the DHA form is about 2–3 times larger than for both VHF forms. These variations of first hyperpolarizability as a function of the substituents as well as the associated contrasts have been explained in terms of donor/acceptor strengths and geometrical parameters.