The sedimentation of quaternized poly(4-vinyl pyridine) (QP4VP) or poly(N-methyl 4-vinyl pyridine iodide) in aqueous solution has been investigated by using an analytical ultracentrifuge (AUC) via sedimentation velocity (SV). When NaI is introduced, either the sedimentation coefficient (s) or apparent molar mass (Mw) of QP4VP increases with NaI concentration because the polyelectrolyte chains aggregate or even form precipitates due to hydrophobic interaction. Upon addition of NaCl, either s or Mw exhibits a maximum as NaCl concentration increases. The sedimentation in NaBr solution falls in between. Besides, the diffusion coefficient (D) shows a minimum as the added salt concentration increases when either of the salts is added. Isothermal titration calorimetry (ITC) measurements show that the enthalpy change increases from NaI over NaBr to NaCl when each of them is mixed with QP4VP, revealing that the order of the strength of the anion–pyridinium interaction is I− > Br− > Cl−. When I− ions are added, they only screen the electrostatic interaction and no counterion competition happens. Upon addition of Cl− or Br− ions, the original counterions (I−) win the competition at low Cl− or Br− concentration, so that the added anions only screen the electrostatic interaction. When the concentration of Cl− or Br− ions is high enough, they replace I− ions to interact with the polyelectrolyte chains due to osmotic pressure. The present study demonstrates that the complex dynamics of the polyelectrolyte is greatly influenced by the ion-specific effect.