Polymerized ionic liquids (PolyILs) are promising materials for applications in electrochemical devices spanning from fuel cells to capacitors and batteries. In principle, PolyILs have a competitive advantage over traditional electrolytes in being single ion conductors and thus enabling a transference number close to unity. Despite this perceived advantage, surprisingly low room temperature ionic conductivities measured in the lab raise an important fundamental question: how does the molecular structure mediate conductivity? In this work, wide-angle X-ray scattering (WAXS), vibrational sum frequency generation (vSFG), and density functional theory (DFT) calculations were used to study the bulk and interfacial structure of PolyILs, while broad band dielectric spectroscopy (BDS) was used to probe corresponding dynamics and conductive properties for a series of the PolyIL samples with tunable chemistries and structures. Our results reveal that the size of the mobile anions has a tremendous impact on chain packing in PolyILs that wasn’t addressed previously. Larger mobile ions tend to create a well-packed structure, while smaller ions frustrate chain packing. The magnitude of these changes and level of structural heterogeneity are shown to depend on the chemical functionality and flexibility of studied PolyILs. Furthermore, these experimental and computational results provide new insight into the correlation between conductivity and structure in PolyILs, suggesting that structural heterogeneity helps to reduce the activation energy for ionic conductivity in the glassy state.