To improve the photocatalytic activity of a coordination polymer in the visible light region, polyaniline (PANI) was loaded onto its surface through a facile in situ chemical oxidation polymerization process. The resulting PANI loaded coordination polymer composite materials with excellent stability exhibit significantly higher photocatalytic activities than the pure coordination polymer photocatalyst on the degradation of methyl orange (MO) under visible light irradiation. This enhancement can be ascribed to the introduction of PANI on the surface of the coordination polymer, which leads to efficient separation of photogenerated electron–hole pairs as well as a significant expansion of the photoresponse region. Finally, we discussed the influence of acidity on the morphology and photocatalytic activity of the composite material. An optimal condition to obtain the PANI loaded coordination polymer composite material with excellent photocatalytic activity has been obtained.