To enhance the photocatalytic property of coordination polymers (CPs) in the visible light region, Ag loaded coordination polymer composite materials (Ag/CPs) were synthesized successfully through a photoreduction reaction of Ag+ on the surface of CPs. Photoluminescence (PL) was used to investigate the separation of photogenerated electron–hole pairs and the results illustrated Ag/CPs display higher quantum yields than CPs. This can be attributed to the strong interactions between Ag nanorods and coordination polymers, which lead to electron–hole pair separation between Ag nanorods and CPs. The degradation of Rhodamine B (RhB) was investigated to study the photocatalytic activities. Ag/CPs exhibited excellent photocatalytic activity in the UV and visible light region, while CPs can only decompose RhB under the irradiation of UV light. Furthermore, Ag/CPs showed outstanding stability during degradation of RhB.