SERS-active silver hierarchical assemblies are synthesized in solution by the assistance of small acid molecules. We here demonstrate the acid-directed self-assembly of metal nanoparticles (MNPs) into large systems with complex structures, without the application of any polymer surfactant or capping agent. It is verified that small acid molecules (citric acid, mandelic acid, etc.) incorporated into conventional solution chemistry can direct the assembly of MNPs into well-defined hierarchical structures. The constructed assembled structures with highly roughened surfaces can be highly sensitive SERS platforms, and the fabricated core–shell Ag wires show especially high SERS sensitivity toward the analyte melamine. The prepared Ag particles with hierarchical structures show no evident polarization-dependent SERS behavior, and this isotropic feature may be an advantage for highly sensitive SERS tags, since no certain incident polarization is required for molecule detection. We believe the subsequent addition of acid to induce formation of self-assembled structures can be a general synthetic platform to fabricate metal structures with complex morphologies.