Reactions of zinc sulfate and (R)- or (S)-(1-phenylethylamino)methylphosphonic acid (pempH2) in the presence of 1,4-benzenedicarboxylic acid (H2bdc) result in homochiral enantiomers (R)- and (S)-[Zn4(pempH)4(bdc)2]·2H2O (1). When H2bdc is replaced by 1,2,4,5-benzenetetracarboxylic acid (H4btec), enantiomers (R)- and (S)-[Zn3(pempH)2(btec)(H2O)2]·H2O (2) are obtained. Compounds 1 have a layer structure in which the chiral inorganic double chains, made up of corner-sharing {ZnO4} and {PO3C} tetrahedra, are connected by bdc2−. While in compounds 2, chiral chains of zinc phosphonate are each connected to their four equivalent neighbors through btec4− linkages, resulting in an open framework structure. All these compounds are optically active, exhibiting second harmonic generation (SHG) responses 0.2 or 0.8 times that of urea.