There are many new approaches to designing complex anisotropic colloids, often using droplets as templates. However, droplets themselves can be designed to form anisotropic shapes without any external templates. One approach is to arrest binary droplet coalescence at an intermediate stage before a spherical shape is formed. Further shape relaxation of such anisotropic, arrested structures is retarded by droplet elasticity, either interfacial or internal. In this article we study coalescence of structured droplets, containing a network of anisotropic colloids, whose internal elasticity provides a resistance to full shape relaxation and interfacial energy minimization during coalescence. Precise tuning of droplet elasticity arrests coalescence at different stages and leads to various anisotropic shapes, ranging from doublets to ellipsoids. A simple model balancing interfacial and elastic energy is used to explain experimentally observed coalescence arrest in viscoelastic droplets. During coalescence of structured droplets the interfacial energy is continuously reduced while the elastic energy is increased by compression of the internal structure and, when the two processes balance one another, coalescence is arrested. Experimentally we observe that if either interfacial energy or elasticity dominates, total coalescence or total stability of droplets results. The stabilization mechanism is directly analogous to that in a Pickering emulsion, though here the resistance to coalescence is provided via an internal volume-based, rather than surface, structure. This study provides guidelines for designing anisotropic droplets by arrested coalescence but also explains some observations of “partial” coalescence observed in commercial foods like ice cream and whipped cream.