We formulate, solve computationally and study experimentally the problem of collecting solar energy in three dimensions. We demonstrate that absorbers and reflectors can be combined in the absence of sun tracking to build three-dimensional photovoltaic (3DPV) structures that can generate measured energy densities (energy per base area, kWh/m²) higher by a factor of 2–20 than stationary flat PV panels for the structures considered here, compared to an increase by a factor of 1.3–1.8 for a flat panel with dual-axis sun tracking. The increased energy density is countered by a larger solar cell area per generated energy for 3DPV compared to flat panels (by a factor of 1.5–4 in our conditions), but accompanied by a vast range of improvements. 3DPV structures can mitigate some of the variability inherent to solar PV as they provide a more even source of solar energy generation at all latitudes: they can double the number of peak power generation hours and dramatically reduce the seasonal, latitude and weather variations of solar energy generation compared to a flat panel design. Self-supporting 3D shapes can create new schemes for PV installation and the increased energy density can facilitate the use of cheaper thin film materials in area-limited applications. Our findings suggest that harnessing solar energy in three dimensions can open new avenues towards Terawatt-scale generation.