Lateral diffusion plays a crucial role in numerous processes that take place in cell membranes, yet it is quite poorly understood in native membranes characterized by, e.g., domain formation and large concentration of proteins. In this article, we use atomistic and coarse-grained simulations to consider how packing of membranes and crowding with proteins affect the lateral dynamics of lipids and membrane proteins. We find that both packing and protein crowding have a profound effect on lateral diffusion, slowing it down. Anomalous diffusion is observed to be an inherent property in both protein-free and protein-rich membranes, and the time scales of anomalous diffusion and the exponent associated with anomalous diffusion are found to strongly depend on packing and crowding. Crowding with proteins also has a striking effect on the decay rate of dynamical correlations associated with lateral single-particle motion, as the transition from anomalous to normal diffusion is found to take place at macroscopic time scales: while in protein-poor conditions normal diffusion is typically observed in hundreds of nanoseconds, in protein-rich conditions the onset of normal diffusion is tens of microseconds, and in the most crowded systems as large as milliseconds. The computational challenge which results from these time scales is not easy to deal with, not even in coarse-grained simulations. We also briefly discuss the physical limits of protein motion. Our results suggest that protein concentration is anything but constant in the plane of cell membranes. Instead, it is strongly dependent on proteins' preference for aggregation.