A compendium of different solid carbonaceous materials detected in space is presented, focussing on the search for organic matter of prebiotic interest. This journey takes us from the carbon grains likely formed in the atmospheres of evolved stars to organic grain mantles made from ice processing thought to be present in dense interstellar clouds and circumstellar regions, making a stop in solar system objects that could have delivered organic species to the early Earth. The most abundant carbon materials detected to date in space appear to be of little biological relevance. On the other hand, organic refractory residues, made in the laboratory from UV-photoprocessing followed by warm-up of interstellar ice analogs, are a hydrocarbon material rich in O and N containing chemical compounds that could act as initiators of prebiotic chemistry. A similar material might be present in dust grains inside dense clouds or circumstellar regions, some comets, and as a minor component in carbonaceous chondrites. We use infrared spectroscopy as a tool to spot organic refractory matter in various space environments. The delivery of organic materials via comets, (micro-) meteorites, and interplanetary dust particles to the primitive Earth might have contributed as a starting material for prebiotic chemistry. To test this hypothesis, it is first essential to characterize the composition of exogenous organic matter.