To develop environmentally friendly solvent processed high performance polymer solar cells (PSCs), an alcohol soluble narrow-band-gap conjugated polymer PCDTBT-N and an alcohol soluble C70 fullerene derivative PC71BM-N, which were functionalized with pendant tertiary amino groups, were developed and used as active layers in PSCs and as buffer layers in metal–semiconductor interfaces. Though solar cells with active layers containing amino groups showed no photovoltaic properties regardless of the processing conditions, both PCDTBT-N and PC71BM-N performed well as buffer layers to improve electron collection in PSCs. Space charge limited current (SCLC), field effect transistor (FET) and cyclic voltammetry (CV) studies revealed that the amino groups act as hole traps and disable hole transport in active layers. Moreover, ultraviolet photoelectron spectroscopy (UPS) indicates electronic structure changes and Fermi level shifts upon complexation of amino groups and the fullerene core, which may diminish the electron-accepting capacity of fullerenes, and may potentially provide a different kind of hole trap in the devices. The underlying mechanisms of the puzzling behavior of PCDTBT-N and PC71BM-N in different layers of PSCs were studied by numerical simulation, which indicates that hole traps distributed in the BHJ layer reduce device performance, conversely hole traps concentrated near the metal electrode can improve device performance. Besides, the formation of an amino group:C70 complex is another important cause of the performance improvements of the resulting solar cells using PCDTBT-N and PC71BM-N as buffer layers, due to the reduced transport loss for efficient electron collection through the n-doping of PC71BM at the interface of the buffer layer and the active layer.