Carbon Nanotubes (CNTs) have been functionalized with a layer of poly(sulfopropyl methacrylate) (PSPM) synthesized from silane initiators attached to the CNT walls. The PSPM brush was coated with a layer of poly(allyl amine hydrochloride) (PAH) and on top of the PAH layer lipid vesicles composed of 75% 1,2-dioleoyl-sn-glycero-3-phosphocholine and 25% 1,2-dioleoyl-sn-glycero-3-[phospho-L-serine] were assembled. The surface modification of the CNTs and lipid assembly were followed by Transmission Electron Microscopy (TEM) and ζ-potential measurements. TEM and Confocal Raman Microscopy (CRM) were used to study the uptake and localization of the surface modified CNTs in HepG2 cells. PSPM modified CNTs were present in the cytoplasm. Proliferation studies based on the MTT assay were used to assess toxicity of the CNTs for the different modifications. Cell proliferation decreases for oxidized CNTs and polymer coated CNTs, having the lowest value for PSPM coated CNTs. The lipid coating on top of the polymers significantly reduces the toxic effect of CNTs.