The edges of pristine graphite were covalently grafted with para-poly(ether-ketone) (pPEK) in a mildly acidic polyphosphoric acid (PPA)/phosphorus pentoxide (P2O5) medium. The resulting pPEK grafted graphite (pPEK-g-graphite) showed that the pristine graphite had been exfoliated into a few layers of graphene platelets (graphene-like sheets), which were uniformly dispersed into a pPEK matrix. As a result, the tensile properties of pPEK-g-graphite films were greatly improved compared to those of controlled pPEK films. The origins of these enhanced mechanical properties were deduced from scanning electron microscope (SEM) images of fracture surfaces. Upon tracing wide-angle X-ray scattering (WAXS) patterns of the film under strain, the graphene-like sheets were further exfoliated by an applied shear force, suggesting that a toughening mechanism for the pPEK-g-graphite film occurred. This approach envisions that the “direct” edge grafting of pristine graphite without pre-treatments such as corrosive oxidation and/or destructive sonication is a simple and efficient method to prepare graphene-based polymer composites with enhanced mechanical properties.