Techniques for producing cold and ultracold molecules are enabling the study of chemical reactions and scattering at the quantum scattering limit, with only a few partial waves contributing to the incident channel, leading to the observation and even full control of state-to-state collisions in this regime. A new R-matrix formalism is presented for tackling problems involving low- and ultra-low energy collisions. This general formalism is particularly appropriate for slow collisions occurring on potential energy surfaces with deep wells. The many resonance states make such systems hard to treat theoretically but offer the best prospects for novel physics: resonances are already being widely used to control diatomic systems and should provide the route to steering ultracold reactions. Our R-matrix-based formalism builds on the progress made in variational calculations of molecular spectra by using these methods to provide wavefunctions for the whole system at short internuclear distances, (a regime known as the inner region). These wavefunctions are used to construct collision energy-dependent R-matrices which can then be propagated to give cross sections at each collision energy. The method is formulated for ultracold collision systems with differing numbers of atoms.