Energy landscapes for machine learning

Machine learning techniques are being increasingly used as flexible non-linear fitting and prediction tools in the physical sciences. Fitting functions that exhibit multiple solutions as local minima can be analysed in terms of the corresponding machine learning landscape. Methods to explore and visualise molecular potential energy landscapes can be applied to these machine learning landscapes to gain new insight into the solution space involved in training and the nature of the corresponding predictions. In particular, we can define quantities analogous to molecular structure, thermodynamics, and kinetics, and relate these emergent properties to the structure of the underlying landscape. This Perspective aims to describe these analogies with examples from recent applications, and suggest avenues for new interdisciplinary research.


I. Introduction
Optimisation problems abound in computational science and technology. From force field development to thermodynamic sampling, bioinformatics and computational biology, optimisation methods are a crucial ingredient of most scientific disciplines. 1 Geometry optimisation is a key component of the potential energy landscapes approach in molecular science, where emergent properties are predicted from local minima and the transition states that connect them. 2,3 This formalism has been applied to a wide variety of physical systems, including atomic and molecular clusters, biomolecules, mesoscopic models, and glasses, to understand their structural properties, thermodynamics and kinetics. The methods involved in the computational potential energy landscapes approach amount to optimisation of a nonlinear function (energy) in a high-dimensional space (configuration space). Machine learning problems employ similar concepts: the training of a model is performed by optimising a cost function with respect to a set of adjustable parameters.
Understanding how emergent observable properties of molecules and condensed matter are encoded in the underlying potential energy surface is a key motivation in developing the theoretical and computational aspects of energy landscape research. The fundamental insight that results has helped to unify our understanding of how structure-seeking systems, such as 'magic number' clusters, functional biomolecules, crystals and self-assembling structures, differ from amorphous materials and landscapes that exhibit broken ergodicity. [3][4][5][6] Rational design of new molecules and materials can exploit this insight, for example associating landscapes for self-assembly with a well defined free energy minimum that is kinetically accessible over a wide range of temperature. This structure, where there are no competing morphologies separated from the target by high barriers, corresponds to an 'unfrustrated' landscape. 2,5,7,8 In contrast, designs for multifunctional systems, including molecules with the capacity to act as switches, correspond to multifunnel landscapes. 9,10 In this Perspective we illustrate how the principles and tools of the potential energy landscape approach can be applied to machine learning (ML) landscapes. Some initial results are presented, which indicate how this view may yield new insight into ML training and prediction in the future. We hope our results will be relevant for both the energy landscapes and machine learning communities. In particular, it is of fundamental interest to compare these ML landscapes to those that arise for molecules and condensed matter. The ML landscape provides both a means to visualise and interpret the cost function solution space and a computational framework for quantitative comparison of solutions.

II. The energy landscape perspective
The potential energy function in molecular science is a surface defined in a (possibly very high-dimensional) configuration space, which represents all possible atomic configurations. 2,11 In the potential energy landscape approach, this surface is divided into basins of attraction, each defined by the steepest-descent pathways that lead to a particular local minimum. 2,11 The mapping from a continuous multidimensional surface to local minima can be very useful. In particular, it provides a route to prediction of structure and thermodynamics. 2 Similarly, transitions between basins can be characterised by geometric transition states (saddle points of index one), which lie on the boundary between one basin and another. 2 Including these transition states in our description of the landscape produces a kinetic transition network, 3,12,13 and access to dynamical properties and 'rare' events. [14][15][16][17] The pathways mediated by these transition states correspond to processes such as molecular rearrangements, or atomic migration. For an ML landscape we can define the connectivity between minima that represent different locally optimal fits to training data in an analogous fashion. To the best of our knowledge, interpreting the analogue of a molecular rearrangement mechanism for the ML landscape has yet to be explored.
Construction of a kinetic transition network 3,12,13 also provides a convenient means to visualise a high-dimensional surface. Disconnectivity graphs 4,18 represent the landscape in terms of local minima and connected transition states, reflecting the barriers and topology through basin connectivity. The overall structure of the disconnectivity graph can provide immediate insight into observable properties: 4 a single-funnelled landscape typically represents a structure-seeking system that equilibrates rapidly, whereas multiple funnels indicate competing structures or morphologies, which may be manifested as phase transitions and even glassy phenomenology. Locating the global minimum is typically much easier for single funnel landscapes. 19 The decomposition of a surface into minima and transition states is quite general and can naturally be applied to systems that do not correspond to an underlying molecular model. In particular, we can use this strategy for machine learning applications, where training a model amounts to minimisation of a cost function with respect to a set of parameters. In the language of energy landscapes, the machine learning cost function plays the role of energy, and the model parameters are the 'coordinates' of the landscape. The minimised structures represent the optimised model parameters for different training iterations. The transition states are the index one saddle points of the landscape. 20 Energy landscape methods 2 could be particularly beneficial to the ML community, where non-convex optimisation has sometimes been viewed as less appealing, despite supporting richer models with superior scalability. 21 The techniques described below could provide a useful computational framework for exploring and visualising ML landscapes, and at the very least, an alternative view to non-convex optimisation. The first steps in this direction have recently been reported. [22][23][24] The results may prove to be useful for various applications of machine learning in the physical sciences. Examples include fitting potential energy surfaces, where neural networks have been used extensively [25][26][27][28][29][30][31][32] for at least 20 years. [33][34][35] Recent work includes prediction of binding affinities in protein-ligand complexes, 36 applications to the design of novel materials, 37,38 and refinement of transition states 39 using support vector machines. 40 In the present contribution we illustrate the use of techniques from the energy landscapes field to several ML examples, including non-linear regression, and neural network classification.
When surveying the cost function landscapes, we employed the same techniques and algorithms as for the molecular and condensed matter systems of interest in the physical sciences: specifically, local and global minima were obtained with the basin-hopping method 41-43 using a customised LBFGS minimisation algorithm. 44 Transition state searches employed the doubly-nudged 45,46 elastic band 47,48 approach and hybrid eigenvector-following. 49,50 These methods are all well established, and will not be described in detail here. We used the python-based energy landscape explorer pele, 51 with a customised interface for ML systems, along with the GMIN, 52 OPTIM, 53 and PATHSAMPLE 54 programs, available for download under the Gnu General Public Licence.

III. Prediction for classification of outcomes in local minimisation
Neural networks (NN) have been employed in two previous classification problems that analyse the underlying ML landscape, namely predicting which isomer results from a molecular geometry optimisation 23 and for patient outcomes in a medical diagnostic context. 24 Some of the results from the former study will be illustrated here, and we must carefully distinguish isomers corresponding to minima of a molecular potential energy landscape from the ML landscape of solutions involved in predicting which of the isomers will result from geometry optimisation starting from a given molecular configuration. We must also distinguish this classification problem from ab initio structure prediction: the possible outcomes of the geometry optimisation must be known in advance, either in terms of distinct isomers, or the range that they span in terms of potential energy or appropriate structural order parameters for larger systems. The ability to make predictions that are sufficiently reliable could produce significant savings in computational effort for applications that require repeated local minimisation. Examples include basin-sampling for calculating global thermodynamic properties in systems subject to broken ergodicity, 55 construction of kinetic transition networks, 56 and methods to estimate the volume of basins of attraction for jammed packings, which provide measures of configurational entropy in granular packings. 57 Here the objective would be to terminate the local minimisation as soon as the outcome could be predicted with sufficient confidence to converge the properties of interest. 23,58 The test system considered here is a simple triatomic cluster with four distinguishable local minima that represent the possible outcomes for local minimisation. This system has previously served as a benchmark for visualising the performance of different geometry optimisation approaches. [59][60][61] The potential energy, V, is a sum of pairwise terms, corresponding to the Lennard-Jones form, 62 and the three-body Axilrod-Teller function, 63 which represents an instantaneous induced dipole-induced dipole interaction: 1 þ 3 cos y 1 cos y 2 cos y 3 r ij r ik r jk À Á 3 where y 1 , y 2 and y 3 are the internal angles of the triangle formed by atoms i, j, k. r ij is the distance between atoms i and j. The influence of the three-body term is determined by the magnitude of the parameter g, and we use g = 2 in reduced units, where the equilateral triangle (V = À2.185e) competes with three permutational isomers of a linear minimum (V = À2.219e). In the triangle the bond length is 1.16875s, and in the linear minima the distance from the centre atom to its neighbours is 1.10876s. The objective of our machine learning calculations for this system is a classification, to predict which of the four minima a local minimisation would converge to, given data for one or more configurations. The data in question could be combinations of the energy, gradient, and geometrical parameters for the structures in the optimisation sequence. Our initial tests, which are mostly concerned with the structure of the ML solution landscape, employed the three interparticle separations r 12 , r 13 and r 23 as data. 23 Inputs were considered for separations corresponding to the initial geometry, and for one or more configurations in the minimisation sequence.
A database of 10 000 energy minimisations, each initiated from different atomic coordinates distributed in a cube of side length 2 ffiffi ffi 3 p s, was created using the customised LBFGS routine in our OPTIM program 53 (LBFGS is a limited memory quasi-Newton Broyden, 64 Fletcher, 65 Goldfarb, 66 Shanno, 67 scheme). Each minimisation was converged until the root mean square gradient fell below 10 À6 reduced units, and the outcome (one of the four minima) was recorded.
The neural networks used in the applications discussed below all have three layers, corresponding to input, output, and hidden nodes. 68 A single hidden layer has been used successfully in a variety of previous applications. 69 A bias was added to the sum of weights used in the activation function for each hidden node, w bh j , and each output node, w bo i , as illustrated in Fig. 1. For inputs we consider X = {x 1 ,. . .,x Ndata }, where N data is the number of minimisation sequences in the training or test set, each of which has dimension N in , so that x a ¼ x a 1 ; . . . ; x a N in n o . For this example there are four outputs corresponding to the four possible local minima (as in Fig. 1), denoted by y NN i , with for a given input x and link weights w (1) ij between hidden node j and output i, and w (2) jk between input k and hidden node j. The four outputs were converted into softmax probabilities as This formulation is designed to reduce the effect of outliers.
In each training phase we minimise the cost (objective) function, E NN (W;X), with respect to the variables w (1) ij , w (2) jk , w bh j and w bo i , which are collected into the vector of weights W. An L 2 regularisation term is included in E NN (W;X), corresponding to a sum of squares of the independent variables, multiplied by a constant coefficient l, which is chosen in advance and fixed. This term is intended to prevent overfitting; it disfavours large values for individual variables. We have considered applications where the regularisation is applied to all the variables, and compared the results with a sum that excludes the bias weights. Regularising over all the variables has the advantage of eliminating the zero Hessian eigenvalue that otherwise results from an additive shift in all the w bo i . Such zero eigenvalues are a consequence of continuous symmetries in the cost function (Noether's theorem). For molecular systems such modes commonly arise from overall translation and rotation, and are routinely dealt with by eigenvalue shifting or projection using the known analytical forms for the eigenvectors. 2,70 Larger values of the parameter l simplify the landscape, reducing the number of minima. This result corresponds directly with the effect of compression for a molecular system, 71 which has been exploited to accelerate global optimisation. A related hypersurface deformation approach has been used to treat graph partitioning problems. 72 For each LBFGS geometry optimisation sequence, d, with N in inputs collected into data item x d , we know the actual outcome or class label, c(d) = 0, 1, 2 or 3, corresponding to the local minimum at convergence. The networks were trained using either 500 or 5000 of the LBFGS sequences, chosen at random with no overlap, by minimising Results were compared for different values of l and in some cases for regularisation excluding the bias weights. These formulations, including analytic first and second derivatives with respect to W, have been programmed in our GMIN global optimisation program 52 and in our OPTIM code for analysing stationary points and pathways. 53  using the same customised LBFGS routine that was employed to create the database of minimisation sequences for the triatomic cluster.
In the testing phase the variables W are fixed for a particular local minimum of E NN (W;X train ) obtained with the training data, and we evaluate E NN (W;X test ) for 500 or 5000 of the minimisation sequences outside the training set. The results did not change significantly between the larger and smaller training and testing sets.
We first summarise some results for ML landscapes corresponding to input data for the three interparticle distances at each initial random starting point. The number of local minima obtained 23 was 162, 2559, 4752 and 19 045 for three, four, five and six hidden nodes, respectively, with 1504, 10 112, 18 779 and 34 052 transition states. The four disconnectivity graphs are shown in Fig. 2. In each case the vertical axis corresponds to E NN (W;X train ), and branches terminate at the values for particular local minima. At a regular series of thresholds for E NN we perform a superbasin analysis, 18 which segregates the ML solutions into disjoint sets. Local minima that can interconvert via a pathway where the highest transition state lies below the threshold are in the same superbasin. The branches corresponding to different sets or individual minima merge at the threshold energy where they first belong to a common superbasin. In this case we have coloured the branches according to the misclassification index, discussed further in Section V, which is defined as the fraction of test set images that are misclassified by the minimum in question or the global minimum, but not both. All the low-lying minima exhibit small values, meaning that they perform much like the global minimum. The index rises to between 0.2 and 0.4 for local minima with higher values of E NN (W;X train ). These calculations were performed using the pele 51 ML interface for the formulation in eqn (2), where regularisation excluded the weights for the bias nodes. 23 When more hidden nodes are included the dimensionality of the ML landscape increases, along with the number of local minima and transition states. This observation is in line with well known results for molecular systems: as the number of atoms and configurational degrees of freedom increases the number of minima and transition states increases exponentially. 73,74 The residual error reported by E NN (W;X train ) decreases as more parameters are included, and so there is a trade-off between the complexity of the landscape and the quality of the fit. 23 The opportunities for exploiting tools from the potential energy landscape framework have yet to be investigated systematically. As an indication of the properties that might be of interest we now illustrate a thermodynamic analogue corresponding to the heat capacity, C V . Peaks in C V are particularly insightful, and in molecular systems they correspond to phaselike transitions. Around the transition temperature the occupation probability shifts between local minima with qualitatively different properties in terms of energy and entropy: larger peaks correspond to greater differences. 75 For the ML landscape we would interpret such features in terms of fitting solutions with different properties. Understanding how and why the fits differ could be useful in combining solutions to produce better predictions. Here we simply illustrate some representative results, which suggest that ML landscapes may support an even wider range of behaviour than molecular systems.
To calculate the C V analogue we use the superposition approach 2,73,76-79 where the total partition function, Z(T), is written as a sum over the partition functions Z a (T), for all the local minima, a. This formulation can be formally exact, but is usually applied in the harmonic approximation using normal mode analysis to represent the vibrational density of states. The normal mode frequencies are then related to the eigenvalues of the Hessian (second derivative) matrix, m a (i), for local minimum a: here k is the number of non-zero eigenvalues, b = 1/k B T, k B is the Boltzmann constant, and E NN (W a ;X train ) is the objective (loss) function for minimum a. T plays the role of temperature in this picture, with C V (T) = (1/k B T 2 )q 2 ln Z(T)/qb 2 . Many molecular and condensed matter systems exhibit a C V peak corresponding to a first order-like melting transition, where the occupation probability shifts from a relatively small number of low energy minima, to a much larger, entropically favoured, set of higher energy structures, which are often more disordered. However, low temperature peaks below the melting temperature, corresponding to solid-solid transitions, can also occur. These features are particularly interesting, because they suggest the presence of competing low energy morphologies, which may represent a challenge for structure prediction, 80 and lead to broken ergodicity, 55,79,[81][82][83][84][85][86] and slow interconversion rates that constitute 'rare events'. 14,87,88 Initial surveys of the C V analogue for ML landscapes, calculated using analytical second derivatives of E NN (W a ;X), produced plots with multiple peaks, suggesting richer behaviour than for molecular systems.
One example is shown in Fig. 3, which is based on the ML solution landscape for a neural network with three hidden nodes and inputs corresponding to the three interparticle distances at the initial geometries of all the training optimisation sequences. These results were obtained using the pele 51 ML interface for the neural network formulation in eqn (2), where regularisation did not include the weights for the bias nodes. 23 The superposition approach provides a clear interpretation for the peaks, which we achieve by calculating C V from partial sums over the ML training minima, in order of increasing E NN (W a ;X train ). The first peak around k B T E 0.2 arises from competition between the lowest two minima. The second peak around k B T E 9 is reproduced when the lowest 124 minima are included, and the largest peak around k B T E 20 appears when we sum up to minimum 153. The latter solution exhibits one particularly small Hessian eigenvalue, producing a relatively large configurational entropy contribution, which increases with T. The harmonic approximation will break down here, but nevertheless serves to highlight the qualitative difference in character of minima with exceptionally small curvatures. In molecular systems competition between alternative low energy structures often accounts for C V peaks corresponding to solid-solid transitions, and analogues of this situation may well exist in the ML scenario. Some systematic shifts in the C V analogue could result from the density of local minima on the ML landscape (the landscape entropy [89][90][91][92] ). Understanding such effects might help to guide the construction of improved predictive tools from combinations of fitting solutions. Interpreting the ML analogues of molecular structure and interconversion rates between minima might also prove to be insightful in future work.
A subsequent study investigated the quality of the predictions using two of the three interatomic distances, r 12 and r 13 , and the effects of memory, in terms of input data from successive configurations chosen systematically from each geometry optimisation sequence. 93 The same database of LBFGS minimisation sequences for the triatomic cluster was used here, divided randomly into training and testing sets of equal size (5000 sequences each). The quality of the classification prediction into the four possible outcomes can be quantified using the area under curve (AUC) for receiver operating characteristic (ROC) plots. 69 ROC analysis began when radar receiver operators needed to distinguish signals corresponding to aircraft from false readings, including flocks of birds. The curves are plots of the true positive rate, T pr , against the false positive rate, F pr , as a function of the threshold probability, P, for making a certain classification. Here, P is the threshold at which the output probability p NN 0 (W;X) is judged sufficient to predict that a minimisation would converge to the equilateral triangle, so that where Y is the Heaviside step function and d is the Kronecker delta. The area under the curve can then be obtained by numerical integration of T pr ðW; X; PÞdF pr ðW; X; PÞ: AUC(W;X) can be interpreted as the probability that for two randomly chosen data inputs, our predictions will discriminate between them correctly. AUC values between 0.7 and 0.8 are usually considered 'fair', 0.8 to 0.9, 'good', and 0.9 to 1 'excellent'. Fig. 4 shows the AUC values obtained with the LBFGS database when the input data consists of r 12 and r 13 values at different points in the minimisation sequence. Here the horizontal axis corresponds to s, the number of steps from convergence, and the AUC values therefore tend to unity when s is small, where the configurations are close to the final minimum. Each panel in the figure includes two plots, which generally coincide quite closely. The plot with the lower AUC value is the one obtained with the global minimum located for E NN (W a ;X train ) for configurations s steps for convergence. The AUC value in the other plot is always greater than or equal to the value for the global minimum, since it is the maximum AUC calculated for all the local minima characterised with the same neural net architecture and any s value. Minimisation sequences that converge in fewer than s steps are padded with the initial configuration for larger s values, which is intended to present a worst case scenario. Fig. 4 shows that the prediction quality decays in a characteristic fashion as configurations move further from the convergence limit. It also illustrates an important result, namely that the performance of the global minimum obtained with the training data is never surpassed significantly by any of the other local minima, when all these fits are applied to the test data. Hence the global minimum is clearly a good place to start if we wish to make predictions, or perhaps construct classification schemes based on more than one local minimum of the ML Fig. 3 Heat capacity analogue for the ML landscape defined for the dataset using only the three initial interatomic distances with three hidden nodes. The insets illustrate the convergence of the two low temperature peaks. In each plot the black curve corresponds to C V calculated from the complete database of minima. The red curves labelled '2', '124' and '152' correspond to C V calculated from truncated sums including only the lowest 2, 124, and 152 minima, respectively. landscape obtained in training. The corresponding fits and AUC calculations were all rerun to produce Fig. 4 using the fitting function defined in eqn (2) and regularisation over all variables, for comparison with the simplified bias weighting employed in ref. 93. There is no significant difference between our results for the two schemes.

IV. Non-linear regression
Regression is perhaps the most well-known task in machine learning, referring to any process for estimating the relationships between dependent and independent variables. As we show in this section, even a relatively simple non-nonlinear regression problem leads to a rich ML landscape. As in the standard regression scenario, we consider a set of N data data points D = ((x 1 ,t 1 ),. . .,(x N data ,t N data )), and a model y(x;q) that we wish to fit to D by adjusting M parameters q = (q 1 ,q 2 ,. . .,q M ). In this example, we investigate the following non-linear model: Our regression problem is one-dimensional (x is a scalar), with a five-dimensional vector q that parameterises the model. We performed regression on the above problem, with a dataset D consisting of N data = 100 data points with x i values sampled uniformly in [0,3p], and corresponding t i values given by our model with added Gaussian white noise (mean zero, s = 0.02): with a particular ad hoc parameter choice q = (0.1, 2.13, 0.0, 1.34, 0.0). The cost function we minimise is a standard sum of least squares: The objective of this regression problem is to find a best-fit mapping from input (x) to target variables (t). Intuitively, we expect minimisation of eqn (10) with respect to the parameters q to yield an optimal value q E q*. However, since E is a nonconvex function of q, there are multiple solutions to the equation rE(q) = 0 and hence the outcome depends on the minimisation procedure and the initial conditions. We explored the landscape described by E(q), and display the various solutions in Fig. 5 alongside our data and y(x;q = q*). In this case, the global minimum is a fairly accurate representation of the solution used to generate D. However, 88 other solutions were found which do not accurately represent the data, despite being valid local minima. In Fig. 6 we show the disconnectivity graph for E(q). Here the vertical axis corresponds to E(q), and branches terminate at the values for corresponding local minima, as in Section III. The graph shows that the energy of the global minimum solution is separated from the others by an order of magnitude, and is clearly the best fit to our data (dotted curve in Fig. 5). The barriers in this representation can be interpreted in terms of transformations between different training solutions in parameter space, and could be indicative of the distinctiveness of the minima they connect. The minima of this landscape were found by the basin-hopping method, [41][42][43] as described in Section II.

V. Digit recognition using a neural network
The next machine learning landscape we explore here is another artificial neural network, this time trained for digit recognition on the MNIST dataset. 94 Our network architecture consists of 28 Â 28 input nodes corresponding to input image pixels, a single hidden layer with 10 nodes, and a softmax output layer of 10 nodes, which represent the 10 digit classes. This model contains roughly 8000 adjustable parameters, which quantify the weight of given nodes in activating one of their successors. The cost function we optimise for this classification example is the same multinomial logistic regression function that is described above in Section III, which is standard for classification problems. Here 'logistic' means that the dependent variable (outcome) is a category, in this case the assignment of the image for a digit, and 'multinomial' means that there are more than two possible outcomes. An L 2 regularisation term was again added to the cost function, as described in Section III. Unless otherwise mentioned, all the results described below are for a regularisation coefficient of l = 0.1.
The neural network defined above is quite small, and is not intended to compete with well-established models trained on MNIST. 95 Rather, our goal in this Perspective is to gain insight into the landscape. This aim is greatly assisted by using a model that is smaller, yet still behaves similarly to more sophisticated implementations. To converge the disconnectivity graph, Fig. 7, in particular the transition states, the model was trained on N data = 1000 data points. The results assessing performance, Fig. 8 and 9, were tested on N data = 10 000 images.
We explored the landscape of this network for several different values of the L 2 regularisation parameter l. The graph shown   (8)]. Each branch terminates at a local minimum at the value of E(q) for that minimum. By following the lines from one minimum to another, one can read off the energy barrier on the minimum energy path connecting them. in Fig. 7, with l = 0.01, is representative of all the others: we observe a single funnel landscape, where, in contrast to the nonlinear regression example, all of the minima occur in a narrow range of energy (cost) values. This observation is consistent with recent work suggesting that the energies of local minima for neural networks are spaced exponentially close to the global minimum 96,97 with the number of variables (number of optimised parameters or dimensionality) of the system. We next assess the performance profile of the NN minima by calculating the misclassification rate on an independent test set. Judging by average statistics the minima seem to perform very similarly: the fraction f of misclassified test set images is comparable for most of them, with 0.133 r f r 0.162 (mean % f = 0.148, standard deviation s f = 0.0043). This observation is also consistent with previous results where local minima were found to perform quite similarly in terms of test set error. 98,99 When looking beyond average statistics, however, we uncover more interesting behaviour. To this end we introduce a misclassification distance c between pairs of minima, which we define as the fraction of test set images that are misclassified by one minimum but not both. 100 A value c ij = 0 implies that all images are classified in the same way by the two minima; a value c ij = c max ij = f i + f j implies that every misclassified image by i was correctly classified by j, and every misclassified image by j was correctly classified by i. In Fig. 8 we display the matrix c ij , which shows that the minima cluster into groups that are self-similar, and distinct from other groups. So, although all minima perform almost identically when considering the misclassification rate alone, their performance looks quite distinct when considering the actual sets of misclassified images. We hypothesise that this behaviour is due to a saturated information capacity for our model. This small neural network can only encode a certain amount of information during the training process. Since there are many training images, there is much more information to encode than it is possible to store. The clustering of minima in Fig. 8 then probably reflects the differing information content that each solution retains. Here it is important to remember that each of the minima were trained on the same set of images; the distinct minima arise solely from the different starting configurations prior to optimisation.
The misclassification similarity can be understood from the underlying ML landscape. We investigated correlations between misclassification distance and distance in parameter space. In Fig. 9 we display the joint distribution of the misclassification distance and Euclidean distance (L 2 norm) between the parameter values for each pair of minima. We see that for a range of values these two measures are highly correlated, indicating that the misclassification distance between minima is determined by their proximity on the underlying landscape. Interestingly, for very large values of geometric distance there is a large (yet seemingly random) misclassification distance.
The seemingly random behaviour could possibly be the result of symmetry with respect to permutation of neural network parameters. There exist a large number of symmetry operations for the parameter space that leave the NN prediction unchanged, yet would certainly change the L 2 distance with respect to another reference point. A more rigorous definition of distance (currently unexplored), would take such symmetries into account. There are at least two such symmetry operations. 68 The first of these results from the antisymmetry of the tanh activation function: inverting the sign of all weights and bias leading into a node will lead to an inverted output from that node. The second symmetry is due to the arbitrary labelling of nodes: swapping the labels of nodes i and j within a given hidden layer will leave the output of a NN unchanged.

VI. Network analysis for machine learning landscapes
The landscape expressed in terms of a connected database of local minima and transition states can be analysed in terms of  network properties. [101][102][103] The starting point of such a description is the observation that for a potential energy function with continuous degrees of freedom, each minimum is connected to other minima through steepest-descent paths mediated by transition states. Hence, one can construct a network 104 in a natural way, where each minimum corresponds to a node. If two minima are connected via a transition state then there is an edge between them. In this preliminary analysis we consider an unweighted and undirected network; this approach will be extended in the future to edge weights and directions that are relevant to kinetics. In this initial analysis we are only interested in whether or not two minima are connected, and multiple connections make no difference. The objective of working with unweighted and undirected networks is to first focus on the global structure of the landscape, providing the foundations for analysis of how emergent thermodynamic and kinetic properties are encoded in future work.
After constructing the network we can analyse properties such as average shortest path length, diameter, clustering coefficients, node degrees and their distribution. For an unweighted and undirected network, the shortest path between a pair of nodes is the path that passes through the fewest edges. The number of edges on the shortest path is then the shortest path length between the pair of nodes, and the average shortest path length is the average of the shortest path lengths over all the pairs of nodes. The diameter of a network is the path length of the longest shortest path. For the network of minima, the diameter of the network is the distance, in terms of edges, between the pair of minima that are farthest apart. The node degree is the number of directly connected neighbours.
For the non-linear regression model, we found 89 minima and 121 transition states. The resulting network consists of 89 nodes and 113 edges (Fig. 10). Although this is a small network we use it to introduce the analysis. The average node degree is 2.54 and the average shortest path length is 6.0641. The network diameter, i.e. the longest shortest path, is 15. Hence, on average a minimum is around 6 steps away from any other minimum, and the pair farthest apart are separated by 15 steps.
Thus, a minimum found by a naive numerical minimisation procedure may be on average 6 steps, and in the worst case 15 steps, from the global minimum. Both the average shortest path and network diameter of this network are significantly larger than for a random network of an equivalent size. 105 The network of minima for the neural network model in Section V has 142 nodes and 643 edges (Fig. 11). The nodes have on average 9.06 nearest neighbours, the average shortest path is 3.18, and the network diameter is 8. Hence, a randomly selected minimum is on an average only 3 steps away from any other minimum in terms of minimum-transition state-minimum triples. Therefore, on average, the global minimum is only 3 steps away from any other local minimum. The networks of minima defined by molecules such as Lennard-Jones clusters, Morse clusters, and the Thomson problem have also been shown to have small (of order O(log[number of minima])) average shortest path lengths, meaning that any randomly picked local minimum is only a few steps away from the global minimum. [101][102][103]106 Moreover, these networks exhibit small-world behaviour, 107 i.e. the average shortest path lengths of these networks are small and similar to equivalent size random networks, whereas their clustering coefficients are significantly larger than those of equivalent size random networks. We have conjectured that the small-world properties of networks of minima are closely related to the singlefunnel nature of the corresponding landscapes. [108][109][110] Some networks also exhibit scale-free properties, 111 where the nodedegrees follow a power-law distribution. In such networks only a few nodes termed hubs have most of the connections, while most other nodes have only a small number.
The benefits of analysing network properties of machine learning landscapes may be two-fold. One can attempt to construct more efficient and tailor-made algorithms to find the global minimum of machine learning problems by exploiting these network properties, for example, by navigating the 'shortest path' to the global minimum. We also obtain a quantitative description of the distance between a typical minimum from the best fit solution.  In the future, we plan to study further properties of the networks of minima for a variety of artificial neural networks and test the small-world and scale-free properties. We hope that these results may be useful in constructing algorithms to find the global minimum of non-convex machine learning cost functions.

VII. The p-spin model and machine learning
Many machine learning problems are solved via some kind of optimisation that makes use of gradient based methods, such as the stochastic gradient descent. Such algorithms utilise the local geometry of the landscape, and eventually iterations stop progressing when the norm of the (stochastic) gradients approach zero. This leads to the following general question: for a real valued function, what values do the critical points have? The answer certainly depends on the structure of the function we have at hand. In this section, we will examine two classes of functions: the Hamiltonian of the p-spin spherical glass, and the loss function that arises in the optimisation of a simple deep learning problem. In this section, the deep learning problem is the same as the one introduced in Section V with a larger hidden layer and zero regularisation. The functions have very different structures, and at first sight they do not appear to resemble one another in any meaningful way. However, we find that the two systems may exhibit similar characteristics in terms of the values of their critical points, in spite of the apparent differences.

A. Concentration of critical points of the p-spin model
The Hamiltonian of a mean-field spin glass is a homogeneous polynomial of a given degree, where the coefficients of the polynomial describe the nature and strength of the interaction between the spins. Since the polynomial is homogeneous, a common choice is to restrict the variables (spins) to the unit sphere. We investigate what can be said about the minimisation problem if we choose the coefficients of this polynomial at random and independently from the standard normal distribution.
First, we define the notation for the rest of the section. We consider real valued polynomials, H(w), where w is the vector of variables of H; the degree of the polynomial H is p. We define the dimension of the polynomial by the length of the vector w, so if w = (w 1 ,. . .,w N ) then H is an N-dimensional polynomial. A degree p polynomial is homogeneous polynomial if it satisfies the following condition for any real t: Finally, a degree p polynomial of N variables will have N p coefficients (some of which may be zero). The coefficients will be denoted x i 1 ,. . ., i p , where each index runs from 1 to N.
Having defined the notation, we now clarify the connection between polynomials and spin glasses. Suppose the vector, w = (w 1 ,. . .,w N ), describes the states of N Ising spins that are +1 or À1. Then P w i 2 = N, so that the distance to the origin is ffiffiffiffi N p .
The continuous analogue of this model is a hypercube embedded in a sphere of radius ffiffiffiffi N p . Therefore, we can interpret the Hamiltonian of a spherical, p-body, spin glass by a homogeneous polynomial of degree p. This formulation for spin systems has been studied extensively in ref. [112][113][114]. From here on, we will explicitly denote the dimension and the degree of the polynomial in a subscript.
The simplest case is when the degree is p = 1 and the polynomial (Hamiltonian) becomes where the spins (w 1 ,. . .,w n ) w A R N are constrained to the where N is the number of spins. The coefficients x i B N(0,1) are independent and identically distributed standard normal random variables. For p = 1 there exist only two stationary points, one minimum and one maximum. When p = 2 the polynomial becomes This is a simple quadratic form with 2N stationary points located at the eigenvectors of the matrix X with elements X ij x ij , with values (energies) equal to the corresponding eigenvalues. 115,116 The picture is rather different when we look at polynomials with degree p 4 2. When p = 3 the polynomial becomes The normalisation factor 1/N for coupling coefficients x ijk B N(0, 1), is chosen to make the extensive variables scale with N. In other words, when P N i¼1 w i 2 ¼ N, the variance of the Hamiltonian is proportional to N. With this convenient choice of normalization, the results of Auffinger et al. 112 show that H N,3 (w) has exponentially many stationary points, including exponentially many local minima (see ref. 117 for a complementary numerical study). In Fig. 12 we show the distribution of minima for the normalised H N,3 (w) obtained by gradient descent for various system sizes, N, and for a single realisation of the coefficients. Since the variance of the Hamiltonian scales with N, dividing by N enables us to compare energies for systems with different dimensions. The initial point is chosen uniformly at random from the sphere. The step size is constant throughout the descent, until the norm of the gradient becomes smaller than 10 À6 . For small N the energies of the minima are broadly scattered. However as the number of spins increases, the distribution concentrates around a threshold. Further details of the calculations can be found in Sagun et al. 99 (and ref. 118 for a  complementary study).
To obtain a more precise picture for the energy levels of critical points, we will focus on the level sets of the polynomial 12596 | Phys. Chem. Chem. Phys., 2017, 19, 12585--12603 This journal is © the Owner Societies 2017 and count the number of critical points in the interior of a given level set. Here, the polynomial is assumed to be nondegenerate (its Hessian has nonzero eigenvalues). Then, a critical point is defined by a point whose gradient is the zero vector, and a critical point of index k has exactly k negative Hessian eigenvalues. Finally, our description of the number of critical points will be in the exponential form and in the asymptotic, large N, limit.
where Y is the complexity function, explicitly given in ref. 112. Note that the complexity function Y is non-decreasing and it is flat above some level. This result indicates that there are no more finite index critical points at high levels, or to be more precise, it is far less probable to find them. We denote this level as ÀE N . The second crucial quantity is when the complexity becomes negative. This level has the property that there are no more critical points of a specified index below it. We denote this level by ÀE k , where k is the given index. For example, there are no more local minima below level ÀE 0 , which in turn means that the ground state is bounded from below by ÀE 0 . In particular, Y approaches a constant for u 4 À E 1 ¼ 2 ffiffiffiffiffiffiffi ffi 2=3 p % À1:633 and is bounded from below by ÀE 0 E À1.657. We therefore have a lower bound for the value of the ground state, and all stationary points exist in the energy band ÀE 0 r u r ÀE N .
An inspection of the complexity function provides insight about the geometry at the bottom of the energy landscape (Fig. 13). The ground state is roughly at u = À1.657. For u Z ÀE N we do not see any local minima, because they all have values that are within the band ÀE 0 r u r ÀE N . Moreover, in the same band the stationary points of index k 4 0 are outnumbered by local minima. In fact, this result holds hierarchically for all critical points of finite index (recall that the result is asymptotic so that by finite we mean fixing the index first and then taking the limit N -N). If we denote the Fig. 12 Histogram for the energies of points found by gradient descent with the Hamiltonian defined in eqn (14), comparing low-dimensional and highdimensional systems. ÀE 0 denotes the ground state, and ÀE N denotes the large N limit of the level for the bulk of the local minima. x-axis intercept of the corresponding complexity function Y k as ÀE k , with Below the level ÀE k the function only has critical points of index strictly less than k. This is consistent with the 'glassiness' or 'frustration' that one would expect for such a system: a random quench is most likely to locate a minimum around the ÀE N threshold and to find a lower energy minimum numerous saddle points need to be overcome. This result suggests the following scenario for finding local minima below the threshold. First identify an initial local minimum through some minimisation algorithm. Since these points are dominant at the ÀE N threshold, probabilistically speaking, the algorithm will locate one around this value. Now we wish to jump over saddle points to reach local minima with lower energies. Since the number of saddles is much less than the number of local minima below the threshold, it may take a lot longer to find them. This feature of the landscape could make finding the global minimum a relatively difficult task.
However, since basin-hopping 41-43 removes downhill barriers, this approach might still be effective, depending on the organisation of the landscape. Testing the performance of basin-hopping for such landscapes is an interesting future research direction. On the other hand, if the band (ÀE 0 , ÀE N ) is narrow, which is the case for the spherical 3-spin glass Hamiltonian described above, then it may be sufficient to locate the first local minimum and stop there, since further progress is unlikely. This scenario holds for the p-spin Hamiltonian with any p Z 3 where the threshold for the number of critical points is obtained asymptotically in the limit N -N. To demonstrate that it holds for reasonably small N Fig. 14 shows the results for the p = 3 case with increasing dimensions. The concentration of local minima near the threshold increases rather quickly.
In Fig. 15 we show disconnectivity graphs for p = 3 spin spherical glass models with sizes N A [50,100] and fixed coefficients x ijk B N(0,1). The landscape appears to become more frustrated already for small N, and the range over which local minima are found narrows with increasing system size, in agreement with the concentration phenomenon described in ref. 112.  Interestingly, the concentration of stationary points phenomenon does not seem to be limited to the p-spin system. Related numerical results 99 show that the same effect is observed for a slightly modified version of the random polynomial, ð3Þ k x ijk defined on the threefold product of unit spheres. We do not yet have an analogue of the above theorem for Ĥ, so guidance from numerical results is particularly useful.

B. Machine learning landscapes and concentration of stationary points
The concept of complexity for a given function, as defined by the number and the nature of critical points on a given subset of the domain, gives rise to a description of the landscape as outlined above. If the energy landscape of machine learning problems is complex in this specific sense, we expect to see similar concentration phenomena in the optimisation of the corresponding loss functions. In fact, it is not straightforward to construct an analogue of the y function as in eqn (15). However, we can empirically check whether optimisation stalls at a level above the ground state, as for the homogeneous polynomials with random coefficients described above.
Let D := (x 1 ,y 1 ),. . .,(x n ,y n ) be n data points with input x i A R N , and label y A R; and let G(w,x) = ŷ describe the output that approximates the label y parametrized by w A R M . Further, let c(G(w,x),y) = c( ŷ,y) be a non-negative loss function. Once we fix the dataset, we can focus on the averaged loss described by The function in eqn (17) is non-negative, but it is not obvious where the ground state is located, and an empirical study could be inconclusive. The following procedure fixes this problem.
(1) Create two identical models and split the training data in half.
(2) Using the first half of the data, train the first network, thereby obtaining a point w* with a small value of L(w*).  (3) Using w*, create new labels for the second half of the data, replacing the true labels with the output of the first model G(w*). (4) Using these new data pairs, (x,G(w*)) train the second network. This procedure ensures that the loss function for the second network over the new dataset has configurations that have exactly value zero. Simply by finding a copy of the first network, w*, the loss for the second network will be L(w*) = 0. In fact, due to the permutation symmetry in the parameters (see Fig. 1) the loss value remains zero for all the points in the correct permutations of w*. Now the optimisation on the second loss function has a known ground state at zero, and we can check empirically whether optimisation stalls above that level (Fig. 16). 99 We emphasise that the similarity between the p-spin Hamiltonian and the machine learning loss function lies only in the concentration phenomena, perhaps because the two functions share some underlying structure. It is likely that this observation of concentration in the two systems, spin glasses and deep learning, are due to different reasons that are peculiar to their organisation. It is also possible that such concentration behaviour is universal, in the sense that it can be observed in various non-convex and high dimensional systems for which the two systems above are just examples. However, if we wish to describe the ML landscape in terms of glassy behaviour, we might seek justification through the non-linearities in neural networks (the hidden layer in Fig. 1). In some sense, the nonlinearity of a neural network is the key that introduces competing forces in which neurons can be inhibitory or exhibitory. This behaviour may introduce a similar structure to the signs of interaction coefficients in spins, introducing frustration. We note that these interpretations are rather speculative at present. Another problem that requires further research is identification of all critical points, not only the ones with low index. A more systematic way to identify the notion of complexity is through finding all of the critical points of a given function. A further challenge lies in the degeneracy of the systems in deep learning. The random polynomials that we considered have non-degenerate Hessian eigenvalue spectra for stationary points at the bottom of the landscape. However, a typical neural network has most Hessian eigenvalues near zero. 119 Recently, a complementary study of the minima and saddle points of the p-spin model has been initiated using the numerical polynomial homotopy continuation method, 120,121 which guarantees to find all the minima, transition states and higher index saddles for moderate sized systems. 116,117 An advantage of this approach is that one can also analyse the variances of the number of minima, transition states, etc., providing further insight into the landscape of the p-spin model. A study for larger values of p, analysing the interpretation in terms of a deep learning network, is in progress. 118

VIII. Basin volume calculations: quantifying the energy landscape
The enumeration of stationary points in the energy landscape provides a direct measure of complexity. This quantity is directly related to the landscape entropy [89][90][91][92] (to be distinguished from the well entropy associated with the vibrational modes of each local minimum 122 ) and is crucial for understanding the emergent dynamics and thermodynamics. In this context other important questions include determining the level of the stationary points (as we discussed in Section VII) and the volume of their basins of attraction. These volumes are of great practical interest because they provide an a priori measure of the probability of finding a particular minimum following energy minimisation. This probability is particularly important within the context of non-convex optimisation problems and machine learning, where an established protocol to quantify the landscape and the a priori outcome of learning is lacking.
The development of general methods for enumerating the number of accessible microstates in a system, and ideally their individual probabilities, is therefore of great general interest. As discussed in Section VII, for a few specific cases there exist methods -either analytical or numerical -capable of producing Fig. 16 Training on linear, fully-connected MNIST networks (student networks) of various sizes. The labels for the network sizes are relative to the first network that is used to create new labels. For the larger and equally sized networks the ground state is known to lie at zero, yet the training stalls at a value around 0.016. 12600 | Phys. Chem. Chem. Phys., 2017, 19, 12585--12603 This journal is © the Owner Societies 2017 exact estimates of these numbers. However, these techniques are either not sufficiently general or simply not practical for problems of interest. To date, at least two general and practical computational approaches have been developed. 'Basin-sampling' 55 employs a model anharmonic density of states for local minima organised in bins of potential energy, and has been applied to atomic clusters, including benchmark systems with double funnel landscapes that pose challenging issues for sampling. The mean basin volume (MBV) method developed by Frenkel and co-workers 57,123-125 is similar in spirit to basin-sampling, but is based on thermodynamic integration and, being completely general, requires no assumptions on the shape of the basins (although thus far all examples are limited to the enumeration of the minima). MBV has been applied in the context of soft sphere packings and has facilitated the direct computation of the entropy in two 123,124 and three 57 dimensions. Furthermore, the technique has allowed for a direct test of the Edwards conjecture in two dimensions, 126 suggesting that only at unjamming -when the system changes from a fluid to a solid, which is the density of practical significance for many granular systems -the entropy is maximal and all packings are equally probable.
Despite the high computational cost, the MBV underlying principle is straightforward. Assuming that the total configuration volume V of the energy landscape is known (simply V = V N for an ideal gas of non-interacting atoms), if we can estimate the mean basin volume of all states, the number of minima is simply where hv basin i is the unbiased average volume of the basins of attraction. We distinguish the biased from the unbiased distribution of basin volumes because, when generating the minima following minimisation from uniformly sampled points in V, they will be sampled in proportion to the volume of the basin of attraction, and therefore the observed distribution of v basin is biased. A detailed discussion of the unbiasing procedure for jammed soft-sphere packings is given in ref. 57. The Boltzmannlike entropy of the system is then simply S B = ln O À ln N!.
Similarly, from knowledge of the biased (observed) distribution of basin volumes v i alone, one can compute the Gibbs-like (or Shannon) entropy S G ¼ À is the relative probability for minimum i. The computation of the basin volume is performed by thermodynamic integration. In essence, we perform a series of Markov chain Monte Carlo random walks within the basin applying different biases to the walkers and, from the distributions of displacements from a reference point (usually the minimum), compute the dimensionless free energy difference between a region of known volume and that of an unbiased walker. In other words where the dimensionless free energy is f = Àln v and the hat refers to quantities estimated up to an additive constant by the free energy estimator of choice, either Frenkel-Ladd 57,127 or the multi-state Bennet acceptance ratio method (MBAR). 125,128 The high computational cost of these calculations is due to the fact that, in order to perform a random walk in the body of the basin, a full energy minimisation is required to check whether the walker has overstepped the basin boundary.
Recently the approach has been validated when the dynamics determining the basin of attraction are stochastic in nature, 129 which is precisely the situation encountered in the training by stochastic optimisation of neural networks and other non-convex machine learning problems. The extension of these techniques to machine learning is another exciting prospect, as it would provide a general protocol for quantifying the machine learning landscape and establishing, for instance, whether the different solutions to learning occur with different probabilities and, if so, what their distribution is. This characterisation of the learning problem could help to develop better models, as well as better training algorithms.

IX. Conclusions
In this Perspective we have applied theory and computational techniques from the potential energy landscapes field 2 to analyse problems in machine learning. The multiple solutions that can result from optimising fitting functions to training data define a machine learning landscape, 23 where the cost function that is minimised in training takes the place of the molecular potential energy function. This machine learning landscape can be explored and visualised using methodology transferred directly from the potential energy landscape framework. We have illustrated how this approach can be used through examples taken from recent work, including analogies with thermodynamic properties, such as the heat capacity, which reports on the structure of the equilibrium properties of the solution space as a function of a fictitious temperature parameter. The interpretation of ML landscapes in terms of analogues of molecular structure and transition rates is an intriguing target for future work.
Energy landscape methods may provide a novel way of addressing one of the most intriguing questions in the machine learning research, namely why does machine learning work so well? One way to ask this question more quantitatively is to investigate why we can usually find a good candidate for the global minimum of a machine learning cost function relatively quickly, even in the presence of so many local minima. The present results suggest that the landscape for a range of models are single-funnel-like, i.e. the largest basin of attraction is that of the global minimum, and the downhill barriers that separate it from local minima are relatively small. This organisation facilitates rapid relaxation to the global minimum for global optimisation techniques, such as basin-hopping. Another possible explanation is that many local minima provide fits that are competitive with the global minimum. 96,97,99 In fact, these two scenarios are compatible, so that global optimisation leads us downhill on the landscape, where we encounter local minima that provide predictions or classifications of useful accuracy.