Over the last forty years many computer simulations of water have been performed using rigid non-polarizable models. Since these models describe water interactions in an approximate way it is evident that they cannot reproduce all of the properties of water. By now many properties for these kinds of models have been determined and it seems useful to compile some of these results and provide a critical view of the successes and failures. In this paper a test is proposed in which 17 properties of water, from the vapour and liquid to the solid phases, are taken into account to evaluate the performance of a water model. A certain number of points between zero (bad agreement) and ten (good agreement) are given for the predictions of each model and property. We applied the test to five rigid non-polarizable models, TIP3P, TIP5P, TIP4P, SPC/E and TIP4P/2005, obtaining an average score of 2.7, 3.7, 4.7, 5.1, and 7.2 respectively. Thus although no model reproduces all properties, some models perform better than others. It is clear that there are limitations for rigid non-polarizable models. Neglecting polarizability prevents an accurate description of virial coefficients, vapour pressures, critical pressure and dielectric constant. Neglecting nuclear quantum effects prevents an accurate description of the structure, the properties of water below 120 K and the heat capacity. It is likely that for rigid non-polarizable models it may not be possible to increase the score in the test proposed here beyond 7.6. To get closer to experiment, incorporating polarization and nuclear quantum effects is absolutely required even though a substantial increase in computer time should be expected. The test proposed here, being quantitative and selecting properties from all phases of water can be useful in the future to identify progress in the modelling of water.