Irreproducibility in hydrogen storage material research

Abstract

The storage of hydrogen in materials has received a significant amount of attention in recent years because this approach is widely thought to be one of the most promising solutions to the problem of storing hydrogen for use as an alternative energy carrier in a safe, compact and affordable form. However, there have been a number of high profile cases in which erroneous or irreproducible data have been published. Meanwhile, the irreproducibility of research results in a wide range of disciplines has been the subject of an increasing amount of attention due to problems with some of the data in the literature. In this Perspective, we provide a summary of the problems that have affected hydrogen storage material research. We also discuss the reasons behind them and possible ways of reducing the likelihood of further problems occurring in the future.

---

D. P. Broom

Darren Broom is a product manager for Hiden Isochema Ltd in the UK. He received a PhD in materials physics from the University of Salford, UK, in 2003 and was a postdoctoral research fellow at the European Commission's Institute for Energy in the Netherlands from 2004 to 2007. He is the author of “Hydrogen Storage Materials: The Characterisation of Their Storage Properties”, a book published by Springer in 2011, and is currently a UK representative on the International Energy Agency Hydrogen Implementing Agreement (IEA-HIA) Task 32 “Hydrogen-based Energy Storage”.

M. Hirscher

Michael Hirscher is group leader in “hydrogen storage” at the Max Planck Institute for Intelligent Systems, Stuttgart, Germany. He studied physics at the University of Stuttgart, Germany, and at Oregon State University, Corvallis, USA. For his achievements during his doctoral studies he was awarded the Otto Hahn Medal of the Max Planck Society in 1988. Prior to taking his position in Stuttgart, he spent a post-doctoral fellowship at the University of Pennsylvania, Philadelphia, USA. Recently he edited the “Handbook of Hydrogen Storage”. Since 2013 he has been the Operating Agent for International Energy Agency Hydrogen Implementing Agreement (IEA-HIA) Task 32 “Hydrogen-based Energy Storage”. In 2015 he received the Hydrogen & Energy Award in Switzerland.

---

Broader context

Irreproducibility has been attracting a growing amount of attention recently in disciplines as diverse as computer science, psychology and biomedical research. Many reasons lie behind the problems but there is little doubt that researchers are often under intense pressure to find and report new breakthroughs in their field. Hydrogen storage material research is a prime example of this, since new materials could provide the solution to the problem of the safe and efficient storage of hydrogen and their discovery would bring both rich rewards and kudos. In the rush to identify new candidates, however, a number of claims of high storage capacities have been published for which the data have later been shown to be irreproducible. Publication of such results wastes time, effort and money, as other groups work to reproduce the results and research grants are awarded on the basis of flawed data. In light of the above, this Perspective examines some of the cases of irreproducibility that have affected hydrogen storage material research, and discusses both the attempts to address the issue and possible ways forward for the future.

Introduction

The storage of hydrogen in a compact, safe and affordable form is considered to be one of the technical barriers currently preventing its widespread adoption as an alternative energy carrier, primarily in fuel cell cars. Most of the current demonstration vehicles, and those now being delivered to the first customers, use compressed hydrogen in pressurised vessels¹ but this approach has limitations in terms of the volumetric and gravimetric storage densities that can be achieved at practical pressures. Another alternative, liquid hydrogen storage, requires low temperatures around 20 K and has now largely been abandoned,² although it has been seriously considered in the past, most notably by BMW AG (Munich, Germany).³ One of the main reasons for the move away from pure liquid H₂ storage is the occurrence of boil-off, which results in the loss of hydrogen from the tank with time, after a limited dormancy period. The use of low temperature storage tanks is also rather expensive because of the required insulation.²

One of the most promising solutions is thought to be solid state hydrogen storage^4–6 and this topic has thus attracted a vast amount of attention. Fig. 1, for example, shows the increase in the annual number of publications on hydrogen storage materials since 1991. In the rush to find and report new breakthroughs in this field, however, there have been a number of high profile cases in which erroneous or irreproducible data have been published. This has led the hydrogen storage material research community down a number of expensive and time-consuming blind alleys.

Fig. 1 A plot of the annual number of publications on hydrogen storage materials since 1991, determined using the search terms ‘hydrogen storage’, ‘hydrogen adsorption’ and ‘hydrogen absorption.’ The search was performed on the title, abstract and keywords of papers in the Science Citation Index Expanded, Conference Proceedings Citation Index – Science, and Book Citations Index – Science of the Web of Science by Thomson Reuters.

Meanwhile, questions surrounding reproducibility and the replication of results, the cornerstone of science,^7,8 in a diverse range of research fields have been attracting an ever-increasing amount of attention. A number of recent articles and editorials, for example, in prominent journals have focused on this issue,^9–21 and a piece published in Science in 2014, covering a dispute in the field of social psychology, gave examples of initiatives in biology, psychology and computer science, amongst others, aimed at the replication of research.²² A large scale replication study in the field of psychology was published in Science the following year, as part of one such initiative,²³ and an interesting discussion of the issues in biomedical research – many of which have wider implications – can be found in a report from a recent UK Academy of Medical Sciences symposium.²⁴ Efforts to address the amount of irreproducible data in the scientific literature therefore seem to be gaining momentum.

The problems that have occurred in hydrogen storage material research, which is itself interdisciplinary, involving materials scientists, physicists, chemists and chemical engineers, provide another practical illustration of the ease with which irreproducible data can creep into the literature. In this article, we therefore provide an overview of these problems, with three main aims. Firstly, to provide an informative overview for both existing researchers and newcomers to the field; secondly, to offer a cautionary tale in order to help reduce the likelihood of further errors occurring in the future; and, thirdly, to provide a case study that will hopefully contribute constructively to the recent literature on the irreproducibility of data and problems regarding the replication of research.

We begin by emphasising the importance of reproducibility. We then provide an introduction to potential hydrogen storage materials and hydrogen sorption measurement techniques. In the following section, we describe some of the instances of irreproducibility that have occurred in hydrogen storage material research. We then discuss recent attempts to investigate and address the problems, and the reasons that lie behind them, before concluding by considering potential solutions and proposing a way forward for the future.

The importance of reproducibility

As noted above, reproducibility and the replication of results is a cornerstone of science. The use of experiment has a long history²⁵ but various events and developments, including the long-running debate between Boyle and Hobbes in the 17th century,²⁶ have led to widespread acceptance of the central role that the independent experimental testing of results plays in science. The ability of scientists to replicate the results of others has thus become a guiding principle of the scientific endeavour.²⁷

It would be difficult to express this more clearly than Sir Karl Popper, who wrote in The Logic of Scientific Discovery,²⁸ that:

“Only when certain events recur in accordance with rules or regularities, as is the case with repeatable experiments, can our observations be tested – in principle – by anyone. We do not take even our own observations quite seriously, or accept them as scientific observations, until we have repeated and tested them. Only by such repetitions can we convince ourselves that we are not dealing with a mere isolated ‘coincidence’, but with events which, on account of their regularity and reproducibility, are in principle inter-subjectively testable.”

and,

“Thus I may be utterly convinced of the truth of a statement; certain of the evidence of my perceptions; overwhelmed by the intensity of my experience; every doubt may seem to me absurd. But does this afford the slightest reason for science to accept my statement? Can any statement be justified by the fact that K. R. P. is utterly convinced of its truth? The answer is ‘No’; and any other answer would be incompatible with the idea of scientific objectivity.”

He later reiterated the point that “non-reproducible single occurrences are of no significance to science.”

Unfortunately, however, there has been a tendency in recent years to undervalue the replication of previously reported results, in favour of the publication of attention-grabbing data; despite the fact that if any of these results later turn out to be, in the words of Popper, “non-reproducible single occurrences” then they “are of no significance to science.” In light of the above, in this article we shall look in more detail at problems with the reproducibility of some of the research into hydrogen storage materials.

Potential hydrogen storage materials

Potential hydrogen storage materials can be classified according to the type of binding involved in the hydrogen sorption process.² The different types thus include interstitial metal hydrides,^29,30 complex hydrides,^31–33 salt-like metal hydrides³⁴ and nanoporous adsorbents.^35–41

Interstitial metal hydrides include those formed from elemental metals, such as PdH_x and TiH₂, and AB₅, AB₂ and AB intermetallic compounds, where A and B are hydride-forming and non-hydride-forming elements, respectively.^29,30,42 The gravimetric hydrogen storage capacities of most elemental metals are rather low, approximately 0.6 wt% for PdH_x, for example. TiH₂ has a higher capacity (∼4 wt%), but its decomposition temperature is >770 K, which is too high for mobile storage applications. Some intermetallic compounds, such as LaNi₅, multicomponent materials based on TiMn₂, and TiFe, can reversibly absorb hydrogen at near ambient temperatures and pressures but their reversible gravimetric capacities are generally limited to ∼2 wt%. Many of these materials, however, have remarkably high volumetric storage densities that can exceed that of liquid H₂.

Complex hydrides, meanwhile, include sodium alanate (NaAlH₄), lithium borohydride (LiBH₄), and the Li–N–H system.^32,33 These materials have impressive gravimetric hydrogen storage capacities, but they require higher temperatures for operation. NaAlH₄, for example, has a reversible capacity of 5.6 wt% at temperatures around 420 K, providing it is doped with Ti.³¹ LiBH₄ and the Li–N–H system have total capacities of 18.5 wt% and 10.5 wt%, respectively, but only a fraction of this is reversible, and the temperatures required are even higher than that of NaAlH₄.

The main salt-like metal hydride is MgH₂. It has a high gravimetric storage capacity of 7.6 wt%, but the thermodynamics of hydride formation and decomposition limit its operation to temperatures above 570 K. The kinetics are also rather slow although they can be improved by nanostructuring via ball-milling, for example, together with the use of additives, such as V₂O₅ or Nb₂O₅.^34,43

Finally, nanoporous materials include zeolites,⁴⁴ porous carbons,^39,45,46 Metal–Organic Frameworks (MOFs),^35–38,41 and microporous organic polymers.^40,47 These materials have high surface areas and store hydrogen in molecular form (H₂) adsorbed on their internal surfaces via physisorption. High gravimetric capacities can be achieved using H₂ adsorption at low temperatures, around 77 K, providing the material has a high surface area and large pore volume. For example, MOF-177, which has a BET area of ∼4600 m² g⁻¹ and a pore volume >1.5 cm⁻³ g⁻¹, has a saturation (excess) uptake of around 7.5 wt% at 7.0 MPa and 77 K.⁴⁸ The volumetric capacities of nanoporous adsorbents are relatively low, in comparison to hydrides, and their gravimetric capacities at ambient temperature do not generally exceed 2 wt% even at 20 MPa; however, they have some significant advantages such as rapid kinetics and the full reversibility of the H₂ adsorption and desorption process.^49,50

Research in recent years has focussed on the use of new compounds and nanoporous materials for hydrogen storage, but also on improving the hydrogen storage properties of each of these material types using different methods. A prime example is the use of nanoconfinement to prepare nanocrystals that can be stabilised in porous scaffolds. This can alter the kinetics and thermodynamics of the interaction of hydrogen with different metal or complex hydrides.^33,51 An important part of this work involves the accurate characterisation of the hydrogen sorption properties of each material, using one of a number of techniques that we will now describe.

Hydrogen sorption measurement

The amount of hydrogen absorbed or adsorbed by a material can be quantitatively determined using a number of techniques. They can be broadly separated into three categories: volumetric or manometric, gravimetric, and temperature-programmed desorption. Details of these methods can be found elsewhere⁶ but we will briefly summarise the principles below.

Volumetric or manometric measurements typically involve the determination of hydrogen uptake by measuring the drop in pressure in a system of a fixed, known volume and applying appropriate molar balance expressions using the real gas law. This approach is also known as Sieverts' method. It is susceptible to a range of errors including those originating from poorly calibrated internal volumes, leaks in the apparatus, poor temperature control or stability, inaccurate pressure measurement, and the accumulative errors associated with isotherm determination using multiple gas doses.⁵² Furthermore, the sample size must be large enough when making measurements in a system of a given volume.⁵³ Instruments that implement this technique are commercially available, but it is important not to treat them as “black boxes” due to the errors that are inherent to this measurement type. A variation, which, to the best of our knowledge, is not currently available commercially, is the differential volumetric technique.^54,55 This approach exploits the higher accuracy of differential pressure transducers compared to their absolute counterparts, and involves charging two arms of the apparatus, one containing the sample and the other containing a non-interacting “dummy” material of matching volume. The pressure differential between the arms then provides a measure of the amount of hydrogen adsorbed or absorbed by the material.

Gravimetric measurements typically involve the use of a microbalance to determine hydrogen uptake by measuring the weight change of a sample exposed to different pressures of H₂ gas. This technique is also susceptible to a range of errors, which can originate from instability of the microbalance and the sorption of gas phase impurities that will result in a larger weight signal per mole than H₂ itself. Furthermore, the raw weight data must be carefully corrected for buoyancy effects that change as a function of pressure, although it is worth noting that the equivalent in the volumetric or manometric case are the dead volume corrections that are necessary to account for the presence of the sample in the sample cell. In the gravimetric case, the sample size is limited by both the capacity and resolution of the microbalance; in practice, the latter is defined by the long term stability of the weight signal in the instrument, including both random variations, as a function of time, and drift. Gravimetric instruments are also available commercially and, again, should not be treated as “black boxes”. A cruder approach to gravimetric measurement is to simply measure the change in weight of an isolated sample cell at the chosen hydrogen pressure.⁵⁶

Temperature-Programmed Desorption (TPD), or Thermal Desorption Spectroscopy (TDS), involves the application of a temperature ramp to a sample that has previously been exposed to H₂, and the detection of the amount of hydrogen subsequently desorbed from the material, typically using a quadrupole mass spectrometer.^57–59 The quantification of the amount of hydrogen desorbed, in this case, requires calibration of the mass spectrometer signal. Errors can be introduced into the measurement by inaccurate calibration, but one of the key advantages of TPD or TDS is the small sample size that can be used. This is only limited, in principle, by the sensitivity of the mass spectrometer signal in any given system. A practical limitation is the amount of sample that can be weighed and handled with sufficient accuracy; however, measurements have been reported on small samples of the order of milligrams.^57–59 Other approaches to temperature-programmed measurements are also possible, including Thermogravimetric Analysis (TGA) type experiments, the determination of the pressure increase in a closed system following the application of a temperature ramp, and the use of a mass flow meter to measure the quantity of desorbed hydrogen.⁶⁰

Irreproducibility in hydrogen storage material research

Nanostructured carbons

In the late 1990s, a number of reports of impressive hydrogen storage capacities for carbon nanostructures appeared in the literature. Firstly, in 1997, it was claimed that single-walled carbon nanotubes have a potential hydrogen storage capacity at ambient temperature of 5–10 wt%.⁶¹ This was a spectacular result that caused much excitement because it offered the promise of solving the hydrogen storage problem. To put this figure in context, the US Department of Energy (DOE) had recently set hydrogen storage targets, which included a gravimetric hydrogen storage capacity of 6.5 wt%.⁶² The paper therefore suggested that nanotubes could represent a major breakthrough. However, there were problems with the data, to the extent that the reported results could not be reproduced or, rather, the high reported capacities could not be independently confirmed. The figure of 5–10 wt% was the result of a rather questionable extrapolation because the sample itself was not pure. It consisted of an estimated 0.1–0.2 wt% of single-walled carbon nanotubes, the rest being uncharacterised soot and Co nanoparticles. The data presented in support of their conclusion were TPD spectra, and it was assumed that the desorbed hydrogen was stored only on the nanotubes, which constituted a very small proportion of the entire sample. Any contribution from the Co nanoparticles was also discounted. Other reports then rapidly followed and this led to what Harris described, in his 2009 book on carbon nanotubes,⁶³ as “the most controversial episode in nanotube science.” Amongst the subsequent work, a collaborative research project funded by the Federal Ministry for Education and Research in Germany, the BMBF, aimed at reproducing these results. The conclusion was that the uptake of hydrogen by single-walled carbon nanotubes was instead less than 1 wt% at ambient temperature.⁶⁴

In 1998, an even more spectacular result was published.⁶⁵ The claim was that carbon nanofibres, which are closely related to multi-walled carbon nanotubes, could store up to ∼67 wt% of hydrogen at ambient temperature, but other groups were unable to reproduce this incredible figure.^66–68 In this case, the problems appear to be due mainly to a lack of care taken in the performance of the hydrogen sorption measurements, which were made volumetrically. In their report, Chambers et al.⁶⁵ validated their measurements using a number of metal hydrides. However, given the erroneous nature of the main carbon nanofibre results, it is notable that the sorption data reported for the hydrides were themselves inaccurate. For example, they reported an uptake of 2.07 wt% for Pd, which is known to have a hydrogen storage capacity of less than 0.6 wt%,³ and an uptake for MnNi_4.5Al_0.5 of 3.33 wt%, which is, again, far greater than the known figure of 1.2 wt%.²⁹

Boron nitride nanotubes

In the following decade, a series of other results in the hydrogen storage literature were also found to be irreproducible, although it is worth noting that none of these attracted quite as much attention. In 2002, an uptake of 4.2 wt% at 10 MPa and ambient temperature by “collapsed” Boron Nitride (BN) nanotubes was reported.⁶⁹ The measurements were performed gravimetrically. It was later acknowledged, however, by the same authors, that the results were affected by the presence of Pt catalyst particles in the samples.⁷⁰ Later work has concluded that pure BN nanotubes are unlikely to be good hydrogen storage materials, with an uptake of 1.2 wt% at ∼3 MPa and 77 K being reported by Terao et al.⁷¹ for a surface-modified sample.

Conducting polymers

Also in 2002, it was reported that HCl-treated conducting polymers, polyaniline and polypyrrole, could store 6–8 wt% of hydrogen at ambient temperature, but these results could also not be reproduced.⁷² Work in this area has been pursued by other authors, however, with questionable data published in a number of papers, including an uptake of 2.2 wt% at 9 MPa and 77 K, for a mesoporous sample with a specific surface area of ∼50 m² g⁻¹.⁷³ This is notable because the hydrogen adsorption capacities of porous materials at 77 K typically follow the trend of approximately 1 wt% per 500 m² g⁻¹.^74,75 This is often called Chahine's rule.^39,76,77

Metal–Organic Frameworks

In 2003, the first report of hydrogen storage in Metal–Organic Frameworks appeared.⁷⁸ The uptake of MOF-5 (or IRMOF-1) was found to be 4.5 wt% at ambient pressure and 77 K; however, this was reduced to 1.3 wt%, under the same conditions, in a subsequent paper.⁷⁹ In this case, contaminated hydrogen was suspected as the cause of the problem, and the erroneous nature of the original result was also confirmed independently.⁸⁰ Later work revealed that significantly different uptakes are found for MOF-5 samples that have been synthesised and handled differently,⁸¹ although we should emphasise that the initial result has not been reproduced.

Spillover

The latest chapter in this story involves a mechanism known as spillover, which is a well-documented, but poorly understood, phenomenon in heterogeneous catalysis.⁸² It involves the dissociation of molecular hydrogen (H₂) on catalytically active particles and the subsequent migration of atomic hydrogen (H) on to a solid state support that would not otherwise adsorb or absorb H under the same conditions.

In 2002, the first report of the use of spillover for hydrogen storage in carbons was published.⁸³ The storage capacity of multi-walled carbon nanotubes was reported to be below detection limits without a catalyst but increased to 0.6 wt% when a catalyst was present. Since then, Yang and various co-workers have published a series of papers on the use of this phenomenon to enhance the hydrogen storage properties of various nanoporous materials, including MOFs, Covalent Organic Frameworks (COFs) and zeolites.^84,85 Large increases in the ambient temperature hydrogen storage capacity of materials using the spillover mechanism have been claimed. According to analysis of the literature by Luzan and Talyzin,⁸⁶ the increases have ranged from a factor of 1.5 to 10. This is also shown in Fig. 7 of Wang et al.⁸⁵ The reported isotherms are invariably linear up to a pressure of ∼10 MPa – see, for example, the various datasets presented in the review by Wang and Yang.⁸⁴ It has been claimed that this is a positive feature of hydrogen storage using spillover^84,87 because it means that the use of higher pressures will result in higher hydrogen storage capacities; however, it is difficult to explain because some curvature should be expected at these pressures. Otherwise, the hydrogen storage capacity of a material would continue indefinitely with increasing pressure, which is physically unreasonable. Furthermore, the process is purported to be fully reversible at ambient temperature, which implies that no strong C–H bonds are formed.

Amongst the many studies were two papers published in 2006.^87,88 These are now highly cited and reported a significant increase in hydrogen uptake by MOFs that had been mixed with a commercial activated carbon/Pt catalyst with carbonised sucrose used to bridge the particles.^87,88 Uptakes of up to 4 wt% at 10 MPa and ambient temperature were claimed. Other groups have tried to replicate these results; however, no enhancement was found and no spillover occurred within the experimental detection limit.^86,89 Similarly, for doped carbon, no significant enhancement has been found by other authors.^90,91

The dispute has not yet been fully resolved, since no samples have been exchanged or measured in independent laboratories; however, independent investigation of the microscopic nature of the purported phenomenon in doped carbons using different techniques has found only a small amount of hydrogen uptake that can be attributed to the formation of water, surface hydroxyls, or C–H bonds in the vicinity of the noble metal particles.^92,93 For example, room temperature hydrogen uptake at sub-ambient pressure by Pd-doped templated carbon was found by Ghimbeu et al.⁹⁴ to occur via both PdH_x formation and the reduction of PdO to form Pd and water. No long-range hydrogen diffusion on the surface or technologically relevant hydrogen uptake has been found.⁹² Furthermore, the enhanced room temperature uptake of hydrogen by a Pd-doped COF was attributed by Kalidindi et al.⁹⁵ to the hydrogenation of residual organic compounds present on the surface of the Pd nanoparticles.

The lack of detailed microstructural characterisation of the samples in the studies by Yang and co-workers was emphasised by Luzan and Talyzin.⁹⁶ More specifically, the claimed “bridging” between catalytic particles and the host nanoporous material has not been practically demonstrated or evidenced using appropriate microscopic techniques.

Unfortunately, the spillover story is not yet over. This can be seen, for example, in the uncritical support still being given to the idea in recent work.⁹⁷ However, we consider it to be yet another clear example of the presence of irreproducible hydrogen sorption data in the literature, particularly in view of the wide range of reported capacity increases, as noted by Luzan and Talyzin.⁸⁶

Reproducibility studies

Compared to the extensive literature on hydrogen storage, in general, published studies on the reproducibility of results, in terms of the experimental investigation of hydrogen uptake by the same material in multiple laboratories, are relatively scarce. One of the earliest examples that we are aware of predates the controversy over hydrogen storage by carbon nanostructures, and followed the observation that there was considerable variation in the measured hydrogen sorption kinetics of hydriding alloys.⁹⁸ Data measured by seven different research groups in Japan showed considerable variation between the results obtained for the same material (LaNi_4.9Al_0.1) under ostensibly the same conditions. It is worth noting, however, that this did not result in dramatically different predicted total uptakes (or storage capacities), although there was some variation, with the measured hydrogen-to-metal ratios ranging between 0.9 and 1.12. The reported single step kinetic measurements are not necessarily the most accurate way of determining total equilibrium uptakes so these errors of >20% are perhaps not quite as striking as they would be using more accurate approaches. It nevertheless provides an excellent illustration of the care that must be taken in measuring hydrogen uptake.

As described above, following the initial reports of the high potential storage capacity of carbon nanotubes, a study was performed as part of a collaborative project funded by the BMBF,⁶⁴ which was unable to reproduce the earlier work. Around the same time, Ansón et al.⁹⁹ also published a comparative study using three different techniques – two gas phase (volumetric and gravimetric) and one electrochemical – in three different laboratories on a single-walled carbon nanotube material that reached similar conclusions. This work reported good general agreement between the techniques, with a very low value of 0.01 wt% found at room temperature using the gas phase techniques; although higher values of 0.1–0.3 wt% were found electrochemically so a significant percentage error was still present. In 2005, Rzepka et al.⁶⁸ used three different gas phase techniques – two volumetric variants and one gravimetric – to investigate the uptake of hydrogen at room temperature by various carbon nanofibre samples, including six batches supplied by Rodriguez and Baker, two of the authors from the original 1998 publication.⁶⁵ Low uptakes, which were all below 0.4 wt%, were found using the two high accuracy approaches at pressures up to 14 MPa and ambient temperature. Two years later, in 2007, a comparative study of hydrogen adsorption by the nanoporous material MOF-177 was published, with good agreement found between data measured using different techniques (volumetric and gravimetric) in different laboratories.⁴⁸

In 2009, the first of two interlaboratory exercises performed as part of the European Commission's 6th Framework NESSHY (Novel Efficient Solid Storage for H₂) project was published.¹⁰⁰ This work examined the reproducibility of hydrogen adsorption measurements on the commercial carbon molecular sieve, Takeda CMS 4A, which was also used for a similar exercise related to the characterisation of its surface area and pore structure properties.¹⁰¹ In contrast to the report of Furukawa et al.,⁴⁸ which involved only two laboratories, the results in this first NESSHY report, involving 14 different European laboratories, showed a very significant variation. The data for both 77 K and ambient temperature are shown in Fig. 2. The second study was published in 2013.¹⁰² In this case, the interlaboratory exercise involved a MgH₂-based material and 14 laboratories, mostly in Europe but also two in the US, one in China and one in Japan. The spread in the data was not as dramatic, but there was still a rather surprising variation for such a well-studied material. Some of the data are shown in Fig. 3. It can be seen that the total reported uptakes, under essentially the same conditions, at two different temperatures, 553 K and 593 K, were 5.1–6.4 wt% and 5.3–6.6 wt%, respectively. This is a significant uncertainty for measurements made on a specific material from the same batch. Interestingly, this variation is of a similar magnitude to that reported by Wang and Suda.⁹⁸ It would seem likely that if similar materials were prepared by different groups, thus introducing subtle differences in both the preparation and handling of samples, then the spread in the resultant data could be even greater. It is worth noting that, in contrast to the kinetic study reported by Wang and Suda,⁹⁸ the equilibrium sorption measurement techniques used in the paper by Moretto et al.¹⁰² are the conventional methods used in the field,⁶ as described in Section 4. In addition, Moretto et al.¹⁰² reported kinetic measurements that also showed significant variation.

Fig. 2 Hydrogen adsorption data for Takeda CMS 4A, measured at (a) ambient temperature and (b) 77 K in different European laboratories.¹⁰⁰ The numbers in the legends indicate each anonymous laboratory, which were numbered 1 to 14. (Reprinted from C. Zlotea, P. Moretto, T. Steriotis, A Round Robin characterisation of the hydrogen sorption properties of a carbon based material, Int. J. Hydrogen Energy, 34(7), 3044–3057, Copyright (2009), with permission from Elsevier).

Fig. 3 Hydrogen absorption data for a doped MgH₂ sample measured at (a) 553 K and (b) 593 K in different laboratories in Europe, the US, China and Japan.¹⁰² The numbers in the legends indicate each anonymous laboratory. (Reprinted from P. Moretto, C. Zlotea, F. Dolci, A. Amieiro, J.-L. Bobet, A. Borgschulte, D. Chandra, E. Enoki, P. De Rango, D. Fruchart, J. Jepsen, M. Latroche, I. Llamas Jansa, D. Moser, S. Sartori, S. M. Wang, J. A. Zan, A Round Robin Test exercise on hydrogen absorption/desorption properties of a magnesium based material, Int. J. Hydrogen Energy, 38(16), 6704–6717, Copyright (2013), with permission from Elsevier).

The most recent addition to the literature is a multi-laboratory investigation published this year,¹⁰³ which reported H₂ adsorption measurements on two porous carbons and found much better agreement between datasets than the 2009 study by Zlotea et al.¹⁰⁰ The authors note that “unlike Zlotea's effort, all of the participating laboratories involved have extensive experience and a thorough understanding of their instrument-specific measurement sensitivities for manometric hydrogen sorption capacity characterization.” However, it is worth noting that one of the four participating laboratories reported a skeletal density of 3.686 g cm⁻³, which is greater than the literature value for diamond (∼3.5 g cm⁻³), while another used the ideal gas law to represent the behaviour of H₂ up to 10.0 MPa, which might be expected to result in large measurement errors.^6,52 Nevertheless, good agreement between the data in the study¹⁰³ was reported.

Reasons behind the problems

The reasons behind the irreproducibility of the data discussed above are most likely manifold. The susceptibility of hydrogen sorption measurements to a number of different sources of error^6,52 is undoubtedly partly to blame. However, the sensitivity of hydrogen sorption measurements to differences between samples of essentially the same material has also played a significant role. The presence of impurities in samples, for example, which can interact chemically or physically with hydrogen in different ways under different conditions of temperature and pressure, can significantly affect the measured hydrogen uptake, or the conditions under which a particular material will adsorb or absorb hydrogen.

Methodological errors

Some of the erroneous data in the carbon nanotube story summarised above appear to be due to simple methodological errors in the hydrogen uptake measurements. Another prominent study,¹⁰⁴ for example, claimed uptakes of 14 and 20 wt%, for potassium and lithium-doped carbon nanotubes, respectively, but these high capacities were later shown to be most likely due to moisture contamination in the hydrogen supply.^105,106 More specifically, the alkali salts used for doping transform into highly hygroscopic alkali oxides that will readily react with water.¹⁰⁷ The lesson learned in this case, therefore, was that the gas supply used in hydrogen sorption studies must always be of very high purity.¹⁰⁸

On the more general topic of the accuracy of the common characterisation techniques, a number of recent studies have investigated the practical issues relating to measurement error or uncertainty. In 2014, for example, papers were published that analysed the effects of measurement uncertainty¹⁰⁹ and inaccurate volume calibrations¹¹⁰ on hydrogen uptake measurements performed manometrically (or volumetrically), which was one of the experimental approaches used in the NESSHY interlaboratory studies. This followed earlier work in which the sensitivity of results to the assumed or measured density of a material was investigated.¹¹¹ The key point in the latter case is that the errors increase with decreasing sample density. Carbon nanotubes and nanofibres, carbon molecular sieves and MOFs are all low density materials. It is also worth noting that errors associated with the density of samples can have a significant effect on high pressure gas adsorption measurements, in general,¹¹² so this could well be a more widespread problem. Work has also been published on measurement uncertainties in gravimetric measurements.¹¹³ It seems possible that any of the error or uncertainty sources addressed in the papers referenced above could have contributed to some of the erroneous data published previously in the literature, and that they could also have played a role in the spread of the data found in some of the interlaboratory studies.

Sample purity

Sample purity can be affected by the presence of chemical impurities or different phases, for example, precipitates of differing structure and stoichiometry or amorphous regions in an otherwise crystalline material. Lattice imperfections, such as vacancies, dislocations and grain boundaries, can also alter the behaviour of crystalline materials, together with their crystallinity, which can vary from nanocrystalline to large single crystals, depending on the material and its preparation. Problems relating to sample purity are perhaps more difficult to treat in general terms and are unique to each case. The original single-walled carbon nanotube samples of Heben and co-workers,⁶¹ for example, contained both uncharacterised soot and Co nanoparticles, both of which could interact with hydrogen. Later, the same group began sonicating their samples using a Ti alloy horn, which resulted in the deposition of the alloy onto the samples. The higher temperature peak in the hydrogen TPD spectra of the sonicated samples was later shown to be due to hydrogenation of the contaminant, rather than the nanotubes themselves.^64,114

Differences observed in the uptakes of different MOF samples⁸¹ are also essentially related to sample purity. In the case of MOF-5, this can involve framework interpenetration and the formation of Zn hydroxide species, for example, in the pore network.¹¹⁵ Both of these can be considered another type of impurity. Significant differences between samples, in this case, can be assessed to a certain extent by measuring the N₂ BET areas of different samples. This is a common characterisation method for nanoporous materials¹¹⁶ and is a key indicator of the amount of hydrogen they will adsorb.³⁶ The reported surface areas of different MOF-5 samples differ considerably. Hafizovic et al.,¹¹⁵ for example, report a range from 700 to 3400 m² g⁻¹. In contrast, the theoretical accessible surface area for H₂ of this material is 3882 m² g⁻¹.¹¹⁷

The thermodynamics of the hydrogen absorption process in hydride-forming alloys and intermetallics are also very sensitive to the elemental composition of the host material.⁶ This therefore provides another example of the potential effect that impurities can have on the hydrogen sorption properties of a material. Surface contamination, including poisoning or oxidation, is a further example. The surface state of a material was one of the important factors identified by Wang and Suda.⁹⁸ On a related note, the handling of air-sensitive samples is a crucial consideration in the study of complex hydrides for hydrogen storage. Errors relating to sample handling, in this case, can therefore not be ruled out.

‘Publish or perish’ and other pressures

Despite the sensitivity of hydrogen sorption measurements to error and differences between samples, there are clearly other factors at play too. The role of the current academic climate (‘publish or perish’)^118,119 can probably not be ignored. The perceived need to work in this manner does not encourage researchers to exhaust all obvious and likely sources of error before publishing data, and to perform further complementary measurements and checks, whenever appropriate; although, given the variation seen in the data from some of the recent interlaboratory exercises described above, it is almost certainly not sufficient in isolation to explain the problems because in the case of the interlaboratory studies there was presumably no rush to publish. The studies just needed to be performed and the results prepared for publication. Furthermore, this is certainly not unique to researchers working in hydrogen storage. Another possible culprit could be the pressure of competition. Solving the hydrogen storage problem by discovering a breakthrough material would certainly bring rich rewards and kudos but, again, this is not unique to hydrogen storage. Nevertheless, either of these pressures could potentially lead, for example, to wishful thinking and it does not seem to be too much of a stretch to implicate this as a possible reason behind some of the more optimistic, and ultimately irreproducible, results that have been reported.

Improving reproducibility in the future

The issue of reproducibility will always be present because it is an integral part of the scientific process. As Begley and Ioannidis¹²⁰ say: “It is reasonable to expect that there will be some level of uncertainty and irreproducibility as investigators genuinely push the boundaries of current knowledge and extend into the unknown.” However, it should be possible to reduce the amount of irreproducible data that appears in the literature by avoiding, for example, the repetition of straightforward methodological mistakes. Work carried out on measurement methodology is important in this respect. A recent US DOE project aimed to address this issue¹²¹ and a book written by one of us covers the main experimental considerations in the accurate measurement of hydrogen sorption.⁶ Increased vigilance by reviewers would also be welcome. Despite the problems we have documented above, data that are clearly erroneous can still be found in the recent literature.

In a 2007 review, Thomas¹⁰⁸ noted that “some of the early work on hydrogen adsorption on porous materials did not include enough information to confirm the validity of the measurements.” This is a crucial point, and the more information provided by authors, the greater the likelihood that other researchers will be able to reproduce the results. In many fields^122–125 there are stipulated sets of information that must be provided at a minimum, but no such list yet exists for hydrogen sorption measurements. It is therefore up to authors to decide what information to include and for editors and reviewers to assess whether any given manuscript contains enough detail. This situation is bound to result in inconsistencies. Furthermore, in some cases, the hydrogen sorption measurements may constitute only a small part of a manuscript that also includes synthesis, crystallographic structure determination or microstructural characterisation, computer simulations, gas adsorption characterisation, and so forth, and it is perhaps unreasonable to expect every reviewer to possess expertise in each of these areas; although authors should anyway bear the brunt of the responsibility.

In another review article, published the following year, Férey¹²⁶ noted with regard to hydrogen storage in MOFs that:

“…every published value, even unprecedented, must be taken with care and needs to be verified by other groups before becoming credible, using for instance round robin procedures, as is done in other disciplines prior to acceptance…For the future, there is an urgent need of normalization, with a complete set data including isotherms of adsorption (classical) and desorption (currently rare), gas capacity at high pressure (wt%; cm³g⁻¹; cm³cm⁻³) as well as surface areas (BET, Langmuir). At the laboratory scale, to the best of my knowledge, the performances of only three MOFs have been validated: MOF-5, MIL-53 and HKUST-1.”

It is not clear from the literature whether this situation has since changed significantly. In a more recent review, for example, Suh et al.³⁸ tabulated the reported hydrogen sorption capacities of around 200 different MOFs. These mostly consist of only single values, for any given temperature and pressure, although in some cases, including MOF-5, more were quoted; however, it is notable that the erroneous figure of 4.5 wt%, at 77 K and 1 bar, still appears. The question of how many of the other tabulated values would be reproducible is an intriguing one, particularly as these materials are known to be so sensitive to synthesis, handling and activation conditions. It is also worth noting that work on the use of the spillover mechanism to enhance the hydrogen storage properties of MOFs was treated uncritically by Suh et al.³⁸ This review is already highly cited, so erroneous data continues to propagate through the literature.

On a related note, the literature search reported in Fig. 1 was extended to identify retracted articles. 3234 articles were found to be marked as retracted in the Web of Science database. Only one paper¹²⁷ of the 19 622 results for the topic search for ‘hydrogen storage’ contained a clear retraction notice. This paper was published in 2004 and remained uncited until 2010 when it was retracted,¹²⁸ thus receiving its first and only citation. However, the article was withdrawn due to duplication, rather than for the inclusion of problematic data. None of the other studies discussed in this article have been retracted, even though some of them contain results that are clearly irreproducible.

In terms of sample purity, the microstructural characterisation of nanostructured and nanoporous materials is fraught with difficulties.¹²⁹ It is therefore important that materials are characterised as carefully as possible before conclusions on the material that the samples are purported to represent are drawn. The problems that have occurred previously with this issue should act as motivation for researchers to thoroughly characterise their samples and to carefully consider the effects that impurities could have on their hydrogen sorption properties.

Standardisation of the measurement methods is another possibility, although this would be difficult to achieve effectively. Nonetheless, work is underway on this, with regard to adsorption measurement, at the National Institute of Standards and Technology (NIST) in the US.¹³⁰ Measurement guidelines, some of which have been proposed by one of us,⁶ are another related option. However, a consensus would have to be reached and they would also need to be endorsed by an organisation such as the International Union of Pure and Applied Chemistry (IUPAC) to gain widespread acceptance. To be fully effective, they would also need the support of the editors and reviewers for the vast majority of journals that publish hydrogen sorption data. It is worth noting that if a list of such journals were to be compiled it would be rather long and would cross a number of different disciplines.

More careful consideration of the physical plausibility of some of the more fanciful results in the carbon nanotube and nanofibre story would certainly have helped in that case. For example, the claimed 67 wt% uptake by carbon nanofibres corresponded to a hydrogen-to-carbon ratio of ∼24, and it is not clear how a carbon structure would be able to support such a large number of H atoms. More careful consideration of this general point in the future would obviously help, and the increased use of complementary computer simulation using sufficiently accurate methods to model the sorption process could potentially aid this task.¹³¹ Increased collaboration between theoreticians and experimentalists should also therefore be encouraged.

Collaboration between different laboratories also offers the possibility of publishing reproducibility studies on hydrogen sorption data when new materials are reported, perhaps including measurements performed using different techniques; for example, both gravimetrically and volumetrically determined uptakes. This would help reduce the likelihood of problems occurring due to errors associated with a single piece of apparatus or a particular measurement methodology. Also, encouraging authors to more thoroughly characterise the hydrogen sorption properties of materials being reported for the first time for hydrogen storage applications could be of considerable benefit. This could involve short term cycling studies, for example, measurement of repeated isotherms at the same temperature, but also isotherms measured at different temperatures, in order to check the temperature dependence of the uptake and to calculate thermodynamic parameters such as the enthalpy and entropy of H₂ sorption. Demonstration of the physical plausibility of such results would help authors show that results are reasonable and repeatable. Both editors and reviewers could help encourage authors to be more rigorous in this respect, and this may ultimately lead to better interlaboratory reproducibilty.

What seems clear, in general, however, is that more experimental interlaboratory reproducibility studies are needed, regardless of the adoption or otherwise of best practice in isolated reports of the hydrogen storage properties of particular materials. The problems that have been revealed both by the documented controversies in the field and the recent interlaboratory studies certainly, in our opinion, warrant further investigation.

It is also worth emphasising that there are a number of straightforward tests that can be performed to check the likely validity of a particular hydrogen sorption measurement. These include the performance of measurements on more than one sample size, to check for sample size dependency, which could indicate problems with the calibration of apparatus or the measurement methodology. More than one isotherm can be measured with different dosing steps in each case. In the absence of significant errors, this should result in good repeatability. Varying the equilibration times can also be used to check for issues with the achievement of both thermal and sorption equilibrium at each isotherm point. This is important because different sorption processes can occur over significantly different timescales and it is crucial that equilibrium is achieved at each point. If it is possible to vary the assumed density of the sample in the calculation of the hydrogen uptake, then a realistic range of density values can be used to estimate the magnitude of likely errors due to this issue, which can be a particular problem for materials with very low densities. Degassing times and conditions, for nanoporous materials, or the activation process, in general, can also be varied to assess related problems with this aspect of the measurement methodology. We therefore recommend that such tests are performed for each material before data are published. Any differences between data measured under different conditions should then be addressed prior to publication, and the results reassessed.

Although these practical suggestions are specific to hydrogen sorption measurement, and include some that are perhaps rather obvious, it is likely that some of the reasons we mention at the end of the previous section apply more generally in other fields as well. If this is true, it points to problems that are more fundamental than a simple lack of appreciation of the sensitivity of hydrogen sorption measurements to measurement methodology or sample purity. Some of these have been discussed in more detail elsewhere^{13,120,123,132–135} and their solution would require a significant shift in the way that science is currently performed. Further discussion of this is certainly beyond the intended scope of this article; however, we hope that our summary of the problems in hydrogen storage material research will at least provide further evidence that something is perhaps amiss.

Before we conclude, it is worth emphasising the cost, in both time and money, of the publication of irreproducible data. This includes the laboratory time spent on attempted replication and the grants that are awarded as a direct consequence of the publication of headline-grabbing, but ultimately irreproducible, results. Financial support for subsequent research, based on such results in our field, has certainly totalled several tens of millions of US Dollars or Euros. Any strategies that can be implemented more generally to reduce this cost would therefore be most welcome.

Conclusions

In this article, we have summarised some of the problems with the irreproducibility of data in hydrogen storage material research that have occurred over the last couple of decades. We have seen that there have been problems with carbon nanotubes and nanofibres, MOFs, and various materials purported to store significant amounts of hydrogen via spillover, amongst others. Many of the problems can be attributed to the sensitivity of hydrogen sorption measurements to measurement error and the purity of samples. Recent interlaboratory studies have shown that even measurements on well understood materials, including both nanoporous materials and hydrides, can be susceptible to a significant amount of error. This perhaps suggests that more, as yet unidentified, irreproducible data is likely to exist in the current literature. We have discussed some of the reasons that we believe are behind the problems and proposed some ways forward for the future. We hope that this article will be of interest to those already working in the field and to newcomers, but also that it will help prevent the occurrence of further problems in the future by raising awareness of past mistakes in the field. In addition, given the increasing amount of attention in the literature with regard to irreproducibility and the replication of data, we also hope it will contribute constructively to the discussion of this issue more generally in other disciplines by highlighting and summarising the problems that have arisen in our own field. Regardless of the precise reasons behind each of the cases documented above, it seems likely that other occurrences of irreproducibility in related fields will share at least some common ground with those found in ours.

Acknowledgements

We are very grateful to Werner Marx and Robin Haunschild of the Information Retrieval Service (IVS-CPT) of the Max Planck Society for their help with the analysis of the hydrogen storage material literature.

Notes and references

1. International Energy Agency (IEA), Technology Roadmap: Hydrogen and Fuel Cells, OECD/IEA, Paris, 2015, http://www.iea.org/publications/freepublications/publication/TechnologyRoadmapHydrogenandFuelCells.pdf, accessed May 2016.
2. U. Eberle, M. Felderhoff and F. Schüth, Angew. Chem., Int. Ed., 2009, 48(36), 6608.
3. L. Schlapbach and A. Züttel, Nature, 2001, 414, 353.
4. Solid-State Hydrogen Storage: Materials and Chemistry, ed. G. Walker, Woodhead Publishing, Cambridge, 2008.
5. Handbook of Hydrogen Storage: New Materials for Future Energy Storage, ed. M. Hirscher, Wiley-VCH, Weinheim, 2010.
6. D. P. Broom, Hydrogen Storage Materials: The Characterisation of Their Storage Properties, Springer, London, 2011.
7. R. Heller, M. Bogomolov and Y. Benjamini, Proc. Natl. Acad. Sci. U. S. A., 2014, 111(46), 16262.
8. W. C. Hines, Y. Su, I. Kuhn, K. Polyak and M. J. Bissell, Cell Rep., 2014, 6(5), 779.
9. B. R. Jasny, G. Chin, L. Chong and S. Vignieri, Science, 2011, 334, 1225.
10. R. D. Peng, Science, 2011, 334, 1226.
11. Editorial, Nat. Biotechnol., 2012, 30(9), 806.
12. Editorial, Nature, 2012, 483, 509.
13. C. G. Begley and L. M. Ellis, Nature, 2012, 483, 531.
14. E. Yong, Nature, 2012, 485, 298.
15. J. F. Russell, Nature, 2013, 496, 7.
16. Editorial, Nature, 2013, 496, 398.
17. Editorial, Nature, 2014, 515, 7.
18. M. McNutt, Science, 2014, 343, 229.
19. L. P. Freedman, M. C. Gibson, S. P. Ethier, H. R. Soule, R. M. Neve and Y. A. Reid, Nat. Methods, 2015, 12(6), 493.
20. R. Nuzzo, Nature, 2015, 526, 182.
21. J. V. Sweedler, Anal. Chem., 2015, 87(23), 11603.
22. J. Bohannon, Science, 2014, 344, 788.
23. Open Science Collaboration, Science, 2015, 349, aac4716,  DOI:10.1126/science.aac4716.
24. The Academy of Medical Sciences, Reproducibility and reliability of biomedical research: improving research practice, Symposium report, October 2015, http://www.acmedsci.ac.uk/policy/policy-projects/reproducibility-and-reliability-of-biomedical-research/, accessed May 2016.
25. S. O. Hansson, in The Role of Technology in Science: Philosophical Perspectives, ed. S. O. Hansson, Springer, Dordrecht, 2015, ch. 5, pp. 81–110.
26. S. Shapin and S. Schaffer, Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life. Princeton University Press, Princeton, New Jersey, 1985.
27. B. Gower, Scientific Method: An Historical and Philosophical Introduction, Routledge, Abingdon, 1997.
28. K. Popper, The Logic of Scientific Discovery, Hutchinson, London, 1959.
29. G. Sandrock, J. Alloys Compd., 1999, 293–295, 877.
30. B. Sakintuna, F. Lamari-Darkrim and M. Hirscher, Int. J. Hydrogen Energy, 2007, 32(9), 1121.
31. B. Bogdanović and M. Schwickardi, J. Alloys Compd., 1997, 253–254, 1.
32. S. Orimo, Y. Nakamori, J. R. Eliseo, A. Züttel and C. M. Jensen, Chem. Rev., 2007, 107(10), 4111.
33. E. Callini, Z. Ö. K. Atakli, B. C. Hauback, S.-i. Orimo, C. Jensen, M. Dornheim, D. Grant, Y. W. Cho, P. Chen, B. Hjörvarsson, P. de Jongh, C. Weidenthaler, M. Baricco, M. Paskevicius, T. R. Jensen, M. E. Bowden, T. S. Autrey and A. Züttel, Appl. Phys. A: Mater. Sci. Process., 2016, 122, 353.
34. J.-C. Crivello, B. Dam, R. V. Denys, M. Dornheim, D. M. Grant, J. Huot, T. R. Jensen, P. de Jongh, M. Latroche, C. Milanese, D. Milčius, G. S. Walker, C. J. Webb, C. Zlotea and V. A. Yartys, Appl. Phys. A: Mater. Sci. Process., 2016, 122, 97.
35. L. J. Murray, M. Dincă and J. R. Long, Chem. Soc. Rev., 2009, 38(5), 1294.
36. K. M. Thomas, Dalton Trans., 2009, 1487.
37. M. Schlichtenmayer and M. Hirscher, J. Mater. Chem., 2012, 22(20), 10134.
38. M. P. Suh, H. J. Park, T. K. Prasad and D.-W. Lim, Chem. Rev., 2012, 112(2), 782.
39. J. Wang and S. Kaskel, J. Mater. Chem., 2012, 22(45), 23710.
40. Z. Xiang and D. Cao, J. Mater. Chem. A, 2013, 1(8), 2691.
41. H. W. Langmi, J. Ren, B. North, M. Mathe and D. Bessarabov, Electrochim. Acta, 2014, 128, 368.
42. A. Züttel, Mater. Today, 2003, 6(9), 24.
43. G. Barkhordarian, T. Klassen and R. Bormann, J. Phys. Chem. B, 2006, 110(22), 11020.
44. H. W. Langmi, A. Walton, M. M. Al-Mamouri, S. R. Johnson, D. Book, J. D. Speight, P. P. Edwards, I. Gameson, P. A. Anderson and I. R. Harris, J. Alloys Compd., 2003, 356–357, 710.
45. Y. Yürüm, A. Taralp and T. N. Veziroglu, Int. J. Hydrogen Energy, 2009, 34(9), 3784.
46. V. Presser, M. Heon and Y. Gogotsi, Adv. Funct. Mater., 2011, 21(5), 810.
47. R. Dawson, A. I. Cooper and D. J. Adams, Prog. Polym. Sci., 2012, 37(4), 530.
48. H. Furukawa, M. A. Miller and O. M. Yaghi, J. Mater. Chem., 2007, 17(3), 3197.
49. S. Tedds, A. Walton, D. P. Broom and D. Book, Faraday Discuss., 2011, 151, 75.
50. M. Bastos-Neto, C. Patzschke, M. Lange, J. Möllmer, A. Möller, S. Fichtner, C. Schrage, D. Lässig, J. Lincke, R. Staudt, H. Krautscheid and R. Gläser, Energy Environ. Sci., 2012, 5(8), 8294.
51. P. E. de Jongh and P. Adelhelm, in Handbook of Hydrogen Storage: New Materials for Future Energy Storage, ed. M. Hirscher, Wiley-VCH, Weinheim, 2010, ch. 10, pp. 279–340.
52. D. P. Broom, Int. J. Hydrogen Energy, 2007, 32(18), 4871.
53. D. E. Demirocak, S. S. Srinivasan, M. K. Ram, D. Y. Goswami and E. K. Stefanakos, Int. J. Hydrogen Energy, 2013, 38(3), 1469.
54. J. M. Zielinski, C. G. Coe, R. J. Nickel, A. M. Romeo, A. C. Cooper and G. P. Pez, Adsorption, 2007, 13(1), 1.
55. A. Qajar, M. Peer, R. Rajagopalan and H. C. Foley, Int. J. Hydrogen Energy, 2012, 37(11), 9123.
56. J. M. Zielinski, P. McKeon and M. F. Kimak, Ind. Eng. Chem. Res., 2007, 46(1), 329.
57. F. von Zeppelin, M. Haluška and M. Hirscher, Thermochim. Acta, 2003, 404(1–2), 251.
58. B. Panella, M. Hirscher and B. Ludescher, Microporous Mesoporous Mater., 2007, 103(1–3), 230.
59. K. E. Hurst, M. J. Heben, J. L. Blackburn, T. Gennett, A. C. Dillon and P. A. Parilla, Rev. Sci. Instrum., 2013, 84(2), 025103.
60. F. J. Castro and G. Meyer, Rev. Sci. Instrum., 2000, 71(5), 2131.
61. A. C. Dillon, K. M. Jones, T. A. Bekkedahl, C. H. Kiang, D. S. Bethune and M. J. Heben, Nature, 1997, 386, 377.
62. S. Hynek, W. Fuller and J. Bentley, Int. J. Hydrogen Energy, 1997, 22(6), 601.
63. P. J. F. Harris, Carbon Nanotube Science: Synthesis, Properties and Applications, Cambridge University Press, Cambridge, 2009.
64. M. Becher, M. Haluska, M. Hirscher, A. Quintel, V. Skakalova, U. Dettlaff-Weglikovska, X. Chen, M. Hulman, Y. Choi, S. Roth, V. Meregalli, M. Parrinello, R. Ströbel, L. Jörissen, M. M. Kappes, J. Fink, A. Züttel, I. Stepanek and P. Bernier, C. R. Phys., 2003, 4(9), 1055.
65. A. Chambers, C. Park, R. T. K. Baker and N. M. Rodriguez, J. Phys. Chem. B, 1998, 102(22), 4253.
66. C. C. Ahn, Y. Ye, B. V. Ratnakumar, C. Witham, R. C. Bowman Jr. and B. Fultz, Appl. Phys. Lett., 1998, 73(23), 3378.
67. G. G. Tibbetts, G. P. Meisner and C. H. Olk, Carbon, 2001, 39(15), 2291.
68. M. Rzepka, E. Bauer, G. Reichenauer, T. Schliermann, B. Bernhardt, K. Bohmhammel, E. Henneberg, U. Knoll, H.-E. Maneck and W. Braue, J. Phys. Chem. B, 2005, 109(31), 14979.
69. C. Tang, Y. Bando, X. Ding, S. Qi and D. Golberg, J. Am. Chem. Soc., 2002, 124(49), 14550.
70. D. Golberg, Y. Bando, C. Tang and C. Zhi, Adv. Mater., 2007, 19(18), 2413.
71. T. Terao, Y. Bando, M. Mitome, K. Kurashima, C. Y. Zhi, C. C. Tang and D. Golberg, Phys. E, 2008, 40(7), 2551.
72. B. Panella, L. Kossykh, U. Dettlaff-Weglikowska, M. Hirscher, G. Zerbi and S. Roth, Synth. Met., 2005, 151(3), 208.
73. N. F. Attia, S. M. Lee, H. J. Kim and K. E. Geckeler, Int. J. Energy Res., 2014, 38(4), 466.
74. R. Chahine and T. K. Bose, Hydrogen Energy Progress XI: Proceedings of the 11th World Hydrogen Energy Conference, 1996, 1259.
75. B. Panella, M. Hirscher and S. Roth, Carbon, 2005, 43(10), 2209.
76. M. Sevilla, R. Mokaya and A. B. Fuertes, Energy Environ. Sci., 2011, 4(8), 2930.
77. J. Goldsmith, A. G. Wong-Foy, M. J. Cafarella and D. J. Siegel, Chem. Mater., 2013, 25(16), 3373.
78. N. L. Rosi, J. Eckert, M. Eddaoudi, D. T. Vodak, J. Kim, M. O'Keefe and O. M. Yaghi, Science, 2003, 300, 1127.
79. J. L. C. Rowsell, A. R. Millward, K. S. Park and O. M. Yaghi, J. Am. Chem. Soc., 2004, 126(18), 5666.
80. B. Panella and M. Hirscher, Adv. Mater., 2005, 17(5), 538.
81. S. S. Kaye, A. Dailly, O. M. Yaghi and J. R. Long, J. Am. Chem. Soc., 2007, 129(46), 14176.
82. W. C. Conner and J. L. Falconer, Chem. Rev., 1995, 95(3), 759.
83. A. Lueking and R. T. Yang, J. Catal., 2002, 206(1), 165.
84. L. Wang and R. T. Yang, Energy Environ. Sci., 2008, 1(2), 268.
85. L. Wang, A. J. Lachawiec and R. T. Yang, RSC Adv., 2013, 3(46), 23935.
86. S. M. Luzan and A. V. Talyzin, Microporous Mesoporous Mater., 2010, 135(1–3), 201.
87. L. Li and R. T. Yang, J. Am. Chem. Soc., 2006, 128(25), 8136.
88. L. Li and R. T. Yang, J. Am. Chem. Soc., 2006, 128(3), 726.
89. R. Campesi, F. Cuevas, M. Latroche and M. Hirscher, Phys. Chem. Chem. Phys., 2010, 12(35), 10457.
90. N. P. Stadie, J. J. Purewal, C. C. Ahn and B. Fultz, Langmuir, 2010, 26(19), 15481.
91. H. Oh, T. Gennett, P. Atanassov, M. Kurttepeli, S. Bals, K. E. Hurst and M. Hirscher, Microporous Mesoporous Mater., 2013, 177, 66.
92. C. I. Contescu, C. M. Brown, Y. Liu, V. V. Bhat and N. C. Gallego, J. Phys. Chem. C, 2009, 113(14), 5886.
93. J. L. Blackburn, C. Engtrakul, J. B. Bult, K. Hurst, Y. Zhao, Q. Xu, P. A. Parilla, L. J. Simpson, J.-D. R. Rocha, M. R. Hudson, C. M. Brown and T. Gennett, J. Phys. Chem. C, 2012, 116(51), 26744.
94. C. M. Ghimbeu, C. Zlotea, R. Gadiou, F. Cuevas, E. Leroy, M. Latroche and C. Vix-Guterl, J. Mater. Chem., 2011, 21(44), 17765.
95. S. B. Kalidindi, H. Oh, M. Hirscher, D. Esken, C. Wiktor, S. Turner, G. Van Tendeloo and R. A. Fischer, Chem. Eur. J., 2012, 18(35), 10848.
96. S. M. Luzan and A. V. Talyzin, Microporous Mesoporous Mater., 2011, 139(1–3), 216.
97. S. K. Konda and A. Chen, Mater. Today, 2016, 19(2), 100.
98. X.-L. Wang and S. Suda, J. Alloys Compd., 1995, 231(1–2), 660.
99. A. Ansón, M. Benham, J. Jagiello, M. A. Callejas, A. M. Benito, W. K. Maser, A. Züttel, P. Sudan and M. T. Martínez, Nanotechnology, 2004, 15(11), 1503.
100. C. Zlotea, P. Moretto and T. Steriotis, Int. J. Hydrogen Energy, 2009, 34(7), 3044.
101. J. Silvestre-Albero, A. Sepúlveda-Escribano, F. Rodríguez-Reinoso, V. Kouvelos, G. Pilatos, N. K. Kanellopoulos, M. Krutyeva, F. Grinberg, J. Kaerger, A. I. Spjelkavik, M. Stöcker, A. Ferreira, S. Brouwer, F. Kapteijn, J. Weitkamp, S. D. Sklari, V. T. Zaspalis, D. J. Jones, L. C. de Menorval, M. Lindheimer, P. Caffarelli, E. Borsella, A. A. G. Tomlinson, M. J. G. Linders, J. L. Tempelman and E. A. Bal, in Characterisation of Porous Solids VIII: Proceedings of the 8th International Symposium on the Characterisation of Porous Solids, ed. S. Kaskel, P. Llewellyn, F. Rodríguez-Reinoso and N. A. Seaton, RSC Publishing, Cambridge, 2009, pp. 9–16.
102. P. Moretto, C. Zlotea, F. Dolci, A. Amieiro, J.-L. Bobet, A. Borgschulte, D. Chandra, H. Enoki, P. De Rango, D. Fruchart, J. Jepsen, M. Latroche, I. Llamas Jansa, D. Moser, S. Sartori, S. M. Wang and J. A. Zan, Int. J. Hydrogen Energy, 2013, 38(16), 6704.
103. K. E. Hurst, P. A. Parilla, K. J. O'Neill and T. Gennett, Appl. Phys. A: Mater. Sci. Process., 2016, 122, 42.
104. P. Chen, X. Wu, J. Lin and K. L. Tan, Science, 1999, 285, 91.
105. R. T. Yang, Carbon, 2000, 38(4), 623.
106. F. E. Pinkerton, B. G. Wicke, C. H. Olk, G. G. Tibbetts, G. P. Meisner, M. S. Meyer and J. F. Herbst, J. Phys. Chem. B, 2000, 104(40), 9460.
107. V. Skákalová, A. Quintel, Y.-M. Choi, S. Roth, M. Becher and M. Hirscher, Chem. Phys. Lett., 2002, 365(3–4), 333.
108. K. M. Thomas, Catal. Today, 2007, 120(3–4), 389.
109. C. J. Webb and E. M. Gray, Int. J. Hydrogen Energy, 2014, 39(1), 366.
110. C. J. Webb and E. M. Gray, Int. J. Hydrogen Energy, 2014, 39(5), 2168.
111. T. P. Blach and E. M. Gray, J. Alloys Compd., 2007, 446–447, 692.
112. D. P. Broom and K. M. Thomas, MRS Bull., 2013, 38(5), 412.
113. C. J. Webb and E. M. Gray, Int. J. Hydrogen Energy, 2014, 39(13), 7158.
114. M. Hirscher, M. Becher, M. Haluska, U. Dettlaff-Weglikowska, A. Quintel, G. S. Duesberg, Y.-M. Choi, P. Downes, M. Hulman, S. Roth, I. Stepanek and P. Bernier, Appl. Phys. A: Mater. Sci. Process., 2001, 72(2), 129.
115. J. Hafizovic, M. Bjørgen, U. Olsbye, P. D. C. Dietzel, S. Bordiga, C. Prestipino, C. Lamberti and K. P. Lillerud, J. Am. Chem. Soc., 2007, 129(12), 3612.
116. M. Thommes, K. Kaneko, A. V. Neimark, J. P. Olivier, F. Rodriguez-Reinoso, J. Rouquerol and K. S. W. Sing, Pure Appl. Chem., 2015, 87(9–10), 1051.
117. H. Frost, T. Düren and R. Q. Snurr, J. Phys. Chem. B, 2006, 110(19), 9565.
118. M. Gad-el-Hak, Phys. Today, 2004, 57(3), 61.
119. D. Fanelli, PLoS One, 2010, 5(4), e10271.
120. C. G. Begley and J. P. A. Ioannidis, Circ. Res., 2015, 116(1), 116.
121. K. J. Gross, R. K. Carrington, S. Barcelo, A. Karkamkar, J. Purewal, S. Ma, H.-C. Zhou, P. Dantzer, K. Ott, T. Burrell, T. Semeslberger, Y. Pivak, B. Dam and D. Chandra, Recommended Best Practices for the Characterization of Storage Properties of Hydrogen Storage Materials, v3.34 US DOE Hydrogen Program, 2012, http://www1.eere.energy.gov/hydrogenandfuelcells/pdfs/best_practices_hydrogen_storage.pdf, accessed May 2016.
122. A. L. Spek, Acta Crystallogr., Sect. D: Biol. Crystallogr., 2009, 65, 148.
123. B. A. Nosek, J. R. Spies and M. Motyl, Perspect. Psychol. Sci., 2012, 7(6), 615.
124. A. Kenall, S. Edmunds, L. Goodman, L. Bal, L. Flintoft, D. R. Shanahan and T. Shipley, BMC Neurosci., 2015, 16, 44.
125. Nature guidelines, http://www.nature.com/authors/policies/availability.html, accessed May 2016.
126. G. Férey, Chem. Soc. Rev., 2008, 37(1), 191.
127. S. Fukada, S. Morimitsu, N. Shimoozaki and M. Nishikawa, J. Alloys Compd., 2004, 381(1–2), 258.
128. S. Fukada, S. Morimitsu, N. Shimoozaki and M. Nishikawa, J. Alloys Compd., 2010, 495(1), 292.
129. C. Weidenthaler, Nanoscale, 2011, 3(3), 792.
130. National Institute of Standards and Technology (NIST), Measurement Needs in the Adsorption Sciences, Workshop Summary Report, April 2015, http://nist.gov/mml/fact/, accessed May 2016.
131. Z. Xiang, D. Cao, J. Lan, W. Wang and D. P. Broom, Energy Environ. Sci., 2010, 3(10), 1469.
132. S. Kepes and M. A. McDaniel, Ind. Organ. Psychol., 2013, 6(3), 252.
133. M. Derksen and E. F. Rietzschel, Ind. Organ. Psychol., 2013, 6(3), 295.
134. G. B. Schmidt and R. N. Landers, Ind. Organ. Psychol., 2013, 6(3), 305.
135. W. H. Starbuck, Adm. Sci. Q., 2016, 61(2), 165.

---