Emma S.
Walker
a,
Ruth A.
Roberts
*bc and
Jason H.
Gill
de
aBBSRC ESRC Soc-B Centre for Doctoral Training in Biosocial Research, UCL Institute of Epidemiology and Health Care, London WC1E 6BT, UK
bApconiX, Alderley Park, Alderley Edge, SK10 4DG, UK. E-mail: ruth.roberts@apconix.com; Tel: +44 (0)7733 014396
cSchool of Biosciences, University of Birmingham, B15 2TT, UK. E-mail: r.roberts.4@bham.ac.uk
dNorthern Institute for Cancer Research (NICR), Paul O'Gorman Building, Newcastle University, Newcastle upon Tyne, NE2 4HH, UK
eSchool of Pharmacy, King George VI Building, Newcastle University, Newcastle upon Tyne, NE1 7RU, UK
First published on 13th May 2019
To ascertain attitudes to resourcing, collaboration and publication in toxicology, a survey was developed and distributed to British Toxicology Society (BTS) members. The survey comprised 14 questions with 5 response options (strongly agree; agree; conflicted; disagree; strongly disagree) and a free text box. One hundred completed surveys were received by the cut-off date for data analysis. Unsurprisingly, 60% of participants disagreed or strongly disagreed that toxicology research is adequately funded in the UK; only 12% agreed with this statement. A similar proportion of participants (53%) disagreed with the statement that funding councils give equal opportunity to toxicology whereas 31% were conflicted on this point. An overwhelming 97% of respondents agreed that collaboration is important in driving toxicology research whereas only 38% agreed that competition is important. When this question was broadened out beyond the discipline of toxicology, a similar profile was seen suggesting that participants held similar views on toxicology versus other types of research. Many respondents were conflicted regarding the role of competition both in toxicology and in other research disciplines. Free text comments suggested that some competition is good to drive quality but can be counterproductive when competing for limited resources. Most participants were in favour of making toxicology research data openly available (86%) and in favour of open access publication (89%) although there were reservations about the cost of open access. Many (60%) thought the current system of peer review is fair but 65% also supported the idea of double-blind peer review (where both reviewer and author are anonymized). Others suggested a step in the opposite direction towards increased transparency (revealing and holding reviewers to account) would be preferable. Overall, there was a broad theme in free text responses that the need for experienced toxicologists has increased at a time when training and investment in the discipline has declined. However, not all respondents held that view with some noting that toxicology both as a research and as an applied discipline is strong within the UK scientific community.
To address current attitudes of toxicologists to challenges to their discipline, we developed a survey intended to elicit the views of current practitioners of the discipline. This larger survey was based on the outcome from a pilot survey distributed to a small group of toxicologists seeking responses to posed questions but also seeking feedback on process and survey format (Masters’ Research Project Study conducted by E Walker; unpublished data). Based on the outcome of this pilot, a larger survey was developed and distributed to British Toxicology Society (BTS) members and toxicology conference participants asking for opinions on resourcing, collaboration, competition, research infrastructure and peer review. Herein we present the outcome of this research. Conclusions are focused on UK toxicology but are clearly relevant to many aspects of scientific research in the UK and elsewhere.
Number | Question | Response options |
---|---|---|
1 | I believe toxicology research is adequately resourced in the UK | Strongly agree |
2 | When compared to other disciplines, I believe the research funding councils and other funding bodies provide equality of opportunity to facilitate toxicology research | Agree |
3 | I believe collaboration to be important in driving toxicology research in the UK | Conflicted |
4 | When compared to other disciplines, I believe collaboration to be especially important in driving toxicology research | Disagree |
5 | I believe competition to be important in driving toxicology research in the UK | Strongly disagree |
6 | When compared to other disciplines, I believe competition to be especially important in driving toxicology research | |
7 | I believe there is a good infrastructure to support research between academia, industry and the regulatory authorities | |
8 | I believe making toxicology research data openly available with as few restrictions as possible in a timely and responsible manner would improve the impact and efficiency of toxicological research | |
9 | When reading research publications, I consider whether there is a conflict of interest when assessing the validity of research | |
10 | I believe the current system of peer review for research outputs is a fair and appropriate system | |
11 | When peer reviewing research publications, I consider whether there is a conflict of interest when assessing the validity of research | |
12 | I believe reviewers are often not sufficiently trained to adequately and fairly judge the merit, quality and impact of toxicology research | |
13 | I believe double-blind peer review, where both reviewer and author are anonymised, is a good idea | |
14a | I believe open access publication, where articles are available free at source, is a good idea | |
14b | Have you published research outputs using an open access route in the last two years? | Yes |
No | ||
14c | If so, how was the cost for publication supported? | Institute or employer |
Collaborator or industry sponsor | ||
Competitive research award | ||
Competitive research award (co-author) | ||
Funded personally | ||
Not applicable | ||
Other (free text) |
The survey was accompanied by a participants’ information sheet (see ESI 2†) that explained the purpose of the study and information around data protection and confidentiality. The survey and participants’ information sheet were reviewed and approved by the Executive Committee of the BTS for distribution to BTS members and BTS conference participants. The survey was distributed by the BTS Secretariat via a web link.
Anonymized responses were collated into an excel spreadsheet for analysis and review. At the date preset for the close of the survey, one hundred responses had been received. Thus, data are presented as actual numbers of respondents but can also be quoted as percent response. Since the survey was set up as an exploratory study the data are presented as a descriptive analysis of results. Not all data add up to the 100 respondents since there were some non-responders (0–4 people) for some questions. Although data were anonymized, each respondent had a ‘respondent number’ preserved across all questions allowing for an analysis of internal pairings of answers.
Raw data, plots and pivot tables are available in ESI 3.†
The disagree/strongly disagree category for Q1/Q2 yielded more free text answers than any other questions in the survey. Comments fell broadly into three categories (Table 2) around reductions in training/resourcing/career guidance, lack of appreciation of the value of the discipline and a third category of relatively emotive statements. One comment summarized an overall view expressed by those that disagreed/strongly disagreed that toxicology is adequately resourced: ‘Academic centres have reduced, government spending over a long period has been minimal, pharmaceutical companies have relocated or contracted, CRO's do not have development of the science at their core purpose’. Several respondents highlighted that many pharmaceutical companies have contracted in the UK. While this may be true, the Association of the British Pharmaceutical Industry (ABPI) data5 shows that the number of pharmaceutical enterprises in the UK rose steadily between 2012 and 2015 (the last year in the survey). ABPI data also show that the number of people employed has stayed relatively constant at around 60–65 thousand employees over the last decade since 2009.5 A possible explanation of this apparent discrepancy in views is that while large pharma is contracting, the UK small and medium enterprise (SME) biotech sector is booming with 2019 figures showing a 40% increase in investment.6
Theme | Overall point | Example answers |
---|---|---|
Disagree/strongly disagree | Lack of appreciation of the value of the discipline | It is virtually impossible to obtain research council funding for toxicology-based research they are just not interested |
Not enough input | ||
Not enough being done to drive the discipline forward | ||
It's like the forgotten science, despite being front and central to many areas | ||
Training, research and career development much reduced | As a early career researcher, I have noticed that any toxicology-related education or career advice have been effectively absent throughout my undergraduate and postgraduate studies | |
Not promoted as an effective career | ||
No resources for research | ||
Emotive statements | We are screaming out for studies to be done | |
Why would drug companies want to identify problems! |
Overall, there was a broad theme that the need for experienced toxicologists has increased at a time when training and investment in the discipline has declined. Also notable in the free text comments was a concern around sustainability and a perception that toxicology in the US is still well resourced as summarized by this participant: ‘Although there is a fair amount of toxicology research underway in the UK it is clearly not as well-resourced as toxicology research elsewhere in the world such as the USA’. Also notable was the perspective from one early career scientist who stated that ‘As a student hoping to enroll on a toxicology associated PhD, my experience in looking for such opportunities suggests that projects of a toxicological nature are far less common than those in alternative biological/pharmaceutical areas’.
Of the 12 participants that agreed or strongly agreed that toxicology was adequately resourced in the UK, several provided free text comments primarily on two themes. Firstly, that hazard and risk assessments are adequately done in the UK. Secondly, that toxicology is a multidisciplinary science and therefore training in other disciplines feeds the pool of toxicologists. This second theme touches on a concept explored in the initial opinion article1 where we proposed that toxicology must be willing to redefine its boundaries as a discipline and draw strength and diversity from other sciences. One important discussion point regarding resourcing of toxicology is the distinction between education and training. In general terms, training refers to imparting a special skill or behavior to a person and is often delivered ‘on the job’ and/or in applied training courses. It is not the same as education, which is usually delivered by an institution with the aim of developing a sense of judgment and reasoning that can be applied to any topic or problem. Thus, scientists who have received an education in topics such as biochemistry, pharmacology, medicine, veterinary sciences, physiology and mathematics are all great potential toxicologists, but a training course coupled with significant ‘on the job’ experience is required to transform such individuals into toxicologists. Thus, postgraduate training courses in toxicology should be supported and expanded.
A pairing analysis (Table 3) produced a notable pattern in concordance around collaboration: of the 71 participants who strongly agreed with Q3 (collaboration is important in toxicology), 42 of them also strongly agreed with Q4 (collaboration is uniquely important in toxicology). Looking at this the other way around, 92% (46/50) of those who strongly agreed with Q4 also strongly agreed with Q3.
Questions (5 & 6) regarding competition in toxicology and other disciplines produced a high number of conflicted responses (32 and 37, respectively). A pairing analysis showed that 26 respondents were conflicted over both questions. The overall theme of the ‘conflicted’ free text comments on the role of competition in driving research was that some competition is good to drive quality but that it can be counterproductive when groups are competing for limited resources. This raises an interesting point in that competition is regarded as intrinsic to the conduct of research in any discipline, including toxicology. For example, researchers compete for grant funding, journal space, training positions and jobs; this is generally regarded as an important step in ensuring quality. However, if competition is taken to extremes it can prevent effective collaboration; these aspects of toxicology (when to compete and when to collaborate) are more about scientists possessing the interpersonal skills to ensure appropriate and constructive behaviour rather than about the scientific discipline per se. In this regard, perhaps ‘responsible competition’ should be highlighted in degree courses and career and professional training programmes.
To understand if there is a relationship between concern around resources and concern around the role of competition, a pairing analysis was conducted to see how those that disagreed with Q1 (toxicology research Is adequately funded) responded to Q5. Of the 41 respondents who disagreed that toxicology is adequately resourced, 17 of them agreed that competition plays a positive role in driving toxicology research and 8 of them were conflicted on this statement. Thus, it would seem that concern over adequate resourcing is not the main driver for a dislike of competition among some participants.
Theme | Overall point | Example answers |
---|---|---|
Conflicted | Mixed – room for improvement | The collaboration is present at the BTS congress and other meetings. Otherwise interactions can be limited especially with regulatory agencies |
There are good industry/academic links, but we should harness these links more broadly rather than company by company | ||
There could be more done to support this | ||
In some instances I agree, however more collaboration between more academic departments and institutes would be welcome instead of just a few working with industry | ||
Yes and No – my issue is that on the whole regulatory authorities do a good job, but they are often difficult to deal with and can be fairly unhelpful. It should be more a collaborative approach | ||
It's about individuals | I think it depends on the individual authorities/institutions | |
I have experience of when the different areas have worked well together. But this has been down to the good will of the individuals concerned, rather than anything that stems from government funded actions | ||
Where it works well it's about networks & personal relationships rather than infrastructure – depends on individuals’ commitment to making something happen … | ||
Real or perceived COI | Mixed feelings here. On the one hand academic institutions are forced to cooperate with other partners due to declining funding from government bodies. On the other hand society is suspicious to this cooperation. Moreover, many regulatory authorities want to be “independent” and not very keen on cooperation | |
For some collaborations this exists but the perception of the conflicting interests of these 3 organisations often prevents effective collaboration |
Some of the comments regarding open access were as expected with strong support for scientists to access useful information. Others commented that data sharing can help avoid duplication of animal experimentation. However, there were some potentially controversial free text comments largely from those who agreed that it was a good idea; specifically, one participant stated that ‘Most publishers bring very little value added to the table, and are exploiting the voluntary efforts of others to make large profits’ and another that ‘…publishing houses are taking the scientific community for a ride. You don't even get access to your work unless you have subscription or access to one’. These opinions reflect a recent trend in pushing back on publishing strategies as illustrated by the University of California which has decided not to renew its journal subscriptions with Elsevier in a disagreement over ‘the rapidly escalating costs associated with for-profit journals’.7
Free text comments offered more detail on opinions on COI. Several participants commented that considering COI was a very important issue and was ‘self-evident’. Others stated that they would decline to review if there was COI. There were also several free text comments to clarify what COI meant to individual participants: ‘…only if it is a REAL conflict of interest rather than inferred just because of employer or sources of grant funding’. Rather than decline to review, others suggested that if they perceived COI then they would ‘be more insistent on approaching every sentence with the mindset of an auditor – Can the statement be fully supported backed up by data that is presented if it can't be supported, are more data required or should the statement be edited or removed?’ The topic of COI came up also in the free text responses to Question 7 on research infrastructure (Table 4). Specifically, participants commented that real or perceived COI could be a block to the creation and maintenance of the effective research infrastructure networks needed to drive a multidisciplinary subject such as toxicology.
There were a few participants that were conflicted on questions 9 and 11, but only 3 participants reported being conflicted on both of these statements. Most of the free text comments on ‘conflicted’ for questions 9 and 11 were around the participant feeling unable to comment due to lack of knowledge of current systems.
The data and freetext responses show that COI was clearly an issue of concern amongst survey participants, with several highlighting the need to distinguish between real and perceived COI. COI was helpfully defined in 2017 as ‘…a set of circumstances that creates a risk that professional judgement or actions regarding a primary interest will be unduly influenced by a secondary interest’.8 Clearly, circumstances that create a real risk of COI are to be avoided. However, what is meant by perceived COI? It was pointed out more than a decade ago that COI is being used in gamesmanship where, for example, ‘strong assertions of conflict of interest are used to justify particular points of view’.9 This challenge still persists today with ongoing attention on COI and some unfortunate consequences such as the exclusion of industry scientists from governmental panels and review committees.8
There were some interesting points raised in the free text from those both in favour of and against double blind peer review (Table 5). Many disputed the value and feasibility; others suggested that a step in the opposite direction towards increased transparency (revealing and holding reviewers to account) would be preferable.
In favour | Against | |
---|---|---|
Anonymity | Gives more opportunity to unknown and up-and-coming groups | It's good to know the group's history and reputation |
It doesn't take much to work out who the authors are and where they are from | ||
I am happy to be identified as a reviewer and would prefer that rather than blinding the authors | ||
Process | Not sure how this would work | It's too complex |
Already too time consuming – this would make it worse | ||
Is this even feasible? | ||
Value | It seems fairer good to decide purely based on what is presented | I can't see how this would help |
Is there a problem to fix? |
In this survey we only asked about the currently prevalent system of peer review (where reviewers are anonymous) and attitudes to double blind peer review. However, there are other systems that can be considered such as “Open Peer Review” (OPR) and triple blind peer review (where authors and their institutions are reciprocally anonymous to reviewers and editors).10 In their review, Tennant et al.10 present a detailed analysis to consider the pros and cons of these various systems.
Some participants agreed (30) or strongly agreed (7) that reviewers are not sufficiently trained to judge the merit, quality and impact of toxicology research. However, 21% disagreed/strongly disagreed with this statement. A large number (41%) were conflicted on this point, the highest number of conflicted answers for any of the questions in the survey. There were very few free text answers offered by those who disagreed with this question. However, those who agreed/strongly agreed (that reviewers are not sufficiently trained) offered free text answers mainly drawn from negative personal experience with comments such as ‘this is very clear from some of the referees’ comments I received in the past’ and ‘Often they seem to miss point’. Many of the conflicted respondents offered similar views that were well summarized by one participant; ‘It is mixed. Some are excellent. Some very dismissive and some simply do not understand the science’.
It may be useful to follow this survey with a more detailed analysis of views of the BTS towards peer review. There is also the opportunity to offer training to BTS members and other toxicologists based around the 6-red flags rules described by Begley.11 These flags include such fundamental issues as inappropriate reagents, lack of reproducibility and absence of controls. Steps have been proposed to overcome this for toxicology as a discipline;12 the BTS could play a role in delivering an integrated and unified approach to address COI, peer review and unreproducible data across all basic and applied pharmacology and toxicology.1
ITTP (MRC Integrative Toxicology Training Partnership)13 was also mentioned by several participants as a great scheme, typified by the following comment: ‘It took huge effort to get the ITTP programme to work – but it is very limited in comparison to other sectors and review panels rarely have toxicologists evaluating the merit of applications for funding’. Two other key themes were: 1. the decline of Centres of Excellence for Toxicology Research such as Central Toxicology Laboratory and Bibra which were instrumental in many aspects such as driving research, training and collaboration and 2. The decline of toxicology training centres as illustrated by this quote: ‘There are not enough training locations for Tox in the UK. Only Birmingham offers a full-time and part-time MSc course. The community is still reeling from the closure of the Surrey course’.
This work was supported by a BTS Summer Vacation Scholarship awarded to Emma Walker and supervised by Jason Gill.
Footnote |
† Electronic supplementary information (ESI) available. See DOI: 10.1039/c9tx00063a |
This journal is © The Royal Society of Chemistry 2019 |