Analysis 2018 – Frequently Asked Questions
This document is intended to explain or to help clarify some of the key areas of the HRCS and the planning for the production of the 2018 report. It acts as a companion to the main Submission Guidance and is subdivided into five main sections:
- The Basics – which provides information on the UKCRC, HRAF and background to the report series
- Participation – some further explanation for our aims and how to take part in the report.
- The Coding – more details on how awards can / should be coded.
- Complex Awards – common questions regarding how to report on awards that are more complicated (e.g. extensions, block grants and co-funding).
- Data Specifics – answers common questions about some of the requested data fields.
If there are additional questions arising as the project begins, this will be added to the document and displayed via the HRCS website.
This FAQ has been produced by the HRCS Project Management Team and approved by the HRAF. Please contact us if you have any additional questions.
Who are the UKCRC and what is the HRCS?
The UK Clinical Research Collaboration (UKCRC) is a group of representatives from the major public, industry and charitable funders of UK Health Research. Its role is to co-ordinate strategic approaches to clinical research to benefit patients, researchers and the nation. In 2004, the UKCRC established the bespoke Health Research Classification System (HRCS) to categorise all types of health research across all disease and areas of health. More information is available on the UKCRC website and HRCS website.
Who are the HRAF?
The UKCRC Health Research Analysis Forum (HRAF) is a group of representatives from the 12 largest funders of public and charitable health research and the Association of Medical Research Charities (AMRC), representing over 140 medical research charities. The forum was established in 2009 to take over the maintenance and development of the HRCS. This includes the co-ordination and production of the nationwide reporting; the UK Health Research Analysis series.
What is the UK Health Research Analysis 2018?
The 2018 report will be the fourth in a series of reports on health research funding in the UK begun in 2004. These UK Health Research Analyses (often referred to informally as ‘HRCS Reports’) have been used extensively by the participant organisations, UK Government and stakeholders in biomedical research. More information on the HRCS Reports and their wider use is on the reports section of our website.
I/My organisation have not been in previous HRCS reports. Why am I being asked to be involved?
Since the first report in 2004, the use of HRCS coding has become a routine procedure for many biomedical funders and each report has widened the participation to provide a more comprehensive picture of biomedical funding across the UK. Our aim for the 2018 report is to make this the most complete analysis of the UK health research to date, which includes contacting more funders than we ever have before. This includes additional AMRC members as well as other Government and not-for-profit funders whose portfolio may include funding for health R&D alongside other disciplines.
In addition, and only if historical data is available, you would be welcome to submit award data for the previous reporting periods (2014, 2009/10 and 2004/05). This is entirely optional but would allow us to perform the same trends-over-time assessment as those funders who have been part of the previous UK Health Research Analysis reports.
What does participation in the 2018 report involve?
To be part of the analysis, all we will need you to provide details of the health-relevant awards, grants or projects you funded that were/are active in the calendar year 2018 (01/01/18 to 31/12/18). Full details of the data requirements and inclusion criteria can be found on our accompanying Submission Guidance.
Please note that how each funder defines ‘health relevant’ varies so we ask that each organisation provides us with a short descriptive explanation for how data was chosen and collated. Examples of this can be found in Annex 1 of the last report (2014).
I/My organisation does not provide traditional awards/grants, but we do support research by other means (e.g. commissioned research). How should we report these sources of funding?
To ensure as complete a picture of health-relevant research as possible, all types of funding can be submitted; the precise mechanism / type of funding is not too important. All we require is the minimum data fields (see below) including a value (e.g. contract amount), dates/duration of the work (that meets the active in 2018 inclusion criteria), and a description (e.g. tender specification) to allow coding. In addition, there are some (non-mandatory) columns where you can provide further details on funding mechanism if you would like to provide them.
I/My organisation is not a dedicated health funder. How do we determine which awards are health relevant for this analysis?
As we expand the analysis to include more research funding organisations this will include a greater proportion where health or medicine is not a primary topic for funding. This analysis series has, from the outset, included research council data from other disciplines such as engineering, physics, biology, economics and sociology.
Determining what is ‘health relevant’ is a considerable undertaking and there are no set criteria for this analysis. However, as with HRCS coding in general, we would look for any type of funding for research where health is a consideration, either as an indicator or direct outcome of the project (e.g. impact of pollutants on disease). We would generally exclude funding for topics where the health component is only implied or a long-term consideration (e.g. climate change will have eventual health impacts, but not all climate change research would be considered health relevant).
In practical terms, we can accept any award information you consider health relevant. If there are no pre-existing criteria you can use to help with this, the project management team can help guide you determine health relevance should you need it. To be included into the main analysis will require some post-submission assessment and classification of award data under the HRCS, so please note that some data submitted may eventually be excluded or analysed separately. However this is an iterative process and the project management team will happy to work with you during this data cleaning period.
I/My organisation is not a dedicated research funder. Can we still participate?
The focus of this analysis is on research funding, so awards that fund healthcare services or support groups would not normal be included (else we’d also include the entire NHS budget). What we are looking for is funding that asks a question or poses a theory and seeks to answer it. So any comparison, trial or assessment of services, interventions or methodologies would all be considered research here, even if they fall outside the realms of academia.
As with the decision of what is health relevant above, we happy to work with participating funders to determine how best to approach this with each organisation individually.
What data fields are required for submission?
What if my organisation does not record the data required for this analysis?
To be included in the main analysis requires sufficient detail (e.g. start/end dates, value, titles and abstracts or equivalent project summaries) to allow the funded research to be classified using the Health Research Classification System (HRCS), the bespoke coding system developed for these nationwide landscaping analyses. However if such data is unavailable, we can incorporate bulk data on health-relevant expenditure into our broader analysis of health R&D funding. Absence of coding or code-able data does not prevent inclusion in this assessment. Please get in touch with the project management team for more details of this process.
What about data publicity? Who will be able to see the final data?
The short explanation is that, as with previous reports, the final dataset (a spreadsheet of each award included in the analysis) will be made publicly available via the data pages of the HRCS website.
We have also prepared a Data Publicity Statement which provides a more detailed explanation of the data protection and conditions of use for the data.
The longer answer is that as more and more data on awards have become publicly available the UK Health Research Analyses data has kept track with the consensus regarding data availability, with progressively more of the core data collected being made publicly available.
Many funders have already begun this process of data transparency. UKRI make all awards available via Gateway to Research (https://gtr.ukri.org/ ). In addition, many public sector organisations and charities also make their own awards available, either via their own websites or through centralised databases such as EuroPubMed, WHO’s Global Observatory or NIH’s World RePORT. This includes greater levels of detail, including award values, grant codes, titles and abstracts.
From a data analysis perspective, there are distinct benefits to having a greater amount of data publicly available, as it allows for further analysis of awards. For example, a sub group analysis of mental health funding specifically for ‘depression’ can be achieved using keywords, data-mining or related machine learning techniques. An excellent example of the further use of the last report’s data (2014) is Marie Curie’s analysis of palliative care in the UK. In conclusion, a publicly accessible dataset would allow the UK Health Research Analysis 2018 to become an even more valuable resource beyond its original intended purpose.
Therefore in keeping with most participating funders’ support for open accessibility of data, and the potential post-analysis benefits of doing so, our aim is to make final data, including grant codes, titles and abstracts, publicly available. However please note that this does not include every data field. The Data Dictionary in the submission guidance includes a column to show which fields will be included in the publicly available final version.
There will be an option for specific exceptions where funders have reasons to exclude details – e.g. where awards identify personal details (e.g. salaries), contain confidential details (e.g. patent details) or there are concerns about the safety of researchers (e.g. highly detailed in vivo experiments).
If you have concerns regarding data publicity, please contact the HRCS Project Management Team via the HRCS website.
What is the timeline for the project?
Formal data collection begins on 02-JAN-2018 with the aim to capture all the currently available awards. This ensures the ‘final’ dataset needs to be provided by the 28-FEB 2019 and will contain the remaining awards not included in any WIP datasets (e.g. awards activated later in 2018 which were not ready for submission). Data analysis will continue March, with the draft report going to the UKCRC board at the beginning of April 2019. Our aim is to have a final published version available in May 2019.
Isn’t collecting data in Jan/Feb 2019 on awards made in 2018 too soon?
We are aware that award management systems do have a degree of lag time between an award being agreed in principle and it appearing in databases. This can mean awards made late in 2018 may not be available for extraction for a few days, weeks or even months.
However our inclusion criteria for this project states we are looking for awards ‘active’ in 2018. By this we generally mean awards incurring spend in 2018, meaning the award itself has already passed a series of administrative hurdles and funding is either ready to be or has been transferred from you, the funder, to the awardee. The precise definition of ‘active’ will vary as funders record spend in a variety of ways, but so long
How do we submit data?
Data submissions are achieved via our Standardised Excel Spreadsheet (2018), emailed to the HRCS project management team. Further advice on the process of data extraction can be found in our Submission Guidance.
Do we require a data transfer agreement to send you our data?
Our Data Publicity Statement provides some basic information on what we can/cannot do with any data you provide to us. If you require us, i.e. MRC on behalf of UKCRC, to sign a formal data transfer agreement this is absolutely fine so long as it meets our standard requirements for participation.
How is the award coding being completed?
In most cases, funders have systems in place for coding their awards periodically. However, as we are requesting full award data for the complete 2018 calendar year (01/01/18 to 31/12/18) without much lead time we may need to ‘code as we go’ for the more recent awards. Our estimate for total number of awards that would be eligible from all funders is between 17,000 to 20,000 awards.
But how will up to 20,000 awards be coded?
The good news is that the majority of this coding has already been done, as most larger funders code award data routinely. We are advising funders with outstanding coding to ensure their data is coded ahead of the data collection deadline, and there may be additional options for coding. This includes additional training sessions for manual coding, advice on seeking pay-per-award ‘contract’ coding and potential for access to automated HRCS coding. Any awards that remain uncoded can still be included in the wider assessment of total expenditure but can’t be incorporated into the main analysis.
I/My organisation is new the HRCS Reports and don’t have any awards HRCS coded. What do we do?
There is very limited central capacity for additional manual coding for this project so, where possible, we would ask any participating funder to provide their submission already coded. There are some options available to help with this;
- Training staff in manual coding, which can be provided by the Project Management Team
- Paying for external manual coding (‘contract coding’)
- Submitting awards to be auto-coded
Please note we would still ask individual funders to determine which awards are of health relevance, although the central HRCS Project Management Team may be consulted to help in this process for new funders if required.
I/My organisation does not have capacity for HRCS coding, but we would still like to participate. Is this possible?
Of course. Absence of coding or code-able data does not prevent inclusion in the analysis, only how the data is reported. Without coding at a project level it would not be possible to incorporate your data into the main analysis. However we do a separate assessment of the wider funding environment, including support for infrastructure, capacity and other uncoded data sources. The data would be presented as an aggregate figure and would ensure your contribution to the health research environment is included.
My organisation has provided funding / is co-funding an award with another organisation. How do we collectively report this?
This will be very dependent on how the collaboration works between your organisations, and how each report the award in their own database.
Ideally, each organisation will report only the amount they have spent, and will record the award with the same details for PI, Title and Abstract. This should ensure both are coded identically.
If the collaboration involves the funders providing different areas of support, then the title/abstract should reflect this and allow coders to provide coders relevant to the areas supported. For example if one funder provides the funding for the research itself, and the other provides funding for equipment of admin, the former would be coded as normal and the latter would fall under the IndirectAward category of ‘Infrastructure’.
The problem will be if one funder provides a TotalAward value including the funding from the other, leading to duplication and double counting of award funding.
For this reason, we’d prefer funders to indicate the award is a collaboration in the Comment column, and (if possible) provide their co-funder’s name/acronym and grant number in this section so we can pair up the awards correctly. This is difficult to do, but it would be of huge benefit if achieved.
My organisation provides ‘block grants’; awards designed to provide the recipient organisation the flexibility to redistribute funding themselves. Common examples of this include Doctoral Training Partnerships (DTPs) which can fund multiple studentships from a single award. However these awards can come with little, if any, award information with which to HRCS code. What should I do?
In an ideal world the original block grant would be sufficiently focused to allow a ‘best guess’ coding approach. However in reality this is not always possible and in most situations the IndirectAward category should be applied. More details on this categorisation is available via the Submission Guidance. If your organisation has sufficient information on the individual projects funded through the block grant (such as PhD studentship abstracts), you are welcome to submit these instead.
How do I handle award extensions?
Sometimes funders provide additional funding to an established award, to extend the duration or provide specific funding for additional resources. In these cases the title/abstract for both original (parent) and extension (child) awards will be the same/very similar. The awards may also be linked in some fashion within a funder’s database (e.g. partially matched grant references).
The optimal method for reporting this is to combine the additional funding to the original award, and apply HRCS coding only once to the combined award. If the further funding changes the lifetime of the award, make sure this is incorporated into your StartDate, EndDate, Duration and TotalAward/AnnualAward values.
If this combined award is too difficult or resource intensive to edit within your portfolio / spreadsheet submission, our main requirement is that both parent and child awards MUST BE coded identically and a note made in the Comment column to indicate that the two awards are linked.
There are two exceptions to these rules:
- If the additional funding falls into the category of IndirectAward (e.g. if the extra funding is for administrative support, not research) – see the guidance on this for more details.
- If the additional funding is for a new area of research that would have to be coded differently.
In both these situations the title/abstract should make this clear and will, in all likelihood, be different to the original. In these cases, the extension is handled as its own separate award and therefore you should follow your normal procedures for coding. It would still be useful to indicate the link between the two awards in the Comment column, but this is not mandatory.
I have awards where there are two or more principal investigators (Co-PIs), which may include co-PIs from different institutions. Which PI details / Institution do I use?
If there is some indication of which PI has main responsibility for the award, then use that person’s details and institution. For example, if designated as the primary point of contact (PoC), corresponding author etc..
If this information is not available, I’m afraid the best we can suggest is pick one. In both cases this is an imperfect solution, but will be simplest for funders to provide in the limited timescale of the project. It is one area we hope to review in the future.
How do we handle reporting studentships?
Past experience has shown that funders support studentships in a variety of different ways. Many use block grants (see above) and as a result finding sufficient detail to HRCS code individual studentships can be difficult. In addition there are often greater data protection / sensitivity issues on studentships than other award types.
We would advise that if you can include studentships with sufficient details to allow HRCS coding. This would give us the most nuanced picture of health research. However if such details are unavailable, we are happy to accept awards in whatever format you can provide. Note that if there are insufficient details to HRCS code, we may have to handle these studentships in the separate indirect/support funding assessment; see our Studentships guidance on IndirectAwards for more details.
Why are you analysing data from a calendar year, rather than financial year?
The short answer is that different funders collect data at different times of the year, and we’ve established that finalising data early in Q1 of a calendar year is the optimal time to do so.
While it’s true that many UK funders use the standard UK financial year (1st of April to 31st of March) this is not true of all. In addition, many funders will not have data on the previous financial year fully available until some months after the financial year end, as it has to be collated from a range of different stakeholders. Finally, several funders collect award data and/or HRCS coding of their awards at set times of the year, with several using the relatively quiet Christmas/New Year period to do so.
More practically the way we report on funding (see the TotalAward vs AnnualAward information below) is a little different to ensure we can get a standard approach across all funders, so we tend to avoid using true expenditure as a means to value awards.
Why are you requesting TotalAward or AnnualAward instead of spend?
The HRCS report is intended as a snapshot of health research funding in the UK. As such, getting a value for research spending is critical and a precise figure of ‘spend’ has merits as a better metric to use for this than an estimation.
However the practicalities of achieving this make acquisition of this ‘spend’ difficult. Many funders/awardees do not report regularly, and if they do it is at specific times within a year, most commonly the end of the financial year.
As a result, most funders will not be able to provide an accurate value to amount spent on any particular award. However all funders should have the total award value (i.e. how much funding has been allocated) or, less commonly, how much they assign to a particular award annually.
Furthermore, most research funding is ‘front loaded’ meaning the majority of actual spend occurs earlier in the lifetime of the award, which if used as a metric would give greater weighting to new awards over those reaching the end of the award period.
Therefore, in the interests of simplifying the process for all funders, and to reduce the impact of front loading in awards, we are requesting TotalAward (or AnnualAward if your funder uses this system). From either of these values we can generate an estimate of award value for the 2014 calendar year.
Some of our awards do not have a set total value (e.g. on-going investments without a fixed end date), and do not have a predetermined annual allocation either. How do we account for these types of award?
This is difficult and will vary from funder to funder. Clearly we would like awards to match the provided criteria as best we can, but more practically there will be certain types of award where this is not possible and we would prefer a ‘best we can’ answer than awards without values at all.
In these circumstances we would ask you provide a value that best reflects the award, which could include:
- The recommended budget from the last review; which would be best represented by this value entered in the TotalAward column, with StartDate and EndDate being the review period.
- The last full annual expenditure from a large investment; which would be best represented in the AnnualAward column, even if it’s not strictly within the required calendar year inclusion criteria.
Whatever the circumstances we recommend you contact the project team to discuss the specifics involved, and that any deviations from the prior criteria be noted in the Comments column on submission.
My award has start and end dates, but no duration in our database. How do I calculate this value?
So long as you have provided an award StartDate and EndDate in DD/MM/YYYY format, we will calculate durations for all submitted awards centrally, using the following excel formula to generate a value for duration in days (rounded up):
However if you are able to do so we recommend that funders use this method to check their data for anomalies before submitting. This is because if there are any errors (e.g. start and end dates mixed up) it is easier for funders to identify them themselves and fix them before submitting, rather the project management team identify a problem and having to query it
I have newly awarded grants in 2018, but we do not have a confirmed start date or whether there is any spend. What do I do?
Generally we would define StartDate as the ‘actual’ start date (i.e. date at which first spend is incurred) if it’s available but in many cases this is not. As a result, we would suggest using a predicted/expected start date.
Please note we can’t include awards in the 2018 data without a StartDate but remember that you will have until the end of the data collection period to submit awards. Thus, if you have newly awarded grants and are currently unsure if they will meet our inclusion criteria, keep an eye on them and submit as a ‘top up’ if you confirm their eligibility before the close of the data collection period.
My database does not have abstracts available for all awards (e.g. they are not stored in electronic format / extractable from our databases). Am I still able to submit them?
In an ideal world, and if it was not too arduous for the analysts to do so, we would want all abstracts in the electronic dataset you provide. The more information we have available in the final combined database, the more use it will be for further analysis. It’s also worth noting that Lay Abstracts can also be submitted for this analysis. While less suitable for coding, they can be utilised if no technical abstracts are available.
However so long as you have been able to code awards from other sources (e.g. from paper applications) then you are still very welcome to submit the awards to the main analysis. However, any awards submitted without coding AND insufficient detail to apply coding by other means can still be counted but only as IndirectAwards, effectively adding to the total value of R&D funding but not the main analysis.
If you have any other questions regarding the UK Health Research Analysis 2018, please contact Dr. Jim Carter at the MRC via the HRCS website.