The revised version of the CERIF4REF schema, designed to make the compilation of CERIF-compliant data for the REF exercise easier, is now available on the CERIF4REF page of this site. This version has been revised to incorporate impact statements as proposed by the MICE model outlined below. The associated XSLT stylesheet cerif4refimpact2cerif.xsl generates CERIF files conforming to the MICE model in the CERIF format which we have proposed (although note, that these have no official status yet as part of the CERIF standard – hopefully this will change).

Two documents produced by Jacqueline Cooke and Tahani Nadim at Goldsmiths, which analyse the proposed MICE methodology in the context of live information environments, are now available on the Documents page. The first, Measuring impact under CERIF at Goldsmiths, also available here, assesses Goldsmiths’ current data and information provisions vis-à-vis the impact measure schema and makes several recommendations based on their findings. The second, available here, documents the findings of an extensive survey carried out by the Goldsmiths team on the feasibility of adopting this approach.

Many thanks to Jacqueline and Tahani for this excellent work.

The feasibility study on integrating the CERIF impact model advocated here in PURE is now available on the Documents page, or by clicking here. Many thanks to the team at St. Andrew’s and Denmark for this excellent work!

The MICE project’s core output, the data model for impact under CERIF, is now available: this can be found as item 2 A proposed implementation of CERIF for encoding impact indicators and measures on the Documents accessible from the menu link above, or directly by clicking here.

The model outlines an overall architecture as follows:-

The basic linkage expressed is between any CERIF base object (OrgUnit, Project, Person etc) , an impact indicator and an impact measure. In turn the impact indicator and impact measure are themselves linked. In addition, it is possible to further link the impact indicator and impact measure to another CERIF base object which is the recipient or beneficiary of the impact.

For further details, including some XML examples illustrating the model in more detail, please see the data model document.

Keith Jeffery, Richard Gartner, Brigitte Joerg and Mark Cox met for a second time on 14th April to work further on the mapping of impact indicators and measures to CERIF. This was another very productive meeting, at which we drew up an agreed methodology: I’ll write this up over the next few days and post it here for comments. Watch this space!

Draft conclusions of first MICE workpackage 3 meeting 28 March 2011

These are the tentative conclusions drawn up at a meeting between Mark Cox, Richard Gartner and Keith Jeffery to assess the potential of mapping the list of impact indicators and measures drawn up at the output of workpackage 2 to CERIF.

The following initial observations were made:-

• the list of indicators and measures needs to be reduced to make it feasible to map to CERIF
• a temporal component has to be included as REF measures impact from 1993 to 2013
• the scheme should aim for the finest level of granularity (allowing aggregation if necessary)
• the scheme should aim for the finest level of granularity (allowing aggregation if necessary)
• impact must relate to all CERIF base entities, including orgUnit (UOA), person and project.

Role of CERIF

We envisage CERIF fitting into the overall information environment as shown in Figure 1: a variety of information sources may be used to aggregate data to fill CERIF’s impact measure components, or these may be input directly. Information users, such as the REF itself, will access these measures via impact indicators which will be recorded semantically in CERIF’s linking mechanisms.

Overall architecture

The overall architecture proposed is shown in Figure 2. Each base entity is joined to impact measures via a CERIF linking table: these include the impact indicators as semantics. Optionally, the impact measure may be linked to another entity (shown on the right of the diagram) which experiences the effect of the impact. So, for example, a research project may be joined by a semantic link of the type ‘improved healthcare’ to the NHS via an impact measure for decreased mortality rates.

Semantics of measures

A taxonomy for measures divides them into a small number of overall outcomes and forms of impact as follows:-

Each component is identified below by the letter given in brackets in the table above..

At the moment we envisage creating nine tables for each combination of the above – WN, WB, WS, QN, QB, QS, FN, FB, FS – which is necessitated by the varying semantics of each, although we may investigate simplifying this.

We envisage the following overall structure for tables of impact measures:-

This will apply to all tables except for those with B as their type of impact, in which cases the Value measure column will contain a change in value rather than an absolute value (for instance, increased profits).

These interim conclusions will now be discussed at our next meeting on April 14^th.

The results of our survey of existing work on impact indicators and measures is now available in the Documents section of this site. This presents a lengthy list of these taken from a number of key source documents: this list is arranged hierarchically in a single tree diagram.

This will now form the basis of our work on the next workpackage, in which it will be mapped to CERIF: work on this will start on Monday 28th with a meeting between Richard Gartner, Mark Cox and Keith Jeffery at which we aim to make a solid start on the mapping.

A meeting held this afternoon between Richard Gartner, Mark Cox, Anna Clements and Brigitte Joerg reviewed the mapping of impact indicators and measures which had come out of our earlier meeting. All were happy with what we’d put together, and, apart from a few minor tweaks, we’ll be publishing it on this site early next week.

The resulting document will then form the basis of the next workpackage, in which we’ll be mapping our findings to CERIF.

Richard Gartner, Mark Cox, Anna Clements and Brigitte Joerg from the the DFJI met in London on 28th February to look at the results of our research so far on existing work on impact indicators. The materials mentioned in the earlier blog post were examined in detail, and the indicators and evidence measures they suggested were extracted and put into a hierarchical tree structure, which allowed us to gain an overall view of the impact landscape as it currently stands.

This tree will now be examined and revised  before we publish it to this blog by the end of March. This will then form the basis of the next stage of our work, which will examine the mapping of these indicators to CERIF.

We held our project start-up meeting at King’s College on February 9th. This was intended to allow the project partners to get to know one another and to plan out our work for the coming months. Richard Gartner and Mark Cox  from King’s College. Jacqueline Cooke from Goldsmith’s and Josh Brown from JISC were there in person, and were joined virtually by Thomas Vestdam and Henrik Rasmussen by Skype from Denmark and Anna Clements by phone from St. Andrews.

After looking at the project plan and working out the timetable of work for the coming months, we decided on which pieces of work on impact already undertaken will be looked at in stage one of the project. These include:-

• the REF Consultation document on impact, specifically the indicators in Appendix D (http://www.hefce.ac.uk/pubs/hefce/2009/09_38/09_38.pdf)

• REF impact pilots, such as those on Earth and Environmental Sciences (http://www.hefce.ac.uk/research/ref/impact/EarthSystems_EnvironmentalSciences.pdf) and English Language and Literature (http://www.hefce.ac.uk/research/ref/impact/EnglishLang_Lit.pdf)
• Draft impact case studies already undertaken  at King’s
• The Pure activities model, based at St. Andrew’s, which includes  impact indicators and a  hierarchy hierarchy for them
• The ENQUIRE project final report (http://www.jisc.ac.uk/media/documents/programmes/RIM/RIMEnquire_FinalReport.pdf)
• An Australian paper on assessing impact, Measuring the Impact of Research (02/2007) (http://www.atn.edu.au)/docs/Research%20Global%20-%20Measuring%20the%20impact%20of%20research.pdf)
• The EU publication Assessing Europes’s University-based Research
  (http://ec.europa.eu/research/science-society/document_library/pdf_06/assessing-europe-university-based-research_en.pdf)

Our next meeting on 28th February will examine these in detail and attempt a synthesis of their findings.