Processbeskrivning - Universitetet i Oslo

Processbeskrivning - Universitetet i Oslo

ASSESSMENT OF SOCIETAL IMPACT VR CONNECTIONS STEN SDERBERG NRF 2016-11-08 ENRESSH 2017-01-19--20 NOS-HS 2017-03-27 The concept of societal impact of research delimitations Impact beyond academia rather than impact within academia (e.g,, citations) Impact not the same as co-operation and partnerships with partners outside academia although impact normally presupposes co-operation

Impact not the same as communication and outreach - although impact normally presupposes outreach activities Impact not the same as relevance (an ex-ante judgment) VR definition of the concept of societal impact of research VR definition: effects of research beyond academia which in some contexts and over time could amount to concrete influence on society by the application of research results to achieve social, economic, environmental or cultural effects. Impact assessment not a part of the daily evaluation agenda at VR (neither ex-ante nor ex-post)

Growing interest for impact assessment on European level on national political level at VR and other public research funding bodies at local universities and university colleges A number of proposals and initiatives on impact assessment from VR in recent years Pilot study in the HS area Criminology 2012

Recent VR proposals including impact assessment VR FOKUS proposal 2014: A model for allocating resources to universities and university colleges involving peer review of research quality and societal relevance, enabling resource allocation such that quality and performance are rewarded (not adopted by Swedish government) VR ALF proposal 2013: A proposal for evaluation of ALF-funded clinical research presented by the Swedish Research Council to the Government, September 2013 (adopted/decided, and now being prepared with the first evaluation planned for 2017/2018) (ALF is a Swedish acronym for an agreement between the state and regions on the funding of medical training and clinical research) FOKUS proposal in a nutshell FOKUS Research evaluation in Sweden Purpose: To improve the quality of research and ensure that high quality research is of benefit to society Background

information (not to be graded) 70 % Scientific quality 15 % Quality enhancing factors 15 % Impact outside academia All research to be evaluated in a cohesive manner every sixth year ALF proposal in a nutshell (modified) ALF a model for peer review-based quality evaluation of ALF-financed clinical research in Sweden

50 % Scientific quality 25 % Clinical relevance and societal utility of the research 25 % Preconditions for (clinical) research Proposed/preferred methods for impact assessment in VR:s proposals Expert review by panels of national and/or nordic experts with experience from societal use or application of research findings

Material for expert assessments: - FOKUS: Case studies complemented by self-evaluations (British model) - ALF: Self-evaluations complemented by case studies (Dutch model) Quantitative indicators can be used but only as parts of the selfevaluations or/and case studies; and can vary between research fields Indicators should not directly inform resorce or funding allocation Pilot study on impact of a research programme in the HS area - criminology

https:// publikationer.vr.se/produkt/att-utvardera-effekter-av-ett-grundforskningsp rogram-en-pilotstudie/ Pilot study on criminology methodological conclusions Programme theory/Logic model (useful tool) Case study approach (successful) Tracking of references (possible but time-consuming; hardly costeffective)

Up-stream approach more successful than down-stream apparoach (start with the effect rather than the cause) Some issues to consider in impact assessment Time-frames Attribution vs. contribution Positive effects vs. avoiding negative effects Unintended effects

COMPONENTS AND METHODS FOR SSH EVALUATION IN THE VR PERSPECTIVE AN OVERVIEW Evaluation of scientific quality, e.g - Novelty & Originality - Significance for the field - Scientific reliability and methodological rigour Peer review - Reading and assessment of actual scientific texts (samples) Classification of journals and publishers Citations and references Narratives/case studies Quantitative indicators A

Evaluation of societal impact and relevance, e.g. - Reach - Significance N/A A N/A N/A N/A N/A A? A N/A VR Standpoint on the issue of societal impact of

research as expressed to Science Europe There is a need to continue efforts to develop tools and methods that can demonstrate the value and impact research and innovation has in society. This is important in order to facilitate better informed research policy decisions, to gain and maintain trust from politicians to receive continued large funding but also to inform, gain and maintain trust from citizens. We recognize that this is not an easy task. Our premise is that research does benefit society from the perspective of a knowledge economy. Assessing the relevance and societal benefit of research can then be done by means of an assessment of how it contributes to knowledge and skills development in society. We propose that the term impact of research be used to describe the effects of research beyond academia. Impact means, in a broad sense, effects of research beyond academia which in some contexts and over time could amount to concrete influence on society by the application of research results to achieve social, economic, environmental or cultural effects. Impact beyond academia thus refers to the dissemination, further refinement, commercialisation, patenting, licensing or other practical use of research results. Methodological diversity is recommended for the proper assessment of the impact of research on society. Evaluation techniques and the availability of specific indicator sets should not drive assessments. Any indicators used as part of an assessment should be integrated into narratives, quantitative models, or both, that attempt to show causal links between the research and observable effects. Sources of inspiration: National systems in other countries UK: Research Execellence Framework (REF; 2014, 2020)

From 2014, impact is part of REF assessed by impact case studies Determines 20 % of the resource allocation Ca 6700 case studies 2014 Several meta-evaluations, positive NL: Standard Evaluation Protocol (SEP) Societal relevance equally important as scientific quality (50 %) No connection to resource allocation Methodological flexibility between research fields allowed AUS: Excellence for Research in Australia (ERA, every 3rd year) So far, no impact assessment Has determined a small part (4-7%) of the resorce allocation; now abolished An impact component is now being developed for the next ERA as a consequence of political pressure and decisions

Several university-based pilot studies of impact e.g. Excellence in Innovation for Australia Trial 2012 (EIA; case studies) Research Engagement for Australia Trial 2015 (REA; indicators/metrics showing industrial investments in university research) Other international initiatives Payback Framework, research-based framework originally developed in the UK to examine the impact or payback of health services research (Hanney, Donovan et. al) International and national networks: AESIS (Network for Advancing & Evaluating the Societal Impact of Science) biennial conferences (next conference Stockholm June 2017) BIN (Broader Impacts Network) formed at MU, USA, in 2012 in response to initiatives from the American NSF ISRIA (International School of Research Impact Assessment) - an Initiative founded by the Catalonian Agency of Health Quality and Assessment (AQuAS), RAND Europe and the Canadian Alberta Innovates Health Solutions (AIHS) in 2013

Recently Viewed Presentations

  • High Risk Pregnancy - WordPress.com

    High Risk Pregnancy - WordPress.com

    High-Risk Pregnancy. Jeopardy to mother, fetus, or both. Condition due to pregnancy or result of condition present before pregnancy. Higher morbidity and mortality. Risk assessment with first antepartalvisit and each subsequent visit . Risk factors
  • This template is UND branded. There are two

    This template is UND branded. There are two

    Choose Custom, and enter the size you want your poster. 42" is the width of our paper; your poster can be wider or longer than this in the other dimension. DO use charts, graphs, pictures. DO use the Insert, Picture,...
  • Assisted vaginal delivery via vacuum extraction Unit 1:

    Assisted vaginal delivery via vacuum extraction Unit 1:

    The occiput is lower than the sinciput in the pelvis when the fetal head is well-flexed. A fetal head in a well-flexed vertex position has a diameter of 9.5 cm. Abnormal positions and defelxion can prolong the progress of labor...
  • PHIL 201 - facstaff.cbu.edu

    PHIL 201 - facstaff.cbu.edu

    Explanans: what does the explaining. Ex. Navel oranges are called by that name because they have a growth that resembles a human navel on the end opposite the stem. ...
  • (Rev 5:8-11) And when he had taken the

    (Rev 5:8-11) And when he had taken the

    Our Hands. 1. Playing Instruments - Psalm 33:2-3 says, "Praise the LORD with harp: sing unto him with the psaltery and an instrument of ten strings, Sing unto him a new song; play skillfully with a loud noise"
  • Being Kinder to Myself Elaine Beaumont, Lecturer University

    Being Kinder to Myself Elaine Beaumont, Lecturer University

    Learning to observe and monitor physical, emotional, behavioural and cognitive reactions using thought records/charts/cost benefit analysis. Cognitive restructuring to examine core beliefs, dysfunctional assumptions and negative automatic thoughts. Compassionate letter writing - focusing on being kind, supportive and nurturing to...
  • Industrialization and Economic Development

    Industrialization and Economic Development

    Brandt Line. Level of Development. GDP= Gross Domestic Product, an approximation of the total value of the final good (finished product) and services produced by a country per year.
  • Atomic Physics Driving Doppler Down Doppler-free methods Laser

    Atomic Physics Driving Doppler Down Doppler-free methods Laser

    Excitation can be to a virtual level, midway between the initial and final state (of opposite parity) - e.g. a 1s to 2s transition in hydrogen. Lyman-α has a wavelength of 126 nm; hence each of the 2 photons needs...