Activities of the Working Group


Recommendations on How to teach health IT evaluation published

The recommendations describe scope and content for health IT evaluation courses on master and postgraduate level:

Ammenwerth E, de Keizer N, Brender J, Craven C, Eisenstein E, Georgiou A, Khairat S, Magrabi F, Nykänen P, Otero P, Rigby M, Scott P, Weir C. How to Teach Health IT Evaluation: Recommendations for Health IT Evaluation Courses. Stud Health Technol Inform. 2017;243:3-7.


Workshops on "Recommendations for Evaluation Courses" at MIE 2015, AMIA 2015 and Medinfo 2015

Together with the IMIA WG “Technology Assessment and Quality Development”, the AMIA WG “Evaluation” and the EFMI WG “Education in Health Informatics”, the WG organized a workshop at MIE 2015 in Madrid entitled "Courses on Health IT Evaluation: Development of Recommendations”. This workshop continued the discussions on recommendations for health IT evaluation courses.

Together with the AMIA WG “Evaluation”, the AMIA WG “People and Organizational Issues”, the AMIA WG “Education” and the IMIA WG “Technology Assessment and Quality Development”, the WG organized the workshop “Biomedical and Health Informatics Curricula: Development of Recommendations for Evaluation Courses” at AMIA 2015. This workshop continued the discussions on recommendations for health IT evaluation courses.

Together with the IMIA WG “Technology Assessment and Quality Development”, the WG organized a workshop entitled “Courses on Health IT Evaluation: Development of Recommendations” at Medinfo 2015 in Sao Paolo. This workshop continued the discussions on recommendations for health IT evaluation courses.

Together with the IMIA WG “Technology Assessment and Quality Development”, the WG organized a workshop entitled “Why evidence-based health informatics should have theoretical foundations: Exploring the implications for policy and evaluation” at Medinfo 2015 in Sao Paolo.


Workshop at MIE 2014, Istanbul

During the Medical Informatics 2014 conference in Istanbul, the EFMI WG EVAL organized a workshop on "Health IT Evaluation in Health Informatics Curricula". In this workshop, recommendations for the structure, content and methods for an health IT evaluation course were discussed. In addition, examples of health IT evaluation courses were presented.

Here you can find the description of nine health IT evaluation courses from different countries (ZIP, 150 KB).

In addition, a business meeting took place to discuss future activities. Participants of the business meeting at MIE2014:

(from left to right: Philip Scott, Hannele Hyppönen, Christopherx Pearce, Nicolet de Keizer, Elske Ammenwerth, Andrew Georgiou, Usman Iqbal).


Medinfo2013 in Kopenhagen

The WG organized a workshop with 40 - 50 participants: "Evaluation and Evidence: Applying guidelines for health IT evaluation in developed and developing countries":
Slides of the presentations
(PDF, 3 MB)

The IMIA and EFMI WGs, with participation of AMIA WG, also had a business meeting to discuss future activities. Please find all information and slides here.

At this opportunity, the WG members thanked Nicolette de Keizer for her long-year work as IMIA WG chair. She has stepped down now and is followed by Farah Magrabi and Andrew Georgiou as new IMIA WG co-chairs.

The core team of the EFMI WG and the IMIA WG:


(from left to right: Michael Rigby, Marie-Catherine Beuscart-Zephir, Pirkko Nykänen,
Catherine Craven, Elske Ammenwerth, Jytte Brender, Nicolet de Keizer, Farah Magrabi, Andrew Georgiou).


MIE2012 in Pisa

Workshop: Developing quality indicators for IT interventions in health care (N de Keizer, E Ammenwerth, H. Hyppönen, M. Rigby)

Workshop programm

Workshop slides

Workshop results


MIE 2011

The working group organized or participated in several activites at MIE2011:

Hannele Hypponen, Elske Ammenwerth, Christian Nøhr, Åke Walldius. Towards an International Minimum Dataset for Monitoring National Health Information System Implementations. WORKSHOP. Slides:

Jan Talmon, Michael Rigby, Sylvia Pelayo, Elske Ammenwerth. The Medical Device Directive and Software: The implications. PANEL. Slides of the panel.

Michael Rigby, Jytte Brender, Marie-Catherine Beuscart-Zephir, Hannele Hyppönen, Pirkko Nykänen, Jan Talmon, Nicolette de Keizer, Elske Ammenwerth. Next Steps in Evaluation and Evidence - from Generic to Context-Related. PRESENTATION.

Jan Talmon, Michael Rigby, Nicolette de Keizer Reporting of Health Informatics Evaluation Studies. WORKSHOP.

Evaluation WG Business Meeting. Here you can find a transcript of the discussion of this business meeting, including a list of planned and optional activities.


Medinfo 2010

Members of the working group organized the following activities at Medinfo2010:


MIE 2009

The working group had a successful workshop on GEP-HI: Introducing guidelines for good evaluation practice in health informatics at MIE2009 in Sarajevo, Aug30-Sept2 2009. Please find some slides of this workshop here.


MIE 2008

The working group had a successfull workshop with > 60 participants on "Guidelines for good evaluation practice and reporting" at MIE 2008 in Göteborg, 26. - 28. May 2008.

Please find here the workshop schedule and the minutes of the workshop.

Please find here the slides: GEP-HI (jytte Brender), STARE-HI (Jan Talmon), Future Plans (Nicolette de Keizer)


Medinfo 2007

Scientific presentation by Jan Talmon, Elske Ammenwerth, Thom Geven: The quality of reporting of Health Informatics evaluation studies. Describes a first study using STARE-HI to evaluated the quality of evaluation reports.

Workshop presentation by Pirkko Nykänen: Good evaluation practice guidelines for health informatics GEP-HI. Presents the state of work on GEP-HI.


MIE 2006

The working group had a successful workhop at MIE 2006 on Monday August 28th, 2006 in Maastricht on "Publication of Evaluation Studies - Challenges and Guidelines for Authors".

You can find the programme of this workshop here.

Here are the materials of the workshop:


MIE 2005

The working group had a successful workhop at MIE 2005 on Tuesday 30th August in Geneva on "Development of Good Evaluation Practice Guidelines in Health Care".

You can find the programme of this workshop here.

Here are the materials of the workshop:

NEW: Report on the outcome of the workshop

In this context, please find here the slides of the MIE2005 Keynote on "Without evaluation no proper implementation" (1,1 MB). This track keynote was held by Jytte Brender as stand-in for Jan Talmon.


Medinfo 2004 San Fransisco

The Working Group has co-organized a workshop at Medinfo 2004 (September 7th 2004) on "Good evaluation of Health Informatics Applications", together with the IMIA Working Group on Technology Assessment and Quality Improvement. You can here find the links to the handhouts and slides:

Workshop programm (Doc) and Agenda (PPT)

Introduction: EFMI WG EVAL and the Declaration of Innsbruck (Ammenwerth)

The Declaration of Innsbruck (published in IJMI 73, 2004, 479-491)

GEP-HI: Approaches towards Good Evaluation Practice Guidelines - Slides (Nykänen)

GEP-HI: Approaches towards Good Evaluation Practice Guidelines - Handout (Nykänen)

STARE-HI: Standards for Reporting on Evaluation Studies (Talmon)

Overview an assessment methods: Handout and Slides (Brender)

We also had our business meeting there at Medinfo 2004, at Wednesday, 8 September 2004.


EFMI STC 2004: Contribution of Medical Informatics to Health

The working group has co-organized the EFMI Special Topic Conference 2004 in Munich, June 14th - 15th 2004. The topic of this event was "The contribution of Medical Informatics to Health". A special issue of this confernece, co-edited by the working group, is just being prepared.

The working group organized a tutorial on "introduction to evaluation of health information systems" on Sunday June 13th. You can find the programme as well the slides and other materials in the next paragraphe:

Part 1: Introduction to Evaluation

Introduction (Ammenwerth)

Basic of Evaluation of ICT in Health Care - Part 1 (Talmon)

Basic of Evaluation of ICT in Health Care - Part 2 (Brender)

Part 2: Recent and future activities of EFMI WG Evaluation of Health Information Systems

Overview on aims and activities of EFMI WG EVAL (Ammenwerth)

Good Evaluation Practice Guidelines for health informatics - GEP-HI (Nykänen)

Standards for Reporting of Evaluation Studies in Health Informatics - STARE-HI - and Evaluation Portal (Talmon)

Additional material

Evaluation of Health Informatics Applications in a Lifecycle Perspective (Brender; published in EFMI STC 2004 proceedings)

Visions and Strategies to improve evaluation of health information systems (published in IJMI 73, 2004, 479-491)


MIE 2003

The working group organizes two workshop and one tutorial during MIE 2003, 4 - 7 May 2003, St. Malo, France:

More information


Workshop 2004

On 4th - 6th April 2003, in Innsbruck an ESF Exploratory Workshop on New Approaches to the Systematic Evaluation of Health Information Systems took place. The number of participants (many of them were members of this EFMI working group) was restricted. Some more information.


Homepage of the Working Group
Webmaster: elske.ammenwerth@umit.at
Last change: 22.09.2017