The METRIQ Research Agenda

The MedEd LIFE Research Collaborative has been hard at work on a research agenda evaluating blogs and podcasts for the past three years. We have attempted to quantify their growth and development (studies 1-3), quantify the impact (study 4), outline ways to improve them (studies 6-7), and develop tools to evaluate their quality (studies 8-13). The METRIQ study will bring all of this work together in an effort to validate the quality evaluation tools that have been developed.


We understand that free access to research is important - particularly so when it is on the topic of Free Open Access Medical education (FOAM). For that reason we have done everything in our power to make the publications that led to and result from the METRIQ Study accessible. This page links to our formal journal publications as well as freely archived PDF versions of the articles in our research agenda. When free access to the published PDF is not available through a recognized repository and publication rules allow, we will self-archive a post-refereed version of our manuscripts. If you ever have difficulty accessing literature that has been published by our group, please CONTACT US so that we can personally share a copy with you. We are readily available via email and on Twitter.

1 This study quantified the number of emergency medicine and critical care blogs and podcasts.


Read it in EMJFree PDF from eScholarship!

2 This survey study evaluated the use of various open educational resources by emergency medicine residents and program directors.

Read it in CJEMFree PDF from CJEM!

3 This scoping review provided an overview of the landscape of digital products that were historically used for medical education.

Read it in The WinnowerFree PDF from The Winnower!


4 This study derived and provided validity evidence for the use of an impact factor for blogs and podcasts: the Social Media Index.


Read it in WestJEMFree PDF from eScholarship!


5 Building on historical paradigms, this innovation report explored how ALiEM incorporated expert peer review could be into a medical education blog.

Read it in CJEMFree PDF from CJEM!


6Building on 5, this innovation report explored how a new model of "Coached Peer Review" was implemented at CanadiEM to foster junior authors and editors.

Read it in Academic Medicine


7 This study shifted the focus to defining quality. A systematic review and quality analysis were conducted to develop quality indicators for blogs and podcasts.

Read it in JGME


8 This Delphi study built consensus among a group of emergency medicine physicians on which quality indicators were the most important.


Read it in Annals of EMFree PDF from Annals of EM!


9 This Delphi study built consensus among a group of medical educators on which quality indicators were the most important.


Read it in PMJFree PDF from PMJ!


10 This paper translated the information in studies 8 and 9 by developing user-friendly checklists for blogs and podcasts.


Read it in The WinnowerFree PDF from The Winnower!


11 This study evaluated the ALiEM AIR Score for validity and reliability as a measure of assessment for blog posts.


Read it in Annals of EMFree PDF from Annals of EM!


12 This study evaluated the gestalt recommendations of blog posts by medical students, residents, and attendings.


Read it in Perspectives on MedEdFree PDF from Perspectives on MedEd!


13 This study combined the data from studies 8 and 9 along with new information to derive two quality "decision rules", METRIQ-5 and METRIQ-8.


Read it in WestJEMFree PDF from WestJEM!


16 This study compares the reliability of the METRIQ-8 and ALiEM AIR Score to the community standard of average gestalt quality assessment.

Read it in Teaching and Learning in Medicine


14 This paper describes how the METRIQ Study recruited 309 participants from a virtual community of practice to complete a 90-120 minute survey.


Read it in AEM: E&TFree PDF from AEM: E&T!


17 This study compares average gestalt, METRIQ-8, and ALiEM AIR to the Social Media Index to investigate the latter's relationship with quality.


Read it in Annals of EM


15 The first results from the METRIQ Study reveal that gestalt quality evaluation of blog posts quality is inadequate. We need to do better!


Read it in Annals of EMFree PDF from Annals of EM!


18 This study assessed the editorial processes and affiliations of the top 100 emergency medicine blogs.


Read it in Academic Emergency Medicine: E&T


13 This study combined the data from studies 8 and 9 along with new information to derive two quality "decision rules", METRIQ-5 and METRIQ-8.


Read it in WestJEMFree PDF from WestJEM!


16 This study compares the reliability of the METRIQ-8 and ALiEM AIR Score to the community standard of average gestalt quality assessment.

Read it in Teaching and Learning in Medicine


14 This paper describes how the METRIQ Study recruited 309 participants from a virtual community of practice to complete a 90-120 minute survey.


Read it in AEM: E&TFree PDF from AEM: E&T!


17 This study compares average gestalt, METRIQ-8, and ALiEM AIR to the Social Media Index to investigate the latter's relationship with quality.


Read it in Annals of EM


15 The first results from the METRIQ Study reveal that gestalt quality evaluation of blog posts quality is inadequate. We need to do better!


Read it in Annals of EMFree PDF from Annals of EM!


18 This study assessed the editorial processes and affiliations of the top 100 emergency medicine blogs.


Read it in Academic Emergency Medicine: E&T


19 This analysis of METRIQ Blog Study data led to the creation of the revised METRIQ (rMETRIQ) Score.


Read it in AEM: E&T


20 This commentary provides an overview of the literature on the critical appraisal of emergency medicine open educational resources such as blogs and podcasts.

Read it in AEM: E&T


21 This manuscript outlines the first systematic review of emergency medicine open educational resources. It utilized the rMETRIQ Score to assess resource quality.

Read it in AEM: E&T


22 A real-world observational survey about podcast usage for medical and health professions education.

Read it in the Canadian Journal of EM


23 The reliability of rating the quality of podcasts is reported by our group in this study.


Read it in Perspectives on Medical Education


24 A scoping review of how social media has been used for knowledge translation and education.

Read it in Perspectives on Medical Education


25 A rapid review of recent literature for tools and decision aids for assessing quality of open access medical education resources.

Read it in Seminars in Nephrology


26 A knowledge translation primer for social media usage for scientific communication.



Read it in the African Journal of EM


27 A narrative commentary on the founding and evolution of #FOAMed and FOAM resources.


Read it in ATS Scholar


28 An international group of educators propose guidelines for the acknowledgement of online education resources in academic promotion.

Read it in Seminars in the WestJEM


30 An overview of good practices in harnessing social media for scholarly discourse, knowledge translation, and education.


Read it in Perspectives on Medical Education


29 This analysis of METRIQ Blog Study data resulted in the creation of the revised Aprroved Instructional Resources (rAIR) score.

Read it in AEM: E&T