Construct Validity of an Instrument for Assessment of Reflective Writing-Based Portfolios of Medical Students
Construct Validity of an Instrument for Assessment of Reflective Writing-Based Portfolios of Medical Students
Blog Article
Salah Eldin Kassab,1 Mubarak Bidmos,1 Michail Nomikos,1 Suhad Daher-Nashif,2 Tanya Kane,2 Srikant Sarangi,3 Marwan Abu-Hijleh1 1Department of Basic Medical Sciences, College of Medicine, QU Health, Qatar University, Doha, Qatar; 2Department of Population Medicine, College of Medicine, Qatar University, QU Health, Doha, Qatar; 3Danish Institute of Humanities and Medicine (DIHM), Aalborg University, Aalborg, DenmarkCorrespondence: Salah Eldin KassabPhysiology & Medical Education, College of Medicine, QU Health, Qatar University, PO Box 2713, Doha, QatarTel +974 – 4403 7843Email skassab@qu.edu.qaPurpose: Assessment of reflective writing for medical students is challenging, and there is lack of an available instrument with good psychometric properties.The authors jolly rancher filled gummies developed a new instrument for assessment of reflective writing-based portfolios and examined the construct validity of this instrument.
Methods: After an extensive literature review and pilot testing of the instrument, two raters assessed the reflective writing-based portfolios from years 2 and 3 medical students (n=135) on three occasions.The instrument consists of three criteria: organization, description of an experience and reflection on the experience.We calculated the reliability of scores using generalizability theory with a fully crossed design and two facets (raters and occasions).In addition, we measured criterion validity by testing correlations with students’ scores using other assessment methods.
Results: The dependability (Φ) coefficient of the portfolio scores was 0.75 using two raters on three occasions.Students’ portfolio scores represented 46.6% of the total variance across all score comparisons.
The variance due to occasions was airpods in jacksonville negligible, while the student–occasion interaction was small.The variance due to student–rater interaction represented 17.7%, and the remaining 27.7% of the variance was due to unexplained sources of error.
The decision (D) study suggested that an acceptable dependability (Φ = 0.70 and 0.72) can be achieved by using two raters for one and two occasions, respectively.Finally, we found moderate to large effect-size correlations between students’ scores in reflective writing-based portfolios and communication skills (r = 0.
47) and PBL tutorials (r = 0.50).Conclusion: We demonstrated the presence of different sources of evidence that support construct validity of the study instrument.Further studies are warranted before utilizing this instrument for summative assessment of students’ reflective writing-based portfolios in other medical schools.
Keywords: G-theory, validity, reflective writing, portfolio, reliability, student assessment.