The Direct Observation of Procedural Skills (DOPS) Validity and Reliability Set for Occupational Therapy of Splint Fabrication

Purpose: The objectives of this study were to investigate the content validity, test-retest and intra-rater reliability characteristics of the Directly Observed Procedural Skills (DOPS), measuring Occupational Therapy Students in performing the fabricating skill of splint and to help inform decisions about the outcome of students' and postgraduate staffs' training. Methods: We developed Occupational Therapist DOPS for Taiwan, established content items and verify the validity, reliability by conducted 14 Registered Occupational Therapists (ROT) from Chung Sun Medical University Hospital in Taichung City Taiwan. Results: DOPS for Taiwan has excellent content validity, total CVI =0.94. Individual item-CVI, had 4 items to give total agreements. For reliability analysis, we conducted the Item-Deleted Cronbach's Alpha from 0.61 to 0.98 and the correlation coefficient from 0.55 to 0.90. For test-retest reliability showed good stability over time, Pearson's correlation coefficient α was 0.812 for the two separate time raters, and intra-rater reliability, there was satisfactory with good internal consistency for subscales, 'Cronbach's Alpha = 0.906, 95% C.I. = 0.800-0.968 (p<0.01). Conclusion: This is the first attempt to evaluate the use of comprehensive workplace assessment across occupational therapists in Taiwan. The methods are feasible to conduct and can make reliable distinctions between OTs performances. With our adaptation, they may be appropriate for assessing the performance of other grades and specialties of ROT. This may be helpful in informing foundation assessment.