TY - JOUR
T1 - Comparing fidelity monitoring methods in an evidence-based parenting intervention
AU - Tiwari, Ashwini
AU - Whitaker, Daniel
AU - Self-Brown, Shannon
N1 - Funding Information:
Funding. These data were part of CDC funded study. The authors were responsible for all data analyses and interpretation.
Publisher Copyright:
© 2021, Emerald Publishing Limited.
PY - 2021
Y1 - 2021
N2 - Purpose: Two common methods in community settings of assessing program fidelity, a critical implementation component for program effectiveness, are video and audio recordings of sessions. This paper aims to examine how these two methods compared when used for a home-based behavioral parenting-training model (SafeCare®). Design/methodology/approach: Twenty-five SafeCare video-recorded sessions between home visitors and parents were scored by trained raters either using the video or audio-only portions of recordings. Sessions were coded using fidelity checklists, with items (n = 33) classified as one of two fidelity aspects, content [delivery of program components (n = 15)], or process [communication and rapport building (n = 11)]. Seven items were considered to overlap between constructs. Items were coded as having been done or not done appropriately. Coders rated items as “technological limitation” when scoring methods hindered coding. Analyses compared percent agreement and disagreement between audio and video coders. Findings: Overall agreement between coders was 72.12%. Levels of agreement were higher for content items (M = 80.89%, SD = 19.68) than process items (58.54%, SD = 34.41). Disagreements due to technology limitations among audio coders were noted among 15 items; particularly, higher levels of disagreement were seen among process items (42.42%) than content items (9.64%). Originality/value: Compared to video, fidelity monitoring via audio recordings was associated with some loss of process-related fidelity. However, audio recordings could be sufficient with supplements such as participant surveys, to better capture process items. Research should also examine how content and process fidelity relate to changes in family behavior to further inform optimal fidelity monitoring methods for program use.
AB - Purpose: Two common methods in community settings of assessing program fidelity, a critical implementation component for program effectiveness, are video and audio recordings of sessions. This paper aims to examine how these two methods compared when used for a home-based behavioral parenting-training model (SafeCare®). Design/methodology/approach: Twenty-five SafeCare video-recorded sessions between home visitors and parents were scored by trained raters either using the video or audio-only portions of recordings. Sessions were coded using fidelity checklists, with items (n = 33) classified as one of two fidelity aspects, content [delivery of program components (n = 15)], or process [communication and rapport building (n = 11)]. Seven items were considered to overlap between constructs. Items were coded as having been done or not done appropriately. Coders rated items as “technological limitation” when scoring methods hindered coding. Analyses compared percent agreement and disagreement between audio and video coders. Findings: Overall agreement between coders was 72.12%. Levels of agreement were higher for content items (M = 80.89%, SD = 19.68) than process items (58.54%, SD = 34.41). Disagreements due to technology limitations among audio coders were noted among 15 items; particularly, higher levels of disagreement were seen among process items (42.42%) than content items (9.64%). Originality/value: Compared to video, fidelity monitoring via audio recordings was associated with some loss of process-related fidelity. However, audio recordings could be sufficient with supplements such as participant surveys, to better capture process items. Research should also examine how content and process fidelity relate to changes in family behavior to further inform optimal fidelity monitoring methods for program use.
KW - Child maltreatment
KW - Fidelity
KW - Implementation
KW - Parent-training programs
UR - http://www.scopus.com/inward/record.url?scp=85101430297&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85101430297&partnerID=8YFLogxK
U2 - 10.1108/JCS-01-2020-0005
DO - 10.1108/JCS-01-2020-0005
M3 - Article
AN - SCOPUS:85101430297
SN - 1746-6660
VL - 16
SP - 104
EP - 116
JO - Journal of Children's Services
JF - Journal of Children's Services
IS - 2
ER -