Self-Report After Randomly Assigned Supervision Does not Predict Ability to Practice Motivational Interviewing

R. Morgan Wain, Bryan A. Kutner, Jennifer L. Smith, Kenneth M. Carpenter, Mei Chen Hu, Paul Amrhein, Edward V. Nunes

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

The objective of this study was to investigate the relation between self-report and objective assessment of motivational interviewing (MI) skills following training and supervision. After an MI workshop, 96 clinicians from 26 community programs (age 21-68, 65% female, 40.8% Black, 29.6% Caucasian, 24.5% Hispanic, 2.0% Asian, 3.1% other) were randomized to supervision (tele-conferencing or tape-based), or workshop only. At four time points, trainees completed a self-report of MI skill, using items from the MI understanding questionnaire (MIU), and were objectively assessed by raters using the Motivational Interviewing Treatment Integrity (MITI) system. Correlations were calculated between MIU and MITI scores. A generalized linear mixed model was tested on MIU scores, with MITI scores, supervision condition and time as independent variables. MIU scores increased from pre-workshop (mean = 4.74, SD= 1.79) to post-workshop (mean = 6.31, SD= 1.03) (t= 8.69, p< .0001). With supervision, scores continued to increase, from post-workshop to week 8 (mean = 7.07, SD= 0.91, t= 5.60, p< .0001) and from week 8 to week 20 (mean = 7.28, SD= 0.94, t= 2.43, p= .02). However, MIU scores did not significantly correlate with MITI scores, with or without supervision. Self-reported ability increased with supervision, but self-report was not an indicator of objectively measured skill. This suggests that training does not increase correspondence between self-report and objective assessment, so community treatment programs should not rely on clinician self-report to assess the need for ongoing training and supervision and it may be necessary to train clinicians to accurately assess their own skill.

Original languageEnglish
Pages (from-to)96-101
Number of pages6
JournalJournal of Substance Abuse Treatment
Volume57
DOIs
StatePublished - 1 Oct 2015

Fingerprint

Motivational Interviewing
Aptitude
Self Report
Education
Therapeutics
Hispanic Americans
Linear Models
Surveys and Questionnaires

Keywords

  • Community-based supervision
  • Dissemination
  • Evidence-based practice
  • Motivational interviewing
  • Training

Cite this

Wain, R. Morgan ; Kutner, Bryan A. ; Smith, Jennifer L. ; Carpenter, Kenneth M. ; Hu, Mei Chen ; Amrhein, Paul ; Nunes, Edward V. / Self-Report After Randomly Assigned Supervision Does not Predict Ability to Practice Motivational Interviewing. In: Journal of Substance Abuse Treatment. 2015 ; Vol. 57. pp. 96-101.
@article{a100568147b24f52aecd48e467a150e6,
title = "Self-Report After Randomly Assigned Supervision Does not Predict Ability to Practice Motivational Interviewing",
abstract = "The objective of this study was to investigate the relation between self-report and objective assessment of motivational interviewing (MI) skills following training and supervision. After an MI workshop, 96 clinicians from 26 community programs (age 21-68, 65{\%} female, 40.8{\%} Black, 29.6{\%} Caucasian, 24.5{\%} Hispanic, 2.0{\%} Asian, 3.1{\%} other) were randomized to supervision (tele-conferencing or tape-based), or workshop only. At four time points, trainees completed a self-report of MI skill, using items from the MI understanding questionnaire (MIU), and were objectively assessed by raters using the Motivational Interviewing Treatment Integrity (MITI) system. Correlations were calculated between MIU and MITI scores. A generalized linear mixed model was tested on MIU scores, with MITI scores, supervision condition and time as independent variables. MIU scores increased from pre-workshop (mean = 4.74, SD= 1.79) to post-workshop (mean = 6.31, SD= 1.03) (t= 8.69, p< .0001). With supervision, scores continued to increase, from post-workshop to week 8 (mean = 7.07, SD= 0.91, t= 5.60, p< .0001) and from week 8 to week 20 (mean = 7.28, SD= 0.94, t= 2.43, p= .02). However, MIU scores did not significantly correlate with MITI scores, with or without supervision. Self-reported ability increased with supervision, but self-report was not an indicator of objectively measured skill. This suggests that training does not increase correspondence between self-report and objective assessment, so community treatment programs should not rely on clinician self-report to assess the need for ongoing training and supervision and it may be necessary to train clinicians to accurately assess their own skill.",
keywords = "Community-based supervision, Dissemination, Evidence-based practice, Motivational interviewing, Training",
author = "Wain, {R. Morgan} and Kutner, {Bryan A.} and Smith, {Jennifer L.} and Carpenter, {Kenneth M.} and Hu, {Mei Chen} and Paul Amrhein and Nunes, {Edward V.}",
year = "2015",
month = "10",
day = "1",
doi = "10.1016/j.jsat.2015.04.006",
language = "English",
volume = "57",
pages = "96--101",
journal = "Journal of Substance Abuse Treatment",
issn = "0740-5472",
publisher = "Elsevier Inc.",

}

Self-Report After Randomly Assigned Supervision Does not Predict Ability to Practice Motivational Interviewing. / Wain, R. Morgan; Kutner, Bryan A.; Smith, Jennifer L.; Carpenter, Kenneth M.; Hu, Mei Chen; Amrhein, Paul; Nunes, Edward V.

In: Journal of Substance Abuse Treatment, Vol. 57, 01.10.2015, p. 96-101.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Self-Report After Randomly Assigned Supervision Does not Predict Ability to Practice Motivational Interviewing

AU - Wain, R. Morgan

AU - Kutner, Bryan A.

AU - Smith, Jennifer L.

AU - Carpenter, Kenneth M.

AU - Hu, Mei Chen

AU - Amrhein, Paul

AU - Nunes, Edward V.

PY - 2015/10/1

Y1 - 2015/10/1

N2 - The objective of this study was to investigate the relation between self-report and objective assessment of motivational interviewing (MI) skills following training and supervision. After an MI workshop, 96 clinicians from 26 community programs (age 21-68, 65% female, 40.8% Black, 29.6% Caucasian, 24.5% Hispanic, 2.0% Asian, 3.1% other) were randomized to supervision (tele-conferencing or tape-based), or workshop only. At four time points, trainees completed a self-report of MI skill, using items from the MI understanding questionnaire (MIU), and were objectively assessed by raters using the Motivational Interviewing Treatment Integrity (MITI) system. Correlations were calculated between MIU and MITI scores. A generalized linear mixed model was tested on MIU scores, with MITI scores, supervision condition and time as independent variables. MIU scores increased from pre-workshop (mean = 4.74, SD= 1.79) to post-workshop (mean = 6.31, SD= 1.03) (t= 8.69, p< .0001). With supervision, scores continued to increase, from post-workshop to week 8 (mean = 7.07, SD= 0.91, t= 5.60, p< .0001) and from week 8 to week 20 (mean = 7.28, SD= 0.94, t= 2.43, p= .02). However, MIU scores did not significantly correlate with MITI scores, with or without supervision. Self-reported ability increased with supervision, but self-report was not an indicator of objectively measured skill. This suggests that training does not increase correspondence between self-report and objective assessment, so community treatment programs should not rely on clinician self-report to assess the need for ongoing training and supervision and it may be necessary to train clinicians to accurately assess their own skill.

AB - The objective of this study was to investigate the relation between self-report and objective assessment of motivational interviewing (MI) skills following training and supervision. After an MI workshop, 96 clinicians from 26 community programs (age 21-68, 65% female, 40.8% Black, 29.6% Caucasian, 24.5% Hispanic, 2.0% Asian, 3.1% other) were randomized to supervision (tele-conferencing or tape-based), or workshop only. At four time points, trainees completed a self-report of MI skill, using items from the MI understanding questionnaire (MIU), and were objectively assessed by raters using the Motivational Interviewing Treatment Integrity (MITI) system. Correlations were calculated between MIU and MITI scores. A generalized linear mixed model was tested on MIU scores, with MITI scores, supervision condition and time as independent variables. MIU scores increased from pre-workshop (mean = 4.74, SD= 1.79) to post-workshop (mean = 6.31, SD= 1.03) (t= 8.69, p< .0001). With supervision, scores continued to increase, from post-workshop to week 8 (mean = 7.07, SD= 0.91, t= 5.60, p< .0001) and from week 8 to week 20 (mean = 7.28, SD= 0.94, t= 2.43, p= .02). However, MIU scores did not significantly correlate with MITI scores, with or without supervision. Self-reported ability increased with supervision, but self-report was not an indicator of objectively measured skill. This suggests that training does not increase correspondence between self-report and objective assessment, so community treatment programs should not rely on clinician self-report to assess the need for ongoing training and supervision and it may be necessary to train clinicians to accurately assess their own skill.

KW - Community-based supervision

KW - Dissemination

KW - Evidence-based practice

KW - Motivational interviewing

KW - Training

UR - http://www.scopus.com/inward/record.url?scp=84940575466&partnerID=8YFLogxK

U2 - 10.1016/j.jsat.2015.04.006

DO - 10.1016/j.jsat.2015.04.006

M3 - Article

C2 - 25963775

AN - SCOPUS:84940575466

VL - 57

SP - 96

EP - 101

JO - Journal of Substance Abuse Treatment

JF - Journal of Substance Abuse Treatment

SN - 0740-5472

ER -