Abstract: This article details how the YES! program team applied modified definitions of "positive outcomes" and "improved outcomes" to National Outcome Measures (NOMs) data, to more accurately measure young adults' situations.
"Wisconsin's Youth Empowered Solutions Monitors and Improves Participant Outcomes" (2019)
Janae Goodrich, Robin Lecoanet, & Delaney Peterson
Wisconsin's Youth Empowered Solutions (YES!) is administered by the Wisconsin Department of Health Services and is funded by a five-year Now is the Time – Healthy Transitions grant from the Substance Abuse and Mental Health Services Administration (SAMHSA). Two Wisconsin counties, Jefferson County and Outagamie County have been serving youth and young adults (ages 16–25) since April 2015. YES! sites serve youth and young adults with, or at risk for, serious mental health and/or substance use conditions, and who are experiencing challenges related to poverty, high rates of psychiatric hospitalization, and unmet mental health service needs. The Wisconsin Department of Health Services contracted with the University of Wisconsin Population Health Institute (UWPHI) for the purposes of program evaluation.
Methods
As a part of the Now is the Time – Healthy Transitions grant, YES! site staff collect data using the federally-required National Outcome Measures (NOMs) tool when a participant is admitted to YES!, every six months after admission, and when a participant is discharged from YES! services.1 As a part of the program evaluation, UWPHI staff use SAMHSA's Outcome Measure Report Guide as a framework to inform analyses conducted to monitor and report participant outcomes.2 These analyses include participant outcomes between the intake and six-month follow-up interviews, and the intake and discharge interviews. Participant outcomes analyses are conducted and reported to state and local partners on an annual basis.
Improving participant outcomes and defining "positive outcomes" and "improved outcomes" for youth and young adults served through YES! has been a focus of the YES! initiative and evaluation. YES! staff and stakeholders have discussed this at length and considered what is appropriate and realistic for this population. This discussion and review of SAMHSA's outcome measures resulted in a modification of SAMHSA's definitions of "improved outcomes." For example, SAMHSA considered a change in drug and alcohol use to abstinence from drugs and alcohol as an improved outcome; however, YES! staff agreed that the definition of an improved outcome should include sustained abstinence and reductions in alcohol and drug use.
Similarly, SAMHSA's definition of an improved outcome for stable housing in the community includes changing housing situations to an "owned or rented house, apartment, trailer, room," and a "group home" for this population. YES! staff decided to assess changes in housing, without placing an "improved" definition on it since it varies for this population. In addition, YES! site staff began gathering separate, more in-depth, housing stability measures to document whether current housing situations are considered to be stable, as defined by YES! staff and stakeholders. For example, a 17-year-old living in a parent's house without a threat of displacement is considered to be in stable housing under this definition.
To demonstrate how our modification of the SAMHSA indicators impacted our results, we will review our analysis of the outcomes in the domain of illegal drug use. While SAMHSA defines an improved outcome as using illegal drugs within the 30 days before baseline, and then never using any illegal drugs in the 30 days prior to the second interview, YES! staff agreed that less frequent use of illegal drugs in the 30 days prior to the second interview (as compared to the baseline interview), and sustained abstinence from illegal drugs (no use in the 30 days prior to the baseline and second interview) should be included in a definition of improved outcomes. In the initial analysis of the use of illegal drugs domain according to SAMHSA's definition at six-month follow-up, 20% of our total YES! participants were abstinent from illegal drugs in the 30 days prior to the six-month follow-up interview. This was largely due to our participants not using illegal drugs within the 30 days prior to the baseline interview. When we modified SAMHSA's definition of improved outcomes and used our local definition, 64% of our total YES! participants remained abstinent and/or have improved outcomes.
Outcomes analyses were conducted using the domains outlined by SAMHSA, and analyses conducted included the expanded definitions of improved outcomes defined by YES! staff and stakeholders.
Results
Using the definitions developed, UWPHI evaluation staff conducted participant outcomes analyses between the intake and six-month follow-up interviews, and the intake and discharge interviews (see Table 1). These analyses were conducted based on all of the information collected by YES! site staff during the first three years of YES! implementation.
Table 1. Participant Outcomes
Baseline interviews were completed with 136 individuals during the first three years. At the six-month follow-up interview (n = 63), at least half of YES! participants experienced improved outcomes in 10 of the 11 outcome areas measured, with more than 75% of YES! participants reporting improved outcomes in seven of the areas.
Outcomes measured at discharge (n = 33) were similar to those reported at the six-month follow-up interview, and rates of improved outcomes in the areas of mental and physical health were higher than those at the six-month follow-up. It should also be noted that the average length of stay for YES! participants who were discharged from YES! during the first three years of implementation was 8.5 months, and 47% of those discharged were involved for nine months or longer.
Limitations
The number of individuals included in the outcomes analyses has been impacted by a series of changes in the required federal participant interview tool, as well as challenges in successfully completing the six-month follow-up and discharge interviews with participants.
A larger number of participants will be included in the six-month and discharge outcomes analyses as YES! continues to be implemented. Future analyses of other follow-up interviews (12-month, 18-month, etc.) will also be considered.
Additional information about the YES! initiative and the program evaluation can be found in the YES! Evaluation Report for Grant Years 1–3.3
References
- Substance Abuse and Mental Health Services Administration. (2017). Center for Mental Health Services NOMs Client-Level Measures for Discretionary Programs Providing Direct Services Services Tool for Adult Programs. Retrieved from https://www.samhsa.gov/sites/default/files/programs_campaigns/samhsa_hrsa/noms-client-level-measures.pdf
- Substance Abuse and Mental Health Services Administration. (2017). SAMHSA's Performance Measure Accountability Reporting System (SPARS) outcome measures report guide. Retrieved from https://spars.samhsa.gov/sites/default/files/Ref-399_CMHS_Services%20Outcome%20Measures%20Report%20Guide_1-24-18.pdf
- Goodrich, J., Lecoanet, R., & Dustman, D. (2018). Youth empowered solutions (YES!) evaluation report for grant years 1-3. Retrieved from https://uwmadison.app.box.com/s/srxhrl3ac5pah9eh485l2szkwxdddcsi
Suggested Citation
Goodrich, J., Lecoanet, R., & Peterson, D. (2019). Wisconsin’s Youth Empowered Solutions Monitors and Improves Participant Outcomes. Focal Point: Youth, Young Adults, and Mental Health, 33, 30–32. Portland, OR: Research and Training Center for Pathways to Positive Futures, Portland State University.