Use Evaluation to Guide PDSAs Rather Than Derail Them
A key to making sure that teams are testing changes on the road to improvement is to avoid getting stuck mistaking an evaluation activity for a test of change. This mistake falls under a common pitfall of mistaking a task for a test. That isn’t to say that you should avoid evaluation activities at all costs. Evaluation activities—surveys, interviews, focus groups—can be important steps in formulating a test of change.
Evaluation activities can help you identify the first change you want to test during the “Do” stage of the Plan-Do-Study-Act (PDSA) cycle. For example, let’s say you team is trying to increase referral uptake for children who screen positive for developmental concerns. For an evaluation activity, you might survey families to find out why they didn’t follow up to schedule their referrals. This step helps teams better understand and respond to the most likely problems affecting the ability to schedule referrals in your clinic. Or, put another way, it provides information to help you plan a small test of change to implement.
With that task, let’s say your team discovers families primarily delayed scheduling a referral because they forgot or were unable to call back on their own and schedule. Using this information, the team can test a method that removes the need for families to call on their own. For example: they might have front desk staff coordinate and book referrals during the same appointment that the child screened positive for a potential developmental delay. Here, evaluation as an early step in your “Planning” stage (the survey) informs the next steps in the "Planning" stage, ultimately preparing your team to be able to move forward and run your test.
In the “Study” stage, evaluation can help you identify what modifications should be made to the test during your “Act” stage. Continuing with the developmental screening example, after one week, the team discovers that referral rates have increased by 50 percent. However, the team is worried that this change may have a negative impact on the front desk staff managing the referrals. To find out more, a small focus group is conducted with the front desk staff to get a sense of their experience with the change. This focus group is an evaluation activity (a task). In it, several staff share concerns about longer wait times because they have an additional task to coordinate. In response, your team decides to track and measure wait time during the next round of your PDSA. Depending on the results of this new measure, you may decide to add an extra staff person. Here, evaluation in your “Study” stage (the focus group) informs the actions you will take in your next series of tests.
Evaluation is a powerful tool that can and should transform improvement work. It only becomes risky when it keeps you from jumping into your test of change. Just remember, with quality improvement work, the focus is to learn from doing, not get stuck evaluating the problem.
Looking for more examples on using PDSA cycles to drive improvement? Click here to access our free e-course, Quality Improvement 102, and uncover best practices for moving from one PDSA cycle to the next.
MCH Lead Poisoning Toolkit: Lessons on Using Data for Improvement
The Maternal and Child Environmental Health Collaborative Improvement and Innovation Network (MCEH CoIIN), a national initiative led by the Association of Maternal and Child Health Programs developed the MCH Lead Poisoning Toolkit to share innovative practices and methods that nine different state teams tested out to improve access to systems and services that address the needs for pregnant women, infants, children, and families that are exposed to lead. NICHQ provided quality improvement expertise and technical assistance for the MCEH CoIIN—offering state teams guidance on using data and measurement to determine where improvements have been made and areas that still require change. Read tips and take-aways from the three-year CoIIN for data collection and using data for Improvement.
NICHQ Employee Spotlight: Stacey C. Penny
With NICHQ's Rare As One Network Workstream Facilitation Initiative at a halfway point, Senior Project Director Stacey C. Penny, MSW, MPH shares an inside look at one of NICHQ's most collaborative projects.
Are Screens Making our Children’s Eyes Worse?
Even before the COVID-19 pandemic, children of all ages were spending more screen time than ever before on cellphones, tablets, and laptops. Prolonged periods of time staring at a screen that may be too big, too bright, or too close to our eyes can cause eye fatigue or even lead to dry eye, a chronic eye condition common in older adults. With eye conditions becoming more prominent in children, it's important for health professionals to encourage healthy screen viewing habits and support children’s eye health in the digital age.
NICHQ Employee Spotlight: Olivia Giordano
Olivia Giordano, MPH, Project Manager shares how her work with NICHQ’s Supporting Healthy Start Performance Project (SHSPP) is supporting 101 Healthy Start community sites to harness lessons learned, implement innovative approaches to improvement, and ultimately start to close the disparity gap in maternal and child health.
It Starts with Us and It Starts Now: Healing for Moms and Babies Begins with Ourselves and Our Systems
NICHQ CEO Scott D. Berns, MD, MPH, FAAP shares a message on healing and the ongoing need for equity-designed systems in 2021 and beyond.
Safe Sleep and Breastfeeding Initiative Invites Advocates to Join Communities of Practice
A multi-year initiative to improve infant safe sleep and breastfeeding is launching sector-specific Communities of Practice in 2021 to address policies, improve skills, and learn from other advocates’ experiences.