Lightnews — Scholar-powered news

Coalition for Evidence-Based Policy

@coalition4evidence.bsky.social

61 followers 14 following 130 posts

The Coalition for Evidence-Based Policy is a nonprofit, nonpartisan organization, whose mission is to tackle major U.S. social problems using rigorous evidence about “what works.” www.evidencebasedpolicy.org

Posts Media Videos Starter Packs

Pinned

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · Dec 16

We're excited to announce the launch of our new online publication, the No-Spin Evidence Review. We summarize recent program evaluations and explain what the evidence really shows. See our first review below and check out No-Spin at nospin.evidencebasedpolicy.org.

1 4

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

Here's the original study report: mdrc.org/sites/defaul...

mdrc.org

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

Here's our No-Spin report: nospin.evidencebasedpolicy.org/articles/new....

New York City’s P-TECH 9-14 College and Career Preparatory Schools | No-Spin Evidence Review

nospin.evidencebasedpolicy.org

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

but a 12-year follow-up found sizable earnings gains. Longer-term follow-up of the P-TECH 9-14 study would be valuable to see if similar earnings impacts materialize.

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

5/ COMMENT: The study found the program to be well implemented - for example, providing students with significant workplace experience during high school. A high-quality RCT of a related model - Career Academies - found initially disappointing results (no impact on high school or college completion)

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

The study found a possible impact on college degree completion at the 7-year follow-up (13% T vs 8% C). However, this finding is based on only 16% of the sample (the early cohorts with seven years of available data) and not quite statistically significant (p=.08), so is best viewed as preliminary.

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

The study also found no statistically significant impact on the percent of students either enrolled in college or having completed a degree at the 5-, 6-, and 7-year follow-ups (e.g., 60% T vs 60% C at six years).

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

4/ FINDINGS: The study found no statistically significant impact on the high school graduation rate at follow-ups 4, 5, and 6 years after study entry (e.g., 76% T vs 77% C had graduated at six years).

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

Students were randomly assigned via lottery to a treatment group offered admission to a P-TECH 9-14 school (T) or a control group not offered admission (C). Based on careful review, this was a high quality RCT (e.g., baseline balance, negligible sample attrition).

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

The model has proliferated nationally and internationally since the first P-TECH 9-14 school opened in Brooklyn in 2010.

3/ STUDY DESIGN: The study sample comprised 3,161 rising 9th graders, 86% of whom were Black or Hispanic.

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

2/ PROGRAM: New York City’s P-TECH 9-14 schools are "a partnership among a high school, a community college, and one or more employer partners that focuses on preparing students for both college and careers within six years."

1 1

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 2d

See No-Spin report on an RCT of P-TECH 9-14 College & Career Prep Schools. In brief: Despite positive portrayal of results in the study's overview, it found no discernible impact on high school grad'n or college progression over 7y, but future earnings impacts are possible.🧵

1 1 1

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 7d

And here's the original study report: acf.gov/opre/report/....

Beyond the Early Years: The Long-Term Effects of Home Visiting on Mothers, Families, and Children. Results from the Mother and Infant Home Visiting Program Evaluation (MIHOPE)

This report presents results from the analysis of data from the kindergarten follow-up of the Mother and Infant Home Visiting Program Evaluation (MIHOPE).

acf.gov

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 7d

Here's our No-Spin report: nospin.evidencebasedpolicy.org/articles/hom....

Home Visiting for At-Risk Families, Delivered Under the Federal Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Program | No-Spin Evidence Review

nospin.evidencebasedpolicy.org

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 7d

8/ The study doesn't report the effect sizes for each of the 4 home visiting models individually. But it appears the model-specific effects are smallish since (as the report shows) few were statistically significant. We've asked the study team for more info on these effects.

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 7d

7/ However, the statistically significant effects were so small as to be of little *practical* significance for children or families. Their average effect size was 0.03, which is the equivalent of moving the average child or mother from the 50th to the 51st percentile.

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 7d

6/ 5-7 years after study entry – when most children were in kindergarten – the study found that home visiting produced statistically significant positive effects for 6 of the study’s 8 prespecified research questions. (5 were significant at 0.05; one was significant at 0.10.)

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 7d

5/ Based on careful review, this is a high-quality RCT (e.g., baseline balance, prespecified outcomes and analyses, use of well-established measures of child and parent outcomes, moderate attrition for most outcomes).

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 7d

4/ For this latest report on kindergarten-age outcomes, the study organized its 60 prespecified outcome measures into 8 prespecified research questions, and conducted an omnibus test of statistical significance for the set of outcomes within each question.

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 7d

3/ The programs vary in key features, but all provide home visits (e.g., by a paraprofessional or nurse) to at-risk families that are expecting or have recently given birth. Families entered the study when their children were under six months old or in utero.

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 7d

2/ This is a large, ongoing RCT, with a sample of 4,102 families across 12 states. The RCT is evaluating four MIECHV-funded home visiting program models - Parents as Teachers, Healthy Families, Early Head Start, and Nurse-Family Partnership.

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 7d

See our new No-Spin report on a major RCT of federally-funded home visiting programs for at-risk families with young children. In brief: Despite the positive portrayal of results in the study's overview, it found negligible impacts on child & family outcomes at the 5-7yr mark.🧵

1 1

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 16d

AV also produced an excellent video narrated by the Governor: www.youtube.com/watch?v=wD0r...

Arnold Ventures Partners with Maryland to Scale Evidence-Based Programs

YouTube video by Arnold Ventures

www.youtube.com

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 16d

11/ In short, the Partnership is laser-focused on using strong evidence about "what works" to improve lives at scale. Here's Gov Moore's announcement: governor.maryland.gov/news/press/p....

Governor Moore, Arnold Ventures Announce $20 Million In Grant Awards To Support Programs That Help Maryland Youth Thrive - Press Releases - News - Office of Governor Wes Moore

governor.maryland.gov

1 1

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 16d

9/ Finally, the Partnership will build new evidence about what works specifically in MD: 3 of the 4 programs being expanded will include large RCTs, so as to produce definitive evidence about whether the programs deliver meaningful improvements in the lives of Marylanders.

1 1

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · 16d

8/ The Partnership's policy impact is also unique: AV’s $20M catalyzed $20M in matched gov't funds for proven programs that previously received ~$0 in public funding in MD. This represents substantial gov't investment in proven programs that wouldn't have otherwise occurred.

1 1