|
http://www.researchforum.org/project_general_3.html |
|
|
A Better Chance (ABC) Evaluation
General Information
View a brief abstract of this project.
| Evaluator(s) |
Abt Associates, Inc. |
| Investigator(s) |
David Fein
(Abt Associates, Inc.)
|
| Sponsor(s) |
Delaware Department of Health and Social Services |
| Funder(s) |
Delaware Department of Health and Social Services |
| Subcontractor(s) |
Not applicable |
| Domain |
Income Security/TANF |
| Status | Operational with Findings |
| Duration | Oct 1995 - Mar 2002 |
| Type |
Research and/or Program Evaluation |
| Goal | To evaluate the components of A Better Chance (ABC). |
| Program/Policy Description | ABC involves a four-year limit on AFDC eligibility for nearly all families in which a parent was head of the case. During their first two years of welfare receipt, the plan requires all adult recipients to participate in activities designed to lead to employment. After two years of welfare receipt, the plan provides benefits for up to two additional years contingent on hours worked in a pay-for-performance job which the state provides if no employment opportunity is available. After four years of welfare receipt, no further AFDC assistance is provided to the family. Participation in the program is mandatory for all welfare recipients. |
| Notes | Experimental component (random assignment) was halted after the passing of the Personal Responsibility and Work Opportunities Reconciliation Act of 1996. |
| Last Updated | 03/20/03 |
| Type of Summary | Reviewed |
| External Reviewer(s) |
David Fein
(Abt Associates, Inc.)
|
| Contact(s) |
David Fein (David_fein@abtassoc.com)
Abt Associates, Inc. 4800 Montgomery Suite 600 (T) (301)-913-0548 (F) (301)-652-3618 |
| Publications Department |
Abt Publications (not reported)
Abt Associates, Inc. 4808 Montgomery Lane (T) not reported (F) not reported |
Populations Studied
| Target Population |
Recipients/participants/clients |
| Subgroups Analyzed |
Single parent families Two-parent families |
| Sample Size and Unit |
18,000 families (welfare participants). Random sample of program (with ABC program) and group (with traditional AFDC) members (number in each group not reported). |
Sites Studied
Dover, Delaware;Georgetown, Delaware;
New Castle, Delaware;
Wilmington, Delaware
Program Components, Policies, and Activities Evaluated
Employment activities
- Job readiness activities
- Job search
- Work supplementation programs
Financial incentives
- Earnings disregards
- Earnings supplements/work subsidies
- Elimination of 100 hour rule
- Financial Incentives - misc.
Financial disincentives/Sanctions
- Reduced benefits for non-compliance
- Strengthened JOBS sanctions
Program requirements
- Work requirement
- Community or alternative work
- Enrollment in substance abuse program
- Parenting or social contract
- School attendance
- Living arrangements for unwed pregnant or parenting minors
- Workshop attendance
- Immunizations for children
- Broadened JOBS participation requirement
Social/Support services
- Transitional child care
- Transitional health benefits
- Employment support for job retention
- Parenting classes/training
- Family planning education and services
Administration/Implementation
- Program enforcement of sanctions
- Administration/Implementation - misc.
Time limits
- Time Limits - misc.
Family caps
- Family Caps - misc.
Eligibility
- Eligibility - misc.
| Variation in program components across sites? | No |
| Notes on program components |
Changes in Program components: In January 2000, the program will be altered so that families may receive assistance for a maximum of three years, and only if working in a pay-per-performance job. In addition, children born to unmarried teenage mothers after December 1998 are not to be eligible for any cash assistance, although the State is to provide other services.
Employment activities: During the first two years of welfare receipt, adult clients are required to participate in activities designed to lead to employment. After two years of welfare receipt, the plan provides benefits for up to two additional years contingent on hours worked in a pay-for-performance job which the state provides if no employment opportunity is available.
Family caps: Welfare grants are not increased with the number of additional children born to families who are currently receiving welfare support.
Financial disincentives/sanctions: Clients are encouraged to become more responsible parents through Contracts of Mutual Responsibility and financial penalties for failure to engage in specified activities.
Financial incentives: The financial rewards of working on welfare are increased to ""make work pay".
Program operations: Implementation of program components is studied. Program requirements: Minors are required to attend school, and pregnant/parenting minors must live with parents. Social/Support Services: Child care and health care coverage are provided. Time limits: Welfare recipients are limited to four years of welfare receipt. |
Outcomes Assessed
Benefit termination
- Due to employment
- Due to time limit
- Due to sanctions
Family and relationship outcomes
- Violence in family or other relationships (child abuse and neglect)
- Births/pregnancies
- Parent-child interactions
- Family formation and stability/Living arrangements
Education
- School attendance
Employment
- Job attainment
- Job retention
- Number of hours worked for wages
Income security
- Child support payments
- Earnings
- Food stamps receipt
- Medicaid receipt
- Welfare receipt
Housing
- Housing - misc.
Attitudes towards work, welfare, and program
- Attitudes towards work, welfare, and program - misc.
Service utilization
- Service utilization - misc.
Sanctions
- Sanctions - misc.
Program implementation
- Program Implementation - misc.
Child Outcomes
- Child social/emotional/behavioral outcomes
- Child academic outcomes
- Child overall development
- Child mental/physical health outcomes
Types of Studies
| Type | Impact Study (Controlled Experiment) |
| Aim | To determine whether ABC achieves the intended improvements in the well-being of families and children. Note: Experimental component (random assignment) was halted after the passing of the Personal Responsibility and Work Opportunities Reconciliation Act of 1996. |
| Type | Implementation/Process Study |
| Aim | To describe and understand welfare reform policies, program operations, and experiences. To understand why and how the program had the effects measured. To suggest useful approaches to strengthening the program’s design and/or operations through a critical review of diverse experiences with ABC by different stakeholders. |
| Type | Descriptive/Analytical Study |
| Aim | To report how outcomes change over time. To monitor welfare spell duration, employment experiences, and changes in health and other well-being indicators for children. To contrast outcomes for families with different pre-and post-reform experiences and characteristics. |
Data Sources
| Source | Field Research |
| Title | Site visits |
| Sample Characteristics/Data Collection |
Visits to 5 pilot offices. Periodic visits. |
| Sites | All sites. |
| Response Rate/Attrition Notes | N/A |
| Additional Execution Notes | No notes reported. |
| Source | Survey |
| Title | In-person Background Information Form (BIF) |
| Sample Characteristics/Data Collection |
Records for 5,051 welfare recipients. Sample of 2,652 program and 2,399 control group members. Collected at baseline. |
| Sites | All sites. |
| Response Rate/Attrition Notes |
Reported response rate:
90.6% program group welfare recipients. 90.7% control group welfare recipients. 68.4% program group welfare applicants. 64.4% control group welfare applicants. |
| Additional Execution Notes | No notes reported. |
| Source | Survey |
| Title | Telephone follow-up survey |
| Sample Characteristics/Data Collection |
1,600 welfare applicants and recipients. Sample of 800 program and 800 control group members. Collected at baseline and periodically thereafter (wave 1: March-May 1997; wave 2: completed in July 2000) |
| Sites | All sites. |
| Response Rate/Attrition Notes | Reported response rate: 70% in each wave. Of wave one respondents, 77% reinterviewed in wave 2 |
| Source | Field Research |
| Title | Periodic key informant interviews with senior program managers |
| Sample Characteristics/Data Collection |
7 key administrators at Department of Social Services, Department of Labor, and Delaware Economic Development Agency. Periodic visits. |
| Sites | All sites. |
| Response Rate/Attrition Notes | Reported response rate: 100% |
| Additional Execution Notes | No notes reported. |
| Source | Focus Group |
| Title | Focus group |
| Sample Characteristics/Data Collection |
Key constituencies (i.e. AFDC clients, employers, service providers, and advocates). 2 focus groups with clients who reached time limits: 1) 8 who did not participate in workfare; 2) 11 who did participate in workfare. Data collected March, 1999. |
| Sites | All sites. |
| Response Rate/Attrition Notes | N/A |
| Additional Execution Notes | No notes reported. |
| Source | Field Research |
| Title | Site visits to DOL and DSS contractors |
| Sample Characteristics/Data Collection |
DSS and DOL contractors (number in sample not reported). Judgement sampling. Periodic field visits. |
| Sites | All sites. |
| Response Rate/Attrition Notes | Fielded sample number not reported. |
| Additional Execution Notes | A combination of observations and interviews will be used to assess contractors’ experiences and practices. |
| Source | Administrative data |
| Title |
Delaware Client Information System (DCIS). Unemployment Insurance wage reporting system. State’s immunization registration system. Local school records. Department of Services to Children, Youth, and Their Families databases. |
| Sample Characteristics/Data Collection | Records for 18,000 welfare recipients. |
| Sites | All sites. |
| Response Rate/Attrition Notes | N/A |
| Additional Execution Notes | No notes reported. |
Findings Available
Interim Implementation FindingsInterim Impact Findings
Interim Descriptive/Analytical Findings
Findings
| 02/01/97: A Better Chance Evaluation: First-Year Evaluation Progress Report |
|
Interim Implementation Findings: "Administrators’ perceptions reflect their unique position as intermediaries between policy makers and front-line workers. Interviews suggested that managers endorsed the goals and many provisions of ABC and clearly understood their responsibilities for implementing the program as designed. At the same time, interviewees had serious concerns about a number of the reform’s provisions"(33). "A primary challenge in implementing ABC was preparing staff for their new responsibilities. Two themes consistently emerged in workers’ comments about the preparation for ABC: 1) staff desired additional guidance on how to implement ABC policies; and 2) staff saw a need for consistent communication on policy and procedural changes"(Appdx B, p5). |
| 12/01/97: A Better Chance Evaluation: The Early Economic Impacts of Delaware's A Better Chance Welfare Reform Program |
|
Interim Impact Findings “Impact analyses reveal that the changes induced moderately strong impacts on employment rates, average earnings and welfare payments. By the fourth quarter after random assignment, the employment rate was eleven percentage points higher for treatment (56%) than control (45%) group members. Average earnings for the fourth follow-up quarter were $167 higher (representing a 16% proportionate effect), and average welfare payments were $76 lower (an 18% proportionate effect), These impacts compare favorably with those from several other recent state demonstrations” (ii). “The reduction in cash assistance payment occurred even though there was no statistically significant net impact on the percent of families receiving welfare. ABC’s ‘fill-the-gap’ policy, designed to ‘make work pay,’ allows recipients to receive some assistance when their earnings reach levels that previously made families ineligible. A rough calculation suggests that among treatment group members, as many a six- percent would not have been eligible for cash assistance except for fill-the-gap. Subtracting these families from the observed 56% welfare receipt rate for the treatment group members yields an adjusted rate of 50%- nine percentage points below the 59 percent rate observed for control group members” (ii)” “Treatment group members evidenced broad familiarity with the new policies, but also a need for better comprehension of detailed rules. For example, 84% said they were aware their assistance was time-limited, but only 27% knew the initial time limit was 24 months…Control group members mostly understood that they were not subject to ABC policies. Nearly four-fifths (79%) scored low (saying less than three items applied) on an ABC policy index, compared to 44 percent of treatment group members. Although a substantial treatment-control knowledge gap was created, some control group members may have been led by what they heard in the media or ‘on the street’ to believe they were subject to parts of the reform. The estimated impacts therefore may understate ABC’s full early impacts somewhat” (iii). “A variety of program start-up difficulties resulted in lower-than-desired rates of participation in new work services administered by the state’s Department of Labor. The shift to a ‘work first’ approach nonetheless is evident in the participation statistics. More treatment (15%) than control (9%) group survey respondents said they have participated in work activities. Identical fractions (13%) of the two groups reported participation in education and training activities” (iii)” “A vigorous campaign to enforce financial sanctions was the earliest administrative response to noncompliance with ABC requirements. By March 1997, nearly half (49%) of all recipients enrolled in ABC from October 1995 to September 1996 had received one or more sanctions. The sanction rate for noncompliance with work and training requirements (41%) was higher than the rate for noncompliance with family responsibility rules (22%). These sanction rates are much higher than those characterizing the previous generation of welfare reforms in Delaware and elsewhere, and must be considered as a strong potential contributing factor in any observed impacts” (iv). “ABC’s work and training sanctions progress through three states can lead ultimately to permanent ineligibility for cash assistance. To determine how sanctions were being resolved, a group of clients under sanction in December 1996 was followed for a six-month period. By June 1997, less than a quarter had cured their sanctions, another quarter remained under sanction, and more than half has left assistance. It’s too early to tell whether the low compliance rates implied by these statistics will increase in the longer-term. The recent advent to innovate case management-based ‘compliance service’ contracts may lead to improved sanction outcomes” (iv). “Beneath the surface of ABC’s small net impacts on welfare receipt, descriptive evidence suggests some important changes in welfare dynamics. Treatment group members who left welfare were somewhat less likely to attribute their exits to increased earnings (50% of exits) than control group members (63%). This findings is consistent with the suspicion that fill-the-gap was operating to extend assistance for working recipients to ease their transitions to self-sufficiency. There is also evidence that ABC hastened some families’ exits for reasons other than increased earnings. A majority of treatment group members who said ABC influenced their welfare exits said the reason was related to dislike of, or negative experiences with, new ABC requirements and sanctions”(iv). “Subgroup analyses suggest ABC’s impacts on employment and welfare participation were concentrated among clients with medium duration (one or two years) of previous welfare receipt. There was little evidence of impacts either for clients who had spent less than one years, or three or more years, on assistance of the five preceding random assignment. Reports from earlier welfare-to-work demonstrations finding a similar pattern suggest a plausible explanation. Because the lease welfare-reliant clients are relatively employable and motivated, they may have less need for ABC’s work services and incentives. At the other extreme, long-term recipients have many more serious barriers and may not be able to respond as quickly to ABC’s services and incentives” (v). “Many non-welfare agencies are concerned about the potential ramifications of sanctions and time limits on the demand for services beyond cash welfare assistance. At this early juncture, there is little evidence to increased utilization in a variety of alternative public and private assistance programs. To the contrary, ABC produced slightly lower receipt of food assistance from food stamps and food banks among treatment than control groups members. The program also resulted in a small (4 percentage point) positive impact on reported receipt of child support payments through government agencies. ABC had no impact on average household income after its first year. Positive earnings impacts apparently were offset by program-induced reductions in average welfare payments and other income sources” (v). |
|
Interim Impact Findings: “The findings suggest ABC sanctions are influenced by varied implementation and personal factors. That large office differences remain even after accounting for socioeconomic differences across offices is strong testimony to the adage that local implementation matters. The analyses do not identify specific features of implementation that were responsible. The list of likely candidates includes staffing arrangements, office leadership, sanction procedures, and attitudes- especially belief in sanctions as a tool and willingness to pursue other remedies. The role of implementation practices in varying sanction experiences likely is much greater than the office differences reveal, since it is individuals workers within offices who ultimately implement sanctions (21).” “At the client’s end, measures of understanding and ability to comply with ABC rules were more strongly associated with sanctions than indicators of motivation, suggesting that difficult circumstances have had at least as much to do with noncompliance as unwillingness to play by the rules. It should be acknowledged that only limited indicators for motivation, understanding, and ability to comply were available. Also, effects for several background variables may be reflecting motivational sources of sanctioning: for example, long-term recipients may have lower motivation to meet new requirements than new applicants (21).” “Nonetheless, there is strong evidence that factors other than motivation explain much noncompliance. This finding is important, since sanctions are not likely to be effective for clients who already are willing to meet requirements. Giving workers more tools to better discriminate and differentiate responses when cooperation is due to unwillingness to comply, and when it arises from other circumstances, could lead to a more effective sanction policy (21).” “Finally, the fact that clients who are sanctioned are more socially and economically disadvantaged that those who are not belies the conjecture that many clients accept sanctions because they already have ready access to replacement income. The finding also raises the prospect that sanctions may leave recipients who already have relatively modest means even more disadvantaged than they were before (21).” “For curers, sanctions may have led to improvements in behaviors linked to work and family outcomes- assuming offenses cured were in fact defiant behaviors in the first place, and not merely failures to report these behaviors to the welfare office. Among voluntary leavers, sanction-induced exits may suggest increased efforts to find alternative means of financial support or decisions by clients with alternative financial means already (e.g. unreported income) to cut their losses. Although voluntary leavers technically leave welfare at their own initiative, they presumably do so under duress, and their post-welfare situations therefore cannot automatically be assumed to represent net improvements. This group’s social and economic fortunes will depend on the nature and amount of support available from whatever alternatives to welfare are found, including options as diverse as: regular employment; reliance on family, friends, and other public and private agencies; marriage and cohabitation; and illegal economic activities (24).” “Finally, as a group, outcomes for nonresponders seem unlikely to be as positive as for the other groups of sanctioned clients. Nonresponders’ characteristics suggest relatively lower access to alternative opportunities, and the involuntary nature of their exists itself suggests potential difficulty in responding well to any ensuing financial difficulties. Consequently, potential seems weakest for positive economic outcomes for this group, and any resulting increase in financial stress and/or maternal hardship could have deleterious consequences for family functioning (25).” |
| 08/01/99: A Better Chance Evaluation: Do Welfare Recipients' Children Have a School Attendance Problem? |
|
Interim Descriptive/Analytical Findings: “Absenteeism is greater for welfare children than for other children. Income differences account for a large share of this absenteeism, especially among teenagers” (ii). “Parent surveys yield reasonably accurate estimates of absenteeism for welfare children and of differences in absenteeism between welfare children and other children” (iii). “Most absences among welfare children arise from illness rather than truancy” (iii). “ABC workers have been enforcing the school attendance policy. Low truancy rates suggest that a substantial part of the process currently is devoted to verifying attendance for non-truant children” (iv). “Because they monitor attendance directly and serve all poor children, school are the most logical place to locate interventions to reduce truancy. Family services provided by welfare agency social workers potentially could provide a valuable resource for schools seeking to bring parents into the picture” (v). “Better knowledge of how to help parents address high absenteeism and other problems is needed” (vi). |
| 06/30/99: A Better Chance Evaluation: Will Welfare Reform Influence Marriage and Fertility? Early Evidence from the ABC Evaluation |
|
Interim Impact Findings: ABC offers the first evidence that comprehensive state welfare overhauls can influence welfare recipients’ marriage and childbearing decisions. After only eighteen months of operations, ABC had modest positive impacts on marital cohabitation among women who were under age 25 and those with less than 12 years of education. The reform also raised marriage expectations among women with less than 12 years of education, but led fewer better-educated women to expect to marry. Although ABC had only a small impact on actual fertility in one subgroup, it sharply reduce desires for more children among women with intermediate durations of past welfare receipt, women who were 25 of age or older, and those who had never married. |
| 03/01/99: A Better Chance Evaluation: A Better Chance for Welfare Recipients? What the Public Thinks |
|
Interim Impact Findings: “More than half the public believes that benefits should not increase when welfare recipients have more children, consistent with current ABC policy. Respondents believe recipients will be more careful and have fewer children under such a policy. “Delawareans support continued provision of cash assistance to teen parents, whereas the State stopped providing cash benefits for babies born to unmarried minor parents after December 1998. Eliminating cash welfare also is the least popular route to reducing teen childbearing among a series of alternatives explored. “The public respond favorably to two possible elements of a new welfare diversion strategy — a one-month applicant job-search period and a lump-sum payment of up to six months’ worth of regular cash payments” (iii). |
| 12/01/00: A Better Chance Evaluation: Impacts of Welfare Reform on Child Maltreatment |
|
Interim Impact Findings: The findings show that ABC increased the fraction of families with an incident of child neglect by around one percentage point in the first and third, but not the second, year of follow-up. The program had no statistically significant overall impacts on other kinds of maltreatment, such as physical and emotional abuse or sexual abuse, or on foster care placements. |
| 01/01/01: A Better Chance Evaluation: Turning the Corner: Delaware's ABC Program at Four Years |
|
Interim Impact Findings: Findings provided here summarize the early challenges Delaware faced in implementing ABC, and welfare recipients’ experiences over roughly a two-and-a-half-year follow-up period.2 The study finds that Delaware implemented a strict, work-oriented program that fundamentally altered the State’s welfare system. In refashioning its cash welfare program, the State’s Department of Health and Social Services’ Division of Social Services (DSS) made many basic changes in policies and procedures, administrative arrangements, and services. Delaware strongly enforced Contracts of Mutual Responsibilities requiring clients to participate in work activities and meet specified parenting responsibilities. The program had a number of significant impacts on clients, of which the most striking was a reduction in welfare use. Thanks to Delaware’s strong economy and no-nonsense work program, the vast majority of clients went to work during at least part of the follow-up period. However, few participants achieved economic independence within the study period, and the majority still was struggling to make ends meet. |
| 08/01/01: A Better Chance Evaluation: How Have They Fared? Outcomes After Four Years for the Earliest DABC Clients |
|
Interim Impact Findings:
|
Recommendations
| “Findings in this report suggest several other changes could enhance the effectiveness of ABC sanctions in bringing about desired behavioral responses. First, reducing the number of sanctionable offenses to a few key behaviors would help workers and clients to better understand needed changes and address barriers to making these changes. Second, the State should consider eliminating permanent, full-family sanctions, as these do not appear to be more effective than moderate penalties in use elsewhere. Finally, for clients whose difficulties stem from inability to manage their affairs rather than unwillingness to cooperate, intensive outreach and services may generate better results than sanctions (38).” |
Existing Publications
|
|