|
Canada's Earnings Supplement Project (ESP) Evaluation
General Information
View a brief abstract of this project.
View a complete, printer-friendly profile of this project.
| Evaluator(s) |
Social Research and Demonstration Corporation
|
| Investigator(s) |
Saul Schwartz
(Social Research and Demonstration Corporation)
Howard Bloom
(Social Research and Demonstration Corporation)
|
| Sponsor(s) |
Human Resources Development Canada
|
| Funder(s) |
Human Resources Development Canada
|
| Subcontractor(s) |
Statistics Canada
|
| |
| Domain |
Income Security/TANF
|
| Status |
Completed (final report released)
|
| Duration |
Mar 1995 - Nov 1998
|
| Type |
Research and/or Program Evaluation
|
| Goal |
ESP was designed to: 1) determine whether a test of a re-employment earnings supplement could be implemented within an operating program; 2) measure the impacts of the supplement program on the future employment, earnings, and UI benefit receipt of displaced workers and repeat UI users; 3) compare the costs and benefits of such a program; and 4) document the personal experiences of the displaced workers and repeat UI users involved in the program.
|
| Program/Policy Description |
The Earnings Supplement Project (ESP) is a multi-site demonstration program that is testing an innovative financial incentive designed to stimulate re-employment. ESP provides a re-employment earnings supplement for selected applicants if they leave the unemployment insurance (UI) program quickly for a full-time job, and the new job pays less than the one they had before applying to UI. In general, the supplement is calculated to make up three-quarters of the difference between earnings in a new job and earnings in the job that was lost. For this project, supplements are being offered to two subgroups of UI claimants- those displaced from long-term employment and those with a history of repeat UI use. To encourage a rapid return to work, eligibility is conditional on leaving UI within a specified time period. The goal for the displaced workers is to shorten their often long and painful re-employment process. The goal for repeat UI users is to stimulate "off-season" employment and promote a shift toward year-round jobs.
|
| Notes |
No notes reported.
|
| |
| Last Updated |
08/18/04
|
| Type of Summary |
Reviewed
|
| External Reviewer(s) |
Saul Schwartz
(Social Research and Demonstration Corporation)
|
| Contact(s) |
Saul Schwartz (schwartz@srdc.org)
Social Research and Demonstration Corporation
50 O'Connor Street
Suite 1400
(T) (613) 237-4311
(F) (613) 237-5045
|
| Publications Department |
SRDC publications dept. (srdc@istar.ca)
Social Research and Demonstration Corporation
50 O'Connor Street
Suite 1400
(T) (613) 237-4311
(F) (613) 237-5045
|
Populations Studied
| Target Population |
Recipients/participants/clients
Applicants
|
| Subgroups Analyzed |
None
|
| Sample Size and Unit |
8,144 displaced workers and 3,414 repeat UI users. 4,063 displaced workers and 1,707 repeat UI users were assigned to the treatment group. 4,081 displaced workers and 1,707 repeat UI users were assigned to the control group.NOTE: UI applicants were checked for eligibility (differing on whether they were at a displaced worker site or a repeat UI users site) by CEC staff, and eligible applicants were asked to participate in the study. They were then randomly assigned by computer to either the supplement group or the standard group.
|
| Execution |
Not reported.
|
Sites Studied
9 Canada Employment Centers (CECs) located in seven provinces.
Findings for displaced workers will be based of ESP demonstrations run at CECs in: Granby, Quebec Oshawa, Ontario Toronto, Ontario Winnipeg, Manitoba Saskatoon, SaskatchewanFindings for repeat UI users will be based on ESP demonstration programs run at CECs in: St. Johns, Newfoundland Halifax, Nova Scotia Moncton, New Brunswick Levis, Quebec
Program Components, Policies, and Activities Evaluated
Financial incentives
- Earnings supplements/work subsidies
- Financial Incentives - misc.
| Variation in program components across sites? |
No
|
| Notes on program components |
Financial Incentives: An earnings supplement is offered if the new job pays less than the previous job held (3/4 the difference) and the recipient leaves UI within a specified period of time (26 weeks for displaced workers and 12 weeks for repeat UI users). The supplement cannot exceed $250 (Canadian) a week. Pre-UI earnings are capped at $42,380 (Canadian) per year or $815 (Canadian) per week when ESP began in 1995. Supplement payments can be received for up to 24 months. Jobs must be 30 hours per week to be considered full-time.
|
Outcomes Assessed
Employment
- Job attainment
- Job retention
Income security
- Earnings
- Unemployment Insurance (UI) receipt
Program implementation
- Program Implementation - misc.
Financial costs and benefits/cost-effectiveness
- Financial costs and benefits/cost-effectiveness - misc.
Types of Studies
| Type |
Implementation/Process Study
|
| Aim |
To examine how the supplement program was operated and attempt to identify factors that influenced its operation.
|
| |
| Type |
Impact Study (Controlled Experiment)
|
| Aim |
To compare the experiences of supplement and control group members to estimate how the supplement offer affected employment, earnings, and UI benefit receipt.
|
| |
| Type |
Cost-Benefit Study
|
| Aim |
To compare the economic benefits of the supplement program with its economic costs.
|
| |
| Type |
Descriptive/Analytical Study
|
| Aim |
To further understand the problems confronting these two groups.
|
| |
Data Sources
| Source |
Field Research
|
| Title |
Site visits
|
| Sample Characteristics/Data Collection |
Number of visits not reported.
|
| Sites |
All sites.
|
| Response Rate/Attrition Notes |
N/A
|
| Additional Execution Notes |
Site reports were filled by research team members each time they visited a CEC. On-site reviews were conducted by the research team to obtain feedback about program operations. Telephone conversations with supplement group (ESP) members were monitored periodically. Feedback was also obtained from a wide group of persons involved in the project.
|
| |
| Source |
Survey
|
| Title |
CEC staff survey
|
| Sample Characteristics/Data Collection |
85 CEC staff. Data collected April 1996.
|
| Sites |
All sites.
|
| Response Rate/Attrition Notes |
Not reported.
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Administrative data
|
| Title |
Phone log.
|
| Sample Characteristics/Data Collection |
A phone log was made of all calls by potential ESP applicants to the special toll-free number they were given to learn more about the project. Data collected throughout project duration.
|
| Sites |
All sites.
|
| Response Rate/Attrition Notes |
N/A
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Survey
|
| Title |
Telephone "mini-survey"
|
| Sample Characteristics/Data Collection |
343 diplaced workers and 229 repeat UI users soon after they were selected for the supplement group.
|
| Sites |
All sites.
|
| Response Rate/Attrition Notes |
Not reported.
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Administrative data
|
| Title |
Management Information System Data
|
| Sample Characteristics/Data Collection |
Data were used to document supplement take-up behavior. Data collected throughout study.
|
| Sites |
All sites.
|
| Response Rate/Attrition Notes |
N/A
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Survey
|
| Title |
ESP Project Application and Informed Consent Form
|
| Sample Characteristics/Data Collection |
8,144 displaced workers and 3,414 repeat UI users. Data collected at baseline.
|
| Sites |
All sites.
|
| Response Rate/Attrition Notes |
100%
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Survey
|
| Title |
ESP 15-Month Follow-Up Survey
|
| Sample Characteristics/Data Collection |
8,144 displaced workers. Data collected 15 months after random assignment.
|
| Sites |
All sites.
|
| Response Rate/Attrition Notes |
Not yet available.
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Administrative data
|
| Title |
UI records
|
| Sample Characteristics/Data Collection |
8,144 displaced workers and 3,414 repeat UI users. Data collected 36 months after random assignment.
|
| Sites |
All sites.
|
| Response Rate/Attrition Notes |
N/A
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Focus Group
|
| Title |
Focus groups for displaced workers and for repeat UI users
|
| Sample Characteristics/Data Collection |
Displaced workers: Small sample of displaced workers in supplement group. Data collected halfway through job search (13 weeks) and just after job search time period ends (26 weeks). First round conducted August 1996 (n=36); second round September, 1997 (n=49; 19 persons initiated supplement and 30 did not).Repeat UI users: 65 repeat UI users. 2 groups in each site: those who did not initiate supplement (n=28) and those who were offered opportunity to take part in ESP but declined (n=37). Data collected August, 1996.
|
| Sites |
Displaced workers: Toronto and Oshawa, Canada Repeat UI users: All UI sites.
|
| Response Rate/Attrition Notes |
Not reported.
|
| Additional Execution Notes |
Telephone interviews will be conducted with a small sample of long-term displaced worker supplement receipients just after they reach the end of their 2-year supplement receipt period. Large-scale sample survey of repeat UI users is planned for 1998, followed by another round of focus groups later in 1998.
|
| |
Findings Available
Final Implementation Findings
Final Impact Findings
Findings
|
10/01/97:
Canada's Earnings Supplement Project: Implementing the Earnings Supplement Project: A Test of a Re-employment Incentive
|
|
Final Implementation Findings:
"Enrollment of the ESP sample was conducted by local CEC staff at each study site. Information from on-site reviewed, plus a survey of CEC staff and numerous discussion with CEC managers, indicate that once standard operating procedures were in place, ESP enrollment became a regular part of local CEC routines. However, staff reductions and reorganizations associated with national changes in the UI program sometimes complicated this process. Initiation and payment of the ESP supplement were operated by a small staff at a central payment office in Halifax. This process, separate from the UI payment system, operated smoothly and effectively" (xii).
"During the ESP intake period (roughly one year), 8,144 displaced workers and 3,414 repeat UI users were enrolled in the study sample. Included in this sample were a broad range of sites, occupations, and demographic characteristics" (xii).
"Staff from CECs in the Displaced Worker study indicated clients were quite interested in ESP, whereas staff from CECs in the Repeat UI User study indicated there was little interest. Focus groups conducted with a small sample of the displaced workers supplement group indicated that many of them were interested in the supplement. Among displaced workers, 97% of those who completed an ESP application agreed to participate in the study. Only 41% of repeat UI users who were given a chance to take part in ESP agreed to do so" (xii).
"responses in a special telephone mini-survey of 343 workers and 229 repeat UI users indicated that they found the information easy to understand and quite helpful; had a good working knowledge of the terms and conditions of the ESP supplement offer; and had a good working knowledge of UI. Hence, they understood the financial benefits of the supplement relative to UI, and knew what to do to receive a supplement" (xii).
"The main effect [of the ESP supplement] will likely broaden the range of job opportunities considered by displaced workers. One-third of the displaced workers in the ESP focus groups said the supplement offer had affected their job search. Mainly they felt it expanded the range of possible jobs they would consider rather than caused them to look for work sooner or more intensely. These findings were further supported by responses to the ESP mini-survey" (xiii).
"Of the repeat workers offered the supplement, 3.8% were paid a supplement for work done by the end of their 12-week job search period. This findings, in conjunction with repeat UI users documented lack of interest in the supplement, suggests it is very unlikely the supplement influences their labor market behavior; it simply was not relevant for them" (xiii).
|
| |
|
05/01/99:
Canada's Earnings Supplement Project: A Financial Incentive to Encourage Employment Among Repeat Users of Employment Insurance
|
|
Final Impact Findings:
For repeat UI users:
By the end of the 15th month after the month of random assignment, supplement group members received $7,641 in regular unemployment benefited while control group members received $7,483. The difference…shows that the supplement group received $158 more than the control group. However,…this difference was not statistically significant…This result shows that ESP did not achieve its goal of reducing the amount of unemployment benefits received (13).
Similarly, ESP did not reduce the number of weeks of regular unemployment benefits received. The average supplement group member received unemployment benefits for 27.8 weeks during the first 15 months after random assignment. The average control group members received benefits for 0.4 weeks less. Again the difference was not statistically significant (13).
ESP, did, however, raise the cost to the government in most months…Supplement group members consistently received more in regular UI benefits and supplement payments combined than was received by control group members as regular benefits alone. However, the differences were small and only significant for a few months (15).
In conclusion, ESP was not effective in reducing the use of unemployment benefits by repeat UI users, nor was it effective in decreasing costs to government (18).
|
| |
|
05/01/99:
Canada's Earnings Supplement Project: Testing a Re-employment Incentive for Displaced Workers
|
|
Final Impact Findings:
For displaced workers:
ESP had a small positive effect on rapid re-employment. ESP produced a modest increase in full-time employment toward the end of the six month job-search period. Thus, the program increased the percentage of displaced workers who became re-employed full time during this period by 4.4 percentage points. This reflected, in roughly equal parts, a shift from part-time to full-time employment, and an increase in overall employment (ES-4).
ESPs effect on hastening re-employment may have caused a few displaced workers to take jobs that paid less than the ones they would have taken otherwise. The program may have caused average earnings during the 15-month follow-up period to be $682 or 4.6 percent lower than they would have been otherwise, presumably by inducing a few supplement group members to take lower-paying jobs. Hourly wage rates were $0.33 or 2.5 percent less than they would have been otherwise (ES-4).
ESP had virtually no effect on the amount or duration of unemployment benefits received by supplement group members. The estimated program impact on average weeks of unemployment benefit payments during the first 15 months after random assignment was 0.2 weeks or 0.9 percent. The estimated impact on total benefits received was $90 or 1.4 percent (ES-4).
ESP produced a modest transfer of resources from the Canadian government (taxpayers) to the 2 out of 10 displaced workers who received supplement payments. This helped to compensate recipients for the losses they incurred in a way that did not inhibit re-employment. On average, supplement group members experienced a small financial gain of $569 during the first 15 months after random assignment. This was because the supplement payments they received exceeded the earnings loss they incurred and their unemployment benefit payments made to the small fraction of displaced workers who received them was an important sources of temporary income for this subgroup (ES-4).
ESP produced a net financial cost for the Canadian government of $1,340 per supplement group member during the first 15 months after random assignment. This occurred because supplements paid did not reduce unemployment benefits paid (ES-4).
The findings from ESP indicate that supplements probably will not improve the labor market prospects of displaced workers markedly or reduce their unemployment benefit payments substantially. Implementing an earnings supplement program might be appropriate, however, if the goal is to provide direct financial compensation to displaced workers who must take lower-paying jobs in the future and this will pay a disproportionately higher share of the costs of economic progress (88).
|
| |
|
03/31/04:
Canada's Earning Supplement Project (ESP) Evaluation: Understanding Employment Insurance Claim Patterns: Final Report of the Earnings Supplement Project
|
|
Final Descriptive/Analytical Findings:
- Those who claim EI the most also know how to benefit the most from its rules.
- Employers may play a role in their employees' EI claim patterns.
- EI claim patterns should be examined in the context of household decisions.
- EI does not appear to be a consistent influence on individuals' decisions to move between and within provinces.
- Seasonal workers are not by definition frequent claimants of EI.
- A minority of frequent claimants remain frequent claimants over time.
|
| |
Recommendations
Existing Publications
|