http://www.researchforum.org/project_general_149.html

Canada's Earnings Supplement Project (ESP) Evaluation

General Information

View a brief abstract of this project.

View a complete, printer-friendly profile of this project.

Evaluator(s) Social Research and Demonstration Corporation
Investigator(s) Saul Schwartz (Social Research and Demonstration Corporation)
Howard Bloom (Social Research and Demonstration Corporation)
Sponsor(s) Human Resources Development Canada
Funder(s) Human Resources Development Canada
Subcontractor(s) Statistics Canada
 
Domain Income Security/TANF
Status Completed (final report released)
Duration Mar 1995 - Nov 1998
Type Research and/or Program Evaluation
Goal ESP was designed to: 1) determine whether a test of a re-employment earnings supplement could be implemented within an operating program; 2) measure the impacts of the supplement program on the future employment, earnings, and UI benefit receipt of displaced workers and repeat UI users; 3) compare the costs and benefits of such a program; and 4) document the personal experiences of the displaced workers and repeat UI users involved in the program.
Program/Policy Description The Earnings Supplement Project (ESP) is a multi-site demonstration program that is testing an innovative financial incentive designed to stimulate re-employment. ESP provides a re-employment earnings supplement for selected applicants if they leave the unemployment insurance (UI) program quickly for a full-time job, and the new job pays less than the one they had before applying to UI. In general, the supplement is calculated to make up three-quarters of the difference between earnings in a new job and earnings in the job that was lost. For this project, supplements are being offered to two subgroups of UI claimants- those displaced from long-term employment and those with a history of repeat UI use. To encourage a rapid return to work, eligibility is conditional on leaving UI within a specified time period. The goal for the displaced workers is to shorten their often long and painful re-employment process. The goal for repeat UI users is to stimulate "off-season" employment and promote a shift toward year-round jobs.
Notes No notes reported.
 
Last Updated 08/18/04
Type of Summary Reviewed
External Reviewer(s) Saul Schwartz (Social Research and Demonstration Corporation)
Contact(s) Saul Schwartz (schwartz@srdc.org)
Social Research and Demonstration Corporation
50 O'Connor Street
Suite 1400
(T) (613) 237-4311
(F) (613) 237-5045
Publications Department SRDC publications dept. (srdc@istar.ca)
Social Research and Demonstration Corporation
50 O'Connor Street
Suite 1400
(T) (613) 237-4311
(F) (613) 237-5045

Populations Studied

Target Population Recipients/participants/clients
Applicants
Subgroups Analyzed None
Sample Size and Unit 8,144 displaced workers and 3,414 repeat UI users.
4,063 displaced workers and 1,707 repeat UI users were assigned to the treatment group.
4,081 displaced workers and 1,707 repeat UI users were assigned to the control group.

NOTE: UI applicants were checked for eligibility (differing on whether they were at a displaced worker site or a repeat UI users site) by CEC staff, and eligible applicants were asked to participate in the study. They were then randomly assigned by computer to either the supplement group or the standard group.

Execution Not reported.

Sites Studied

9 Canada Employment Centers (CECs) located in seven provinces.
Findings for displaced workers will be based of ESP demonstrations run at CECs in:
Granby, Quebec
Oshawa, Ontario
Toronto, Ontario
Winnipeg, Manitoba
Saskatoon, Saskatchewan

Findings for repeat UI users will be based on ESP demonstration programs run at CECs in:
St. Johns, Newfoundland
Halifax, Nova Scotia
Moncton, New Brunswick
Levis, Quebec

Program Components, Policies, and Activities Evaluated

Financial incentives

  • Earnings supplements/work subsidies
  • Financial Incentives - misc.
Variation in program components across sites? No
Notes on program components Financial Incentives: An earnings supplement is offered if the new job pays less than the previous job held (3/4 the difference) and the recipient leaves UI within a specified period of time (26 weeks for displaced workers and 12 weeks for repeat UI users). The supplement cannot exceed $250 (Canadian) a week. Pre-UI earnings are capped at $42,380 (Canadian) per year or $815 (Canadian) per week when ESP began in 1995. Supplement payments can be received for up to 24 months. Jobs must be 30 hours per week to be considered full-time.

Outcomes Assessed

Employment

  • Job attainment
  • Job retention

Income security

  • Earnings
  • Unemployment Insurance (UI) receipt

Program implementation

  • Program Implementation - misc.

Financial costs and benefits/cost-effectiveness

  • Financial costs and benefits/cost-effectiveness - misc.

Types of Studies

Type Implementation/Process Study
Aim To examine how the supplement program was operated and attempt to identify factors that influenced its operation.
 
Type Impact Study (Controlled Experiment)
Aim To compare the experiences of supplement and control group members to estimate how the supplement offer affected employment, earnings, and UI benefit receipt.
 
Type Cost-Benefit Study
Aim To compare the economic benefits of the supplement program with its economic costs.
 
Type Descriptive/Analytical Study
Aim To further understand the problems confronting these two groups.
 

Data Sources

Source Field Research
Title Site visits
Sample Characteristics/Data Collection Number of visits not reported.
Sites All sites.
Response Rate/Attrition Notes N/A
Additional Execution Notes Site reports were filled by research team members each time they visited a CEC.

On-site reviews were conducted by the research team to obtain feedback about program operations.

Telephone conversations with supplement group (ESP) members were monitored periodically.

Feedback was also obtained from a wide group of persons involved in the project.

 
Source Survey
Title CEC staff survey
Sample Characteristics/Data Collection 85 CEC staff.
Data collected April 1996.
Sites All sites.
Response Rate/Attrition Notes Not reported.
Additional Execution Notes No notes reported.
 
Source Administrative data
Title Phone log.
Sample Characteristics/Data Collection A phone log was made of all calls by potential ESP applicants to the special toll-free number they were given to learn more about the project.
Data collected throughout project duration.
Sites All sites.
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Survey
Title Telephone "mini-survey"
Sample Characteristics/Data Collection 343 diplaced workers and 229 repeat UI users soon after they were selected for the supplement group.
Sites All sites.
Response Rate/Attrition Notes Not reported.
Additional Execution Notes No notes reported.
 
Source Administrative data
Title Management Information System Data
Sample Characteristics/Data Collection Data were used to document supplement take-up behavior.
Data collected throughout study.
Sites All sites.
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Survey
Title ESP Project Application and Informed Consent Form
Sample Characteristics/Data Collection 8,144 displaced workers and 3,414 repeat UI users.
Data collected at baseline.
Sites All sites.
Response Rate/Attrition Notes 100%
Additional Execution Notes No notes reported.
 
Source Survey
Title ESP 15-Month Follow-Up Survey
Sample Characteristics/Data Collection 8,144 displaced workers.
Data collected 15 months after random assignment.
Sites All sites.
Response Rate/Attrition Notes Not yet available.
Additional Execution Notes No notes reported.
 
Source Administrative data
Title UI records
Sample Characteristics/Data Collection 8,144 displaced workers and 3,414 repeat UI users.
Data collected 36 months after random assignment.
Sites All sites.
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Focus Group
Title Focus groups for displaced workers and for repeat UI users
Sample Characteristics/Data Collection Displaced workers:
Small sample of displaced workers in supplement group.
Data collected halfway through job search (13 weeks) and just after job search time period ends (26 weeks).
First round conducted August 1996 (n=36); second round September, 1997 (n=49; 19 persons initiated supplement and 30 did not).

Repeat UI users:
65 repeat UI users.
2 groups in each site: those who did not initiate supplement (n=28) and those who were offered opportunity to take part in ESP but declined (n=37).
Data collected August, 1996.

Sites Displaced workers:
Toronto and Oshawa, Canada
Repeat UI users:
All UI sites.
Response Rate/Attrition Notes Not reported.
Additional Execution Notes Telephone interviews will be conducted with a small sample of long-term displaced worker supplement receipients just after they reach the end of their 2-year supplement receipt period.

Large-scale sample survey of repeat UI users is planned for 1998, followed by another round of focus groups later in 1998.

 

Findings Available

Final Implementation Findings
Final Impact Findings

Findings

10/01/97: Canada's Earnings Supplement Project: Implementing the Earnings Supplement Project: A Test of a Re-employment Incentive
Final Implementation Findings:

"Enrollment of the ESP sample was conducted by local CEC staff at each study site. Information from on-site reviewed, plus a survey of CEC staff and numerous discussion with CEC managers, indicate that once standard operating procedures were in place, ESP enrollment became a regular part of local CEC routines. However, staff reductions and reorganizations associated with national changes in the UI program sometimes complicated this process. Initiation and payment of the ESP supplement were operated by a small staff at a central payment office in Halifax. This process, separate from the UI payment system, operated smoothly and effectively" (xii).

"During the ESP intake period (roughly one year), 8,144 displaced workers and 3,414 repeat UI users were enrolled in the study sample. Included in this sample were a broad range of sites, occupations, and demographic characteristics" (xii).

"Staff from CECs in the Displaced Worker study indicated clients were quite interested in ESP, whereas staff from CECs in the Repeat UI User study indicated there was little interest. Focus groups conducted with a small sample of the displaced workers supplement group indicated that many of them were interested in the supplement. Among displaced workers, 97% of those who completed an ESP application agreed to participate in the study. Only 41% of repeat UI users who were given a chance to take part in ESP agreed to do so" (xii).

"responses in a special telephone mini-survey of 343 workers and 229 repeat UI users indicated that they found the information easy to understand and quite helpful; had a good working knowledge of the terms and conditions of the ESP supplement offer; and had a good working knowledge of UI. Hence, they understood the financial benefits of the supplement relative to UI, and knew what to do to receive a supplement" (xii).

"The main effect [of the ESP supplement] will likely broaden the range of job opportunities considered by displaced workers. One-third of the displaced workers in the ESP focus groups said the supplement offer had affected their job search. Mainly they felt it expanded the range of possible jobs they would consider rather than caused them to look for work sooner or more intensely. These findings were further supported by responses to the ESP mini-survey" (xiii).

"Of the repeat workers offered the supplement, 3.8% were paid a supplement for work done by the end of their 12-week job search period. This findings, in conjunction with repeat UI users’ documented lack of interest in the supplement, suggests it is very unlikely the supplement influences their labor market behavior; it simply was not relevant for them" (xiii).

 
05/01/99: Canada's Earnings Supplement Project: A Financial Incentive to Encourage Employment Among Repeat Users of Employment Insurance
Final Impact Findings:

For repeat UI users:

“By the end of the 15th month after the month of random assignment, supplement group members received $7,641 in regular unemployment benefited while control group members received $7,483. The difference…shows that the supplement group received $158 more than the control group. However,…this difference was not statistically significant…This result shows that ESP did not achieve its goal of reducing the amount of unemployment benefits received” (13).

“Similarly, ESP did not reduce the number of weeks of regular unemployment benefits received. The average supplement group member received unemployment benefits for 27.8 weeks during the first 15 months after random assignment. The average control group members received benefits for 0.4 weeks less. Again the difference was not statistically significant” (13).

“ESP, did, however, raise the cost to the government in most months…Supplement group members consistently received more in regular UI benefits and supplement payments combined than was received by control group members as regular benefits alone. However, the differences were small and only significant for a few months” (15).

“In conclusion, ESP was not effective in reducing the use of unemployment benefits by repeat UI users, nor was it effective in decreasing costs to government” (18).

 
05/01/99: Canada's Earnings Supplement Project: Testing a Re-employment Incentive for Displaced Workers
Final Impact Findings:

For displaced workers:

“ESP had a small positive effect on rapid re-employment. ESP produced a modest increase in full-time employment toward the end of the six month job-search period. Thus, the program increased the percentage of displaced workers who became re-employed full time during this period by 4.4 percentage points. This reflected, in roughly equal parts, a shift from part-time to full-time employment, and an increase in overall employment” (ES-4).

“ESP’s effect on hastening re-employment may have caused a few displaced workers to take jobs that paid less than the ones they would have taken otherwise. The program may have caused average earnings during the 15-month follow-up period to be $682 or 4.6 percent lower than they would have been otherwise, presumably by inducing a few supplement group members to take lower-paying jobs. Hourly wage rates were $0.33 or 2.5 percent less than they would have been otherwise” (ES-4).

“ESP had virtually no effect on the amount or duration of unemployment benefits received by supplement group members. The estimated program impact on average weeks of unemployment benefit payments during the first 15 months after random assignment was 0.2 weeks or 0.9 percent. The estimated impact on total benefits received was $90 or 1.4 percent” (ES-4).

“ESP produced a modest transfer of resources from the Canadian government (taxpayers) to the 2 out of 10 displaced workers who received supplement payments. This helped to compensate recipients for the losses they incurred in a way that did not inhibit re-employment. On average, supplement group members experienced a small financial gain of $569 during the first 15 months after random assignment. This was because the supplement payments they received exceeded the earnings loss they incurred and their unemployment benefit payments made to the small fraction of displaced workers who received them was an important sources of temporary income for this subgroup” (ES-4).

“ESP produced a net financial cost for the Canadian government of $1,340 per supplement group member during the first 15 months after random assignment. This occurred because supplements paid did not reduce unemployment benefits paid” (ES-4).

“The findings from ESP indicate that supplements probably will not improve the labor market prospects of displaced workers markedly or reduce their unemployment benefit payments substantially. Implementing an earnings supplement program might be appropriate, however, if the goal is to provide direct financial compensation to displaced workers who must take lower-paying jobs in the future and this will pay a disproportionately higher share of the costs of economic progress” (88).

 
03/31/04: Canada's Earning Supplement Project (ESP) Evaluation: Understanding Employment Insurance Claim Patterns: Final Report of the Earnings Supplement Project
Final Descriptive/Analytical Findings:

  • Those who claim EI the most also know how to benefit the most from its rules.
  • Employers may play a role in their employees' EI claim patterns.
  • EI claim patterns should be examined in the context of household decisions.
  • EI does not appear to be a consistent influence on individuals' decisions to move between and within provinces.
  • Seasonal workers are not by definition frequent claimants of EI.
  • A minority of frequent claimants remain frequent claimants over time.
 

Recommendations

Canada's Earnings Supplement Project: Implementing the Earnings Supplement Project: A Test of a Re-employment Incentive (10/01/97)
Recommendations not yet available.
 

Existing Publications

10/01/97 Canada's Earnings Supplement Project: Implementing the Earnings Supplement Project: A Test of a Re-employment Incentive SRDC
05/01/99 Canada's Earnings Supplement Project: A Financial Incentive to Encourage Employment Among Repeat Users of Employment Insurance SRDC
05/01/99 Canada's Earnings Supplement Project: Testing a Re-employment Incentive for Displaced Workers SRDC
03/01/01 Canada's Earnings Supplement Project: The Frequent Use of Unemployment Insurance in Canada SRDC
03/01/02 Canada's Earnings Supplement Project: Essays on the Repeat Use of Unemployment Insurance SRDC
03/31/04 Canada's Earning Supplement Project (ESP) Evaluation: Understanding Employment Insurance Claim Patterns: Final Report of the Earnings Supplement Project SRDC
05/01/04 Canada's Earnings Supplement Project Evaluation: Employment Insurance and Family Response to Unemployment: Canadian Evidence From the SLID SRDC