Evaluation Research

Evaluations That Achieve Results:  Many SPR studies have significantly influenced programs and legislation, such as our recent research on Ontario nature and recreation trails which prompted the introduction of Bill 100 -- the Supporting Ontario Trails Act in 2015; research on family-friendly work policies, which aided the introduction of new parental leave provisions under EI in 2000; extensive research in the area of occupational health and safety in Ontario aiding the OHS Act of 1990 (and development of certification training, with 400,000 workers certified as of 2019); and the Canada Pension Plan (return-to-work initiatives).

Key Evaluation Issues:  Evaluations can help determine whether programs are achieving their objectives and provide direction on how programs can be improved.  Many of the evaluations which SPR has conducted for Federal, Provincial and other agencies over the past 25+ years have enhanced or streamlined existing programs and provided improved tools for managers.  SPR's evaluation studies have:

  • measured results using administrative data, macro and micro-economic studies, surveys and audits.  Many SPR evaluations have developed new indicators of results directly related to program goals;

  • assessed client and stakeholder satisfaction through the use of client satisfaction surveys (for example, SPR's surveys of users of federal information clearinghouses);

  • identified economic and social impacts using experimental, longitudinal and 'after-the-fact' designs, such as SPR's analysis of macro-economic factors affecting public use of the Canada Pension Plan (Disability);

  • evaluated cost-effectiveness in program funding, Federal-Provincial-Municipal cost-sharing agreements;

  • evaluated horizontal programs (inter-departmental initiatives), such as the Federal Contractor's Program (HRDC), which covers nearly all Federal agencies; and

  • recommended alternative program approaches and strategies for program delivery, and needs for programs and services.
Methodologies:  SPR evaluations have identified strategies to improve the cost-effectiveness and impact of programs in business and human services, and have included a variety of methodologies such as:

  • Descriptive studies of programs:  examining formal and operational views of programs; reviewing program goals and objectives as seen by different stakeholders; or providing analyses of services;

  • Case studies:  providing an overview of specific cases, such as an occupational health and safety program in a particular industrial facility, or operations of a Native Friendship Centre;

  • Client surveys:  assessing client views of and satisfaction with programs, or determining client priorities for program improvements;

  • Staff surveys:  assessing the operations of health & safety programs for organization Surveys of staff of organizations such as Labour Canada and the Canadian Weather Service; surveys of staff at Ontario corrections facilities and surveys of youth protection workers at Ontario Children's Aid Societies;

  • Multi-variate analyses of program impacts:  comparing the impacts of programs (for example, through 'quasi-experimental' designs), or assessing mediating impacts of participant characteristics (such as the way firm size or export experience mediate the impact of Federal export control programs).