Studies in a Balanced Scorecard Framework

SPR has conducted many public sector studies within a Balanced Scorecard (BSC) Framework, several resulting in major program, organizational development, or public policy impacts. Generally, these studies have focused on measuring systemic changes and making strategic changes for future improvements, based on the linkages between organizational results and capacity (resources) and internal processes. BSC studies usually move away from a classic "measurement perspective" typical of evaluations, towards a "management perspective" which emphasizes performance improvement.

BSC studies also require a causal model of key processes or, in the BSC framework, strategy maps, so that findings can show how management decisions can improve performance. These studies often focus on "the bottom-line" (profits) in the private sector, but more broadly on the achievement of goals or "results" in the public sector.

BSC studies differ from other studies as they generally have been developed within organizations which are engaged in major change and improvement initiatives. BSC studies also require the development of broader organizational or multi-constituency buy-in to a change process (with dissemination of results and, ideally, impact throughout the organization). Many of SPR's BSC studies have been multi-year efforts, resulting in major organizational or public program development.

Some of SPR's balanced scorecard-type studies include:

  • Study of Truck Parking, Public Safety and OHS Impacts:  This $280,000 study examined the need for long-haul truck parking and rest areas along Provincial highways in Southern Ontario. The project included an on-line survey of 2,300+ truck drivers using Southern Ontario highways, consultations with trucking companies and industry associations, and engineering studies. The research addressed the need for additional parking and the linkages between driver fatigue and collisions (for Canadian Trucking Associations and the Ontario Ministry of Transportation, 2018-2019);

  • Evaluation of the Victim/Witness Assistance Program (V/WAP):  Utilizing a multi-method approach, this balanced scorecard-type evaluation assessed the quality and effectiveness of the program which provides assistance in court to 300,000 Ontarians annually, mainly women who are victims of family violence. The evaluation identified success factors such as effective staffing and also assessed future priorities. The project involved a web survey of 200 V/WAP staff, Crown Attorneys and 500+ agencies (community services, police, etc.) and identified important issues in the program, including worker burnout (for the Ontario Ministry of the Attorney General, 2010-2011);

  • RCMP Performance Measurement:  This project provided baseline data for the RCMP's performance measurement within a balanced scorecard framework. The project included a number of multi-method surveys. A Canada-wide survey of 30,000 households asked about awareness, experience, and satisfaction regarding services received from the RCMP. Particular attention was given to Western and Atlantic Canada, and the Territories, where the RCMP provides a wider range of services. Over 9,000 RCMP employees, policing agencies, clients of the RCMP's contract policing (selected provinces, territories, municipalities and First Nations), and international policing partners were also surveyed (for the RCMP, 2003-2004);

  • Strategic Review of Exchange Programs:  This review examined a number of youth exchange programs, particularly interprovincial reciprocal visits between high school youth.  The review reflected a balanced scorecard-type framework by examining processes in a through-time context.  Research examined resources, reach, impact and future potential and involved interviews, focus groups, and surveys with exchange providers and youth in every province and territory.  Study results showed that youth exchanges achieve important impacts on national identity, use of official languages, and social and work skills of youth.  This study aided the development of Exchanges Canada, a program providing youth exchanges (for Canadian Heritage, 1997-2000);

  • Evaluation of the Federal Labour Code - Phase I:  This project, implemented in a 'balanced scorecard' type framework, examined labour standards as a tool for the achievement of broader labour and economic goals of the Government of Canada.  The evaluation examined how human and technological resources affected bottom-line outcomes in complaints and costs under the Canada Labour Code.  The evaluation focused on the need for improved information processes to aid the reduction of employer infractions. Key data sources included employer, union, worker and staff satisfaction surveys.  A strategy to allocate new resources to Labour Code communications and information was approved as a result of this project (for Human Resources Development Canada, 1996-1998);

  • Evaluation of the Special Measures Initiatives Program:  This balanced-scorecard-type assessment focused on success in achieving employment equity goals in the Federal Public Service. This was achieved by examining program finances, departmental access and buy-in to the program, client satisfaction, and the impact of some 161 innovative projects on specific Federal Departments and agencies. Data collection included administrative information, financial data, focus groups and stakeholder consultations, which were conducted across Canada. The evaluation resulted in a highly-modified business relationship between Treasury Board and the Public Service Commission (for Treasury Board of Canada Secretariat, 1997-98); and

  • Review of Occupational Health and Safety Training in Ontario:  This study, implemented in a balanced scorecard framework, examined program dynamics and future directions for Ontario's twelve health and safety organizations -- a program with a budget of over $55 million in 1991. The review examined resources, efficiency, staff capabilities and goal achievement under the Occupational Health and Safety Act. Sub-studies examined staff satisfaction, resources, and organizational capacity. Related research continued over a five-year period. The review aided the development of Canada's largest training program in Occupational Health and Safety (with over 400,000 workers and managers certified since 1994) (for the Ontario Workplace Health and Safety Agency, 1994-96).