Home Page Company Clients Reports CallWeb SampleCalc EPlanCalc French

For every complex problem there is an answer that is clear, simple, and wrong.
(H. L. Mencken)

Document T060

Performance Measurement and Evaluation Framework for the Canada Research Chairs Program 2000—2004

Report presented to the Chairs/NCE Evaluation Steering Committee, December 2001

The Canada Research Chairs Program (hereafter, the program) is a key piece in Canada's strategy to become a world leader in the knowledge-based economy. The program's contribution to the strategy is to enable Canadian universities to create research opportunities that will attract and retain the outstanding researchers. It is aimed at developing 2000 university chairs between years 2000 and 2005. The 2000 Budget provided $900 million over five years toward that objective — in addition to $250 million earmarked within the Canadian Foundation for Innovation (CFI) to support the program. The program is managed by the Social Science and Humanities Research Council on behalf of a management committee where the Natural Sciences and Engineering Research Council, the Canadian Institutes of Health Research, the Canada Foundation for Innovation as well as Industry Canada are also represented.

Program logic

The program is based on the pre-allocation of research chairs to universities and disciplines based on funds received from granting councils in the past. Universities present nominations which are assessed through a peer-review process. The review takes into consideration both the established excellence of the researchers (for Tier I chairs, valid for seven years and renewable without limit) or the potential for excellence (for Tier II chairs, valid for five years and renewable once) and the contribution that the nominee could make to the realization of the university strategic research plan.

The program aims at producing two critical short term impacts: retaining the best Canadian researchers as well as attracting excellent researchers from other countries (or Canadian expatriates). The program also pursues a peripheral objective of ensuring the effective use of research resources through strategic planning by the institutions as well as the inter-institutional and inter-sectoral collaboration.

The establishment of a more productive research infrastructure environment is expected to contribute to offsetting "brain-drain" pressures. A series of impacts on the university research environment should flow from the reduction of pressures on key personnel: easing the construction of dynamic research teams; reinforcing the training of graduate students through inclusion in a world-class research environment; producing more and better graduate students and improving retention of highly qualified personnel in Canada.

In the long run, the goal of the program is to lead to a strengthened and more highly competitive research environment in Canada's universities as compared to our competitors in the world, and enhanced Canadian visibility in the global knowledge-based economy. Ultimately, the program should contribute to the overall government objectives of a strong economy and of a better quality of life.

Evaluation issues

This evaluation framework paves the way for ongoing performance monitoring, the mid-term review and the evaluation planned in the fifth year of program implementation. It is based on a review of program documentation, interviews with 30 key informants and a literature scan.

Evaluation issues have been identified within three groups:

Evaluation methods

Priorities have been established within the set of issues and timing of the study of the issues has been laid out. Indicators and data sources have been associated with each issue. The following data sources have been identified as requirements for a full evaluation of the program:

Ongoing performance measurement was given particular attention. It includes program activity data drawn from statistical reports produced by the program as well as information on immediate impacts from special requests to universities and annual chair and university reports.

Evaluation options were developed. They correspond to a minimal but credible package which deals with highest priority issues; a more complete package which provides more qualitative information relative to the incrementality of the program and more complete data collection on the issue of excellence; and a comprehensive package including all indicators and data sources.

52 pages, 647k [PDF format]

Document T078

Joint Evaluation of Research Tools and Instruments Grants (RTI) and Major Facilities Access Grants (MFA)

Prepared for the Natural Sciences and Engineering Research Council, August 2007

The Natural Sciences and Engineering Research Council (NSERC) is a national research-granting agency that funds the direct costs of research. It assists in buying or developing research equipment and in accessing regional or national research facilities via two main programs: the Research Tools and Instruments (RTI) program and the Major Facilities Access (MFA) program. This report presents the findings of the Joint Evaluation of RTI and MFA.

91 pages, 403K [PDF format]

Document T079

Evaluation of the Networks of Centres of Excellence Program

Prepared for the Interagency Evaluation Steering Committee, October 2007

This evaluation study was the Networks of Centres of Excellence (NCE) Program's third in the ten years. It was required as part of the renewal of the program's Terms and Conditions and it was conducted for the Interagency Evaluation Steering Committee on behalf of the NCE Steering Committee.

98 pages, 506K [PDF format]

Document T080

Evaluation of the Tri-Agency Indirect Costs Program

Prepared for the Interagency Evaluation Steering Committee, July 2009

The present evaluation study of the Indirect Costs Program (ICP) was conducted in preparation for the renewal of its terms and conditions. It was conducted for the presidents of the Canadian Institutes of Health Research (CIHR), the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Social Sciences and Humanities Research Council of Canada (SSHRC).

118 pages, 474K [PDF format]

Document T081

Canada Graduate Scholarships (CGS) Program and Related Programs Review

Prepared for the Interagency Evaluation Steering Committee, November 2008

This evaluation study of the Canada Graduate Scholarships (CGS) program was conducted in preparation for the renewal of its Terms and Conditions. It was conducted for the Presidents of the Canadian Institutes of Health Research (CIHR), the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Social Sciences and Humanities Research Council of Canada (SSHRC). It was managed by the Interagency Evaluation Steering Committee which is comprised of program and evaluation representatives from all three Agencies as well as Industry Canada.

167 pages, 714K [PDF format]

Document T089

Industry-Driven Collaborative Research and Development Sub-Program, Final Evaluation Report

Prepared by R.A. Malatest and Associates

This evaluation of Natural Science and Engineering Research Council's (NSERC) Industry-driven Collaborative Research and Development sub-program was conducted by R.A. Malatest and Associates in collaboration with NSERC. Circum's president, Benoît Gauthier, was principal evaluator. The evaluation was based on a variety of sources of information: a secondary data review; an administrative data review; key informant interviews; case studies; a literature review; and a cost-efficiency analysis.

The sub-program represents NSERC's largest suite of initiatives supporting industry-academic partnerships. This evaluation, conducted in fiscal year 2015/16, covered three of the Industry-driven Collaborative Research and Development grants with total expenditures of $643 million from 2009 to 2014. These three funding opportunities are: Collaborative Research and Development (CRD) grants, Industrial Research Chairs (IRC) grants, and Engage grants.

The evaluation concluded that the sub-program was designed to meet the needs of both industrial partners and academic researchers and that the sub-program was well aligned with the priorities of the federal government and NSERC.

The university-industry partnerships were generally successful at fostering meaningful collaborations that last beyond the funding period. The industry-driven funding opportunities have substantially contributed to enhancing the research capacity of the university researchers involved. Industrial partners reported that the grants had significant impacts on competitiveness and productivity. The transfer of knowledge to industrial partners was another benefit that companies obtained. The industry-driven funding opportunities involved substantial numbers of students and fellows in applied industrial research and their involvement was multifaceted,

Recommendation #1: maintain the Industry-Driven funding opportunities.

Recommendation #2: continue efforts to develop common metrics for the measurement of impacts on industry and consider homogenizing vocabulary among grants.

Recommendation #3: consider revising the Engage logic model to improve alignment with the objectives of the grants.

943K [PDF format]

To reach us:

General address : service@circum.com
Benoît Gauthier : gauthier@circum.com, @BGauthierCEEQ
Tel. : +1 819 775-2620, Fax : (no more fax; sorry)
238 Fleming Road, Cantley, Québec J8V 3B4

Benoît Gauthier, Mastodon