Document T083
The National Capital Chapter of the Canadian Evaluation Society (CES-NCC) Board of Directors requires a better understanding of the professional needs of the members of the Chapter in order to develop its multi-year service strategy. Although the CES has traditionally focussed on the training component of professional needs, CES-NCC wants to explore other aspects such as networking, mentoring, communications, advocacy, etc.
The objectives of the project were:
- to provide an understanding of the professional needs of the members of the CES-NCC; and,
- to support the development of the CES-NCC multi-year service strategy.
The study methodology included a Web-based survey of 285 CES-NCC members (out of 633) between August 18 and September 24, 2010 (response rate of 48%; precision of±4.3 percentage points); and, three 2-hour focus groups with non-management federal evaluators, federal evaluation managers, and non-federal evaluators.
KEY RESULTS
Who are CES-NCC members?
- Federal: 65% federal; 26% private sector; 6% not-for-profit.
- Producers: 85% are producers of evaluation: 52% for their own organization and 33% for other organizations.
- Variable experience: 32% have 3 years of experience in evaluation or less; 23%, 4 to 7 years; 43%, 8 or more.
- Involved: 69% devote at least half of their time to evaluation; 29%, less than half.
- Educated: 68% have a Master's degree; 12%, a doctorate.
Reaction to the current CES-NCC offering?
- Mildly satisfied: 68 points overall.
- But loyal: 89 points on likelihood to renew membership.
- 2009 learning event: particularly well received, interactive, dynamic, well run.
- Networking: current events not conducive to networking.
- Web site: has very low visibility; confusion with the national site.
Needs regarding training
- Professional introduction training is sought for new evaluators.
- Advanced training relevant to established evaluators is the most desired service.
- Preferred topics: cost-effectiveness and cost-benefit analyses; assessing data reliability, validity and trustworthiness; data analysis and interpretation; research design; communicating evaluation results.
- Training format: face-to-face, off work site; single-day for managers; multi-day for others.
Needs regarding networking
- Thirst: there is considerable thirst for networking in the form of communities of practice.
- Lunch & Learn: very positive feedback.
Expectations regarding advocacy
- Priority: important CES responsibility for many.
- Meaning: improving the profile of evaluation, publicly demonstrating evaluation value, promoting quality professional values and behaviour, taking positions in public debates.
- Capacity: recognized as a difficult endeavour, particularly without a permanent secretariat.
RECOMMENDATIONS
Training
- Develop a high-level curriculum for new evaluators; support offerings.
- Identify existing training offerings; coach trainers to adapt to the evaluation context.
Networking
- Develop a strategy to implement Niche Interest Groups; e.g., seed events, lists of individuals sharing interests, training on network management, training on Internet-based tools to support networking, making venues available at low or no cost for meetings, etc.
- Revise the format of breakfast sessions to foster networking.
- Consider building a Web service to list evaluators.
Communications
- Define expectations for the Web site and build a relevant development strategy.
- Monthly e-mail communication from its president to members.
- Brand communications from CES-NCC as CES-NCC.
Pricing
- Continue to offer events free of charge for members.
Advocacy
- Take an active role in the advocacy efforts of CES National.
60 pages, 194K [PDF format]
Home Page