Post-secondary Education in the West: British Columbia vs Alberta
My “Run-off” is between the departments of Advanced Education in British Columbia and in Alberta. The Advanced Education (or post-secondary education) programs fall within the responsibility of similar Ministries in both provinces. I reviewed the fiscal cycle for 2012-2013 which was the latest period for which actual results were available in both provinces. During the period between planning and accountability reporting, the Ministry portfolio in both provinces changed slightly, however, advanced education and training remained the primary focus of the Ministry in both provinces. In 2012 – planning year for 2012-2013 - post-secondary education fell under the BC Ministry of Advanced Education, which changed to the Ministry of Advanced Education, Innovation and Technology by the time the results were reported in 2012-2013. In Alberta, during the planning part of the accountability cycle in 2012, post-secondary education was the responsibility of the Ministry of Advanced Education and Technology but changed slightly to be the Ministry of Advanced Education and Enterprise by the reporting time in 2013. In all cases, the Ministries had the responsibility for the oversight of the publically funded universities and other post-secondary institutes within their respective provinces. Since the portfolios changed slightly from planning to reporting, I focused only on the post-secondary objectives, measures, and results.
Overview:
While the governments of the day in BC and Alberta are politically not that similar, they both have well developed planning and reporting processes. As a result, both Ministries that I reviewed scored high in many of the categories. Based on the information available on the government websites, this maturity in process is evident across the government as a whole and not just within the specific Ministries I reviewed. The planning and reporting cycle (accountability cycle) in both provinces begins with an overall government Strategic Plan (or Service Plan) which presents the governments’ 3 year outlooks. This document is updated annually and released around February each year to reflect the changing priorities of the government. The overall strategic plan does not necessarily reflect any of the priority or specific programs of any particular ministry. However, in both the BC and Alberta examples I reviewed (2012-2015 plans), the Finance Minister specifically announced in the budget speech, items which were also priority items or projects for the Advanced Education Ministries. While these items were mentioned in the Government Strategic Plans, there was very little detail (in the government or the Ministry plans) to be able to later assess whether the targets were met.
The next step in the accountability cycle for these Ministries was to create a detailed service plan. In BC, the Ministry prepared a 3 year service plan for 2012-2015 which detailed all of the specific goals, objectives and performance measures for the Ministry. In AB, the Ministry of Advanced Education and Technology prepared a 3 year business plan for 2012-2015 which also stated the Ministry goals, objectives and related performance measures. The Alberta model is very succinct and to the point, while the BC model included a lot of discussion and opinion. Depending on your knowledge of the Ministry in review, the added detail can be seen as useful background information or political story telling.
To complete the accountability cycle, both Governments and Ministries prepared a report on the fiscal results. An Annual Report in Alberta or Annual Service Plan Report in BC which detailed the results for the year. These documents were made available to the public late in the year of 2013. In both cases the documents included results of the specific performance measures as well as some commentary on primary objectives. Both provinces reported on their results and provided a comparison of the actual results to their targets.
Finally, I reviewed some of the other documents that were available from both Governments and Ministries. These include further documentation on their accountability frameworks, summaries of results and Government/Ministry at a glance documents. As I will discuss further, some of these documents add value to understanding the accountability cycle, while others were simply marketing documents for the government of the day. One thing to note is that for the period I compared 2012-2013 neither of the provinces had a change in government. In Alberta, Premier Redford began in the fall of 2011, and in BC, Premier Clark started in spring 2011. I believe that the continuity of the documents may be more difficult to follow had there been a change in government due to the potential for significant adjustments to the Ministerial priorities or portfolios between the planning and reporting periods. For this review, the commonalities in the electoral process made the results easier to compare. Comparison:
To prepare the comparison of the Accountability reports of these two advanced Education departments I went to the respective websites. In both provinces, the latest actual results and accountability report available was for the fiscal year 2012-2013. This is the base year that was used for comparison. I reviewed the strategic plans and budget documents for the government and the specific Ministries for this same fiscal year. For some of the additional background information there was no 2012-2013 fiscal year version online. In these cases I reviewed a more current version of the document which likely reflects the government’s current practices.
To facilitate the comparison, I divided the Accountability Cycle into 4 sections as follows:
1. Availability of Information
2. Review of the Planning process and documents
3. Review of the Annual Accountability Report and Results
4. Appropriateness of Criteria and Objectives.
Within each of these sections I created detailed criteria to use to rate the effectiveness of the process using a simple 0-5 scale. I took the average score within each section to produce the section and overall scores for each Province. A summary of the scale I used can be found in Appendix 1. As you can see from the following Table (see Appendix 1 for graph detail), the Ministry of Advanced Education in Alberta came away with slightly higher overall score than the similar Ministry in British Columbia. In both cases, the Ministries scored higher on the planning part of the cycle than they did on the reporting/accountability part. For detailed scoring and a summary of my observations for each category, see Appendix 2.
The British Columbia accountability cycle of planning, measuring, reporting has the feel of a campaign document or a budget announcement. I feel like the information being provided has been thoroughly reviewed and provided to me only if it helps to paint the picture the Government wants to show. While the Ministry of Advanced Education has a good plan which documents all of the objectives and the rationale for the objectives, it lacks measurability on the reporting side. Within the planning and budget documents there is a large section containing a narrative which sets the environment and provides the connection to the Ministry’s priorities and the Government’s priorities. In the reporting cycle I reviewed, the primary focus of the BC government was job creation. The information provided in the planning documents which was a report on how the Ministry did (or is doing) would have been more effective if it had been included in the Annual Service Report rather than the Service Plan. The Ministry of Advanced Education in BC has a document called the Accountability Framework Standards Manual which is a very good explanation of the data sources for the various performance measures as well as specific timelines and reporting requirements by institution. Overall, the BC Ministry of Advanced Education does an above average job of accountability reporting. The documents are available, accurate and provide the reader with information which is consistent with their strategy and mandate. The Ministry does not seem to reflect expected changes in performance in their goal setting as most of the targets in the performance measures are the same for the entire 3 year planning period. Additionally, I felt that the measurement scale which used language such as “substantially achieved” for performance measures in the 90-100% range misleads the public by giving the impression of accomplishment when in reality the goal was not met. (See Appendix 2 Section 3 for more detail on the BC scale)
In Alberta, the accountability cycle seems to be more about setting out the goals and measuring the results as compared to the analogy of painting a picture used for BC. The planning documents contain much more financial data, included summarized Financial Statement for each entity within the Ministry’s portfolio. The planning documents are not quite as easy to read as they contain more data and less narrative, but they seem to reflect just the strategies, goals and performance measures. For a busy constituent or Minister, the ‘to the point’ presentation helps to guide the reader quickly through the document. The Ministry of Advanced Education seems to put considerable thought into the targets used throughout their performance measures as they often had footnotes explaining a small increase or decrease in the target. The Annual Report provides information about accomplishments and highlights from the Ministry, but like the BC annual report, it is difficult to directly tie the results to specific objectives or priorities. I liked the clear scale used by Alberta which simply denotes if a target was exceeded, met or not met. For me, this scale adds validity to the results as it doesn’t appear that the government is trying to convince the reader of something. While the presentation within the documents is not as easy to read or ‘glossy’ as the BC reports, the substance provides better information for anyone wishing to learn about the Ministry’s activities. Overall, Alberta Ministry of Advanced Education does a good job at following the Accountability Cycle from beginning through to the end.
In conclusion, I was surprised at the amount of information collected and published by both Governments and Ministries. They both appear to have a good understanding of the accountability cycle and the importance of setting goals and measuring results. As government priorities change regularly I was surprised that there were less changes between the planning phase and the reporting phase. I’m not sure if this is a reflecting of the Ministry I chose, the processes within the provinces or perhaps just a reflection of the stability of these governments within the timeframe reviewed. Additional review of other Ministries and time periods may show different results. For citizens of British Columbia or Alberta I believe that they have quite a lot of information at their fingertips (if they choose to find and read it) that explains and justifies many of the decisions and program choices of the Ministry. While there is room for improvement in both provinces, I believe that they are on the right track.

Appendix 1

Scale used for grading the comparison of the Accountability Cycle.
5 - Highest
0 - Lowest

Overall results from comparison

Appendix 2
Results:

AB – the AB website had all information that was required. It was easy to find and laid out in such a way that all the documents relating to a particular period were in the same spot. The ministry had budget information and annual reports from 2005-2006 onward. The government as a whole had budget information published since 1997-98.
BC – The BC government website had all information that was required, however it was a bit difficult to navigate to the section containing the prior year reports. Once in this section, the reports were available for the Ministry from 2001-02 and for the government as a whole since 1995-96.

Summary of Reports Used and Available
Alberta British Columbia
2012-2015 Strategic Plan 2012/13-2014-15 Strategic Plan
Budget 2012 “Investing in People” Budget 2012
Ministry Business Plans – 2012-2015 Ministry of Advanced Education and Technology (AET) Ministry of Advanced Education (AVED)2012/13-2014/15 Service Plan
Ministry of Enterprise and Advanced Education (EAE) Annual Report 2012-2013 Ministry of Advanced Education , Innovation and Technology (AEIT) 2012/13 Annual Service Plan
AET/ EAE Facts and Figures 2012 Accountability Framework Standards Manual 2013/14

AB – The mission statement for the Alberta AET Ministry is “to lead the development of a knowledge driven future through a dynamic and integrated advanced learning and innovation system.” This mission clearly aligns with the AB government’s strategic goal to “Invest in students and research” (in 2012/13). In the Alberta budget they made several announcements and commitments to continue to support learning and research programs and people.
In Alberta, each Ministry has a very clear and concise business plan. The plan, including estimated financial results for AET was only 4 pages. This Ministry in 2012/2013 had 3 high level goals each with several priority initiatives listed below. The goals and the initiatives were written in such a way so as to provide guidance to the people who are implementing these plans. They are not prescriptive and provide general guidance as to where individual institutes should focus their own resources. See example below:
Goal One: A globally recognized, quality advanced learning system that meets the needs of Alberta.
Priority Initiative 1.4: Develop stronger community linkages through regional partnerships and collaboration among post-secondary and community partners.

A typical best practice for goal writing is to ensure that goals are “SMART”. Specific, Measurable, Achievable, Realistic and Timely. (Top Achievement, 2014) Without insight into the details of these programs it is difficult to judge whether this goal and specific initiative is Achievable or Realistic. However, the initiative seems to be Specific as it tell how the linkages will be created. Where this goal falls short is in being Measurable and Timely. Nowhere in the document does it say how the success or failure of these initiatives will be measured. Additionally, the Alberta document doesn’t say when the initiative will be complete. Nearly all of the initiatives in the Ministry’s planning document fall short in this way.
The Alberta planning documents do contain performance measures which clearly identify items of importance to the Ministry. These measures contain prior year actuals, current year forecasts and 3 years’ worth of targets. In most cases the AET has included detailed values for the goals which shows that some effort and consideration has been put in to the preparation of the plan.

BC – The Ministry of AVED in BC specifically refers to the government’s strategic plan and priorities, specifically “Jobs and the Economy”, and “Families First”. The business plan for BC has quite a lot of background information which explains why the Ministry has established the specific goals and objectives. While this helps a reader unfamiliar with the Ministry understand the objectives, there is still the same issue as described above with there being no measurability for the detailed strategies. An example from the BC plan would be as follows:
Objective 1.1: British Columbians are able to fulfill their full potential through access to high quality education and skills and trades training that stimulate innovation and meet future economic needs
Strategies: sharing labour market data, analysis and forecasts with post-secondary education providers.
This amount of detail describes what the Ministry wants to do, but once again provides no way to measure and report on what specifically needs to be done to actually meet the goal during the year.
The BC planning documents also include performance measures with similar 5 year actuals/target. However, not all of the measure have complete information and in many cases, the estimates seem to be rounded and do not reflect any changes within the 3 year planning cycle timeline. This has the appearance that the targets are the minimum required and nothing additional is worth considering or that they haven’t put the effort into to projecting attainable targets.

Overall, both provinces have good planning documents available for the public. The individual Advanced Education Ministry’s planning reports are aligned with the Government’s strategic plan. In this comparison, Alberta gets a higher score setting more realistic targets in their plan. However, the BC report did provide better background information and rationale for the goals.

AB – The AET Annual report provides a clear link from the planning documents and budget to the actual results for the year. Even though the Ministry portfolio changed slightly over the timeframe, the results, particularly the performance measures, still matched the stated goals.

The Ministry has provided a summary of what Ministry’s key responsibilities are and some of the key funding projects that were announced or complete during the fiscal year 2012-2013, however it was difficult to determine whether these items related back to a specific objective or priority in the business plan. Because the priorities in the plan did not contain specific ways to track their completion, it is not surprising that they were not specifically discussed in the report of results.
The Ministry has very detailed and clear reporting of the results of the performance measures that were identified in the planning documents. The Alberta model has a very clear scale for determining the success of each measure. A goal is either exceeded, met or not met. Because of this the processes seems very transparent and reports the results in a way that is clear to all readers. In some cases, the Ministry was able to provide a more detailed breakdown of the results was provided than was initially planned.

BC – The AEIT Annual Service Plan Report also clearly provides a link from the planning documents to the reporting of the actual results. As was the case in Alberta, changes in the Ministry portfolio did not add any confusion to the report.
In BC, the Ministry provided a Highlights section to their report which provided a summary of some of the accomplishments of the Ministry. Some of these highlights are the result of specific items in the planning document however the way in which it is presented makes it difficult to directly tie many of the items to a specific strategy.
One significant difference with the BC accountability reporting model is the scale used to measure success. In this scale 110% or more of target is exceeded; 100-109% is achieved; 90-99% is substantially achieved; and less than 90% is not achieved. I feel that the language used in these ranges artificially creates an environment where more performance measures can be reported as having been met, when in reality they might have been close, but did not meet the target. When this broad and forgiving scale is combined with the limited detail in the performance measure targets, it makes the accountability measurement process seem to be a political exercise and not truly designed to improve Ministerial performance. An example of this is in the reporting of Measure 2.1 “Students satisfied with their transfer experience”. The goal for this measure was to be greater than or equal to 90%. In 2012-2013 BC achieved 83%. Based on the scale, this reports as “substantially achieved”. AB - the Ministry of Advanced Education in Alberta has 9 performance measures that they report on each year. These measures can be grouped into a couple of categories:
a) Student satisfaction surveys,
a. Measure 1.a – Satisfaction of recent post-secondary graduates with the overall quality of their educational experience
b. Measure 1.b – Satisfaction of recent apprenticeship graduates with
• On-the-job training
• Technical training
c. Measure 3.c – Percentage or recent graduates who agree that the program they graduated from was worth the financial cost
b) Counts of spaces or students
a. Measure 1.c – International Visa Students registered at Alberta post-secondary institutions
b. Measure 2.b – Percentage of graduate students studying in priority areas
c. Measure 3.a – Percentage of Albertans age 18-34 participating in post-secondary education
d. Measure 3.b – Percentage of Albertans age 25-64 who have completed post-secondary education
c) Innovation and Research (Economic impacts)
a. Measure 2.a – total sponsored research revenue attracted by Alberta’s comprehensive academic research institutions
b. Measure 2.c – Percentage of Canadian venture capital invested in Alberta
Most of the measures in the Alberta plan, are controlled by government policy or by the Canadian or provincial economies. The satisfaction surveys are a good measure of how well the schools are functioning however it is necessary for this to be a long term measure. Changes in a particular Institution in one year may only be reflected in the survey 4 or more years down the road.
BC- There are 9 performance measures used by the Ministry of Advanced Education in BC. These measures can be grouped into a couple of categories:
d) Student satisfaction surveys,
a. Measure 2 – Graduates reporting their knowledge and skills are useful in their employment
b. Measure 3 – Students satisfied with their education
c. Measure 5 – Student satisfaction with their transfer experience
e) Counts of spaces or students
a. Measure 4 – International Students studying in British Columbia
b. Measure 6 – 3 year transition rate from high school to post-secondary
c. Measure 7 – Credentials awarded to Aboriginal Students
d. Measure 8 – total student spaces in public post-secondary
f) Job creation or Employment success
a. Measure 1 – Unemployment Rates for students with various post-secondary credentials
b. Measure 9 – BC student loan repayment as % of income
These measures reflect items that are within the control of the Ministry, but not necessarily the control of the individual institutions that report to the Ministry. Additionally, a key problem with many of these measures is that they are difficult to change in the short term. This can be problematic for the government as the election cycles can influence decision which may have short term impact on one or more of the measures, but may not be sustainable or even desirable in the long term.
While the Ministry has created performance measures, they do not reflect or encourage any operational changes within the various institutions. References
Alberta Government. Alberta Innovation and Advanced Education. Retrieved March 2014, from http://www.iae.alberta.ca
Alberta Ministry of Enterprise and Advanced Education. (2012) Annual Report 2012-2013. Retrieved from http://www.iae.alberta.ca/media/377957/eae-annual-report-2012-13.pdf
Alberta Ministry of Advanced Education and Technology (2012). Business Plan 2012-15. Retrieved from http://www.iae.alberta.ca/media/309532/aet2012-15.pdf
Alberta Ministry of Advanced Education and Technology (2011). Business Plan 2011-14. Retrieved from http://www.iae.alberta.ca/media/285437/aet2011-14.pdf
Alberta Ministry of Education and Technology (2011). 2011 Facts and Figures pamphlet. Background and Methodology Information. Retrieved from http://www.iae.alberta.ca/media/301378/aet-facts-and-figures-background-2011.pdf
Alberta Budget 2012: Investing in People - Home Page. Alberta Budget 2012 - Highlights. Retrieved March 2014, from http://budget2012.alberta.ca/highlights/index.html
British Columbia Ministry of Advanced Education, Innovation and Technology (2013). 2012/13 Annual Service Plan Report. Retrieved from http://www.bcbudget.gov.bc.ca/Annual_Reports/2012_2013/pdf/ministry/aved.pdf
British Columbia Ministry of Advanced Education (2012). 2012/13- 2014/15 Service Plan. Retrieved from http://www.bcbudget.gov.bc.ca/2012/sp/pdf/ministry/aved.pdf
British Columbia Ministry of Advanced Education (2014). 2013/14 Accountability Framework Standards Manual 2013/2014. Retrieved from http://www.aved.gov.bc.ca/framework/docs/standards_manual.pdf
British Columbia. Province of British Columbia Strategic Plan 2012/13 – 2014/15. Retrieved from http://www.bcbudget.gov.bc.ca/2012/stplan/2012_Strategic_Plan.pdf
"Ministry of Advanced Education." Government of BC. Web. Mar 2014. .
Services for Government - Government. British Columbia Budgets. Retrieved March 2014, from http://www.bcbudget.gov.bc.ca/default.htm
Top Achievement — Goal Setting and Self Improvement Community. Creating S.M.A.R.T. Goals — Top Achievement. Retrieved March 28, 2014, from http://topachievement.com/smart.html