Auditor wonders about impacts of Heritage decisions

The Office of the Auditor General yesterday released a report that included a review of the activities of the Department of Canadian Heritage. Most of the emphasis was placed upon governance and accountability issues related to the Canadian Television Fund which Heritage administers.

Perhaps of particular interest (but no surprise) to the magazine industry, the report referred to the fact that Heritage is managing programs like the Publications Assistance Program (PAP) without knowing, or having a way of estimating, what the impact of its decisions will be (boldfaced emphasis added):

5.112 Long-term impacts. The Department does not have information about the long-term impacts of its programs (Exhibit 5.5). Information of this type is generally produced as part of a Department's program evaluations.

5.113 Canadian Heritage evaluated the Book Publishing Industry Development Program in 2004 and presented the results in its 2003–04 Departmental Performance Report. In 2005, it evaluated its Publications Assistance Program. It is currently completing an evaluation of the Canadian Television Fund program and is preparing to evaluate the Feature Film Policy.

5.114 Our analysis of the two completed evaluation reports showed that the Department found it very difficult to measure the programs' performance because it did not have the necessary data available at the time of the evaluations. The evaluation of the Book Publishing Industry Development Program recommended that the Department needed to improve its ability to measure progress on the program's cultural objectives. The evaluation of the Publications Assistance Program revealed a lack of clarity and consistency among various stated program objectives, and a lack of valid and reliable performance data at the time the Department conducted the evaluation.

5.115 We recognize it is difficult to measure the long-term effects of cultural industry support programs and to know to what extent any observed progress is attributable to departmental programs. However, this difficulty does not diminish the importance of continuing research and evaluation efforts. Program evaluations should allow the Department to obtain a clearer picture of its performance and the results it has achieved, and should allow the Department to determine the best way to improve its programs.

The recent shocks to the industry concerning PAP were thought to be carried out without much of an idea about what impact they would have on individual titles, certain types and sizes of magazines and magazines in certain sectors. Now, it seems, the Auditor General confirms this and suggests something be done about it. But would more robust evaluations of the "cultural objectives" call into question the whole rationale for such things as a postal subsidy and the Canada Magazine Fund?

The report on Support to Cultural Industries was carried out under the supervision of Assistant Auditor General Richard Flageole.