Horizontal Audit of the Agent Supervision Unit - December 2023

Internal Audit and Evaluation Division

As recommended by the Departmental Audit Committee, subject to approval by the Director of Public Prosecutions, on July 11, 2023.
Approved by the Director of Public Prosecutions on December 21, 2023.

Cette publication est également disponible en français.

© His Majesty the King in Right of Canada, 2023.

Cat. No. J79-21/2023E-PDF
ISBN: 978-0-660-68439-0

Table of Contents

1.0 Executive Summary

1.1 Objectives and Scope

The primary objective of this audit was to provide assurance that the regional Agent Supervision Units (ASU) are engaged in the efficient and effective oversight of agents retained by the Director of Public Prosecutions (DPP) of the Public Prosecution Service of Canada (PPSC), and that the guidance provided to agents supports them in fulfilling their objectives. A secondary objective was the evaluation of consistency across ASU operations and determine whether best practices can be leveraged across regions.

The scope of the audit is horizontal, including the eight (8) regions where agents are retained, as well as Headquarters, where additional oversight controls may be performed.

1.2 Audit Conclusion

The Internal Audit and Evaluation Division (IAED) assessed the ASUs to ensure efficient and effective oversight of the agents, compliance with the Agent Agreement - Terms and Conditions (Ts&Cs), and consistency across the regions. Overall, to varying degrees, ASUs are actively monitoring agent compliance largely through informal, reactive, ad hoc processes. However, there are opportunities for improvement in the following areas:

While ASU structures and job classifications appear consistent across regions, underpinning workload capacities is the lack of clear roles and responsibilities, including differing interpretations of the Ts&Cs requirements, and inefficient data collection and analysis/tracking tools. Further, anomalies in resource allocations need management review as there is no national approach to ensure ASUs are staffed for appropriate resources for agent supervision.

Overall there is a need for a more holistic approach across regions in the determination of monitoring processes and templates, which would ensure consistency while allowing some flexibility to adjust for regional circumstances, and for the identification of opportunities to combine efforts.

1.3 Summary of Recommendations

The report contains the following recommendations:

1.4 Statement of Assurance

In my professional judgment as the PPSC's Chief Audit and Evaluation Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the conclusion provided and contained in this report. The audit findings and conclusion are based on a comparison of the conditions, as they existed at the time of the audit, against pre-established and approved audit criteria that were agreed upon with the PPSC's management. The findings and conclusion are applicable only to the entity examined. The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada.

I appreciate the cooperation and assistance provided to the audit team by PPSC staff in Headquarters and regional offices.

Cathy Rodrigue
Chief Audit and Evaluation Executive

_________________

2.0 Introduction

2.1 Background

The DPP of the PPSC retains the services of private-sector lawyers (agents) to conduct prosecutions in areas where the Department does not have a regional office or in circumstances where it is impractical or otherwise not cost-effective for in-house counsel to handle cases. The PPSC's Agent Affairs Program (AAP) includes ASUs, located in eight of the regional offices, that are responsible for the day-to-day supervision of agents and supports them in their work.

In 2020-21, ASUs supervised 124 law firms that employ 584 individuals to assist with prosecutions on behalf of the federal Crown. For the same fiscal year, the PPSC's total expenditures for prosecution services were $192.5 million, of which, approximately 21%, or $45 million was attributable to the use of agents.

The 2020-21 Departmental Results Report indicates that the current state of the PPSC's AAP could negatively affect the PPSC's ability to manage its funds. The PPSC's 2020-212021-22 Corporate Risk Profile ranked the AAP as the number one risk, potentially resulting in inadequate financial stewardship of government funds.

With this in mind, the IAED conducted the Horizontal Audit of the ASU in accordance with the PPSC's 2022-23 and 2023-24 Risk-based Audit Plan, which was approved by the Director of Public Prosecutions on May 10, 2022. It is acknowledged that contingent on the implementation of a new Legal Case Management System (LCMS), efforts have been made to develop common improved business processes; however, the scope of this audit focused only on the current practices in place as the new processes remain pending.

2.2 Objectives and Scope

The primary objective of this audit was to provide assurance that the ASUs are engaged in the efficient and effective oversight of agents retained by the DPP of the PPSC, and that the guidance provided to agents supports them in fulfilling their objectives. A secondary objective was the evaluation of consistency across ASU operations and determine whether best practices can be leveraged across regions.

The scope of the audit is horizontal, including the eight (8) regions where agents are retained, as well as Headquarters, where additional oversight controls may be performed.

2.3 Methodology

The audit complied with generally accepted auditing practices and was conducted in accordance with the Treasury Board Policy on Internal Audit.

The audit work was carried out from July 2022 to February 2023. The audit methodology included the following:

3.0 Observations and Recommendations

3.1 Roles and Responsibilities

Despite ASU structures and job classifications appearing consistent across regions, the anomalies in resource allocations need further management review as there is very limited data metrics to determine workload capacity. Contributing to ASU workload concerns may in part be the lack of clear roles and responsibilities as ASU processes are inconsistent across the regions and include agent supervisory activities as well as non agent-related activities.

We expected to find an effective and appropriate ASU structure to support agent supervision. This structure would have documented roles and responsibilities, which are communicated and adhered to, and adequate personnel in place to supervise agent work.

ASU Structure, Classification, Number of Employee Positions

The audit compared the reporting structures, classification levels, and number of employees to determine if there were consistencies and reasonableness across all ASUs.

ASU reporting relationships are fairly comparable across the regions.

Classification levels are generally consistent except for the Agent Coordinator / Administrative Coordinator role which varies from AS01 to AS03. There are work descriptions for the various positions; however, the tasks identified are largely unrelated to the ASU function across all classifications. Therefore, it is unclear if these classification differences are appropriate. Several ASU supervisors did express a desire to reclassify counsel from LP-02s to LP-03s because of the supervisory implications of the role includes both employees and agents. Most ASU supervisors indicated having sufficient support staff, which varied across regions due to differing file volumes and complexities; however, most consider overall workload to be at or above capacity, with any increases in in-house work, unfilled vacancies and/or absences creating constraints in fulfilling their ASU-related responsibilities and increasing the risks of non-compliant agent activities.

The table below summarizes the agent portfolio that the ASU is responsible for, along with the number of counsel in the unit.

  Sum of total expenditures # of Files Fixed Term Agreements with Agent Firms # of Agents # of ASU Counsel staff *
Region 2020-21 2021-22 2020-21 2021-22 2021-22 2021-22 Total
Ontario $11,377,647 $12,303,654 5584 4456 32 75 6
British Columbia $8,093,459 $8,146,577 2145 1515 9 53 4
National Capital Region $6,896,848 $7,660,346 2898 2964 26 54 4
Alberta $4,220,874 $3,806,919 2270 1392 14 18 7
Atlantic $2,553,585 $2,592,634 907 750 24 60 6
Saskatchewan $1,602,321 $1,576,570 969 738 9 30 1*
Manitoba $456,397 $432,476 142 80 2 5 1*
Quebec $20,061 $22,234 174 179 4 6 1*

* the only counsel in the ASU is the Agent Supervisor

We noted significant disparities across the regions between the number of resources, file volumes, and the number of agents. Acknowledging these are simple metrics that do not reflect the file complexity, their nature (statute), and/or the number of agents and consequential "contact points" with the ASU, it is challenging to determine whether the ASUs are resourced appropriately and commensurate with risks. Further, there is no national approach used to ensure appropriate resources for agent supervision. In conjunction, there are no measurable, relevant, value-add data metrics to guide resource decisions.

From a qualitative perspective, the majority of ASUs felt they did not have the capacity to fully ensure compliance with the Ts&Cs. The intention of ASU counsel is to be dedicated solely to agent related work such as supervision and carriage of an agent file, however, some agent supervisors reported that counsel split their time between other litigation work and agent supervision work.

Roles and Responsibilities

Section 5.2 of the Ts&Cs has a very high-level statement pertaining to the responsibilities of the ASU supervisor and crown counsel. Beyond this we found only generic work descriptions for the positions within the ASU. These descriptions were found to be insufficient to ensure a clear understanding of this unique supervisory function over out-sourced legal activities. The descriptions do not:

The impact of unclear and undocumented roles and responsibilities is significant and includes unnecessary or duplicated work efforts, disengaged and frustrated employees uncertain of expectations, and procedural gaps as employees are uncertain who does what. We noted several instances of lack of clarity regarding: who monitors file closures; who delivers the Employee Protection Training; and who monitors the input of Fine Recovery information into iCase and the obtainment of the Court Order. Compounding the lack of clarity in their responsibilities, several agent supervisors perceived non-agent work as having a higher priority and consequently had to continuously balance both demands often to the detriment of the agent supervisory activities. Clarity of roles and responsibilities will also help establish boundaries and balance workload in those circumstances where ASUs perform both agent supervision work as well as non-agent related work.

Recommendation

The Deputy Directors of Public Prosecutions, in coordination with the ASUs and the Agent Affairs Division (AAD), should clarify ASU roles and responsibilities, with specific focus on the interpretation, necessity, and ability of achievement, of the Ts&Cs agent deliverables/requirements. These roles and responsibilities should be designed to eliminate duplication of efforts with other functions and ensure that required work is assigned to the appropriate function.

3.2 Agent Supervision Processes

The majority of ASUs have established and/or continue to refine their agent supervision processes. However, these processes need to be strengthened to ensure they are documented, deliberate, structured, and their outcomes measurable, ensuring the ability to demonstrate compliance with the Ts&Cs.

We expected to find documented and consistent ASU processes to ensure agents follow required steps when opening prosecution files, during the lifecycle of a file, and for the closing of a prosecution file in accordance with the Ts&Cs.

File opening / File Approval

In accordance with Section 5.3 of the Ts&Cs, the ASU must approve agent activities before the agent proceeds; this process can vary depending on whether the province is one of charge review or charge approval. Each region has developed their own informal, largely undocumented processes to facilitate either the approval or awareness of all files assigned to the agents. The majority of ASUs require the agent to submit file details; however, several Agent Supervisors have indicated the compliance and timeliness of these requests needs improvement.

Despite having these upfront processes, the majority of ASUs have indicated that due to resource restrictions, and in some cases conflicting workload requirements, they are not able to review all files assigned to agents. Further, and to a lesser degree, ASUs may not always be aware of all files that the agents are working on as the onus is largely on the agent to identify these files.

File Lifecycle Monitoring

Ongoing lifecycle monitoring of each file relies largely on the relationship with the agent, either with their assigned Counsel or with the ASU supervisor directly. While the Ts&Cs explicitly outline the requirement to ensure the ASU is advised of noteworthy issues, the when, how, and what largely remains open for interpretation and necessity. To varying degrees, ASUs have put in place undocumented processes, which include detailed file tracking spreadsheets, informal meetings with the agent, periodic agent file summaries prepared by the agent, ad hoc audits of individual agent billings, and data verification against other provincial databases. There has been limited renewal of on-site visits as a monitoring tool since COVID.

ASU monitoring regimes range from a very "hands-off" culture to a more structured verification. Similar to the file approval processes, we noted that it is challenging to conclude whether these individual monitoring processes are sufficient to mitigate the risks to an appropriate level or are over burdensome as none of the ASUs do any statistical reporting to identify trends or performance issues. We further noted that many of the ASU monitoring processes are often reactive, ad hoc and, in some cases, time-consuming to maintain. While these processes are a good start, greater structure to ensure consistency and completeness of information is needed.

We noted partially offsetting the monitoring shortcomings are such factors as the ASU team members experience and duration in the role, inherently developing and performing these processes regularly, as well as their relationship with the agents. While these factors are important, they should not replace having a defined monitoring regime commensurate with risk, that can be leveraged across all regions as appropriate.

File Closure

According to section 13.8 of the Ts&Cs, file closure should be done within 30 days of the final outcome decision. Only two of the ASUs are attempting to monitor the closure of files. Recently the AAD undertook an exercise to close outstanding files as inaccurate file status impacts their ability to appropriately estimate financial liabilities. Despite these steps we found a few "Active" files in our sample remained open when they should have been closed. In addition, there is also some confusion as to who has this responsibility – ASUs or AAD. The ability to run regular iCase reports and review closure statuses appears to be time consuming as there is no specific reporting function to show file changes. Consequently, the ability to monitor and enforce this clause is limited.

Several agent deliverables in the Ts&Cs allow for some flexibility at the ASU's discretion, such as: the collection of the Final Disposition Report (FDR); the communication of Court Imposed Fines and submission of the Court Order to the ASU within 30 days; and the completion of agent Performance Reviews. ASUs noted that, generally, the oversight on the submission of FDRs and Court Orders was limited. Similarly, the deliberateness of performance evaluations was not done consistently or comprehensively. These requirements contribute to the oversight of the file and ensure a comprehensive understanding of final outcomes. While it may be appropriate to limit the oversight/performance of these requirements as they may be addressed through other activities, at a minimum there should be a deliberate and documented decision by each ASU as to what approach they are taking and how that addresses the circumstances of their region.

Overall, the majority of ASUs indicated that files have proceeded forward in the past that were non-compliant, so they have individually been trying to improve their monitoring processes. They consider their processes to mostly mitigate the risk of non-compliant files proceeding to prosecution or continuing inappropriately, albeit the timely identification of these issues may not be in compliance with the Ts&Cs and largely remains with the agent to bring them forward. Further, the majority of ASUs still believe more monitoring at all stages of the file, including the submission of billings, needs to be done. As no ASU is actively tracking non-compliance statistics they cannot definitively ensure the processes in place are fully effective. Combined with only limited and/or threshold validation of agent billings, over 300 agents and approximately $37 million in expenditures, the impact of these monitoring weaknesses could lead to financial and, equally important, reputational risks.

Recently an ASU network was established which was seen as a valuable tool, particularly given that each region develops their own templates and processes. We noted there may be opportunities for a more holistic and efficient approach to the determination of monitoring processes and the development of templates. This approach would also ensure more consistency amongst regions, allow for some flexibility to adjust for regional circumstances, and for the identification of opportunities to combine efforts.

Tools & Documentation

Some regions have established their own agent databases; however, all admit the documentation is not complete and often resides in emails and not a central and accessible location. Current practices and ability to have a fulsome view of agent activities remains cumbersome and impedes the ability to analyse billings, ensure Ts&Cs compliance, and perform supervisory functions. These data limitations can create inefficiencies and potentially limit the ability to do more strategic trend analysis, which is critical to support resource decisions.

While it is understood that efforts are underway to replace the current LCMS, we continued to observe current data challenges in the collection and sharing of key prosecution and administrative information. There is a need to ensure the specific deliverables within established timelines identified in the Ts&Cs are traceable for effective and efficient monitoring and reporting, should the PPSC decide to enforce compliance.

Overall, the ASU function needs to continue to be strengthened with focused training on more administrative activities, such as supervision, file management, and financial management.

Recommendations

The Deputy Directors of Public Prosecutions, in coordination with the ASUs and AAD, should continue to strengthen the oversight tools/mechanisms, with the objective of sharing best practices, streamlining duplicated efforts, and creating consistent and effective agent file analysis across the regions. Where appropriate, the implementation of these tools/mechanisms should include training for the agents as well to ensure their understanding.

The Deputy Directors of Public Prosecutions, in coordination with the ASUs and AAD, should define metrics relating to the Ts&Cs and develop a reporting strategy on activities related to agent supervision, with consideration given to regional circumstances. Through the execution of these reporting strategies, there should be an ongoing assessment of ASU capacity and Ts&Cs compliance.

3.3 Financial Oversight

There continues to be a lack of clarity and inappropriate execution of the pre-approval of certain disbursements and the subsequent approval for their payment.

We expected to find that ASU supervisors and/or designated employees are verifying agent billings and disbursements, as required in the Ts&Cs.

A 2019 Review of Agent Expenditures concluded that the AAD is neither optimally located, nor best able, to assess the adequacy of the legal services received. The 2019 Review proposed five recommendations, four of which were considered high risk, and currently four remain open. Given this, the audit team did not conduct a comprehensive financial accountabilities review.

Specific to Sections 5 - 9 of the Ts&Cs, travel and disbursements require ASU approval in advance. Based on non-statistical samples, the audit found the following results:

While threshold reviews do provide some assurance as to the validity and accuracy of the expenditure, we find this insufficient to ensure a comprehensive understanding of all expenditures submitted and to ensure compliance with the FAA. Current monitoring activities are not sufficient to ensure consistent compliance and mitigate financial risks. We did not conduct an additional review of these processes as part of this audit and will continue to assess this through current follow-up activities related to outstanding action plans from previous audits.

4.0 Conclusion

The IAED assessed the ASUs to ensure efficient and effective oversight of the agents, compliance with Ts&Cs, and consistency across the regions. To varying degrees ASUs are actively monitoring agent compliance. However, there are opportunities for improvement in the following areas: clarify roles and responsibilities, optimise resource allocation according to individual regional needs, and standardise the oversight processes, including monitoring and reporting.

Recommendations have been provided for those areas where changes would benefit the ASU going forward.

5.0 Management Response and Action Plan

In early 2022, a consultant hired by the Corporate Services Branch (CSB) to assist the PPSC at looking at roles and responsibilities. That report did not achieve the expected objectives. As a result, follow-up work was required and undertaken internally under CSB leadership.

Focus group discussions took place between June and September 2022. On the basis of those group discussions, and with the benefit of a national costing of the entire AAP and all its key components provided by the Finance and Acquisitions Directorate (FAD), the intent was to hire a dedicated resource in early January 2023. That resource was to write a document consolidating options and documenting the key features and gaps in the AAP, with a view to informing a decision as to what program functions should be centralized or decentralized.

In November 2022, the decision was made to conduct this work through a team of current personnel. AAD is currently reviewing the roles and responsibilities of all groups involved with agents, including the ASUs, as part of a "Re-imagine the Agent Affairs Program" project.

In addition, a significant amount of work has been conducted with the ASUs, AAD, and FAD between 2020 and 2022 to design common and consistent business processes to supervise agents. Significantly, these cover the supervision and certification of the work performed by agents, both from a quality and financial accountability perspective, as the two are intimately linked and represent the highest level of risk.

Those new business processes have been documented. Their implementation, however, is contingent on the implementation of a new LCMS, which has been significantly delayed.

In the face of the LCMS delays, a working group (WG), led by the Chief Financial Officer and made up of representatives from FAD, AAD and ASUs, was stood up. Its aim is to improve common practices in all ASUs, through a revised process governing supervision and certification of the work performed by agents, as this continues to represent the greatest risk.

This revised process is an interim solution pending the implementation of the LCMS (which will, in turn, allow for the implementation of the more comprehensive new business processes). The revised process must be within the current level of resources and with existing systems (i.e., iCase, generic mailbox, and GCdocs).

Through a Sprint design exercise in May-July 2023, the WG designed a revised process that would be consistent across all the regions and would require an expenditure initiation pre-charge approval decision on the part of the ASUs. The WG conducted testing/validation interviews, and the finalization of the proposed process is scheduled for July 21, 2023.

This revised process should greatly increase compliance with sections 32 and 34 of the FAA.

To increase quality assurance and performance, the WG will be recommending the development of a one-pager model for key indicators and for benchmarking purposes. This will provide valuable information on the performance of agents to the relevant teams.

The draft process will be presented in the fall 2023 and, subject to Senior Management approval, should be rolled out across the regions in the following months during the current fiscal year.

Recommendation Management Response and Action Plan Office of Primary Interest Target Date
  1. The Deputy Directors of Public Prosecutions, in coordination with the ASUs and the AAD, should clarify ASU roles and responsibilities, with specific focus on the interpretation, necessity, and ability of achievement, of the Ts&Cs agent deliverables/requirements. These roles and responsibilities should be designed to eliminate duplication of efforts with other functions and ensure that required work is assigned to the appropriate function.

 

Risk: High

Management agrees with this recommendation.

AAD is currently reviewing the roles and responsibilities of all groups involved with agents, including the ASUs.

Management expects this to be completed by November 2023 so a decision could be made concerning the division of responsibilities between the various actors, including with respect to processes, practices, etc. involved in recommendations 2 and 3 below.
DDPP, with support from CSB (AAD) November 2023
  1. The Deputy Directors of Public Prosecutions, in coordination with the ASUs and AAD, should continue to strengthen the oversight tools/mechanisms, with the objective of sharing best practices, streamlining duplicated efforts, and creating consistent and effective agent file analysis across the regions. Where appropriate, the implementation of these tools/mechanisms should include training for the agents as well to ensure their understanding.

 

Risk: High

Management agrees with this recommendation.

With the benefit of clarified roles and responsibilities between ASUs, AAD and FAD (recommendation #1) and in conjunction with the work contemplated at recommendation #3, efforts will be undertaken to strengthen the oversight tools/mechanisms (sharing best practices, streamlining duplicated efforts, and creating consistent and effective agent file analysis across the regions).
DDPP, with support from CSB (AAD) and FAD September 2023 - presentation of the interim revised process governing expenditure initiation pre-charge approval decision to Senior Management

March 31, 2024 - implementation of the revised process governing expenditure initiation pre-charge approval decision (subject to Senior Management Approval of the revised process)

April 2024 – March 2025: strengthen oversight tools/mechanisms
  1. The Deputy Directors of Public Prosecutions, in coordination with the ASUs and AAD, should define metrics relating to the Ts&Cs and develop a reporting strategy on activities related to agent supervision, with consideration given to regional circumstances. Through the execution of these reporting strategies, there should be an ongoing assessment of ASU capacity and T&C compliance.

 

Risk: Medium

Management agrees with this recommendation.

Realistically, it will be possible to undertake deliberate work towards implementing this recommendation once the roles and responsibilities between ASUs, AAD and FAD have been clarified (recommendation #1) and once an interim revised process governing supervision and certification of the work performed by agents is implemented (recommendation #2). This work will be in conjunction with the efforts under recommendation #2 to strengthen oversight tools/mechanisms.
DDPP, with support from CSB (AAD) and FAD April 2024 – March 2025

Appendix A – Audit Criteria

Audit Criteria

  1. Regional Agent Supervision Units have defined structures, as well as defined roles and responsibilities, that support effective and appropriate agent supervision.
  2. Effective and efficient processes are in place for agent supervision and billing reviews to ensure compliance with relevant requirements.

Appendix B - List of Acronyms/Abbreviations

AAD Agent Affairs Division
AAP Agent Affairs Program
ASU Agent Supervision Unit
CSB Corporate Services Branch
DDPP Deputy Director of Public Prosecution
DPP Director of Public Prosecutions
FAA Financial Administration Act
FAD Finance and Acquisitions Directorate
FDR Final Disposition Report
IAED Internal Audit and Evaluation Division
LCMS Legal Case Management System
PPSC Public Prosecution Service of Canada
Ts&Cs Agent Agreement – Terms and Conditions
WG Working Group
Date modified: