Evaluation of the National Fine Recovery Program - Final Evaluation Report - November 2018

Evaluation & Research

Corporate Planning & External Relations Division

Table of Contents

• Summary:
• Background:
• Evaluation Objective:
• Methodology:
• The National Fine Recovery Program and Fine Recovery Process
• Findings:
  • 1) Performance: Effectiveness and cost of outsourcing
  • 2) Explanations for the NFRP performance
• Recommendations:
• Conclusion:
• Management Response:
• Appendix 1: Methodology

Summary:

The National Fine Recovery Program (NFRP) recovers federal fines ordered against individuals and companies. In 2016 the Public Prosecution Service of Canada (PPSC) outsourced the recovery of overdue fines to a private collection agency. This evaluation was conducted to assess the impact of that change. The evaluation found that outsourcing, and the associated changes to the NFRP, have not met performance expectations, resulting in a lower volume of fines being recovered. Several factors have contributed to this reduction, including inefficiencies in the PPSC’s internal fine recovery process, namely the dependence of the process on the manual verification and input of data and a reduction in the use of CRA set-off, a proven fine recovery method. It is recommended that the PPSC examine the feasibility of getting direct electronic access to provincial courthouse databases in order to automatically update the PPSC database, and reduce the need for NFRP staff to manually input fine data. It is also recommended that the NFRP immediately take steps to increase its use of CRA set-off. In the longer term, the PPSC should analyse the business case for moving to a hybrid model of fine recovery that includes both internal and outsourced collection.

Background:

The National Fine Recovery Program was launched in 2002. Its mandate is to enforce sentences by recovering outstanding federal fines^{Footnote 1} ordered against individuals and companies convicted under federal statutes. Initially, the NFRP was tasked to recover these fines internally through eight regional fine recovery units across Canada. The PPSC is responsible for administering the NFRP under the terms of an assignment issued by the Attorney General of Canada in September 2007.

In 2010, the PPSC conducted a strategic review of the department and identified the outsourcing of fine recovery as a potential method of increasing the efficiency and effectiveness of the NFRP. In January 2015, the PPSC received approval from the Treasury Board to outsource the collection of outstanding fines. In 2016, a private collection company (Partners in Credit) was contracted to collect outstanding fines.

The Treasury Board’s approval to outsource fine recovery included a commitment by the PPSC to evaluate the NFRP in 2018-19, in order to assess the impact of outsourcing on program and cost effectiveness. The evaluation of the NFRP began in February 2018 and was completed in the summer of 2018.

Evaluation Objective:

The objective of the NFRP evaluation was to examine the effectiveness and efficiency of the program, with particular focus on the impact that outsourcing the collection of outstanding fines has had on both the effectiveness and efficiency of the program. Before the evaluation began, ongoing monitoring of fine recovery indicated that the NFRP was collecting a lower volume of fines since it began outsourcing fine recovery. Thus, a key focus of the evaluation was to fully document how performance had changed since outsourcing and to understand why outsourcing has resulted in fewer fines recovered. In addition, the evaluation also sought to understand how the effectiveness of the program might be improved in both the short and long term.

Methodology:

The evaluation utilized multiple lines of evidence including a document review, literature review, analysis of the PPSC’s fine data, jurisdictional research, and interviews of 11 key informants. As part of the evaluation, the Evaluation and Research unit worked in collaboration with the Internal Audit Division (IAD). The IAD conducted a review of the NFRP’s data processes, examining how data on fines is initially recorded in the iCase system. The findings of the IAD’s review have been incorporate into this report. More details on methodology can be found in Appendix 1.

The National Fine Recovery Program and Fine Recovery Process

Before September 1996, provincial authorities generally handled the collection of federal fines. Provincial courts would issue a warrant of committal in cases where a federal fine remained unpaid after it became due and no further collection action would be taken. Thus, if someone defaulted on payments, that person would most likely face incarceration. When the NFRP was implemented in 2002, it began recovering outstanding fines. These fines are recovered through various types of interventions, including an initial demand letter, telephone contacts, set-off of income tax refunds and GST/HST credits (CRA set-off), payment negotiations, seizure of assets, registration of liens on property, and income garnishment. The decision to outsource collection resulted in the majority of this collection work being handled by the collection agency. However, the recovery of fines by way of set-off against the offenders’ income tax refunds and GST/HST credits continues to be the responsibility of NFRP staff.

Currently, the fine recovery process is a multiple step process involving PPSC staff, the NFRP, regional offices, and the finance directorate, as well the collection agency. A high-level summary of the basic phases of the PPSC’s fine recovery process is provided below:

1. Inputting of information on fines levied by the court — Once a fine is levied, information on that fine is entered into the PPSC iCase system by PPSC staff in the relevant regional office^{Footnote 2}.
2. Verification and updating of fine information — NFRP staff review the fine information in iCase for completeness and accuracy, updating information as needed. NFRP staff also check the fine information against the relevant provincial database to ensure it is accurate and current.
3. Recovery of overdue fines — NFRP staff either send overdue fines to the collection agency for recovery, or to CRA for set-off. In general, fines that are less than $1000 are sent to CRA for set-off, and all other fines are sent to the collection agency. Individuals owing fines may make payments directly to the appropriate provincial courthouse, or they may enter into a payment arrangement with the collection agency, usually by providing them post-dated cheques. If the collection agency is unable to recover the fine, the file is returned to the NFRP at which point it may be sent to CRA for set-off or court action may be used to try and recover the fine.
4. Recording payment and transfer of recovered money — NFRP staff verify all payments made to the provincial courthouses and update the PPSC database accordingly. The NFRP also receives post-dated cheques from the collection agency, and transfers the cheques to the PPSC Finance Directorate, which transfers the money to the appropriate courthouse and pays the commission to the collection agency. The provincial courthouses then take their portion of the fine money and transfer the rest of the money to the Receiver General of Canada.

Findings:

1) Performance: Effectiveness and cost of outsourcing

In its 2015 Treasury Board Submission requesting approval to outsource fine recovery, the PPSC laid out two key performance measures to assess the effectiveness of outsourcing: the volume of fines collected and the cost-effectiveness of fine collection. The evaluation found that outsourcing has produced mixed results; fewer fines were recovered, but it was more cost-effective.

1 a) Outsourcing has resulted in a lower volume of fines collected. Although the volume of fines collected more than doubled in the second year of outsourcing, from $1.6 million in 2016-17 to $3.5 million in 2017-18, the volume was still well below the average annual amount of $5.9 million collected by the NFRP during the last 8 years that fines were collected internally^{Footnote 3} (see figure 1).

Figure 1: Fines recovered by year, internal collection vs outsourced collection^{Footnote 4} ,^{Footnote 5}

1 b) Outsourcing has become cost-effective. Although outsourcing was less cost-effective in 2016-17, it became more cost-effective than internal collection in 2017-18. It cost $0.22 per dollar collected through outsourcing compared to the average of $0.27 per dollar collected using the previous approach of internal collection. This improvement in cost-effectiveness is the result of the increase in the volume of fines recovered in 2017-18.

Figure 2: Cost to recover $1 of outstanding fines, internal collection vs the first two years of outsourcing (2016-18)

2) Explanations for the NFRP performance

The evaluation identified several factors throughout the fine recovery process that have contributed to performance of the NFRP. These include factors related to how fine information is initially recorded by the PPSC; the NFRP’s internal processes related to updating and verifying fine data; as well as the tools and methods used to recover outstanding fines.

2 a) Fine data in the PPSC’s case management system is often inaccurate, incomplete, or out of date, which delays fine recovery. This problem is exacerbated by the fact that the NFRP’s process of updating and verifying fine data is highly dependent on manual reviews of files on a file-by-file basis. Interviews conducted with NFRP staff identified updating and correcting fine information entered into iCase by regional PPSC staff as major workload for the program, which takes up time that staff are meant to use to perform other tasks. Problems with the accuracy of iCase data were confirmed by the work of the PPSC’s IAD, which documented and reviewed the process for entering fine information into iCase, analyzing the accuracy and completeness of that data, and timeliness of data entry. The audit review found problems with the completeness and accuracy of data in all regions. In addition, several regions were found to have significant delays in completing data entry and closing files, which causes further delays because files need to be closed before the NFRP can begin the recovery process.

This process is further hindered by the need for NFRP staff to manually update fine data. While the NFRP has electronic access to all provincial courthouse databases (except Manitoba), which the program uses to verify the PPSC’s fine data and to verify if fines have been paid, this requires NFRP staff to manually verify information on a file-by-file basis. NFRP staff report that this takes time, resulting in delays in fine information being sent out for collection, errors and duplicate files being sent to the collection agency, and out-of-date information being sent to the collection agency, which the NFRP then has to verify and resend. At the time of the interviews, the NFRP had five staff (two clerks, two paralegals and a program coordinator). NFRP staff reported that the two clerks were dedicated full-time to updating iCase data. However, one paralegal reported that a majority of her time was also involved in updating iCase data, which kept her from doing other tasks. As well, the other paralegal and the program coordinator both reported having to take time from other work to update and correct iCase data.

The challenges with the process of updating iCase fine data has resulted in the collection agency receiving fewer fines for recovery. At the time of interviews, collection agency staff reported that had they received significantly fewer fines for recovery than they had anticipated when they entered the contract. The NFRP confirmed that the collection agency has not received all overdue fines, in part because of the large backlog of files created by the beleaguered updating process^{Footnote 6}. The collection agency also reported that the delay in getting notification when a fine has been paid at a provincial courthouse has been a challenge for them. They reported that they understood the difficulties involved for the NFRP staff to get access to this information rapidly, but also reported that it would be helpful if they were notified as soon as possible when a fine has been paid.

Three provincial government fine recovery programs were interviewed for the evaluation in order to get a better sense of how the NFRP’s approach and experience with fine recovery compared to fine recovery in other jurisdictions. All three of the provincial fine recovery programs reported that their fine databases automatically updated when a payment was made at a courthouse, allowing them to ensure that they always had up-to-date information. All three stressed that this ability to have automatic updates was crucial to helping them reduce errors and improve their efficiency.

2 b) Data on the performance of fine recovery is not centralized, making it difficult to get consistent, accurate data. The NFRP lacks a centralized performance measurement system of key performance measures for program. Currently fine recovery data is housed in three locations in the PPSC: the NFRP, the iCase system and the PPSC Finance Directorate. The evaluation found multiple instances of disagreement between data in each these locations. In other instances, the NFRP was not able to provide specific information, or had to calculate data from scratch. The result was that the NFRP had difficulty providing current, accurate data on key performance measures including: the number files that had been sent to the collection agency, the number of files in iCase with incomplete or inaccurate data, and the total amount of fines recovered and by which method of collection. Much of this information should be available in iCase, however, given the difficulties with verifying and updating data in iCase described above, the accuracy of the data generated by the iCase system could not be consistently relied upon.

2 c) CRA set-off has been an effective fine recovery tool for the NFRP, but the program is now using it less than it did previously. NFRP staff reported that generally fines under $1000 are sent to CRA set-off for recovery rather than to the collection agency. This is not a strict rule, and NFRP staff use discretion to determine if CRA set-off will be used. NFRP staff also reported that in some instances a fine will be sent to CRA set-off and, if recovery is not successful, it may then be sent to the collection agency. In addition, some fines that the collection agency are not able to collect are then sent to CRA set-off. However, as figure 3 shows, in recent years the NFRP has recovered significantly less through set-off than it had previously. When NFRP staff were asked why less money was being recovered this way, they reported that deliberate decisions had been made to send fewer fines to set-off because the NFRP did not have the resources to handle the volume of phone calls they receive during tax season from individuals whose refunds are being withheld to pay back their overdue fines. One staff person reported that during tax season the NFRP could receive 40 to 50 calls a day from people asking why their refunds were in set-off.

Figure 3: Fines recovered by the NFRP using CRA set-off, 2011-12 to 2016-17

Provincial fine recovery agencies reported that the use of CRA set-off was a very effective fine recovery tool for them, and was one of their primary methods of recovering fines. Ontario uses CRA set-off in conjunction with private collection agencies, putting all overdue clients in set-off and removing them if the clients enter into and maintain payment agreements with collection agencies. Ontario believes this provides an added incentive for people to pay their fines. Saskatchewan also confirmed that using CRA set-off results in a significant increase in phone calls during tax season, so they plan for this and ensure that they have extra resources in place to deal with the call volume during that time.

2 d) The collection agency reported that the ability to use additional tools could help them recover more fines. The collection agency staff were asked about the biggest challenge they face in recovering fines for the PPSC and they reported that there are two tools they usually use to help them collect fines, which they are not able to use with the PPSC. These tools are: the ability to accept on-line payments and the ability to refer people to credit bureaus if they do not pay. They have discussed both of these items with the NFRP and understand that this is currently not an option, but suggested they would be useful tools for them if they could access them in the future. The NFRP confirmed that, under the current contract with the collection agency, the use of either of those tools is not possible. In addition, the NFRP reported that it was advised of potential privacy concerns related to allowing a collection agency to use credit bureaus.

It was beyond the scope of this evaluation to determine if having access to these tools would in fact improve the ability of the collection agency to recover fines for the PPSC. However, the Ontario provincial fine recovery agency reported that it allows collection agencies to refer individuals with outstanding fines to credit bureaus, and that this has been a successful tool for them, especially for recovering older fines. Ontario reported no concerns with respect to privacy, but noted that they do not provide any additional information on fine holders to credit bureaus than what those bureaus already have and that outstanding fines are recorded as being owed to the Ministry of Finance, rather than the Ministry of Justice or the Attorney General, so there is no indication that the fine is related to a criminal matter.

2 e) The experience of provincial fine recovery agencies offer some alternative models to fine recovery which may be valuable for the PPSC. None of the three provincial government fine recovery programs interviewed for the evaluation relied exclusively on private collection agencies to recover fines. Alberta does not use private agencies at all, having studied the issue and determined that private collection would not be as effective as internal collection in the Albertan context. Ontario and Saskatchewan use collection agencies only when they have already tried to recover the fine, or when the fine is small. Saskatchewan only sends fines under $1200 to private collection, collecting larger fines themselves; whereas Ontario collects all fines internally for the first 30 to 60 days, and then sends any outstanding fines to private collection. Both provincial programs reported that, in their experience, collection agencies can be very effective at recovering overdue fines, but not in all circumstances, which is why these agencies use a blended model of both internal and outsourced collection.

In addition, both Ontario and Saskatchewan use more than one collection agency at a time. Ontario uses four agencies and Saskatchewan uses two. Both provinces reported that using multiple agencies was an important factor in their success. They reported that having multiple collection agencies created competition between them and provided options if one agency is not preforming well. This finding is confirmed by a 2017 study of public and private collection conducted by the International Growth Centre at the London School of Economics^{Footnote 7}.

2 f) The NFRP operates with fewer resources than provincial fine recovery agencies. The NFRP currently operates with five Full-Time Equivalents (FTEs). All NFRP staff interviewed reported that they felt they did not have enough staff to handle the workload of the program. It is not possible to directly compare the staffing needs of the provincial fine recovery agencies with those of the NFRP because each jurisdiction is unique in the size, type, and volume of fines they deal with. Indeed, the evaluation found fairly significant variation in staffing levels between the three provincial agencies themselves. Alberta, which collects fines internally, has 32 FTEs. Saskatchewan has three FTEs working directly on fine recovery who are supported by a call center of 15 FTEs, which handles phone calls for the fine recovery program as well as other government programs. Ontario has eight FTEs working on fine recovery, who are supported by another unit of eight employees who work on fine recovery as well as other tasks. However, regardless of their differences in staffing levels, all three provincial agencies stressed the importance of automating as much of the fine recovery process as possible, stating that this was key to their ability to function effectively with the level of staff that they had.

Recommendations:

1. Examine the feasibility of getting direct electronic access to fine information in provincial courthouse databases in order to automatically update the PPSC database and reduce the need for the NFRP staff to manually input fine data. This process could be piloted in one or two provinces and then scaled up from there.

• In the shorter term, the PPSC should consider ways to make better use of the data in provincial courthouse databases, such as prioritizing getting accurate information to the collection agency quickly over updating the PPSC database. Currently the PPSC’s database is updated using the provincial data before fine information is sent to the collection agency, which means the information may be out-of-date by the time it is sent out. The NFRP should consider getting the information out first and then updating the PPSC database.

2. The NFRP should identify key performance measures for fine recovery and put in place a performance measurement system to ensure ongoing monitoring of those measures. The aim of this system should be to allow the NFRP to provide regular, accurate data on each of those measures. At a minimum the evaluation recommends the following measures:

• Number of fine recovery files with incomplete of inaccurate information in iCase;
• Number of those files that have been updated by the NFRP;
• Number of files sent out for recovery to the collection agency and to CRA for set-off;
• Total amount of fines recovered and by what method (i.e. collection agency, CRA set-off and other).

3. The NFRP should immediately increase the use of CRA set-off. The PPSC should look into solutions to address the potential increased workload this might create for NFRP staff during tax season (e.g. the use of automated phone or email messages providing initial information on how and why set-off is used, or hiring temporary staff during tax season).

4. Examine the implications and feasibility of providing the collection agency with more tools to increase its effectiveness (i.e. use of credit bureaus and ability to accept on-line payments).

5. Consider the business case for developing a hybrid model of collection that includes private collection agencies, internal collection, and CRA set-off. Provincial jurisdictions have successfully used blended models to recover fines. Data from the PPSC case management system on the success rates of each of these three methods of collection (internal, CRA set-off and outsourced) could help identify which method has been most effective in which circumstances. As part of this work, the PPSC should also consider using more than one collection agency at the same time—an approach that has been used effectively at the provincial level.

Conclusion:

The outsourcing of fine recovery, and the associated changes to the NFRP, have not met performance expectations. Although its performance has improved since the first year it was implemented, outsourcing is currently collecting just over half the average volume of fines as was recovered previously by the PPSC when it collected fines internally. However, outsourcing is now more cost-effective than internal collection was. Multiple factors appear to have contributed to the lower volume of fines recovered since the outsourcing of fine recovery. Significant inefficiencies exist in the PPSC’s internal process for updating and tracking information on outstanding fines, foremost of which is the NFRP’s reliance on the manual input and verification of fine data. Furthermore, in recent years the NFRP has reduced its use of CRA set-off, which has been an effective recovery tool for the program. Finally, the collection agency has stated that it could be more effective in recovering fines if it had access to additional tools, such as the use of credit bureaus. Evidence from provincial fine recovery agencies supports this assertion. Addressing these factors, especially the internal inefficiencies and the reduction in the use of CRA set-off, is likely to significantly improve the performance of the program.

In terms of the more fundamental question of whether or not the outsourcing of fine recovery is an effective option for the PPSC, evidence from other jurisdictions demonstrates that the use of private collection agencies can be an effective method of fine recovery, but not necessarily in all circumstances. Given the significant internal challenges facing the NFRP that have been identified in this evaluation, it is not possible for this evaluation to determine if or how private collection should be used by the PPSC without first addressing those internal issues. Ongoing monitoring of the performance of the NFRP (and the impact of the implementation of any program changes) will be required to determine what role private collection should play in the future.

Management Response:

The PPSC agrees with the overall findings and recommendations of the evaluation the National Fine Recovery Program. It will use these findings to review and restructure the program. It will also examine the findings provided by the internal audit review of the program to assess the extent to which those recommendations can also be incorporated into the next version of the program. This review will take place during this year, after which a more detailed action plan will be developed to move the program forward.

Appendix 1: Methodology

The evaluation utilized multiple line of evidence.

Document and Literature Review

The NFRP had not been previously evaluated, however, it had gone through variety of other reviews over the course of its existence, including audit and program reviews. The evaluation examined documents and reports associated with these reviews including: an audit review (2015), a strategic review (2011), a program review (2008) and an audit (2006). The evaluation also conducted a literature review of research examining the use private agencies to collect fines for public institutions.

Analysis of PPSC Fine Recovery Data

There are three key sources fine recovery data at the PPSC: the NFRP, the PPSC Finance Directorate and the case management system (iCase). The evaluation reviewed data from all three sources in order to identify the overall value and number of fines recovered, the cost to recover fines, as well as key fine characteristics, including: fine age, size, type of fine and what methods had been used to recover the fine (i.e. internal collection, CRA set-off or collection agency).

The Review of the Fine Recovery Data Process conducted by the PPSC’s Internal Audit Division

The evaluation worked in collaboration with the Internal Audit Division. Audit reviewed the processes used to enter fine information in the iCase system and analyzed that data for completeness, accuracy, and timeliness.

Key Informant Interviews

The interviews were conducted with three of the five NFRP staff, as well as four staff from the collection agency. Interviewees were asked to describe the fine recovery process and their role in it, as well as what, from their point of view, worked well and what improvements could be made. Staff from the collection agency were also asked how their collection work with PPSC compared to other collection work they do on behalf of other organizations, and if there were any additional tools, resources or information that would be helpful to them to recover fines for the PPSC.

Jurisdictional Review

The evaluation reviewed the fine recovery processes in three provincial jurisdictions (Alberta, Saskatchewan and Ontario) and conducted interviews with the managers of each of these programs. Interviewees were asked to describe the fine recovery process in their jurisdictions, how many FTEs are involved and what worked well and what could be improved with their fine recovery approach.