866-448-5871
Do you assess your law firms’ performance on a regular basis? If so, are you certain you’re using the right evaluation metrics? And if you’re not evaluating vendors regularly, is it because you don’t know where to start?
Understanding how well each of the law firms on your panel meets your expectations and how each measures up against the rest in key performance areas is an important management insight that can help you control costs, improve legal outcomes and increase your legal department’s value to the operation.
This paper presents best practices for developing and implementing a data-driven vendor management process that delivers actionable insights into outside counsel performance. Based on the experience gained while helping numerous LexisNexis® CounselLink® customers implement similar programs in the past, the recommendations outlined here provide a robust framework for action modeled on a proven methodology.
A Lot to Gain
A properly structured vendor management process provides many benefits. Among them, it helps corporate legal departments:
- Objectively and consistently measure law firm performance
- Integrate vendor management with expectations set forth in billing guidelines
- Strengthen relationships with law firms
- Select the best mix of law firms
- Deliver and demonstrate greater value to the organization
Whether you refer to it as a scorecard, a vendor dashboard, a vendor assessment, or other related term, the fact is, a comparative measurement tool of some sort is required to evaluate and rate outside counsel equitably and effectively against a consistent set of metrics. When it’s wielded in support of a legal department’s strategic objectives and its expectations regarding outside counsel performance, that tool becomes a powerful instrument for achieving operational improvements that can raise a legal department’s value to the organization.
Characteristics of a Successful Vendor Management Program
Although the specifics of vendor management programs vary widely from one organization to the next, there are certain common characteristics that distinguish the ones that work the best. In general, the most successful programs:
- Secure the buy-in of in-house lawyers throughout the development process
- Manage vendor management as an ongoing process, periodically reviewing data with operations management and lawyers
- Reassess metrics periodically, adding or removing metrics as needed
- Recognize that a small, focused number of metrics (maximum of 12) is more effective than pages of data
- Share expectations and scores with outside counsel to improve relationships and performance
Getting Started
Developing and implementing a vendor management system is a multi-step process. The first step is to choose the metrics that will feed into the evaluation scorecard, and ultimately, affect the law firm selection process. Often, the first metric considered is price-related, such as partner billing rates paid to each firm in the company’s panel. That’s a good start, but managing price is merely one objective of legal operations. A more comprehensive set of metrics is required – one that compares firms across all of the factors that are important when it comes to managing vendors. The challenge is to select metrics that will allow you to understand which firms meet expectations in terms of relevant variables such as price, matter cycle times, outcomes, adherence to billing guidelines, etc.
To arrive at a comprehensive, yet concise, set of metrics, it can be useful to conduct discussions with in-house counsel. Asking what sets a trusted, “go-to” law firm apart from other firms often produces information about expectations that can be translated into metrics. It’s important not to frame the discussion as asking for recommendations for what to measure, but rather, as simply asking the inhouse counsel to articulate their expectations regarding the firms that work for them. Once you’ve established expectations, brainstorm all the different ways that you might measure against those expectations. There are always going to be multiple ways that you can measure something, so your ultimate goal is to get to the one metric that’s the best proxy for what you’re trying to measure.
Subjective Metrics
Sometimes the metric you decide to use will be subjective. For example, if being good at assessing risk is one of your expectations of your law firms, you may decide to measure that by asking your in-house counsel to evaluate each firm on how well the firm does when it comes to risk assessment. There’s nothing wrong with using that subjective evaluation in this process, but you do need a high degree of confidence that all your lawyers will be able to assess vendors consistently.
Figure 1 above illustrates how the transition from in-house counsel expectations to performance metrics could unfold. In this example, the in-house lawyers have said that accurate case assessment is a critical expectation. We’ve listed three things related to case assessment that could be measured. One is “budget accuracy”, because if the law firm provided the budget based on their case assessment it should be an indication of how well the firm anticipated activities required to resolve the case. Another is “reserve adjustments” tied to their assessment of the case. A third metric is a subjective evaluation of how well the firm does at case assessment.
As you can see in the third column, it’s been agreed that of the three options, a subjective evaluation would be the best metric for determining a firm’s case assessment performance. To ensure that individual evaluations are based on the same criteria, we’ve spelled out specific choices in the fourth column, rather than using terms such as poor, average, good, etc., which can mean different things to
different lawyers.
Objective Metrics
Objective metrics are used more frequently than subjective metrics for measuring law firm performance against expectations. And once again, it’s important to first develop a list of possible metrics and then choose the best.
Figure 2 lists a set of expectations and a corresponding set of possible objective metrics. For example, most legal departments want to know that they are paying a fair price for their legal work. If your firms are primarily billing you by the hour, you can use many metrics related to rates. You could use the weighted average bill rate for the work the firm does for you. You could look at just the rates for partners or associates. You could measure how a given firm’s rate varies or compares to the median rate you pay all of your firms. Or, you might choose to measure how much your firms increase their rates each year. All of these are valid metrics, but unless the expectation that your firms charge fair rates carries a lot more weight than your other expectations or objectives, you should pick just one.
Firm staffing efficiency is another expectation that lends itself to measurement via objective metrics. And again, there are multiple ways to measure and compare staffing by firms. One is the weighted average matter hourly rate, which you could use as an indication of staffing. Alternatively, you could look at hours billed to you by different categories of timekeepers. Drilling down further, you could even look at specific tasks performed by particular timekeepers when setting up the metrics for scoring staffing efficiency.
The third expectation shown in Figure 2 relates to how well firms manage matter budgets. Possible metrics here include average matter budget variance, the percentage of matters that result in material budget variances and a count of the budget revisions law firms submit over a specified length of time.
The point is that although there may be several ways to measure performance against an expectation, there often isn’t a perfect metric. Choose the one metric that gets you closest to the insight you’re looking for.
Next Steps
Once you’ve established the metrics, you’ll need to make other decisions related to each type of metric. For the metrics that rely on data, you will first have to determine the source of the data to use and the calculation required to arrive at the metric. Second, you’ll need to think through the sorts of data issues or anomalies that you need to filter out or that may require special handling. For example, operations that work with firms outside of the United States will have to either convert billing data to U.S. dollars before calculating or report their performance separately. An additional consideration might be to choose to look at a relative metric, such as how much firms have increased their rates year over year rather than an absolute one. The examples shown in Figure 3 represent just a few of the things you should consider as you’re preparing to measure your legal vendors’ performance. Individual legal departments will likely have to think through additional considerations that apply to their specific situations.
Thoughts on Data Needs
- It would be difficult to build a data-driven assessment program without an Enterprise Legal Management (ELM) system
- Don’t get hung up on what data fields you are capturing during metrics brainstorming
- Consider what fields you want to capture for vendor management purposes during ELM implementation, if possible
- It’s never too late to set up new fields and capture additional data going forward
- Subjective assessments can be made quantifiable
Settling the Score
Designing an effective scorecard requires effort, but it’s well worth it in the long run. What goes into a good scorecard? For starters, it presents output that uses a common measurement scheme. For example, let’s say you’ve decided to use matter cycle time as a metric on your scorecard. You next have to determine criteria for scoring the range of matter results, from very good to very poor, and use that scoring range for each metric. For example, Figure 4 shows a possible scoring range with associated criteria. For the first metric, cycle time, we’ve decided that matters that require more than eight months to resolve will be scored 0, ranging all the way up to matters that receive a score of 10 if they are resolved in less than three months.
In Figure 4, the next metric is the average outside counsel fee paid per matter. Here you can see that we are using the same measurement scheme, with the best score awarded to firms that resolve matters for less than $15,000, with incrementally lower scores going to firms whose fees range higher, as indicated. You need to go through this process for each metric on your scorecard.
Getting a Historic Perspective
Performing a historic data analysis is the best way to help you establish the individual scoring thresholds. The histogram shown in Figure 5 illustrates this point. Looking at the bar on the far left you can see that it indicates that twelve matters were resolved in less than one month, so it’s a small population. Similarly, each of the other bars indicates how many matters were resolved within each of the labeled time periods. Looking at the data this way gives you a rationale for establishing the breaking points that you’ll use as part of your scoring. You could go through this sort of analysis for each metric, pulling the historic data that’s available.
It’s important to keep in mind that you may not want to use historic data for some of your metrics,
however. For instance, if you were going to include a metric for budget accuracy, you may not care at
all what your historic budget accuracy has been from each of your law firms. What matters is that the
bills from the firms that give you the best value should be within a small band of their budget – say, 10
to 20 percent. So that might be your cutoff for giving somebody a score of 10 or 8.
Presenting Data Visually
There are many different ways to present the final output of a scorecard visually. In the example
shown in Figure 6, the organization has selected nine expectations for its scorecard. The scorecard
shows the individual scores for three law firms, as well as their average weighted scores and relative
rank based on the weighted scores. Notice how the use of color-coding helps to highlight key score
categories visually.
Ultimately, you need to think of how you want the scoring to work in terms of what behavior you’re
trying to drive and what you want to achieve by pulling this data together. At the end of the day, you
want a straightforward way of seeing which firms are meeting which of your expectations.
Weighty Metrics
If some of your expectations are more critical than others, you may want to factor a higher weight into the scores at the end to account for the expectations’ greater relative importance, rather than score them all equally.
The Scorecard as a Management Tool
Viewing a vendor scorecard as a purely administrative or operational exercise won’t net you the types of outcomes that characterize successful vendor management programs. But when looked at as a management tool that can help your legal department build better relationships with your law firms and improve both the firms’ and your own department’s performance, the scorecard comes into its own.
It’s a tool that will allow you to determine which firms need improvement and to also document the qualities that make other firms great partners. And as part of a process that includes periodically reviewing firm performance with your department leaders, general counsel and others, it can guide discussions about how to get firms to improve – and what steps you may take if they don’t
Compare, Share and Improve
Once they compile their data and produce a scorecard, most organizations use the information internally at first to make sure they understand it and have confidence in what it’s telling them. But ultimately, a 360-degree approach to sharing output is the best way to develop trust with your outside counsel and to elicit improvement from them. Showing a firm how it stacks up against the other firms that work for you is a powerful and meaningful information-sharing approach that both your department and your outside counsel can use to improve performance.