Examiners love numbers. We like to work with them, write about them and interpret them. We use numbers to describe pools of collateral and/or the results of our analysis of those pools. Some of our favorite numbers are averages, variances and other statistical metrics.
Often we gain great comfort in the results of testing as well as make bold proclamations about those results due to an average result or “net variance.” Unfortunately, our reliance on these comforting metrics is often misguided and worse, can lead to misinformation in our reports.
In collateral examinations or due diligence reviews the potential for over reliance on broad metrics such as averages, means or net variances is plentiful. Fortunately with a little common sense about when to employ broad metrics and when to ignore them; as well as a few statistical tools the potential for misinterpretation can be mitigated.
First Defense – Common Sense!
The most important thing any examiner can do, (especially new examiners that are just starting out) is to understand why an audit procedure is performed and what the risks are that relate to the procedure being performed.
Many procedures in a collateral audit program have multiple objectives and may try to address multiple risks. For example the review of cancelled checks can be used to determine:
- the nature of material disbursements
- the quality of internal controls surrounding the disbursement process
- the time it takes to pay a trade invoice
- the time it takes for a check to clear
Each of the items mentioned above can be measured during the review of canceled checks and the underlying documentation. Each item also has unique risks. Knowing the risk and the consequences of certain results determines whether a broad metric such as the “average” or the “net variance” should be used.
To further illustrate, lets take a look at two of the items above; the nature of disbursements and the time it takes a check to clear.
Nature of Disbursements
By reviewing the canceled checks, it may be possible to determine the number and amount of payments made to trade and non-trade vendors. If the client being audited is in the service business, the amount of non-trade vendors would in all likelihood be relatively high. If the client were a manufacturer you would expect to see a higher concentration of trade vendor payments. In this type of analysis, broad statistic such as an average or proportion can be very informative (“for the period reviewed the examiner noted that 65% of all disbursements were to non-trade vendors”).
However, if during the review payments to owners, relatives of owners or entities controlled by owners were noted, the use of broad statistics may not be appropriate. If such payments were prohibited per the loan agreement, the comment “the examiner noted that only 2.2% of total disbursements for the period reviewed were made to owners” has little meaning. In this scenario there is no “only” when it comes to disbursements to owners. In this application, the broad statistic should be ignored and the specific items should be discussed in the report.
Check Clearance
By reviewing the canceled checks and determining the amount of time it takes for checks to clear, the examiner can often determine if the company being reviewed is “holding” checks. Held checks pose a problem as they cause the AP to be understated. In addition, a particularly unsavory client can also use held checks to overstate its cash.
Often examiners will use averages for this type of audit procedure to determine if there is the possibility of a risk issue. Often the risk issues are overlooked due to an over reliance on broad statistics
For example, assume an examiner selects 30 payments for review. 25 of the checks cleared within a reasonable time of 6 days. The remaining 5 checks took 20 days to clear. The average days to clear would be 8.3 days. A general rule of thumb is that it should take 7-10 days for a check to clear. Given the average of 8.3 days, an examiner may make the incorrect assumption that the risk of held checks appears low This interpretation ignores the fact that approximately 17% (5/30) of the checks sampled were most likely held for some time. By identifying the percentage of checks that have an unacceptable clearance lag, the risk of held checks is more evident and more likely to be identified.
Some Helpful Tools
Two tools that are fairly simple to implement and use are the “weighted average” and “stratification.” When using the weighted average, items tested that have greater value are given greater importance. With stratification, the results of audit procedures are stratified so that results can be better interpreted.
The examples below illustrate how these tools can help better interpret audit results.
Second Defense – More (Better) Statistics
Weighted Average
When considering the risk of held checks or a particular check being held, one can easily see that checks written for greater amounts have a higher probability of being held. The greater the check the higher the chance that the company would not have enough funds to cover it and the higher the probability that the check would be held until it could be “covered.” This type of scenario is ideal for using a weighted average, as the weighted average places greater importance on items that have a greater value.
Lets assume that the 25 checks with the clearance time of 6 days had a value of $1,000.00 each. Lets also assume that the 5 checks with a clearance time of 20 days had a value of $10,000.00 each. The weighted average time it takes for a check to clear would be 15.3 days. Given the rule of thumb as a reasonable clearance time is 7-10 days, utilizing the weighted average could have a profound effect on the interpretation of the results.
Now lets assume the opposite, lets say the 25 checks with the 6 day clearance were for $10,000.00 each and the 5 checks with the 20 day clearance was for $1,000.00. Here the weighted average would be 6.27 days. Given the results here, its safe to say that there is little likelihood that the company is holding checks. Why would they hold checks for $1,000.00 and not hold checks for $10,000.00?
Stratifying Results
We have already touched on the benefit of stratification when we discussed the percentage of disbursements that cleared in 20 days; however, a better example might be to look at inventory and inventory test counts. When discussing test count results, invariably our collective lending hats are hung on the “net variance.” Although this may be appropriate as the overall risk may be negated by the overages and shortfalls offsetting each other an underlying control risk remains with the client and the underlying collateral when there are large variances in both directions.
This control risk has at least the potential to translate to a financial risk for a lender and should be understood to the extent possible. By stratifying the results of test counts, we can shine a bright light on the net variance and see if it justifies the comfort it gives us when it is low or within acceptable limits.
As an example, lets assume that 20 items are tested. 10 items have no variance, 5 items have a variance of –10% each (test counts under the inventory perpetual or listing) and 5 items have a variance of +10% (test counts above the perpetual). Obviously the overall net variance would be 0%. Normally a 0% variance would be cause for great comfort in the perpetual that is being reported to the lender. Is this comfort justified when there are material variances that offset each other? The answer very well could be no.
To further illustrate, lets assume that the client is a manufacturer of coats. Lets further assume that the coats are segregated on the perpetual by size. The common medium sizes are the items that have the negative variances (counts below the perpetual), the less common XXL and XXS coats are the coats that have the zero and positive variances (counts above the perpetual). If the results were stratified by coat size and/or variance results several significant risks would become apparent:
- inventory that remains in the warehouse may have a greater proportion of the undesirable product then the perpetual would indicate.
- High volume, easily sold items have the highest negative variances, while slower moving items have the lower variances
- inventory controls are poor as the “absolute” variance is high, or alternatively, the span of the variance is large
Statistics and metrics are wonderful tools. They help us understand the world at large as well as our particular piece of it. As lenders they help us understand our portfolios, as well as the individual clients that make up those portfolios, But used incorrectly they can easily mislead us or lead us to a false sense of comfort when concern is warranted.
I hope you found this post informative and useful.
For in depth analysis of collateral please visit ECG Collateral Services at www.ecgcollateralservices.com.
