Benchmarking: an incentive to improve, a distraction or a red herring?

In the question and answer session of the presentation I gave a couple of weeks ago I was asked how I thought a process of benchmarking should fit into the annual budgeting process. I gave an answer at the time but I don't think I expressed my view as well as I could have so I thought I would have a go in writing.

The question was posed to me in terms of unit costs: if we are preparing our budget for a service how should we take into account the fact that our neighbours or peers deliver the service for a lower (or higher) unit cost? On the face of it this seems like a good idea and I'm sure there must be some sound theories to support it. My answer to the group, though, was sceptical, borne from experience rather than theory.

I think that the "information" a public sector organisation gets from such benchmarking is unhelpful. I think carrying out a benchmarking exercise is "busy work"; it feels like you're doing something positive but in practice you're using up a lot of resources and getting very little in return. They say you don't fatten a pig by measuring it. Well, benchmarking is the equivalent of trying to fatten your pig by measuring someone else's.

I've been involved in numerous benchmarking activities over the years (it was a key feature of the Audit Commission's work in the 90s and 00s) and the first problem is agreeing common definitions so that the information is at all comparable. The development of accounting standards and financial reporting standards has no doubt improved the consistency of treatment by public bodies, but that does nothing for the second, and bigger, problem: that public managers will cling to favourable comparisons and find reasons to criticise unfavourable ones. (I once has a manager who was very quick to claim successes were the result of his management but failures were the result of external factors beyond his control. Sound familiar?) If you want an academic reference for this, Demeere et al (2009), writing about the use of activity-based costing in healthcare, said:

"... healthcare managers might argue about the accuracy of the models estimated costs and pro"tability rather than address how to improve the inef"cient processes, unpro"table products, and considerable excess capacity that the model has revealed."

Demeere et al actually touch on the potential negative consequence of benchmarking: complacency. If a public manager's unit costs (or whatever) are good they do not need to try harder; if they are poor, they might try hard to make a defence but they can avoid the need to improve their service.

Many aspects of public management these days are informed by or developed from private sector practice. Do private sector organisations benchmark with each other? Perhaps, but it seems unlikely that an organisation could benchmark with competitors directly so they are more likely to use competitors' prices as the benchmark. They may ask themselves, Can we sell our product for the same price as the competition and still make an adequate return? If the answer is no then they either have to find a way to make their product more cheaply, charge a higher price and compete on the basis of quality/design or cease to produce the product. If the answer is yes then that's great, but it would not stop them from seeking efficiency improvements in order to improve their gross profit margin.

How does that compare with a public service? Well, there are some services where charges are made but generally there is not a market because public bodies usually have a monopoly on the service in some geographic area. But there are some areas where there are comparable private sector providers whose prices could be used as a signal. Except, how many local authorities would compare the unit costs of their schools with fee-paying private schools, or compare their access-for-all leisure centres with members-only private gyms? "They're not offering the same things as us," would be the cry so of course our unit costs are different from theirs.

So, based on my experience, what would I recommend? In my view the answer is much easier to find than carrying out benchmarking exercises because the organisation already has it. The answer is to compare our own unit costs over time. This avoids all the problems of alternative accounting treatments, geogrpahical differences in prices, etc. and it builds on the idea that however good a service is the managers should strive to improve it. I'm sure that if all the time and energy spent carrying out benchmarking work and then defending or criticising the results was spent on improving services the benefit for the public would be significantly improved.

Reference

Demeere, N., Stouthuysen, K., & Roodhooft, F. (2009). Time-driven activity-based costing in an outpatient clinic environment: Development, relevance and managerial impact. Health Policy, 92(2-3), 296–304. doi:10.1016/j.healthpol.2009.05.003

The future of public audit

I've just read a pamphlet called The Future of Public Audit (published by Solace Foundation Imprint). There are about a dozen articles/short essays in the pamphlet and, rather like the stereotypes of auditors, they are worthy but rather dull. Or, perhaps, a better adjective would be "earnest". The articles incorporate interesting ideas and insights about things like trust, transparency, the demise of the Audit Commission and self-improvement but they don't sparkle. Perhaps part of the reason for this is the subject matter itself but should articles about audit will always be so dry?

Perhaps it is the writers rather than the subject. All of the authors of the pamphlet I read were male, white and most were, to be polite about it, substantially experienced. We could do with some different people writing about this subject (and other aspects of financial management) to get some diversity of views and to do for the subject (in a small way) what Brian Cox is doing for science: making people interested in it.

I guess I have to be part of the answer. Here I am blogging about public finance so it is incumbent on me to make what I write interesting. I certainly hope to do that with posts like the one about the film Moneyball. I also try to do it in my lectures by including video and images in presentations and telling stories from my experience in order to illuminate the subject. But, nevertheless, I can't rest on my laurels. During the 1996 European Championships, Ruud Gullit coined the phrase "sexy football" to describe teams whose play was artistic and entertaining as well as effective. I don't think we can have "sexy audit" but could we at least have some sexy articles about audit? That's a challenge I've set myself.

Is Ruud thinking about how to make public audit sexy?

Audit Commission replies to Pickles's plan.

David Walker, journalist and former communication director of the Audit Commission tweeted today (@exauditor77): "Polite language, but Audit Comm is now saying Pickles' plan will ADD expense, leave public finances unguarded http://t.co/Bpk6fVs". I'm sure it will, too, in the longer term. Let's face it, it would be a remarkable fluke if an off the cuff policy decision was actually a good decision.

Competing for public audits

What does the OFT’s announcement that the audit market should be investigated by the Competition Commission mean for the audit of public organisations? When Eric Pickles announced in August 2010 that the Audit Commission would be abolished he stated that public organisations should be free to choose their auditors. One possibility is that the Audit Commission's own audit arm (formerly known as District Audit) could be floated off in some form to compete with the existing audit firms. I believe that Pickles's department has hired some management consultants to advise on how that might happen.

The abolition of the Audit Commission is, in its way, a classic piece of Conservative market deregulation. My personal view is that, generally speaking, market regulations were imposed by governments for good reason and deregulation makes things worse for most people, whilst allowing a few to make a lot of money. There are too many cases of deregulation resulting in a bad deal for consumers (eg price rises) if not worse (Enron-type frauds and scandals). Once local authorities and NHS trusts are in a position to choose their own auditor I am sure that the larger ones—the county councils, London boroughs, metropolitan boroughs, major acute hospitals—will receive suits from the 'Big Four' as well as some of the smaller firms. The process will be a beauty parade. The codes of practice that set out what comprises the audit of a major organisations means all the firms will provide the same service so the client organisation will pick the one they fancy. In the first year or two they might also get a good discount on the fee.

What the practice of the FTSE 350 companies shows is that once an audit firm is hired they very rarely are replaced. I suspect that's because there's little incentive. All the firms do more or less the same thing and charge more or less the same fee so why would a client organisation spend the time and trouble to have a new beauty parade every five or seven years? And if a new firm were selected as a result there would then be an element of disruption as the new auditors found out all they needed to know about the client's business and accounting system and so on. It's rather like changing your bank. Even if you could get a cheaper deal from a different bank, it would have to be a very significant improvement on your current deal for it to be worth the trouble of moving all your direct debits and all the rest of it.

Whilst the larger public organisations can look forward to being enticed by the Big Four audit firms I doubt that the smaller district councils, the ones whose budgets are still £10–20million and so are substantial organisations in their own right, will. They might see much the same effect, though, from the small and medium-sized audit firms who would be happy to have a regular income from organisations that will pay their bills and never go bust.

If the Audit Commission's audit division is floated off as a stand-alone organisation (New District Audit, perhaps?)then I expect it would compete for all shapes and sizes of audit. Its unique selling point would be its specialism in public sector audits and I expect that it would win some of the business. Prior to the Audit Commission's creation, local councils could choose between being audited by the District Audit or by a 'professional auditor'. (Back then hospitals were not part of trusts and the whole of the NHS was audited by the Exchequer and Audit Department, which was renamed the National Audit Office in 1983).  When local government was reorganised in 1974 202 boroughs had moved over to the professional audit, while 119 use the District Audit (Coombs and Edwards, 2004: 82). (There were 21 others who used the antiquated system of electing local people as auditors, a practice which I don’t think the secretary of state, or anyone else, is proposing to reinstate.) So back then the private sector held about 2/3 of the local government audit market. During its tenure the Audit Commission has favoured its own auditors with about 70% of the market. Would a stand-alone ‘New District Audit’ be able to hang on to 70% of the market. I doubt it. At least, I doubt it in terms of fees. There are about a thousand principal councils and NHS bodies currently audited by the Audit Commission (and many thousands more parish councils). The New District Audit might be able to win 70% of them as clients but the professional firms will focus on the bigger, more valuable clients and I can well imagine that they might secure 70% of the fees. And once they are in they’ll be very difficult to replace.

Reference: Coombs and Edwards, (2004) The audit of municipal corporations—a quest for professional dominance. Managerial Auditing Journal, 19(1), 68-83.

Look after the pennies ...

It is an old saying that if you look after the pennies the pounds will look after themselves. Maybe, but one might say that being focused on details means that one misses the big picture. It seems to me that this is the effect of the government's transparency policy: by enabling anyone to see details of payments for £500 swamps them with masses of data but they aren't really holding anyone to account. Accountability, you see, requires explanation as well as information (just as accounting requires words as well as numbers). Knowing what £500 was spent on does not tell you whether it was spent on the right thing. As Carnegie and West put it: “a hospital that is well managed in financial terms cannot be presumed to be meeting a community’s needs for health care.”

I spotted a couple of things in yesterday's news that brought this issue to mind. First I saw that Grant Shapps was criticising the Audit Commission for spending £20,000 on the failed recruitment of a chief executive last spring. The amount doesn't seem unreasonable to me and the fact that no chief executive was recruited had more to do with Shapps's boss than anything else. Shapps was also critical about the various minor payments made by Audit Commission staff on its procurement card. Now, procurement cards are usually normal-looking credit cards that are restricted only to allow purchases of certain classes of goods and services. They are promoted to public sector organizations to make the purchase of low-value goods in a cheap and efficient way by avoiding all the red tape of writing formal orders and processing invoices etc. In my experience their use is well-controlled, often the control being better than under previous systems because the credit card company can provide so much information about when and where the card was used. The fact that someone in the Audit Commission bought something from an HMV shop suggests to me that there was a business reason for buying whatever it was. I would be very surprised if it turned out that someone bought the latest Take That CD to listen to in their car on the way home from work. That sort of thing rarely happens, and it is even less rare, I believe, when it is as easily detectable as a payment made on a procurement card that will have an itemised bill sent to the finance department. All in all, the Audit Commission will probably spend more time and money providing evidence to Grant Shapps that it was a legitimate business expense than the item cost. That's not value for money in anyone's book.

The second story I saw yesterday was about the Department for Education spending £21million on consultants. A much more significant amount of money, you'll agree,  and it may well be that it is much less than Labour spent. But the figure by itself doesn't tell us what the consultants have done so that we can judge the value for money of their services. If I can only have one explanation, I'd rather know what the £21million bought us than the £20,000 spent by the Audit Commission on its procurement cards.

Reference

Carnegie, G. D., & West, B. P. (2005). Making accounting accountable in the public sector. Critical Perspectives on Accounting, 16(7), pp. 905-928. doi:10.1016/j.cpa.2004.01.002

Choose your auditor wisely

About a week ago Eric Pickles announced his proposals for the audit regime for public bodies that would replace the current regime once the Audit Commission has been abolished. Public organizations with a turnover of at least £6.5million would be free to select their own external auditor in the same way that private companies do. As with all the local freedom announced by this government I don't doubt that there will be constraints but the proposals, so far, do not go into the details of the codes of practice or guidelines that would accompany this new freedom. Nevertheless, I have a couple of general observations to make.

Firstly, last week I read a newspaper snippet about a House of Lords report which claimed that the complacency of auditors contributed to the current financial crisis. The newspaper report mentioned that on average the FTSE100 companies change their auditors every 48 years. Given that some of the companies change of auditor would have been because Arthur Anderson collapsed after the Enron scandal one begins to wonder how often a large company changes its auditor when it does not have to. This kind of thing would be good news for the audit firms pitching for business from councils and hospitals because once they are appointed they might expect to receive fees for generations. I appreciate the government might include a requirement in the code of practice, or whatever, that means an audit firm has to be changed every 5 or 7 years but it seems to me unlikely since no such requirement is included in company legislation.

My second comment is to point out that the European Commission is heading in the opposite direction. Last year it published a consultation paper Audit Policy: Lessons from the Crisis where it said it was ‘considering the feasibility of a scenario where the audit role is one of statutory inspection wherein the appointment, remuneration and duration of the engagement would be the responsibility of a third party, perhaps a regulator, rather than the company itself.’