Best Returns

Best Value Guidance
for
Library Authorities in England

6 The role of the Audit Commission

The Audit Commission - the independent organisation that promotes the best use of public money by local authorities and other bodies - has additional powers under best value legislation.  These powers cover two distinct but complementary roles - audit and inspection. 

6.1 Audit of best value performance plans

Every BVPP published by local authorities will be independently audited.  This will normally be carried out by the same external auditor who audits the authority’s accounts each year, appointed by the Audit Commission from District Audit or from one of the private accountancy firms (eg, KPMG, PricewaterhouseCoopers, Robson Rhodes).  As well as auditing the BVPP, the external auditor will continue to audit the accounts and produce an annual management letter on the financial affairs of the authority.

The auditor will assess:

• whether the authority has prepared and published its BVPP in line with legislation and statutory guidance;
• the systems in place for collecting and recording performance information; and 
• whether the authority’s corporate and performance management arrangements for best value comply with legislation and statutory guidance (Exhibit 10).

The auditor will issue a report on the BVPP by 30 June each year, for publication by the authority.

6.2 Best value inspections

All local authority services and functions are subject to independent inspection under best value.  The Audit Commission is responsible for inspecting those services not already covered by an existing inspectorate - including public library services - and for ‘cross-cutting’ inspections. 

The purpose of best value inspection (BVI) is to:

• let the public know whether their authority is achieving best value;
• let the authority know how well it is doing;
• let Government know how well its policies are working on the ground;
• identify failing services where remedial action may be necessary; and
• identify and disseminate good practice.

BVI aims to complement the audit process.  Whereas best value audit is concerned primarily with corporate and management arrangements, BVI focuses on service delivery - particularly from the perspective of service users. 

How inspection will work

Inspections will normally follow the same five-year cycle as authorities’ programmes of best value reviews.   For example, if an authority carries out a single review of the whole of its library service, the inspection that follows is likely to cover the whole of the library service; if an authority reviews its library service as part of a BVR of cultural services, the inspection that follows is likely to cover cultural services.  In the same way that libraries may be included in both service-specific and cross-cutting reviews, inspection may also potentially include library services more than once.  Each authority is assigned to a Lead Inspector who will agree an annual programme of inspections and co-ordinate inspection teams.

There will also be some out-of-cycle inspections.  These will include ‘follow-up inspections’ to assess whether the recommendations of the original inspection have been implemented, and ‘special inspections’ where there are concerns about an authority’s performance, triggered, for example, by the external auditor.  Government can also direct the Audit Commission to carry out inspections.  London borough, unitary and metropolitan authorities can expect a minimum of 3 - 6 inspections per year; county councils, a minimum of 3 - 5. 

An inspection will normally take the BVR as its starting point.  But it will not be limited to the information generated by the BVR.  Nor will it be merely a ‘review of a review’ or a desk-based paper exercise - much of it will be carried out on site.  It will be rigorous and evidence-based, and both critical and supportive.

What inspectors will be looking for

Throughout each inspection - irrespective of the service under review - the inspection team (normally comprising two people) will be assessing:

• the quality of current service delivery;
• the likelihood of future service improvement; and
• the rigour and value of the review process, including application of the ‘4Cs’.

At the end of the inspection the inspectors will make two judgements:

• how good is the service?
• how likely is the service to improve?

To arrive at the two judgements the inspectors will structure their activities and findings around six key questions (Exhibit 11).  The two judgements on the service will be illustrated on a scoring chart (Exhibit 12). 

Focusing on what the inspectors will be looking for should help authorities in their approach to BVRs of library services.  Authorities should ensure that these questions are addressed through their BVRs, and be constantly thinking about - as the inspectors will be - the perspective of users and potential users of their services.

The process of inspection

Best value inspections will normally follow a standard process (Exhibit 13).  A typical - or ‘medium’ - inspection will normally take two inspectors around 15 days each, of which five will normally be spent carrying out ‘reality checks’ on site at the local authority.  Reality checks are a distinctive part of the inspection process.  They are activities designed to test out the actuality of service delivery on the ground and gather evidence to answer the inspection questions.  They will be tailored to each individual inspection.

Every inspection will end with a published report, available on the Audit Commission’s web site and in hard copy.  Publication of the findings of external scrutiny will be a new experience for many library services and can have a significant impact.  Early experiences suggest that very good or very critical reports attract considerable media attention and give the authority a high profile. 

The Audit Commission has refined its inspection process for the second year of best value.  The level of inspection should become more “appropriate” and “proportionate”, according to an assessment of the public importance and risk of the area under review.  This will be reflected in a mix of ‘small’, ‘medium’ and ‘large’ inspections.  The Commission has also planned to pilot ‘popular’ summaries of inspection reports in leaflet form, in an effort to provide information in a format that can be easily understood by local people. 

Both the Government and the Audit Commission have given a commitment to develop further the inspection system, primarily to streamline the process.  Subject to local authority consultation, future changes may include amending the scoring system, providing earlier guidance on reviews and a more “risk-based” approach to inspection. 

Further details on the current best value inspection process are available in the Audit Commission publication Seeing is Believing: how the Audit Commission will carry out Best Value Inspections in England.

 What inspectors will do

Inspectors will tailor the reality checks they carry out on site to each individual inspection.  But reality checks will always be focussed on two areas:

• what the service is like now from the user’s perspective; and
• whether the authority will deliver future improvements that will benefit the public.

Some authorities are surprised at how little attention inspectors pay to the review process itself.  As one of the case study authorities commented; “the inspectors were much more focused on the service than on the best value review process”.  Inspectors are primarily concerned with how far the review and use of the ‘4Cs’ have driven improvement, rather than in the actual process.  Authorities should not expect detailed recommendations on their review processes.

But inspectors will look closely at the improvement plan; it will be critical to the inspectors in reaching their second judgement (likelihood of improvement).  They will expect the plan to demonstrate clearly the improvements that will be delivered to local communities.  They may carry out case studies to track through how, for example, the findings of comparison and consultation activities have informed the plan, how it links to local priorities, and how it will be resourced.

Inspectors will spend most of their time on site meeting staff, managers, elected members, community groups, the public, internal and external partners, and other stakeholders.  Exactly who they will want to see and what they will ask them will vary according to each individual inspection. 

Inspectors will normally interview key elected members responsible for the service and for best value, from the ruling and main opposition parties; senior and middle managers, including the service head; front-line staff; service users or user representatives, such as Friends groups; and main contractors, partners and community representatives.  They may also ask to see the Leader of the Council, Chief Executive, Director of Finance and service accountant.  Inspectors will normally want to hold individual interviews with elected members and key officers, to encourage openness and ensure confidentiality.  They will normally ask authorities to set up separate focus groups of front-line staff, middle managers and service users. 

There is no fixed agenda for interviews or focus group meetings.  But inspectors will always be searching for evidence to support the two judgements and six key questions.  Authorities will also know from the inspectors’ pre-inspection briefing the main areas they will be investigating.  In broad terms, interviews with elected members and senior officers will tend to focus on corporate and service strategies, aims and objectives; focus groups will tend to focus on operational service delivery. 

Inspectors will also visit a selection of libraries, sometimes accompanied, sometimes unaccompanied.  They may visit the best and worst libraries - by the authority’s own assessment - and both mobile and static sites.  Often they will visit libraries with different social profiles and at different times of the day.  During these visits inspectors may assess:

• ease of physical access;
• external and internal signage, layout and ease of navigation;
• the quality and appearance of books and other materials;
• public access computer facilities; and
• public information and promotional displays.

Inspectors may also carry out structured observations of staff carrying out their normal day-to-day work. 

Inspectors will be open about who they are and what they are doing.  The exception is where they are directly testing various aspects of service delivery from the user perspective - by making an enquiry in person, in writing or on the ‘phone, for example - as anonymous, ‘mystery’ customers. 

During their on site week, inspectors will discuss their findings with the authority at key stages in the inspection, often on a daily basis.  They will respect confidentiality.  At the end of the inspection they will present their findings in a formal, confidential meeting - the interim challenge - and make high-level recommendations for action.

How library services are likely to be assessed

The Audit Commission has stated that “where services have recognised standards, inspection will assess how far they are being achieved from the customer’s viewpoint”¹⁶.  Inspectors will draw on performance against the public library standards when inspecting library services - but they will not be responsible for enforcing them.  And the standards will form only a part of what inspectors will be assessing, as discussed below; it is the Audit Commission’s view that “there is a delicate balance between operating a framework that judgements can be made within and so risking criticism, or alternatively be accused of over prescribing standards rather than protecting local discretion and democracy”¹⁷.

The Audit Commission has developed ‘judgement criteria’ and guidance for inspectors in applying the inspection framework to specific service areas.  These should help inspectors to:

• ensure consistency in assessing performance;
• make comparisons between authorities; and
• identify the top 25%. 

Much of what inspectors will be assessing relates to good service management practice - leadership, performance management, use of resources, management capacity and capability, staff commitment - and is generally applicable across all services.  This will be supplemented by assessment of issues relevant to individual services.  Inspectors will be guided in this by service-specific judgement criteria based on what users - and those targeted to receive the service - may reasonably expect from a good service.   

Inspectors are likely to consider and assess three broad areas specific to library services: 

• to what extent the service meets its statutory obligations;
• to what extent the service adopts government and national initiatives; and
• to what extent the service demonstrates professional good practice.

From these broad areas, the Audit Commission has defined a number of judgement criteria based on the expectations of library users and potential users.  For each of these judgement criteria the Commission has identified the key features - not absolutes - that inspectors should expect to find in libraries.  Performance against these judgement criteria and key features will guide their assessment of whether the library service is excellent, good, fair or poor.

The Audit Commission had intended to publish and consult on its judgement criteria for library services during autumn 2000.  It later planned publication and consultation for summer/autumn 2001.  The detail of how library services will actually be assessed is therefore still subject to change.

At the time of writing, no further information was available.  The three broad areas in ‘working draft’ are discussed in more detail below; they have not been materially revised since the first edition of Best Returns.

To what extent the service meets its statutory obligations 

Inspectors are likely to look for evidence that:

• the service is ‘comprehensive and efficient’;
• membership entitlements are in line with statutory guidance; and
• service charges fall within the legal framework.

Inspectors will not try to determine whether a library service is ‘comprehensive and efficient’.  They will consider performance against the public library standards as reported in the Annual Library Plan and the DCMS/CIPFA assessment of the Plan, as part of the information gathered for the context and performance review stage of the inspection.  They will also test the reality on the ground. 

Even where a library service meets the standards, the inspectors may judge that it is not delivering best value.  Performance against library standards will indicate whether an authority is delivering a general entitlement to library provision.  But best value requires authorities to tailor this general entitlement to local circumstances.  And inspectors will be making a more rounded judgement than one simply of legal compliance. 

To what extent the service adopts government and national initiatives

Initiatives that inspectors are likely to expect library authorities to be responding to include:

Annual Library Plans

The Annual Library Plan (ALP) will be a key inspection document.  Inspectors will form a view on an authority’s Plan but will not formally assess it - that will continue to be done by CIPFA/IPF on behalf of DCMS. 

The ALP is likely to be used by inspectors to:

• understand background information about the local authority and its library service;
• understand the library service mission statement, policy statements and objectives;
• evaluate the authority’s performance track record by comparing targets in the ALP with the performance information in the DCMS Planning Profile for the authority; 
• compare performance targets in the ALP with BVPIs;
• assess the overall configuration of the library service;
• assess consultation and customer feedback mechanisms;
• assess comparison information;
• identify probable strengths and weaknesses;
• assess strategic and action planning processes; and
• assess achievement against service standards - Government-endorsed, professional and local - in operation.

As assessment of ALPs has so far been based on the quality of the planning process rather than the service, a good ALP will not be sufficient in itself to deliver best value.  Assessment of the first year’s Plans in 1998 found “no correlation between plan quality and service quality”¹⁸.  It is possible for DCMS/CIPFA to assess an authority’s ALP as good, and for the best value inspectors to judge the quality of the library service on the ground as fair or poor - although a good ALP should provide a sound basis for improving a poor service.

Local Cultural Strategy

The Government is encouraging local authorities to develop a Local Cultural Strategy for their area with a lifespan of five years, by the end of 2002.  Final guidance for local authorities on developing a Local Cultural Strategy was published by DCMS in 2000. 

Inspectors are likely to look for quality library activities and programmes in the Local Cultural Strategy.  These may include reader and literature development, writing and publishing, as well as imaginative and innovative library service involvement in a range of cross-cutting initiatives.

A Modernising ICT Agenda

Inspectors are likely to consider issues around organisational culture, technical competence and employee training in relation to ICT developments and service quality.  They may assess an authority’s response to The People’s Network, its partnership arrangements with local or regional networks of ICT provision, success in bidding for funding to develop ICT networks and support employee training, creation and publication of material on the ICT network, partnerships with local schools, and support for homework clubs.  Inspectors will also expect the library service to be responding to e-government targets, in line with the DTLR/LGA report Modern councils, modern services: access for all.

Social Inclusion

Inspectors are likely to assess whether policies and plans recognise the role of libraries in promoting social inclusion, in line with the DCMS report Libraries for all: social inclusion in public libraries - policy guidance for local authorities in England, and the report by the Cabinet Office Social Exclusion Unit, A New Commitment to Neighbourhood Renewal: National Strategy Action Plan.  They are also likely to assess whether social inclusion policies and plans are being implemented on the ground.  

Lifelong Learning

Inspectors are likely to assess an authority’s ICT-based lifelong learning initiatives, participation in the National Grid for Learning, The People’s Network and University for Industry/learndirect, and efforts to secure external funding.

To what extent the service demonstrates professional good practice

Inspectors are likely to assess policies and their implementation on the ground in such areas as:

• access to the service (eg, service configuration, opening hours, library location and environment, services for people with special needs);
• stock and stock procurement, including the choice available at the point of use, and stock procurement arrangements;
• requests (eg, request policy, involvement of stakeholders in policy development and operation, arrangements for monitoring the performance of the request service);
• reference, information and local studies services;
• children’s and young people’s services, including adoption of the recommendations of Investing in Children: the Future of Library Services for Young People, and the development and implementation of an Integrated Children’s Strategy;
• mobile library services (eg, the process for planning and providing mobile library services, arrangements for monitoring performance and service quality, effectiveness of links with other library and authority services particularly using ICT).

Many of these areas are covered by good practice guidelines published by The Library Association.

Getting the most out of inspection

Some authorities anticipate their forthcoming inspection with fear and trepidation.  But best value inspection should actually help.  And there is much that authorities can do to get the most out of the process, as discussed below.

Authorities that have experienced inspection recommend thorough preparation - both of documentary evidence and of people.  Elected members, officers, front-line staff, service users and others likely to be involved in the inspection should be well briefed in advance.  They will then have a greater understanding of why the inspectors are there and the process they will be following.  They should be encouraged to talk openly and honestly, and reassured that there are no ‘right’ or ‘wrong’ answers; inspectors are not there to catch people out.

Inspection is best approached as a two-way process.  The inspectors will be constantly challenging what an authority does, how and why, and asking for supporting evidence.  Authorities should in turn be prepared to challenge, question and debate with the inspectors the conclusions they have reached, from the initial pre-inspection briefing right through to the interim challenge.  Authorities should also help point inspectors in the direction of what they need, including giving inspectors relevant documents that they may be unaware of, and suggesting activities to attend and people to interview. 

Inspectors will normally want to give daily feedback on each day’s activities and impressions.  Authorities should ensure that these feedback sessions take place; they are valuable opportunities for debate and discussion with the inspectors.  Some authorities find it useful to follow these sessions with a wider internal briefing on the early inspection messages.  They can then respond to - or provide further information on - any areas where the inspectors need further evidence or that they have misunderstood or overlooked. 

It is highly likely that the inspectors will ask authorities to set up a user focus group.  It is in an authority’s own interests to start organising this well in advance.  Early planning will allow time to promote the opportunity for local people to get involved - for example, by contacting residents associations, placing notices in parish magazines, displaying posters, and contacting Friends groups - and to reach a representative sample of the local population.

The inspectors will be clear at the outset about who they want to interview,  and will ask the authority to plan an interview schedule for them.  They are also likely to ask to see others at short notice as issues emerge during the inspection.  Authorities should not hesitate to suggest who else the inspectors should interview, and why.  Nor should they be offended if the inspectors ask interviewees to ‘stand down’; it will normally mean that the inspectors have sufficient evidence in a particular area and have no wish to spend precious time - theirs or the authorities’ - unnecessarily.

The on site stage of best value inspection in particular demands significant officer time and effort from authorities.  It is crucial that authorities identify a single officer contact to support the inspectors, who can make themselves available throughout the week and has sufficient authority within the council.  The contact officer will be asked to set up interviews and meetings, obtain rapidly additional information required, discuss feedback with inspectors, deal with internal communications, and organise logistics.

Authorities find it important to build quickly a good working relationship with the inspectors.  Some authorities recommend locating the inspectors within the library service - rather than, for example, in a corporate policy unit in the town hall.  Experience indicates that this can help to build rapport and enable stronger team-working - and it provides the inspectors with opportunities to observe the service on a day-to-day basis.

At a practical level, authorities can help ensure a smooth-running inspection by providing private office accommodation, with a telephone and internal directory; space(s) for interviews and focus groups; security access codes or passes to buildings; access to a printer and photocopier; car parking space(s); access to refreshment facilities; and maps of the area showing, for example, library buildings and mobile routes.

¹⁶ Audit Commission, Seeing is Believing: how the Audit Commission will carry out Best Value Inspections in England, 2000

¹⁷ Local Government Chronicle, Wendy Thomson, Director of Best Value Inspection Service, 25 May 2001

¹⁸ DCMS, Appraisal of Annual Library Plans, 1998

7. Government powers of intervention