RPS benchmarking metrics for acute hospitals

RPS benchmarking metrics for acute hospitals - 

Definitions to enable meaningful benchmarking of pharmacy services, and the safe and effective use of medicines.

There is currently inconsistency in the way that acute hospitals measure performance when benchmarking. RPS through its Hospital Expert Advisory Group (HEAG) has developed a consensus on definitions for benchmarking metrics relevant to the delivery of pharmacy services and medicines use in acute hospitals.

The aim is to provide a consistent basis for the collection of data which will allow acute hospitals to benchmark performance against each other most effectively. Benchmarking helps identify unwarranted variation and therefore presents opportunity to improve performance within organisations. Definitions look to use existing data sources where possible rather than adding to the burden of data collection. It is acknowledged that this may occasionally lead to compromise in detail/accuracy. 

Defining the target range for the metrics is out of the scope of this work as is accounting for all local variations in service provision (warranted variation); the definitions are national and need to be broadly fit for purpose.

The RPS Hospital Expert Advisory Group will continue identifying and develop further definitions to support the profession in consistent benchmarking. Comments or suggestions for additional definitions can be directed to Catherine Picton ([email protected]).

Sections on this page

1. METRIC = average number of days medicines stock physically held by pharmacy

INTERPRETATION INTERPRETATION

= processing efficiency of the pharmacy – lower figure is good, but if too low then will increase transactional costs and risk of stock outs i.e. needs to be balanced against other metrics.

Note: use the NPPSC* definition: include all stock holding locations for which the Chief Pharmacist/Director of Pharmacy is accountable (satellite dispensaries, aseptic units etc.). Exclude outsourced out-patient dispensaries and ‘stand-alone’ production and pre-packing units etc. Include VAT.

*NPPSC = National Pharmacy Procurement Specialists Committee

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation = 365/stock turn figure (see below)

Stock turn figure = (Annual Hospital Medicines Expenditure minus any expenditure unrelated to/not directly handled by hospital pharmacy e.g. outsourced out-patients, inventory delivered directly to/held on wards, clinics etc, annual homecare expenditure, FP(10) costs) ÷ (year-end pharmacy store stock holding figure).

COMMENTS COMMENTS

The figure for this metric would generally be expected to be a range and hospitals will want to compare with similar peers.

2. METRIC = average number of days medicines stock held (whole system)

INTERPRETATION INTERPRETATION

= effectiveness of pharmacy procurement system in minimising physical store involvement – lower figure is good, but if too low then will increase transactional costs and risk of stock outs i.e. needs to be balanced against other metrics

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation = 365/stock turn figure (see below)

Stock turn figure = (Annual Hospital Medicines Expenditure) ÷ (year-end pharmacy stock holding figure)

COMMENTS COMMENTS

Used as a comparator to metric 1. Offers 2 stock holding options as metrics – each would have its own target defined. Ideally metric 1 is low and metric 2 relatively lower. The greater the relative difference the more effective the system as most stock bypasses a physical pharmacy store.

3. METRIC = percentage of orders sent electronically (e-trading)

INTERPRETATION INTERPRETATION

= processing efficiency of the pharmacy procurement system – high figure is good.

PROPOSED DEFINITION PROPOSED DEFINITION

E-trading of orders is the process whereby orders are sent directly from the purchaser’s computer system to the supplier’s computer system by electronic means. No paper is involved in these transactions which take place electronically and without manual transcription. This excludes fax or e-mailing of orders from the numerator. 

Calculation = (number of orders sent electronically) ÷ (total number of orders created in the same time period) – expressed as a percentage.

Note: it is the number of orders and not the value of orders.

This calculation should be done for (a) all orders sent including homecare and (b) all orders excluding homecare.

It is recognised that a metric that looks at the number of lines ordered, rather than the number of orders, would be more useful, whilst at present it is not possible to measure this easily organisations should be working toward this.

4. METRIC = percentage of invoices processed electronically (e-trading)

INTERPRETATION INTERPRETATION

= processing efficiency of the pharmacy procurement system – high figure is good.

PROPOSED DEFINITION PROPOSED DEFINITION

E-trading of invoices is the process whereby invoices are sent directly from the supplier’s computer system to the purchaser’s computer system by electronic means. No paper is involved in these transactions which take place electronically and without manual transcription. This excludes fax or e-mailing of invoices from the numerator. It is expected that a PEPPOL compliant eCommerce system is used.

Calculation = number of invoices processed electronically/total number of invoices processed in the same time period – expressed as a percentage.

Note: it is the number of invoices and not the value of invoices.

It is recognised that a metric that looks at the number of lines ordered, rather than the number of orders, would be more useful, whilst at present it is not possible to measure this easily organisations should be working toward this.

This calculation should be done for (a) all invoices processed including homecare and (b) all invoices excluding homecare.

5. POINT OF GUIDANCE = % pharmacy store inventory which are not medicines

As a general principle pharmacy stores should only hold medicines and products used as medicines. Holding products not used as medicines (e.g. enteral feeds, dressings) should generally be viewed as non-value adding; if there are exceptions these should be supported with a clear rationale.

6. METRIC = % of pharmacists registered for over three years who are also qualified as prescribers

INTERPRETATION INTERPRETATION

= an indicator of ambition of pharmacy service to be core clinical; modified denominator accounts for pharmacists who are not eligible to access prescribing training; larger figure better.

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation = (number of pharmacists qualified as prescribers) ÷ (total number of pharmacists in service who have been registered a minimum of 3 years) – expressed as a percentage
* Note this is calculated per head-count as oppose WTE.

7. METRIC = % of qualified pharmacist prescribers routinely prescribing

INTERPRETATION INTERPRETATION

Interpretation = an indicator of how well developed pharmacy service is in terms of being core-clinical.

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation = (number of pharmacists qualified as prescribers who routinely prescribe as part of their clinical role) ÷ (total number of pharmacists in service who are qualified to prescribe) – expressed as a percentage. 

It is accepted that the term ‘routinely’ can be open to interpretation it is not easy to put a general value to this. However, the principle is that prescribing is part of the pharmacist’s routine clinical practice i.e. the pharmacist will typically prescribe when working in their clinical role; compared with prescribing only occasionally or rarely which would not be routine.

8. METRIC = Medicines reconciliation by pharmacy team within 24 hours of admission

INTERPRETATION INTERPRETATION

= indicator of extent of pharmacy medicines optimisation practice that is targeted at admission

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation = (number of patients admitted, electively AND non-electively, who have their medicines reconciled by a member of the pharmacy team within 24 hours of the time of admission) ÷ (total number of patients within sample). Sampled monthly as per medicines safety thermometer (100% of patients; admissions during previous 24 hours; last Wednesday of the month). Expressed as a percentage.

COMMENTS COMMENTS

The metric uses the National Institute for Heath and Clinical Excellence (NICE) definition of medicines reconciliation.

For the purpose of this definition, medicines reconciliation is considered complete once the changes have been communicated to the prescriber. However the longer term aspiration is to amend the definition so that medicines reconciliation is complete only when discrepancies are corrected.

Admission is considered to be from the time the decision is taken to admit (not from the day after admission).

This metric is a measurement of the extent of pharmacy team medicines reconciliation, it is recognised that in some organisations other clinical staff may also undertake medicines reconciliation. Where this happens pharmacy should have a programme of training and competency assessment in place to oversee the medicines reconciliation process.

A useful comparison can be made against metric 11 however, to enable this, data should be collected such that it can be broken down between elective and non-elective admissions.

9. METRIC = % patients experiencing an omission of a critical medicine

INTERPRETATION INTERPRETATION

= an important indicator of overall quality of medicines use within an organisation. Lower value better.

It is accepted that this metric reflects many things including:

  • quality and responsiveness of medicines supply system pertaining to inpatient care;
  • nurse/doctor awareness of the importance of administration of critical medicines and related behaviours (e.g. willingness of nurses to source medicines if not available; timing of doctor’s prescribing in relation to ward routines);
  • the robustness of the medicines management system within a complex organisation to deal with variation e.g. how medicines are managed when patients are transferred.

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation: number of patients with an omitted dose of a critical medicine (as defined by medicines safety thermometer) ÷ (total number of patients within same sample from metric 8).

Note: the numerator should exclude omissions due to patient refusal or valid clinical reason for omission – this is different to medicines safety thermometer (but efforts are underway to standardise). Expressed as a percentage.

10. METRIC = % pharmacist time spent on clinical activities (same metric can be applied to pharmacy technicians, noting single exception to calculation listed below)

INTERPRETATION INTERPRETATION

= crude estimate of resource utilisation between clinical and other.  Definition of ‘clinical’ is anything pertaining to an individual patient, but excluding operational activities, details outlined below.

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation = (A) ÷ (A B).  Exclude C.  Present as a percentage.

‘A’ = estimate of overall staff time deployed as a sum of all clinical activities i.e. those provided for individual patients on wards, in  outpatient clinics, care homes and in domiciliary patient facing clinical roles e.g. counselling, drug history taking, medicines reconciliation, organising transfer of care, shared decision-making, prescribing, clinical validation, participation in ward rounds and multidisciplinary team working, medicines administration.

Note: facilitating ‘near patient’ medicines use at ward level including dispensing, use of patients’ own medicines etc. is inextricably linked to a technician’s clinical role and should be included in an estimate for A.  However, satellite dispensaries on or near wards are simply dispensaries and are not clinical.  Any dispensing/supply undertaken by a pharmacist at ward level should be excluded from estimates.

‘A’ should also include an estimate of resource that is contained within departments that provides clinical services for individual patients as defined e.g. medicines information i.e. query answering for specific patients; clinical validation undertaken in providing an aseptic dispensing service.

‘B’ = estimate of overall staff time deployed as a sum of all operational activities including: stock management, dispensing (including staff time working in satellite dispensaries on or near wards, but excluding ‘near patient’ dispensing/supply undertaken by technicians as described), procurement, aseptic dispensing, production, formulary support, administration, quality control, topping up, medicines information (excluding the estimate of individual patient directed resource in ‘A’ as described above).  

‘C’ = resources deployed as chief pharmacist (or equivalent), medicines safety officer (or equivalent), clinical informatics (or equivalent), controlled drug Accountable Officer, quality assurance, audit, practice research, quality improvement and teaching time i.e. C ≠ A or B.  These resources should not be included in the calculation.

COMMENTS COMMENTS

There is no justification for precise measurement of this metric at a service level – there are too many confounding factors to make detailed benchmarking worthwhile. However, this is a useful (albeit crude) estimate of resource utilisation as an indicator of scale of clinical orientation of service.  Staff time is used as the currency as it is easier to understand/work with when compared to financial resource deployment.

Note: in future it will be important to tease out different roles and the resources dedicated to each e.g. clearly not cost effective to use an AfC band 8b (pharmacist) when an AfC band 5 (technician) could do the same job e.g. drug history taking.

11.  METRIC = effectiveness of clinical pharmacy service at weekends versus midweek activity (using medicines reconciliation as a proxy marker)

INTERPRETATION INTERPRETATION

= not all clinical activity is essential 7 days per week.  Using an assumption that emergency care, all non-elective admissions, present a constant demand for clinical pharmacy, then medicines reconciliation at the point of admission can be used as a single proxy measure of appropriate deployment of clinical pharmacy resources across a 7 day time period.

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation = (number of patients admitted non-electively, who have their medicines reconciled by a member of the pharmacy team within 24 hours of the time of admission) ÷ (total number of patients within that sample).  As per metric 8 i.e. 100% of patients, but identified as non-elective; admitted in the previous 24 hours; sampled on a Monday morning). Expressed as a percentage.

COMMENTS COMMENTS

A useful comparison can be made against metric 8. 

12.  METRIC = % injectable chemotherapy sourced and supplied as pre-assembled/ready-made versus local production

INTERPRETATION INTERPRETATION

= use of standard formulations and doses for ‘commonly used chemotherapy’ medicines to enable aseptic capacity to be focused on activity that does require local production i.e. complex, high cost and/or clinical trial medicines.  Assumption is that high volume ‘outsourced’ chemotherapy will provide better value for money and free up valuable local aseptic dispensing capacity.  A high figure is good, but a very high figure may be inappropriate.

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation = (number of pre-assembled/ready-made injectable chemotherapy items dispensed as sourced) ÷ (number of pre-assembled/ready-made injectable chemotherapy items dispensed as sourced + number of injectable chemotherapy items locally produced and dispensed).  Expressed as a percentage.

COMMENTS COMMENTS

‘Commonly used chemotherapy’ includes all chemotherapy that can be prepared in a pharmacy aseptic unit and is included in the national chemotherapy dataset (SACT data submission).

Chemotherapy refers to any systemic anti-cancer therapy, this includes monoclonal antibodies/targeted therapies, intravenous, subcutaneous, intrathecal and oral chemotherapy as well as topical treatments for bladder cancer; hormonal treatment is excluded. NHS standard contract for cancer: chemotherapy (adult) - service specifications.  NHS England. Available at 

Large aseptic manufacturing units should add internal batch prepared products to their figures for pre-assembled/ready-made).

Further background information can be found in Guidance on managing the sourcing and supply of ready to administer chemotherapy doses for the NHS: A ‘how to’ guide (Edition 1A).

13.  METRIC = % adult parenteral nutrition sourced and supplied as pre-assembled/ready-made (or as individual components) versus local bespoke production and supply.  Same metric can be used for paediatric/neonatal nutrition which should be calculated separately to aid benchmarking comparison

INTERPRETATION INTERPRETATION

= sourcing and supply of ready-made standard formulations or, if adult parenteral nutrition bags are supplied to wards as separate components, this removes work from aseptic units and thereby creates capacity for complex, high cost and/or clinical trial medicines.

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation = (number of ready-made standard bags sourced and/or number of parenteral bags supplied to wards without any manipulation) ÷ (total number of parenteral bags supplied in total). Expressed as a percentage.  Calculate performance for adults and paediatrics separately to aid benchmarking comparisons being made.

COMMENTS COMMENTS

Variation in performance will obviously be partially explained by level of clinical complexity within organisations i.e. tertiary/specialist centres may have a lower percentage performance.  Therefore, important to benchmark between similar organisations.

The mixing of individual components at ward level is not being advocated.  A safe system to enable individual components being administered separately is obviously required.

14.  METRIC = Safer use of intravenous potassium salts

INTERPRETATION INTERPRETATION

= indicator of the extent to which an organisation complies with national guidance on the safe handling of high strength intravenous potassium salt solutions

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation = (number of mmols of high strength potassium issued by pharmacy as ready-to-administer infusions) / (number of mmols of potassium issued as ampoules + number of mmols of high strength* potassium salt solutions issued by pharmacy as ready-to-administer infusions).

Note: include all intravenous potassium containing solutions e.g. potassium chloride, potassium acid phosphate.

*High strength applies to concentrations greater than 80mmol/L. Expressed as a percentage.

COMMENTS COMMENTS

The metric represents the proportion of intravenous potassium solutions issued as ready-to-administer infusions (%). The higher the metric value the greater the level of compliance. Data from Pharmacy department stock control systems should be used. 

Centres providing services to paediatrics/neonates may have a lower % performance than those with only with adult patients.

15.  METRIC = number of incident reports and near misses per month in relation to the aseptic preparation of medicines (in-processing quality performance metric)

INTERPRETATION INTERPRETATION

= effectiveness of pharmaceutical quality system; takes account of activity. Lower figure is good.

PROPOSED DEFINITION PROPOSED DEFINITION

Calculation =  number of Datix (or equivalent) and/or in-house (‘near miss’) reports per month ÷ number of aseptic workstation sessions*

* A workstation session is defined as a grade A workstation (laminar air flow cabinet or isolator; 2 gloves represent one station) in use on any morning or afternoon (or part thereof).

Eg. an aseptic unit with one negative pressure 2 glove isolator in use for 5 mornings and 5 afternoons would have 10 workstation sessions per week and 40 per month (assuming a 20 working day month.)

COMMENTS COMMENTS

Datix reports (or equivalent) and/or local in-house ‘near miss’ reports should both be used as Datix reports (or equivalent) are not always used to record in-process ‘near miss’ errors.  The number of errors occurring in production will be higher than the number of actual errors leaving the unit and therefore is a much more useful benchmarking metric.

16. METRIC = % patients who do not need medicines to be dispensed by pharmacy when the decision to discharge from hospital is made for (a) non-elective patients and (b) elective surgical patients

Interpretation Interpretation

= a simple high level measure of ‘preparedness for discharge’ which can potentially demonstrate the effectiveness of the whole system of supply of medicines from the perspective of patient flow.  Patients ‘not ready’ for discharge otherwise have to wait for pharmacy to dispense their medicines from a discharge order/prescription as a step which happens late in the process. This metric is meaningful to patients as it helps prevent avoidable delays in discharge.

Proposed Definition Proposed Definition

A = those patients sampled who do not need any medicines dispensed by pharmacy at the point a decision is made to discharge from hospital

B = total number of patients sampled

Calculation = (A) ÷ (B).  Present as a percentage.

In the absence of any easier way to measure this, use a simple sequential count of patients, large enough to be representative e.g. 100.  Measure (a) non-elective patients admitted and/or (b) elective surgical patients, as performance may differ between the two groups and this approach also allows improvement effort to be differentiated.

If collected continuously/frequently, then this data can be used to help drive improvement.  It can of course be collected less frequently as a means of providing assurance of performance.

Comments Comments

It is acknowledged that multiple factors impact performance with this high level metric e.g. use of patients own medicines, extent of discharge planning, system of supply used e.g. dispensing for discharge, use of ward stocked over-labelled packs, impact of clinical pharmacy at ward level etc.
 
However, lower % performance should prompt further inquiry/action as it would indicate a potential opportunity to improve patient flow.  Higher % of patients not requiring a supply of medicines from pharmacy at discharge will reduce delay at patient discharge due to medicines supply.
 
Higher % of patients not requiring supply of medicines from pharmacy at discharge is more likely achievable in elective surgical patients than other groups due to the greater opportunity to pre plan treatment (e.g. using standardised treatment regimes, positively impacting in pre-admission settings on use of patients’ own medicines etc.).
 
It is acknowledged that other metrics are necessary to aid further detailed improvement work, some of which are very system specific and therefore difficult to measure and benchmark.

17. METRIC = median time of day that patients requiring a supply of medicines from pharmacy on the day of hospital discharge receive them for (a) non-elective patients and (b) elective surgical patients

Interpretation Interpretation

= a simple high level measure of timeliness of TTO processes on the day of a patient’s discharge when a supply by pharmacy needs to be made on the day of discharge. It includes the whole process from writing the TTO to the time the patient is ‘discharge ready’ from a medicines perspective. This can demonstrate the effectiveness of the whole system of supply of medicines from the perspective of patient flow.  This metric is meaningful to patients as it helps determine a component of their experience at the point of discharge and can be a source of complaints.

Proposed Definition Proposed Definition

The median time of day that patients requiring a supply of medicines from pharmacy on the day of hospital discharge receive them. This includes supplies made by pharmacy staff from any location in addition to a dispensary and includes near patient supply from the ward stocks, use of over-labelled medicines, satellites, trollies etc.

In the absence of any easier way to measure this, use a simple sequential count of patients, large enough to be representative e.g. 100.  Measure (a) non-elective patients admitted and/or (b) elective surgical patients, as performance may differ between the two groups.

This data does not need to be collected continuously but will be more useful if it is and this should be an ambition. This measure can be used as a means of measuring improvement and benchmarking internally and externally.

Comments Comments

It is acknowledged that multiple factors impact performance with this high level metric e.g. time of decision to discharge, time of TTO writing, extent of discharge planning, impact of clinical pharmacy at ward level etc.  However, later median time should prompt further inquiry/action as it would indicate a potential opportunity to improve patient flow. An earlier time helps to improve early patient flow and increase the likelihood of discharges to care homes etc. Other ward processes will need to be considered when improving this metric.

An earlier median time of medicines supply on the time of discharge is more likely achievable in elective surgical patients than other groups due to the greater opportunity to pre plan treatment (e.g. using standardised treatment regimes, positively impacting in pre-admission settings on use of patients’ own medicines etc.).

It is accepted that other metrics are necessary to aid further detailed improvement work, some of which are very system specific and therefore difficult to measure and benchmark.