A closer look at the referral secret shopper survey

A mystery shopper paper which criticised pharmacy referral behaviours needs to be seen in its true perspective, says the College

The Australian College of Pharmacy says that the secret shopper paper and subsequent reporting in Australian Doctor may have “cast shadow on the critical role of pharmacists referring patients to medical practitioners when necessary,” but that the paper needs to be interpreted objectively.

“The College has extensive experience in pharmacy mystery shopper research over many years and has published evidence of the profoundly positive effects Australian pharmacists have on patient care,” Dr Brett MacFarlane, education and research manager of the Australian College of Pharmacy, says.

“I would like to draw attention to specifics of the recent mystery shopper paper that will place the outcome reported by the Australian Doctor into perspective.”

Australian Doctor had focused on an outcome of the paper that “Pharmacies are failing to recognise red-flag symptoms that require referral to a doctor in up to 50% of cases”.

Interaction with a pharmacist, either directly or after first speaking to a non-pharmacist staff member, resulted in referral in 80% of interactions (all of which required referral) – but this number dropped to 24% when only a non-pharmacist staff member was spoken to.

Australian Doctor readers were quick to condemn pharmacy over the results.

But the College points out that the rate of doctor referral was actually a secondary objective of the study, which was actually investigating as its primary objective whether repeated performance feedback to pharmacy staff would alter an outcome of doctor referral over time.

“The clinical outcomes do not provide an overall picture of doctor referral rate by Australian pharmacies and simply reflect the actions of certain staff members at certain times,” the College says.

“While the lowest rate of doctor referral was 50%, this was at the start of the nine-week intervention period and by the end had risen to 70% in two of the three scenarios investigated.

“The overall rate of referral was actually 66%. When pharmacists were involved in the interaction, the referral rate was actually 80%.”

The study – a pilot, proof of concept study – only included 13 pharmacies in metropolitan Sydney, the College says: a very small sample of the roughly 5,600 pharmacies and 29,000 registered pharmacists in the country.

It also points out that the mystery shoppers and the scoring sheet used in the study were not validated: “clinical outcomes reported by the paper are not reliable until the methodology is validated”.

An audit of accuracy of the recording of data by mystery shoppers indicated significant error in the data in 6% of visits, it says. One of the 13 pharmacies did not have their mystery shopper visit audio taped and some audio recordings failed.

“Therefore some data collection was reliant on recall of the mystery shopper which is prone to recall bias.

“Details of the ‘red flags’ were only given by mystery shoppers when asked about them by pharmacy staff. Doctor referral rate increased as questioning increased.

“The authors made no conclusion about the actual rate of doctor referral. Their conclusions were oriented around the effect of the study intervention on referral rate.”

President of the College, Georgina Twomey says, “Pharmacy is one of the only professions in Australia that has competency standard based education.

“The pharmacy profession has a long history of supporting research into the quality of our clinical practice and welcomes any opportunity to undertake quality improvement.

“The College welcomes discussion over the role of pharmacists in front line clinical care. I support the interdisciplinary team approach to healthcare and the ongoing close relationship between pharmacists and doctors.

“I call for a mature, evidence based debate.”

Previous $1 discount didn't drive pharmacy choice
Next Hep A & Hep B vaccine shortages

NOTICE: It can sometimes take awhile for comment submissions to go through, please be patient.


  1. Jarrod McMaugh

    This is an excellent article. well done Megan.

  2. Toby

    I totally agree, but the article misses the point.
    The reason doctors don’t allow ‘mystery patient’ surveys, is they don’t want to give their enemies ammunition to bash them with. Pity that pharmacists are too naïve, disunited, and weak, to think this through and act on it.

    • Jarrod McMaugh

      They do actually.

      There are plenty of studies showing that best practice is not achieved in primary care setting, just as there are plenty of others that show that it can be achieved (same as in pharmacy).

      A good example is the information found in the current NPSMedicinewise Detailing on the use of statins. 650 participating general practices show that 35% of all prescribed statins are used in very-low risk patients against guidelines – that’s roughly 7-15 million dollars a year that could be saved if these figures can be extrapolated out to all statin prescribing in Australia.

Leave a reply