  |
|
|
||||
|
||||||
| Journal of the New Zealand Medical Association,
28-October-2005, Vol 118 No 1224
|
||||||
|
Healthline: do primary care doctors agree with the
advice?
Ian St George, Matthew Cullen, Michelle Branney
In the 1990s, although telephone advice formed a significant
part of the workload of US and UK emergency departments, it was often inaccurate
or inadequate.1,2 In 1995, Aitken and coworkers reported phoning 30 New Zealand
public hospital emergency departments and 20 private accident and medical
centres, role-playing the parent of a feverish infant. They received a wide
range of responses. In 14 centres, the doctor on call was contacted at once; in
5 centres, the doctor gave advice; in 26 centres, the nurse gave advice; and in
5 centres, the advisers did not say who they were.
The authors judged the advice from 16 of the 36 centres to
be inadequate.3 Thus such observations led to a desire to standardise telephone
advice for symptomatic callers by using electronic information systems.
In the United States during the 1990s, private insurance
schemes began offering nurse-led, software-supported telephone triage and
advice; and NHS Direct (a free, 24-hour advice and triage line) began in
England.
Meanwhile, in New Zealand in 1994, Tisdale reported a pilot
of a nurse-run telephone advice line,4 and in 1998 Cameron and others reported
high levels of satisfaction by those phoning the National Poisons Centre for
advice, as well as considerable saving of public money.5
Furthermore, the repeated observation that resource-intense
primary medical services were being inappropriately used by those whose symptoms
could have been managed with lower levels of care,6 led to a desire for primary
care demand management, and again, telephone triage seemed to supply an
answer.
The two drivers for freely available telephone triage were
thus standardisation of advice, and improved resource use by directing callers
to lower levels of care.
The New Zealand Health Ministry funded Healthline pilot
began in four regions in 2000. In 2004, it incorporated Plunket Line and in May
2005 it became a national, state-funded, 24x7, primary health service offering
health information, well child, and parenting assistance, and symptom
triage.
About 70% of Healthline callers seek advice on symptoms, and
70% of the calls are outside normal working hours. Many are triaged to lower
levels of care than they had intended before calling Healthline.7
The question arises, then: is it safe? An independent study
of the Healthline pilot, commissioned by the Ministry, exhaustively traversed
the clinical quality activities of the service, and concluded, ‘The
Healthline service has operated at least as safely to date as similar overseas
telephone services,’8 but Moriarty and others expressed concern after a
study of simulated callers.9
Since then, Healthline has changed the software from
‘Personal Health Advisor™’ to a new decision support package
called ‘Care Enhance Call Centre™.’
We decided then to examine the degree of concordance between
primary medical care specialists with Healthline advice using the Care Enhance
Call Centre™ software package.
MethodHealthline nurses triage callers seeking advice on
current symptoms by using a symptom-specific guideline to one of nine
dispositions (‘endpoints’: Table 1). The guideline is
computer-based, prompting the nurse to ask a series of questions designed to
exclude the most serious potential causes of the symptom. The endpoint is
reached when a specific cause cannot be excluded. The nurse makes a full
electronic record including demographic data, free text on the presenting
symptom, the responses to the guideline questions, the endpoint reached, and the
advice and assistance given.
Table 1. Triage
endpoints for symptomatic callers
Beginning from midnight 31 January 2005, we selected the
first 10 cases triaged to each endpoint—and printed the call record,
obliterating the actual endpoint reached, and the advice given. Three primary
care specialists then independently examined the 90 clinical records thus
generated and decided what endpoint they would have recommended, given the same
information. There were two general practitioners (both women, one urban and one
rural) and one accident and medical doctor with a hospital emergency department
background: two were selected on the basis of personal acquaintance with the
service, and one represented the Wellington after hours medical service.
Thus the ‘gold standard’ with which we compared
the nurse decisions, was that of experienced primary care specialists, given the
same clinical information.
We defined agreement as an endpoint at or immediately
adjacent to that actually recommended by the nurse. We calculated concordance by
crude percentage agreement, and by Cohen’s Kappa (K) which provides a
number between 0 and 1: a K of 0.7 or more is regarded as showing satisfactory
inter-rater reliability.
ResultsThe endpoints advised by the three doctors varied: crude
percentage agreement between the highest and lowest was only 51% and
Cohen’s K = 0.43.
For that reason we compared the
median doctor-advised endpoint for each
case with the endpoints reached by the Healthline nurses. For this comparison,
crude percentage agreement was 70%, and Cohen’s K = 0.78.
In 10 cases (11.1%), the median doctor triaged to two
endpoints or more lower (i.e. to a lower level of care) than the Healthline
nurse. In 64 cases (71.1%), the median doctor triaged to within one endpoint of
that reached by the Healthline nurse. Thus, for 82.2% of cases, there were no
safety concerns.
In the other 16 cases (17.8%), the median doctor triaged to
two or more levels of care higher, but the three doctors were unanimous in only
seven of these (7.8%). They were asked to review these seven cases, considering
specifically whether the Healthline nurse endpoint was unsafe. In only one case
(1.1% overall) did all three consider that the lower endpoint posed some risk to
the patient. On detailed review of this case, the lower endpoint reached by the
Healthline nurse was not a function of the software, but an error by the
operator.
In only four cases, the doctors would have triaged to 111
when Healthline did not, and in all of these the Healthline nurse had
recommended immediate medical care. The doctors considered there was no risk to
the patients in these cases.
DiscussionA Kappa of 0.78 between the median doctor and Healthline is
reassuring, as is the perception that Healthline endpoints were clinically safe
in 99% of cases, and the software clinically safe in all cases. The single case
where disagreement was unanimous among the doctors was not a fault of the
software, but an error by the nurse.
One doctor spontaneously commented:
The
exhaustiveness of the questioning that the nurses do before reaching a
conclusion is probably much more accurate in the end than I would do in the
night for example when woken up. The computer prompts them to go into all
possible urgent scenarios and thus I think it would be pretty unusual for them
to miss anything. Having seen the system ‘up close’ it appears
excellent. The patients will appreciate receiving consistent advice rather than
the enormous variety they receive from different individual
doctors.
We observed wide variation among the three doctors (Kappa
0.43), wider than that between the median doctor and Healthline (Kappa 0.78).
Indeed, one doctor wrote:
It
could be that I am a little cautious or that I am used to dealing with patients
who want immediate service/action.
Gribben similarly asked 12 doctors to assess whether they
would have managed patients attending a city emergency department themselves,
and found a Kappa of 0.34 among them; he remarked:
There
was a surprisingly wide range of views on the proportion of cases that the GPs
thought could be completely handled in primary care. We asked for personal
views, and so naturally the skills and experience of individual GPs will have
contributed to the range. These assessments did not appear to be related to age
or gender and the scores of the A&M (accident and medical) doctors and
academic GPs were distributed across the range11
The range of medical practice variation is wide, and with
Marshall Marinker we offer no criticism of that: “Narrowing the range may
give us the illusion of consensus, without telling us anything at all about
whether the consensus is better than the diversity,” he wrote.10
Similarly, we make no judgement as to who is right: triage can be standardised,
but without guidelines it is not a precise science.
Note: We acknowledge
that three is a small group of doctors, and that a larger group may have
provided more reliable data.
Author information:
Ian St George, Medical Director, Healthline, Wellington; Matthew Cullen,
Co-President, McKesson Asia-Pacific, Lane Cove, New South Wales, Australia;
Michelle Branney, General Manager, McKesson NZ Ltd, Wellington
Correspondence: Ian
St George, Healthline, PO Box 10643, Wellington. Fax: (04) 499 2357; email: ian.stgeorge@healthline.co.nz
References:
|
|
||||
| | Current
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads) Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals |
|