|
|
Measuring the Quality of Health Care in Low-Income Settings
Measuring the quality of health care in High-Income Countries (HIC) is deceptively difficult, as shown by work carried out by many research groups, including CLAHRC WM.[1-5] However, a large amount of information is collected routinely by health care facilities in HICs. This data includes outcome data, such as Standardised Mortality Rates (SMRs), death rates from 'causes amenable to health care', readmission rates, morbidity rates (such as pressure damage), and patient satisfaction, along with process data, such as waiting times, prescribing errors, and antibiotic use. There is controversy over many of these endpoints, and some are much better barometers of safety than others. While incident reporting systems provide a very poor basis for epidemiological studies (that is not their purpose), case-note review provides arguably the best and most widely used method for formal study of care quality – at least in hospitals.[3] [6] [7] Measuring safety in primary care is inhibited by the less comprehensive case-notes found in primary care settings as compared to hospital case-notes. Nevertheless, increasing amounts of process information is now available from general practices, particularly in countries (such as the UK) that collect this information routinely in electronic systems. It is possible, for example, to measure rates of statin prescriptions for people with high cardiovascular risk, and anticoagulants for people with ventricular fibrillation, as our CLAHRC has shown.[8] [9] HICs also conduct frequent audits of specific aspects of care – essentially by asking clinicians to fill in detailed pro formas for patients in various categories. For instance, National Audits in the UK have been carried out into all patients experiencing a myocardial infarction.[10] Direct observation of care has been used most often to understand barriers and facilitators to good practice, rather than to measure quality / safety in a quantitative way. However, routine data collection systems provide a measure of patient satisfaction with care – in the UK people who were admitted to hospital are surveyed on a regular basis [11] and general practices are required to arrange for anonymous patient feedback every year.[12] Mystery shoppers (simulated patients) have also been used from time to time, albeit not as a comparative epidemiological tool.[13]
This picture is very different in Low- and Middle-Income Countries (LMIC) and, again, it is yet more difficult to assess quality of out of hospital care than of hospital care.[14] Even in hospitals routine mortality data may not be available, let alone process data. An exception is the network of paediatric centres established in Kenya by Prof Michael English.[15] Occasionally large scale bespoke studies are carried out in LMICs – for example, a recent study in which CLAHRC WM participated, measured 30 day post-operative mortality rates in over 60 hospitals across low-, middle- and high-income countries.[16]
The quality and outcomes of care in community settings in LMICs is a woefully understudied area. We are attempting to correct this ‘dearth’ of information in a study in nine slums spread across four African and Asian countries. One of the largest obstacles to such a study is the very fragmented nature of health care provision in community settings in LMICs – a finding confirmed by a recent Lancet commission.[17] There are no routine data collection systems, and even deaths are not registered routinely. Where to start?
In this blog post I lay out a framework for measurement of quality from largely isolated providers, many of whom are unregulated, in a system where there is no routine system of data and no archive of case-notes. In such a constrained situation I can think of three (non-exclusive) types of study:
- Direct observation of the facilities where care is provided without actually observing care or its effects. Such observation is limited to some of the basic building blocks of a health care system – what services are present (e.g. number of pharmacies per 1,000 population) and availability (how often the pharmacy is open; how often a doctor / nurse / medical officer is available for consultation in a clinic). Such a ‘mapping’ exercise does not capture all care provided – e.g. it will miss hospital care and municipal / hospital-based outreach care, such as vaccination provided by Community Health Workers. It will also miss any IT based care using apps or online consultations.
- Direct observation of the care process by external observers. Researchers can observe care from close up, for example during consultations. Such observations can cover the humanity of care (which could be scored) and/or technical quality (which again could be scored against explicit standards and/or in a holistic (implicit) basis).[6] [7] An explicit standard would have to be based mainly on 'if-then' rules - e.g. if a patient complained of weight loss, excessive thirst, or recurrent boils, did the clinicians test their urine for sugar; if the patient complained of persistent productive cough and night sweats was a test for TB arranged? Implicit standards suffer from low reliability (high inter-observer variation).[18] Moreover, community providers in LMICs are arguably likely to be resistant to what they might perceive as an intrusive or even threatening form of observation. Those who permitted such scrutiny are unlikely to constitute a random sample. More vicarious observations – say of the length of consultations – would have some value, but might still be seen as intrusive. Provided some providers would permit direct observation, their results may represent an 'upper bound' on performance.
- Quality as assessed through the eyes of the patient / members of the public. Given the limitations of independent observation, the lack of anamnestic records of clinical encounters in the form of case-notes, absence of routine data, and likely limitations on access by independent direct observers, most information may need to be collected from patients themselves, or as we discuss, people masquerading as patients (simulated patients / mystery shoppers). The following types of data collection methods can be considered:
- Questions directed at members of the public regarding preventive services. So, households could be asked about vaccinations, surveillance (say for malnutrition), and their knowledge of screening services offered on a routine basis. This is likely to provide a fairly accurate measure of the quality of preventive services (provided the sampling strategy was carefully designed to yield a representative sample). This method could also provide information on advice and care provided through IT resources. This is a situation where some anamnestic data collection would be possible (with the permission of the respondent) since it would be possible to scroll back through the electronic ‘record’.
- Opinion surveys / debriefing following consultations. This method offers a viable alternative to observation of consultations and would be less expensive (though still not inexpensive). Information on the kindness / humanity of services could be easily obtained and quantified, along with ease of access to ambulatory and emergency care.[19] Measuring clinical quality would again rely on observations against a gold standard,[20] but given the large number of possible clinical scenarios standardising quality assessment would be tricky. However, a coarse-grained assessment would be possible and, given the low quality levels reported anecdotally, failure to achieve a high degree of standardisation might not vitiate collection of important information. Such a method might provide insights into the relative merits and demerits of traditional vs. modern health care, private vs. public, etc., provided that these differences were large.
- Simulated patients offering standardised clinical scenarios. This is arguably the optimal method of technical quality assessment in settings where case-notes are perfunctory or not available. Again, consultations could be scored for humanity of care and clinical / technical competence, and again explicit and/or implicit standards could be used. However, we do not believe it would be ethical to use this method without obtaining assent from providers. There are some examples of successful use of the methods in LMICs.[21] [22] However, if my premise is accepted that providers must assent to use of simulated patients, then it is necessary to first establish trust between providers and academic teams, and this takes time. Again, there is a high probability that only the better providers will provide assent, in which case observations would likely represent ‘upper bounds’ on quality.
In conclusion, I think that the basic tools of quality assessment, in the current situation where direct observation and/or simulated patients are not acceptable, is a combination of:
- Direct observation of facilities that exist, along with ease of access to them, and
- Debriefing of people who have recently used the health facilities, or who might have received preventive services that are not based in these facilities.
We do not think that the above mentioned shortcomings of these methods is a reason to eschew assessment of service quality in community settings (such as slums) in LMICs – after all, one of the most powerful levers to improvement is quantitative evidence of current care quality.[23] [24] The perfect should not be the enemy of the good. Moreover, if the anecdotes I have heard regarding care quality (providers who hand out only three types of pill – red, yellow and blue; doctors and nurses who do not turn up for work; prescription of antibiotics for clearly non-infectious conditions) are even partly true, then these methods would be more than sufficient to document standards and compare them across types of provider and different settings.
|
|
|
Which order of the mammals harbours the greatest number of zoonotic species?
Email CLAHRC WM your answer.
|
|
|
Answer to our previous quiz: Alexander Gordon was the doctor who identified puerperal fever as a "contagion" over half a century before Semmelweis. For more information you can read this account by Peter Dunn. Congratulations to Jo Sartori who was first to answer correctly.
|
|
|
Director's Choice - From the Journals
Patient Involvement in Patient Safety:
Null Result from a High Quality Study
Most patient safety evaluations are simple before and after / time series improvement studies. So it is always refreshing to find a study with contemporaneous controls. Lawton and her colleagues report a nice cluster randomized trial covering 33 hospital wards in five hospitals.[1] They evaluate a well-known patient safety intervention based on the idea of giving patients a more active role in monitoring safety on their ward.
The trial produced a null result, but some of the measures of safety were in the right direction and there was a correlation between the enthusiasm/fidelity with which the intervention was implemented and measures of safety.
Safety is hard to measure (as the authors state), and improvement often builds on a number of small incremental changes. So, it would be very nice to see this intervention replicated, possibly with measures to generate greater commitment from ward staff.
Here is the problem with patient safety research; on the one hand the subject of patient safety is full of hubristic claims made on the basis of insufficient (weak) evidence. On the other hand, high quality studies, such as the one reported here, often fail to find an effect. In many cases, as in the study reported here, there are reasons to suspect a type 2 error (false negative result). Beware also the rising tide - the phenomenon that arises where a trial occurs in the context of a strong secular trend - this trend 'swallows up' the headroom for a marginal intervention effect.[2] What is to be done? First, do not declare defeat too early. Second, be prepared to either carry out larger studies or replication studies that can be combined in a meta-analysis. Third, make multiple measurements across a causal chain [3] and synthesise this disparate data using Bayesian networks.[4] Fourth, further to the Bayesian approach, do not dichotomise results on the standard frequentist statistical convention into null and positive. It is stupid to classify a p-value of 0.06 as null if other evidence supports an effect, or to classify a p-value of 0.04 as positive if other data point the opposite way. Knowledge of complex areas, such as service interventions to improve safety, should take account of patterns in the data and information external to the index study. Bayesian networks provide a framework for such an analysis.[4] [5]
-- Richard Lilford, CLAHRC WM Director
Leave a comment
References
“We seek him here, we seek him there, Those Frenchies seek him everywhere.”
The notorious weekend mortality effect is every bit as elusive as the Scarlet Pimpernel. Recent studies have delved deeper into the possibility that the weekend effect is an artefact of admission of sicker patients at the weekend than on week days.[1] First, it has been shown that the mortality of all who present to the emergency department (i.e. admitted plus sent home) is the same over the weekend as over the rest of the week.[2] Second, patients who arrive by ambulance are generally much sicker than patients arriving by other means and the proportion who arrive by ambulance is higher over the weekend than over weekdays.[3] When controlling for method of arrival, most of the weekend effect disappears. Most, but not all. This paper provides further evidence that most estimates of the weekend effect are at least overestimates. Through Professor Julian Bion's HiSLAC Study [4] we are evaluating the effect of weekend admission, not just on mortality, but also on the quality of care and the overall adverse event rate. We will use a Bayesian network to synthesise information across the causal chain and come up with a refined estimate of the effect of weekend admission, not only on mortality, but also on other adverse events.
-- Richard Lilford, CLAHRC WM Director
Leave a comment
References
Patient and Public Involvement in Data Collection
Further to last fortnight's News blog article [1] I have found a further study in which patients participated in data collection.[2] This paper, by and large, corroborates the procedural requirements for public and patient involvement in data collection that I had specified. For example, it was necessary for lay observers to undergo DBS checks; the ethics approval form had to include lay observers; and training had to be arranged for the lay observers. Recruitment of lay observers proved more difficult than anticipated. The lay observers had a positive experience and brought a different perspective to the research according to feedback. The extent to which observer perspective is a good thing is, however, contestable. Generally I think the role of the observer is to collect data for analysis, and not colour it with a ‘perspective’. The professional researchers on the project felt that having lay researchers involved increased their workloads. The thorny issues of payment and selection do not seem to have been fully discussed in this paper. Also not discussed was the idea that, in qualitative research, respondents may be less inhibited to disclose information to a lay observer. Let the debate continue!
-- Richard Lilford, CLAHRC WM Director
Leave a comment
References
Alcohol and its Effects
News blog readers may be familiar with the famous ‘J curve’ relating alcohol consumption to health outcomes, including brain health.[1] The J curve shows a negative correlation between alcohol consumption and cognitive functioning at a low level of alcohol consumption (< 7 units/week), turning to a positive association in quantities exceeding about 28 units/week. One large glass of wine per day should be safe according to this finding. However, the data from which these findings are derived is cross-sectional. The BMJ has recently published a longitudinal study of alcohol and its effect on both cognition and brain structure (as measured by functional MRI).[2] The news is bad I am afraid. In the words of the editor, Fiona Godlee, ‘better’ research flattens the J curve.[3] The study seems to show a linear increase in risk with increasing intake of alcohol. The result was statistically significant for people drinking more than about two small glasses of wine per day. Why was a harmful effect at low dose detected in this longitudinal study but not the cross-sectional studies? So here is the thing – people with higher cognitive functioning tend to have higher alcohol consumption at baseline. In fact, the 'cleverer' the person, the more they tend to drink. The result is a difference in the findings of cross-sectional and longitudinal studies. While cross-sectional studies show no difference in cognition with moderate alcohol intake, the longitudinal studies show that cognition and brain structure decline at relatively low levels of alcohol consumption. To put this another way, moderate alcohol intake abolishes the cognitive advantage that moderate alcohol consumers have at baseline. Interestingly, not all parts of the brain are equally affected on MRI. Likewise the effect on cognition is not global; it affects lexical more than semantic fluency, for example. This is an extremely well-written, detailed and interesting study. The cohort of people who participated in the study were civil servants followed up for 30 years. The results are of immense public health importance. Human happiness, wealth and prosperity all relate to brain function. A person’s intellectual endowment is a precious gift and should not be lightly squandered. I will take these findings too heart, both in my personal life and as a public health practitioner. It is really a question of long-term loss vs. short-term gain - alcohol is a pleasant social lubricant, much beloved of myself, and a small glass of wine has even been shown to improve creative problem-solving![4]
-- Richard Lilford, CLAHRC WM Director
Leave a comment
References
Using the Internet for More Than Just Cat Pictures
The Internet can be a highly useful tool – communicating with old or distant friends, finding out the latest news, purchasing the latest best-seller, looking at photos of cats, etc. People also go online when they, or someone they know, is ill, searching for information or posting on social media. Your Internet search provider tracks all of this, and this data can be used by researchers to track outbreaks and the spread of infectious diseases. A recent paper by Yang and colleagues [1] demonstrated such a feat with regards to dengue fever.
Dengue is quickly becoming one of the most endemic mosquito-borne disease worldwide, infecting around 390 million people each year in 128 countries,[2] and placing the local health services under immense pressure. The Aedes mosquito that transmits dengue thrives in slums / shanty towns.[3] One of the ways to reduce infection rates is to improve early case detection – identifying outbreaks early means that preventive measures, such as mosquito population control, providing mosquito screens or nets, etc., can be undertaken. However, there is no current surveillance system for dengue that is comprehensive, effective and reliable – governments tend to use reports from hospitals that are often delayed and/or inaccurate.
Yang, et al. combined dengue-related Internet searches with historical incidence data to track dengue activity in five areas, Mexico, Brazil, Thailand, Singapore and Taiwan. They were able to successfully estimate dengue activity one month prior to the publication of official local health records, with their method outperforming benchmark models across accuracy metrics in all areas, except Taiwan. The authors note that Taiwan had little previous dengue prevalence on which to base predictions, suggesting the methodology works best in areas where dengue is already endemic.
Diet and Socioeconomic Status
People looking to lose weight and/or get healthy try a wide variety of diets, from fad diets with highly specific restrictions on what can be eaten, to general healthy eating plans. One such nutritional recommendation is the Mediterranean diet, based on the “food patterns typical of Crete… Greece and southern Italy…”,[1] and entails consumption of high amounts of plant foods (fruit, vegetables, cereals, legumes, etc.) and olive oil, moderate amounts of dairy, fish and wine, and low amounts of poultry and red meat. A number of observational studies have shown associations between such a diet and lower incidences of cardiovascular disease (CVD) and associated mortality, cancer, neuro-degenerative disorders, and overall mortality. However, there is uncertainty whether such benefits differ across different socioeconomic groups.
Bonaccio et al. carried out a prospective analysis of nearly 19,000 Italians to see the effect of the Mediterranean diet on CVD.[2] While there was an overall reduction in CVD risk associated with adherence to the diet (HR=0.85, 95% CI 0.73-0.99), this was not seen across all socioeconomic groups – only in those who were educated to a postgraduate or higher level (HR=0.43, 0.25-0.72) and in those with a high (>€40,000) household income (HR=0.39, 0.23-0.66). Those with less education (HR=0.94, 0.78-1.14) and lower income (HR=1.01, 0.79-1.29) had no significant association. Why such a difference? Subgroup analysis of people with similar adherence to the diet showed that there were a number of differences in the diet of those with high compared to low education, and those with high compared to low income. These included consumption of organic vegetables (which would have higher antioxidants and lower levels of pesticides), monounsaturated fatty acids (found in avocado, nuts, olives, etc.), micronutrients, and whole-grain bread, as well as greater dietary diversity.
So perhaps it is more important to make sure the food you are eating is of high quality and varied, than just simple healthy eating. Of course, access to high quality food of high nutritional value is not easy for poor people.
|
|
|
Sally Bradshaw
Mrs Sally Bradshaw is a Research Fellow within the Maternity and Child Health theme, where she is currently focussing on developing support for parents when their child has a life-changing medical diagnosis.
Sally originally trained as a nurse at The University of Leeds University, followed by attaining a first class Bachelor’s degree in midwifery at St Martin’s College in Carlisle (awarded by the University of Lancaster) and a Master’s degree in Public Health at the London School of Hygiene and Tropical Medicine. She has worked in a variety of roles in nursing, midwifery and public health in Leeds, London, the West Midlands and Thames Valley.
Early experiences in nursing and midwifery convinced Sally of the value of a public health approach to disease prevention, and an appetite for further study. Since working in public health she has also developed much more interest in how health and wellbeing can be created, rather than just how ill-health can be managed or avoided.
Following the completion of her Master’s degree in Public Health in 2008 Sally joined the Public Health Higher Speciality Training scheme, the NHS training scheme for consultants in public health. After successful completion and gaining fellowship of the Faculty of Public Health Sally joined the CLAHRC WM team at the University of Birmingham. Here she is able integrate the insights, knowledge and skills gained through her broad clinical and academic experience.
Sally is now leading a CLAHRC WM funded research project, in collaboration with Birmingham Children’s Hospital, to develop the support the hospital provides to parents of children with life-changing long term conditions which will also be submitted as a PhD entitled: “How can parents whose child has a life-changing admission to hospital be supported by the hospital in their parenting role?” This is a mixed methods study which will include primary qualitative research and the development and piloting of a parenting support intervention to be delivered within Birmingham Children’s Hospital.
Sally is particularly interested in working with health care organisations to help them improve the health and wellbeing of service users and the wider public, and to contribute more widely to public health and wellbeing.
Return to top
|
|
|
Re: Cognitive Behavioural Therapy vs. Mindfulness Therapy
Interesting line of argument.
Can’t see what is wrong with letting individuals chose for themselves. Both are easy to try online and the CBT will appeal to those who lead with their cognitive processing, whereas mindfulness addresses both emotions and bodily symptoms of thoughts and emotions as well.
In my understanding attitudes and beliefs that are ‘banned’ either by self or society still impact behaviour and decision making through unconscious bias. Allowing unacceptable thoughts as in mindfulness enables them to be examined and the underpinning issues bought to the fore. It also trains the person to place their focus where they chose leaving them better equipped to focus on the positive than the negative. Ultimately with mindful practice they leave the subconscious as well as the conscious – so a much deeper process.
-- Sarah Stewart-Brown
CLAHRC WM Director's Reply: So a person can train their subconscious mind. I think that is probably true. There are two questions – is there evidence that we are right? Is there evidence that mindfulness is more effective than CBT in this respect? Mind you, I did like Swaran Singh's recent comment to me – "CBT is the scientification of common sense!"
-- Richard Lilford
Re: Patient and Public Involvement
You might be interested in this paper we published addressing some of these issues - Mockford, et al. Research Involve Engage. 2016; 2: 8.
-- Kate Seers
Return to top
|
|
|
|
|
|
|
|