HILJ Club: This edition of HILJ club has been prepared by: Tom Roper, Clinical Librarian, General Surgery and Digestive Diseases, Urology, Acute and Emergency Medicine, Critical Care, Trauma and Orthopaedics, Brighton and Sussex NHS Library and Knowledge Service @tomroper
HILJ Club is a simple way for people to do CPD around with articles published in Health Information and Libraries Journal (HILJ). HILJ is the journal of the Health Libraries Group of CILIP and all members have access. Working with Wiley, the article under discussion is made freely available for a month around the discussion.
We select an article for discussion from each issue of HILJ. The organising group (Alan Fricker, Catherine McLaren, Morag Clarkson, Lisa Burscheidt, Tom Roper) have picked articles thus far but in future people would be welcome to volunteer to host. We now have a permanent home on this WordPress site. Generally the host will offer some questions as prompts and then the discussion can go where interest takes it.
The paper for discussion is Ayiku L, Levay P, Hudson T, Craven J, Finnegan A, Adams R, Barrett E. The Embase UK filter: validation of a geographic search filter to retrieve research about the UK from OVID Embase. Health Info Libr J. 2019 Jun;36(2):121-133. doi:10.1111/hir.12252.
Background
The authors developed a validated geographic search filter to retrieve research about the United Kingdom (UK) from OVID Embase. It was created to be used alongside their previously published OVID MEDLINE UK filter in systematic literature searches for context‐sensitive topics.
Objectives
To develop a validated geographic search filter to retrieve research about the UK from OVID Embase.
Methods
The Embase UK filter was translated from the MEDLINE UK filter. A gold standard set of references was generated using the relative recall method. The set contained references to publications about the UK that had informed National Institute for Health and Care Excellence (NICE) guidance and it was used to validate the filter. Recall, precision and number‐needed‐to‐read (NNR) were calculated using a case study.
Results
The validated Embase UK filter demonstrated 99.8% recall against the references with UK identifiers in the gold standard set. In the case study, the Embase UK filter demonstrated 98.5% recall, 7.6% precision and a NNR of 13.
Conclusion
The Embase UK filter can be used alongside the MEDLINE UK filter. The filters have the potential to save time and associated resource costs when they are used for context‐sensitive topics that require research about UK settings.
Questions for discussion
What? What do you think of this article? What do you think of the research methods? Is there something else that you would have liked to have seen included in the article?
So what? Would you use this filter in practice? Would you use the same methods to develop filters for other searches?
Now what? Will you change your practice as a result of reading this article? If so, how? If not, why not?
What? This is a carefully conducted piece of work, but it’s a little hard to understand the methods for search filter development without referring to the authors’ previous work on a MEDLINE filter. This filter is a translation of the MEDLINE one but importantly, is independently validated.
The authors outline the need for UK specific filters and explain the development of the EMBASE one, the creation of a Gold Standard set against which to test the filter, and the calculation of recall, precision and NNR (Number Needed to Read). It is interesting that 38 of the 599 papers in the Gold standard set were not retrievable as UK papers in EMBASE, as they contained no UK identifiers in searchable fields. Does this matter?
The authors found that the both the Gold Standard and the filter missed two papers (one from each) which had a UK identifier field in the record. In both cases this was due to an Embase indexing error: one paper had been given a United States geographic term, although it was set in Manchester, the other had been assigned the terms Bangladesh and Central America, as it discussed populations that, generations ago, originated from those parts of the world. The authors warn that it is impossible to know how many other Embase records are similarly affected
So what? I have not yet used this filter in any of my searches, though it might be more useful for those conducting searches for public health departments. I consider it to be well constructed and reliable, so would certainly experiment with it under the right circumstances
Now what? It would be helpful if this filter, and the MEDLINE one, could be made available as a filter option in the various search interfaces we use. The incorrect indexing is alarming, and I wonder if the quality of database indexing needs attention, particularly as database providers seem to be automating their processes to greater or lesser extents.
Please join the discussion and add comments below
HILJ Club 
Thanks for kicking off Tom.
I think a big thing for me was that the same indexing errors did not exist for those records in Medline. This makes a good case for searching both databases. There will always be errors in indexing so searching both provides cover against this (provided they don’t just borrow from each others indexing!)
I wondered about how the translation done by the authors might compare to that done by an automated system. (https://medlinetranspose.github.io/documentation.html only works from Ovid Medline to PubMed etc but suspect there must be a translation tool about). I wondered about the impacting of binning lines 10-11 since these are the ones that knock out the poorly indexed articles – I guess it would put the precision down the plughole?
We did a search around geographic filters that started from the LMIC ones (https://epoc.cochrane.org/lmic-filters) but these are not validated.
Accepting the authors arguments about the testing they did of other geographic terms I still wondered about the names of counties given their link to NHS structures. This might be more the case for searches in service management related questions?
Certainly this seems like work well worth doing – the time savings are substantial for those cases where it is an applicable filter. I would be glad to have it include as an option available to me
LikeLike
A comment from Tricia Rey, Library Services Manager at Queen Victoria Hospital, East Grinstead (the web page that hosts HILJ Club appears very poorly formatted using her Trust network):
You mention the issue of indexing of articles in the databases. For me, there is a big issue with regard to the indexing of articles in Embase. The indexing is now carried out electronically and this leads to misindexing caused by synonyms (and also, presumably, the exclusion and inclusion of country names, etc).
For example, the subject heading of TABLET/. This has been problematic for a while as handheld computers have been being indexed under TABLET/ which is intended to be used for pills, capsules, etc. Embase have created a new subject heading of COMPUTER TABLET/ which should deal with this issue but I find that TABLET/ is still being used incorrectly.
In addition, *LANGUAGE DEVELOPMENT/ appears as an indexing term in the record above. I’m not sure that developing an app is what is meant by LANGUAGE DEVELOPMENT/.
I wish that the EMBASE publishers could be brought to see how this incorrect indexing devalues their database for expert searchers.
Slightly off the above topic, I understood that Embase Scope Notes were going to improve following the loading of a medical dictionary, or something similar? I find their Scope notes worse than useless. They frequently just give Used for terms which appear further down the page, anyway, and give no additional help. Looking at the Scope note for TABLET/ just gives the information:
Used For Terms
• Tablets
So still no guidance on what sort of tablet.
I’d be interested in your comments – whether you see this as an issue when searching Embase.
LikeLike
I was curious about the Automatic indexing comment. I had a dig and the EMBASE Indexing policy is here https://www.elsevier.com/__data/assets/pdf_file/0016/92104/Embase-Indexing-Guide.pdf
While there is automated indexing section 4.5 indicates:
“Automatic indexing for selected articles was introduced in 2009 for three types of article:
• Conference abstracts
• Articles in Press
• In-Process records”
And the document states in
“3.2. Process
With the exception of articles designated for automatic indexing (see Section 4.5), indexing for Embase is a manual process performed by trained indexers with a biomedical background.”
So is the potential for machine error but still likely to be human error in the mix too!
LikeLike
One of the concerns that many NHS librarians might potentially have with this article, is that it assumes the use of the OVID version of Medline and Embase databases, when NHS libraries are generally (although not exclusively) provided with the HDAS interface. Since HDAS doesn’t have the same level of functionality as OVID, the filters are less likely to be applicable to NHS librarian’s practice.
LikeLike
I guess the .in field would be an issue for HDAS? And perhaps the very long search set?
LikeLike
Sorry, I don’t mean to interject but I thought I it would be helpful to address the HDAS queries above. We created the UK filters for the OVID platform because this is the interface that we use for systematic literature searches. However, the folks at Shrewsbury and Telford Health Libraries took their own initiative (no involvement from us) to have a go at translating the Medline UK filter for HDAS and they’ve done a pretty good job (although, I’d suggest adding UK cities in too to maximise recall!): https://www.library.sath.nhs.uk/find/searching-for-articles-in-medline-that-are-about-the-uk/
I’m scuttling away again now – feel free to continue the discussion!
LikeLike
Interjections are most welcome, Lynda, thanks!
LikeLike
Very happy to have contributions from the authors – please do promote the discussion and we can try and drive your altmetrics scores up!
LikeLike
Thanks for the link Lynda! Most useful.
I have had difficulty getting the original filter to work in Ovid, and of course dear old HDAS just keels over – so this is a welcome alternative.
LikeLike
Hi Erica, please email me (address is in the Embase UK filter article) and I’ll send the OVID filters to you in a Word document (the search lines can be copied/pasted straight into MEDLINE/Embase from the Word doc).
LikeLike
Interesting article on UK filter, not something I currently need to use, so it was just a quick skim for me – as am still always interested in these kind of developments.
I’m here via Alan’s tweet about your joint editorial piece, and I really wanted to leave a comment to support the excellent work you’re doing with this journal club. I’m in!
LikeLiked by 1 person