article

Skin cancer diagnosis apps are unreliable and poorly regulated, study shows

Tuesday, 11 February 2020

Smartphone apps used as ‘early warning systems’ for skin cancer are poorly regulated and frequently cannot be relied upon to produce accurate results, according to experts.

Skin cancer detection apps are designed to ensure that the right people seek medical attention by providing a risk assessment of a new or changing mole. These apps use specialised algorithms to try to detect possible skin cancers.

Researchers based in the University of Birmingham’s Institute of Applied Health Research in collaboration with the Centre of Evidence-Based Dermatology at the University of Nottingham have analysed a series of studies produced to evaluate the accuracy of six different apps. Their results, published in The BMJ, reveal a mixed picture, with only a small number of studies showing variable and unreliable test accuracy among the apps evaluated.

Skin cancer has one of the highest global incidences of any cancer. Early detection and treatment, particularly of melanoma, can improve survival. According to this new analysis, however, apps may cause harm from failure to identify potentially fatal skin cancers, or from over-investigation of false positive results – for example removing a harmless mole unnecessarily.

The research team was also concerned by the quality of the studies themselves, which evaluated apps using images taken by experts rather than by app users. In addition, many studies did not identify whether lesions identified as ‘low risk’ by the apps were in fact benign, further compromising the conclusions that can be drawn from the evaluations.

Although I was broad minded on the potential benefit of apps for diagnosing skin cancer, I am now worried given the results of our study and the overall poor quality of studies used to test these apps. My advice to anyone worried about a possible skin cancer is “if in doubt, check it out with your GP."
Hywel Williams, Professor of Dermatology at the University of Nottingham

Lead researcher Dr Jac Dinnes said: “This is a fast-moving field and it’s really disappointing that there is not better quality evidence available to judge the efficacy of these apps. It is vital that healthcare professionals are aware of the current limitations both in the technologies and in their evaluations.”

The authors also drew attention to the regulations governing the evaluation of healthcare apps. Manufacturers can currently apply CE marks to smartphone apps without necessarily being subject to independent inspection by bodies such as the UK Medicines and Healthcare Products Regulatory Agency (MHRA). Although this may change with new Medical Device Regulations coming into force in 2020, the researchers noted that stricter assessment processes in operation in the US has resulted in no skin cancer assessment apps receiving regulatory approval.

Co-author Jon Deeks, Professor of Biostatistics in the Institute of Applied Health Research, adds: “Regulators need to become alert to the potential harm that poorly performing algorithm-based diagnostic or risk monitoring apps create. We rely on the CE mark as a sign of quality, but the current CE mark assessment processes are not fit for protecting the public against the risks that these apps present.”

Dr Dinnes added: “As technologies continue to develop, these types of apps are likely to attract increasing attention for skin cancer diagnosis, so it’s really important that they are properly evaluated and regulated. Of course, we also need to emphasise how important it is to go and see your GP if you have concerns – regardless of what an app might tell you.”

The team also has made a series of recommendations for future studies of smartphone apps:

  • Studies must be based on clinically-relevant population of smartphone users who may have concerns about their risk of skin cancer
  • All skin lesions identified by smartphone users must be included – not just those identified as potentially problematic
  • Clinical follow-up of benign lesions must be included in the study to provide more reliable and generalizable results.

The research was supported by the NIHR Birmingham Biomedical Research Centre and is an update of one of a collection of reviews funded by the National Institute for Health Research through its Cochrane Systematic Review Programme Grant.

Story credits

More information is available from Beck Lockwood, in the Press Office at the University of Birmingham, on +44 (0)121 414 2772 Or at r.lockwood@bham.ac.uk

 

Charlotte Anscombe - Media Relations Manager - Faculty of Medicine and Health Sciences
Email: charlotte.anscombe@nottingham.ac.uk
Phone: 0115 748 4417
Location:

Notes to editors:

The University of Nottingham

Our academics can now be interviewed for broadcast via our Media Hub, which offers a Quicklink fixed camera and ISDN line facilities at Jubilee campus. For further information please contact a member of the Press Office on +44 (0)115 951 5798, email pressoffice@nottingham.ac.uk

For up to the minute media alerts, follow us on Twitter

The University of Nottingham is a research-intensive university with a proud heritage. Studying at the University of Nottingham is a life-changing experience, and we pride ourselves on unlocking the potential of our students. We have a pioneering spirit, expressed in the vision of our founder Sir Jesse Boot, which has seen us lead the way in establishing campuses in China and Malaysia - part of a globally connected network of education, research and industrial engagement. Ranked 18th in the UK by the QS World University Rankings 2023, the University’s state-of-the-art facilities and inclusive and disability sport provision is reflected in its crowning as The Times and Sunday Times Good University Guide Sports University of the Year twice in three years, most recently in 2021. We are ranked seventh for research power in the UK according to REF 2021. We have six beacons of research excellence helping to transform lives and change the world; we are also a major employer and industry partner - locally and globally. Alongside Nottingham Trent University, we lead the Universities for Nottingham initiative, a pioneering collaboration which brings together the combined strength and civic missions of Nottingham’s two world-class universities and is working with local communities and partners to aid recovery and renewal following the COVID-19 pandemic.

More news…

Media Relations - External Relations

The University of Nottingham
C Floor, Pope Building (Room C4)
University Park
Nottingham, NG7 2RD

telephone: +44 (0) 115 951 5798
email: pressoffice@nottingham.ac.uk