Medical imaging takes pictures inside the body. Most medical imaging requires bulky and expensive hardware: refrigerator-sized X-ray machines or multi-million-dollar CT and MRI scanners. Ultrasound is different: it is harmless with no ionizing radiation, and images can be obtained using portable handheld probes which now cost just a few thousand dollars and are nearly the size of smartphones (e.g., Exo, Lumify, Clarius, Butterfly).

Unfortunately, because it uses sound waves which obey different rules than visible light, ultrasound images are confusing. For inexperienced users it is difficult to know how to take high quality images and how to interpret them. My research investigates how artificial intelligence (AI) can help.

With proper guidance, AI can learn from the experience of experts on thousands of previous cases to identify anatomic structures and pathology in ultrasound images. AI can be used to inform users when they have obtained diagnostic-quality images, and to suggest diagnoses.

My team is evaluating the combination of ultrasound+AI in various clinical problems to see which ones it can be most helpful in. Some US+AI solutions can save lives, such as 'basic echo' heart ultrasound to assess the cardiac ejection fraction, or lung ultrasound+AI for conditions such as heart failure and Covid19. Others can be commercialized; we now have US-FDA approval for a tool to detect hip dysplasia and another to assess for thyroid cancer.

Portable ultrasound enhanced by AI is a natural way to extend the reach of ultrasound, transforming medical imaging. This could eventually be the 21st-century stethoscope!

Hip dysplasia affects 1-3% of all infants born, and leads to dislocated hips in severe cases and premature osteoarthritis in milder cases if missed. Current screening for hip dysplasia uses conventional 2D ultrasound, which is unreliable because the limited view of the hip it shows depends highly on the skill of the sonographer.

Since 2012 my research team has been performing 3D ultrasound scans of hundreds of infants suspected of having dysplastic hips. 3D ultrasound provides a much more complete view of the hip, which ought to lead to more reliable diagnosis of hip dysplasia. We are developing visual and quantitative ways to make this diagnosis from 3D data, as shown in this video.

In 2015 I co-founded the Collaborative for Ultrasound Deep Learning (CUDL). This was a multidisciplinary, multi-national team of researchers and clinicians who share a vision that advanced machine learning techniques can be used to analyze uploaded ultrasound images (2D or 3D) to help clinicians optimize diagnosis and management of medical problems ranging from hip dysplasia to soft tissue tumors, cardiac and atherosclerotic disease, and other musculoskeletal and solid organ diseases. I describe CUDL, which is partly funded by a grant from the Canadian Medical Association Joule initiative, in this short video. CUDL has transformed into NIDUS.

Our work on implementing hip dysplasia ultrasound/AI is becoming global in scale, from a collaboration in Canada with the Arthritis Society to emerging pilot projects in countries including Scotland, Ecuador, Australia and Ethiopia.

Many forms of arthritis, previously thought to be inevitable consequences of ageing and untreatable, are increasingly recognized to be at least partly due to treatable inflammation. To decide in whom a treatment is likely to be successful, to test whether a new therapy is effective, and to understand the natural history of each form of arthritis, it is important to have measures of disease that are more objective than patient-reported pain. MRI allows reliable tracking of disease status and changes over time, especially when a semi-quantitative scoring system is applied by a trained user.

I work with Dr. Walter Maksymowych, a rheumatologist, and Dr. Rob Lambert, a radiologist, on initiatives applying a modern web-based approach to objective semi-quantitative grading of arthritis. We have applied the approach to spondyloarthritis (SpA), osteoarthritis of the hip (see video of HIMRISS) and knee (see video of KIMRISS), and are investigating its use in juvenile inflammatory osteoarthritis (JIA). Our current focus is on automating these tools, to bring objective assessment of arthritis to the millions of MRI of joints performed worldwide each year.

Tools and further information are available at www.carearthritis.com.