Skip Navigation Bar
NLM logo

NLM Announces Pill Image Recognition Challenge Winners - Video Transcript

August 3, 2016

Michael J. Ackerman
Pill Image Recognition Challenge video for PIR web site

The need for a pill recognition tool is more important today than ever before. Consider the following facts from a 2015 Journal of the American Medical Association published study:

More Americans than ever are taking prescription drugs and using more of them.

Almost 60% of adults used prescription drugs in 2012, up from 50% in 2000. And, 15% of them took 5 or more prescription drugs – up from 10% from the earlier period.

 27% of adults in 2012 were taking high blood pressure drugs, up from 20% from a decade earlier.

 Statin use has more than doubled as well – from 7% to 17%.

 Use of antidepressants has nearly doubled, increasing from 7% to 13%.

 And, all of these increases were seen only in people aged 40 and older, not in younger aged populations.

For older Americans and their caregivers, there are numerous instances when a pill recognition tool would be indispensible.

I’ve gone to older folks and they say things like, ‘get me the pill, and I’ll take the pill,’ and you go and you say, ‘I can’t find it,’

and they respond, "Oh, did I put it in the aspirin bottle or in the vitamin bottle?’

Now, if they can’t remember which bottle they put it in, what confidence do I have that I’m giving them the right pill?

For those in the aftermath of a disaster, the value of a pill recognition tool to assist in identifying pills grabbed during evacuation can be a literal lifesaver.

Pill containers without the original prescription bottle can leave consumers bewildered about the name of the drugs they are regularly taking.

They may be able to tell you that they are “running out of these yellow pills,“ or say,

“I don’t know the name of the pill but it is the yellow one and I take it every day at lunch.”

With this pill recognition tool, the doctor can take a picture, and bingo, he knows what the pill is.

We learned that each prescribed pill is unique.

We noticed when we take pictures of the pills, that depending on what background the pill was on, and depending on the light in the room and which camera we used, the pill’s color changed.

What if you take a picture of a yellow pill with a camera and it looks green?

The first thing we realized is the color changes.

Then we realized that a lot of pills have engravings.

The reason you see the engraving is the shadows, because there is no ink there.

If you get right on top of it and look straight down, it’s very hard to see the shadow, therefore the engraving.

But if you move to the side a little bit so you get a shadow, now all of a sudden you change the geometry so the round pill is slightly oval. So now the yellow round pill with an inscription looks green and oval but the inscription is still the same.

So for a machine to recognize the pill by its unique qualities when its unique qualities are changed in the picture is the challenge.


The National Library of Medicine (NLM) is a leader in research in biomedical informatics and data science and the world’s largest biomedical library. NLM conducts and supports research in methods for recording, storing, retrieving, preserving, and communicating health information. NLM creates resources and tools that are used billions of times each year by millions of people to access and analyze molecular biology, biotechnology, toxicology, environmental health, and health services information. Additional information is available at

Last Reviewed: August 15, 2016