First photorealistic AI avatar for people living with Alzheimer’s and dementia launched by Lenovo and Innovations in Dementia

Alzheimer’s Intelligence project demonstrates how smarter AI can empower people and give different communities a voice around future innovation

Last October 15 at Lenovo Tech World, Lenovo™ in collaboration with Innovations in Dementia, launched Alzheimer’s Intelligence, a photorealistic 3D avatar with custom AI based on the lived experiences of people with dementia and Alzheimer’s. This proof-of-concept project gives people and families navigating a diagnosis of dementia 24-hour access to a conversational avatar that offers curated advice that prioritizes accuracy, privacy, and compassion.

 

“Globally there are around 55 million people living with dementia, and that number is expected to reach 139 million by 2050,” said Dr. David Llewellyn, Professor of Clinical Epidemiology and Digital Health at the University of Exeter. “The AI advancements in Lenovo’s ‘Alzheimer’s Intelligence’ initiative demonstrate how technology has the potential to enhance human interaction, offering a valuable tool to support individuals living with dementia, both in the present and future.”

 

In this first-of-its-kind application, Lenovo pioneered the use of AI made possible by its comprehensive portfolio of technology solutions, from individual to enterprise. The combined technology aggregates firsthand experience and advice from hundreds of real people living with dementia and Alzheimer’s into a responsive, photoreal 3D avatar capable of having an unscripted, natural conversation. With this proof of concept, a person diagnosed with dementia is just a click away from a real-time conversational resource and aid.

 

“With recent and ever-growing advancements in AI, we must harness this technology purposefully and responsibly for communities that may be overlooked by innovative technology,” said Doug Fisher, Chief Security and AI Officer at Lenovo. “This is part of our commitment to delivering Smarter AI for All—developing and deploying technology that can empower everyone at every scale.”

 

Watch the video here: https://youtu.be/9_P9ZyRil1M

 

The avatar was created from a composite of images of 10 people living with dementia and Alzheimer’s. From the initial pose of each image, generative AI extrapolated faces for several thousand other expressions and angles inspired by the subject. These were then aggregated into one dataset, which blends with the face of a filmed performer to create the image of “Liv,” the AI avatar.

 

A large language model (LLM) dataset of the advice Liv can impart was created using data that came directly from experiences of people living with dementia, including entries from Innovations in Dementia’s Dementia Diaries project and in-depth panel interviews. Each time a user asks a question (via Speech-to-text), the LLM queries the dataset to provide text-based answers to the user’s question, expressed using the language of the “persona” created for Liv, and based on the knowledge base built for Liv. The replies are then voiced using a vocal synth created for the project. “Sentiment analysis” is used to analyse the underlying feeling of the reply, which is reflected in the facial expression of the AI when it speaks. Finally, real time 4k visual AI is used to allow the speech generated by the vocal synth to appear to be spoken in real time.

 

Alzheimer’s Intelligence will allow people who have recently been diagnosed with dementia to ask the questions that only someone who has lived with the disease can answer, including understanding day-to-day challenges, discussing the disease with family, and exploring new hobbies. Crucially, the entire experience was designed to be accessible, requiring minimal tech expertise and prioritizing natural, familiar interaction.

 

The avatar is currently being trialled by people with dementia in a controlled environment with the goal of making it more publicly available in the future. The making of the project and initial interactions have been captured in the films linked below.

The Alzheimer’s Intelligence project offers a new way to easily access valuable insight sourced from data based on real people living with dementia and Alzheimer’s. While the information is wide-ranging, the development team took great care to apply clear guardrails to the AI and ensure no medical advice is offered by the avatar. This included working closely with Dr. Llewellyn and Innovations in Dementia, who both advised and consulted on the avatar build and testing.

 

The AI was trained and tested for this specific use case, and the project was designed and brought to life with Lenovo devices, services, software, servers, and cloud solutions. This is the first time ever that Lenovo AI-ready solutions have come together to develop an avatar of this kind. Devices and solutions include:

 

• Lenovo’s tech partner Kaleida, who led the development of the avatar and underlying AI, leveraged Lenovo’s AI professional services to support the processes and security
• ThinkStation P7server, ThinkPad T14s Gen 6, and ThinkPad X1 Carbon Gen 12 AIPC laptops, as well as Lenovo Legion and ThinkVision monitors and accessories.

 

“When people are first diagnosed with dementia, they can feel lost, with so many questions about what is happening. This AI doesn’t replace human contact but what it does do is offer a knowledgeable and compassionate source of assurance and advice based directly on the experiences and wisdom of other people with dementia.” said Steve Milton, Director, Innovations in Dementia. “It’s great to see tech leaders like Lenovo prioritising such an innovative use of AI in a way that not only recognises and the expertise of people with dementia but harnesses it in such a unique way to support others when they need it most.”

 

Innovations in Dementia state that around 20% of people who come to them are under 65 and may live for many years with the condition. The NGO promotes a positive but realistic view of dementia: although it is life changing, it does not have to be life ending.

“We recognize the importance of giving everyone a voice in innovation,” said Doug Fisher at Lenovo. “With huge advancements in AI and smarter technologies, now is the time to explore and pilot creative new ways to use technology like AI thoughtfully and responsibly, for the greater good. Ultimately, it’s about re-imagining a future where technology is for—and from—all, giving everyone a voice in what they want from the most transformative technology of our generation.”   

 

Gail Gregory and Ahmad Krayem, focus group participants comment:

 

• “It’s so important to have real people involved in the creation of the avatar. Because the people with dementia are the experts. We’re the ones living with it. You can’t just grab things from a textbook because it’s not real. It must be real.  And the more realistic it is the more relatable it is. I think it is going to be a very useful tool, especially for people living on their own because they’ve got nobody to ask,” Gail Gregory.
• “I want to help other people who have been recently diagnosed with dementia and reassure them that it is not something to be afraid of. A doctor can tell you what medication to take.  But only people living with dementia, can tell you what they are exactly going through,” Ahmad Krayem.

 

The participants in this film include:

 

• Gail Gregory: Diagnosed with Alzheimer’s at age 55, Gail also subverts the commonly-held stereotypes of a person living with dementia. She can broach serious topics with humour, and is candid in her opinions and struggles.
• Ahmad Krayem: Ahmad was only diagnosed with early onset in his early 60s after insistence from his wife, who was a pharmacist, that he should go for a check-up. Ahmad found stopping work a big challenge and has been through a journey of denial and acceptance.  He talks passionately about helping others who’ve gone through this.

 

Alzheimer’s Intelligence is a proof-of-concept tool and is not publicly available, but for anyone looking to hear from people living with Alzheimer’s or dementia diagnosis, can contact Innovations in Dementia (UK) or Alzheimer’s Association (US).

 

Examples of interactions with the avatar:

 

Q: I keep forgetting to take my pills. How can I remember?

 

A: It’s quite common to forget things like taking medication. Perhaps setting a routine could help, like taking your pills with a meal or at the same time each day. You could also try setting an alarm or reminder on your phone. There are pill organisers, too, which can be useful. Would one of those options suit you, do you think?

 

Q: How do I talk about my diagnosis with friends and family?

 

A: Talking about your diagnosis with friends and family can be a daunting task, but it can also help to bring support and understanding. Start by sharing how you feel today and maybe a bit about what you’ve learned about your condition. Encourage them to ask any questions, as this can help them understand what you’re going through. It’s okay to be honest about any fears or concerns you have, as well. Do you think that might be helpful?