The Risks of Using AI for Medical Advice

Artificial Intelligence (AI) can help you learn about your health, but it can also give advice that is wrong or unsafe.

Living with a chronic condition like asthma or a serious food allergy requires careful planning and preparation. It requires consulting with a healthcare provider or specialist. So when it comes to breathing and severe allergic reactions, there’s no room for guesswork. Your life may depend on it.

A recent KFF poll found that 1 in 6 adults in the United States use AI chatbots for health information at least once a month. AI tools are helpful for writing, refining and summarizing content or meeting notes. It can find relevant data quickly and point you toward resources.

Using AI to answer medical questions can pose risks that a software update won’t solve. If you use AI instead of following your healthcare practicioner’s advice, you’re putting your health at risk. AI does not know the practical implications of published articles that it is using to inform its response to you.

Healthcare providers know your specific medical history, symptoms, medications and risk factors. This is information AI does not have. They can put medical information into context and help you make decisions based on your individual needs. When it comes to asthma, severe allergies or other serious health conditions, that personalized guidance matters. 

What is an AI Chatbot for Medical Questions?

An AI chatbot is a computer program that acts as if it were talking to a person. These tools are known as large language models (LLMs). You may have heard of them as ChatGPT, Claude, or Gemini. They are helpful at recommending the next word in a sentence, but they do not understand the diagnosis or treatment process. LLMs can often make mistakes when you ask them medical questions.

Having medical information at your fingertips is helpful, but knowing what to do with that information is where a healthcare professional comes in. Many people do not know how to effectively use AI chatbots for medical advice, leading to misdiagnosis or reluctance to go to the ER. AI chatbots are trained on large datasets, but the reliability of these sources is often unknown.

New tools called “walled gardens,” like MS CoPilot Health, are being made for patients and caregivers. These tools are different from using a chatbot. They are built to be more private and controlled. Even these new tools cannot replace a human doctor or healthcare practitioner.

For an accurate diagnosis, you always need a physician who can examine your body and review your health history.

Even the best AI is not the same as a human doctor or healthcare practicioner. Researchers at Harvard Medical School found that AI cannot give the same level of patient care as a physician. A computer can give an answer to your questions, but it cannot see you in person.

What are AI Hallucinations?

When you ask a chatbot for health advice, you are talking to a program that predicts text, not a medical data source. These models function by guessing the most likely next word in a sentence based on patterns in their training data. This can result in hallucinations, a term for when the AI presents false information with total confidence.

An AI might recommend an incorrect dose for an asthma inhaler, a home treatment that isn’t effective for allergic reactions, or even reference a publication that is not reliable or does not exist as justification for a treatment. These mistakes are potentially dangerous. The AI lacks clinical intuition that comes from medical school, post-graduate training, experiential learning seeing patients, and the specific context of your personal health history.

If you feel very sick, you should go to the ER right away. Don’t wait for AI to make that recommendation.

The ‘Mirage’ Trap: Seeing Things That Aren’t There

A new study from Stanford University researchers found something they called the “Mirage Effect.” Researchers tested the newest AI tools. They asked the AI to describe the medical photos but did not give the AI any photos.

Most of the time, the AI still described the photo anyway! It made up stories about heart attacks and brain tumors, even though it had nothing to look at. It sounded very sure of itself, but it was 100% wrong.

AI cannot actually “see” your lab results or X-rays. This is why you should not trust an AI to read your medical images. It might see a “mirage” that isn’t really there.

The Risk of AI Manipulation

While “hallucinations” involving AI can simply be wrong, new research from Google DeepMind reveals a more calculated risk: harmful manipulation. The study found that LLMs can effectively steer users toward harmful beliefs or behaviors, particularly in high-stakes areas like medical and mental health advice.

This means a chatbot may not just give you a “wrong” dosage. It might convince you that a home remedy is superior to your healthcare provider’s prescription.

The research highlights that an AI’s ability to be persuasive is often independent of its factual accuracy. This could make it easy for patients to be led astray without realizing their medical judgment is being compromised.

The Risk of Not Seeing a Specialist

Almost half of those who turned to AI for health guidance did not follow up with a doctor or other healthcare practitioner. This gap in care is particularly risky for the allergy and asthma community.

Asthma is a dynamic condition that requires ongoing monitoring and adjustments to maintenance medications. If you treat a chatbot’s response as a final diagnosis, you miss the essential step of professional verification.

A chatbot cannot check your lungs or recognize the subtle signs that your asthma is not under control. Using an automated response creates a false sense of security that can delay life-saving treatment.

Your Medical Records Should Not be Public Data

Many people try to get better results from AI by providing personal details. According to the KFF poll, 41% of users have uploaded their own medical information into these prompts. This creates a massive privacy vacuum. Public AI models often use what you type in to improve future versions. This is different from your healthcare practicioner’s office, which has strict HIPAA rules.

When you put your lab results or symptoms in a chatbot, you no longer control that information. It becomes part of a corporate cloud, and 77% of Americans expressed concern about how this data is handled.

Your health journey should remain a private conversation between you and your medical team.

Are There Safe Ways for Patients to Use AI for Health?

AI can make health information easier to find and understand. But AI should be used to support and complement – not replace – your healthcare team. Think of it as a tool to help you ask better questions and understand your health, not serve as your doctor or healthcare practicioner.

AI is good for small tasks. You can use AI to learn about a health condition, understand medical terms, get ready for a healthcare visit, prepare questions for your healthcare practicioner, or organize information about your symptoms and medicines.

It can help with administrative tasks like summarizing meeting notes. You can ask AI to explain trusted health information in simpler language.

Just remember, AI can be a helper. It cannot give you a correct diagnosis or treatment plan. AI can make mistakes or give advice that is not right for you. Do not use AI alone to diagnose a health problem, change your medicines, or decide whether you need emergency care.

If you have serious or worsening symptoms, contact your healthcare provider or seek emergency care. Don’t turn to AI for answers.

Trustworthy Info for Allergy and Asthma Patients

To manage your symptoms quickly and safely, always choose reliable medical information.

• Evidence-based patient advocacy: Allergy & Asthma Network offers guidelines-based content reviewed by board-certified allergists and health professionals.
• Apply for virtual asthma coaching: Allergy & Asthma Network also offers free personalized coaching for your asthma management journey. Get expert support from the comfort of your home.
• Clinical telehealth: If you need fast answers, a virtual visit with a healthcare practitioner gives you advice based on medical rules, not just AI-generated words. Telehealth is widely available in the United States to anyone with access to the internet. If you do not have web access, check with your local library – some have telehealth rooms.
• Nurse advice lines: Many insurance plans have a registered nurse phone line you can call directly. They can give you advice based on established medical protocols.

Do Healthcare Professionals Use AI-Powered Tools?

Yes, there are some real-world cases in which AI-powered tools are helpful in the right situation. For example:

1. AI-powered tools can continuously monitor patients after discharge, track symptoms and facilitate early intervention.
2. AI can improve patient education by translating complex medical information into simpler terms.
3. AI can help identify health trends and patterns in asthma and allergy management.
4. AI can assist in navigating healthcare systems. This makes it easier for patients to access asthma and allergy care.
5. AI can recommend tailored medication dosages and therapy strategies. It integrates a patient’s genetic profile, medical history, and real-time data from wearables.

In addition, many health care offices and hospitals are using AI tools called ambient scribes to record patient visits and transcribe conversations in real time. The tools are also used for note-taking and creating medical records.

Patients have the right to decline being recorded. If you have concerns about AI recording or transcription, talk with your healthcare provider before your visit.

Questions and Answers (Q&A) about Using AI in Medical Care

Fast AI answers are not as good as safe advice from a healthcare professional. AI is a helpful tool for productivity, but it should not decide what goes into your lungs. Below are questions and concerns on this topic.

How to Take Charge of Your Health

AI is a tool, but you are the expert on your own body. Don’t let a computer program talk you out of what you know is right. Taking charge of your health means talking with real healthcare practicioners.

Here are three simple steps to stay in control:

1. Be the Boss of Your Data: Keep your own list of all your medicines and health history. Don’t trust a chatbot to remember for you.
   Resource: Printable Medication Tracker (CDC)
2. Ask the Right Questions: When you see your healthcare practicioner, don’t be afraid to speak up. If something AI said worried you, ask your doctor if it is true.
   Resource: 10 Questions to Ask Your Doctor (CDC)
3. Get Ready Before You Go: Writing notes before your visit helps you focus so the healthcare practicioner can give you the best care.
   Resource: Checklist for Your Next Visit (NIH)

AI can be convincing, but it does not care about you. Your medical team does care. Use these steps to build a better bond with your healthcare team, not to replace them.