As AI enters exam rooms, states step up oversight

A bipartisan group of Pennsylvania state legislators recently hatched a plan to regulate the use of artificial intelligence in health care.

Four Pennsylvania House Democrats and one House Republican plan to introduce legislation that would require insurers, hospitals and other providers to follow certain rules when using AI for patient care, billing and coding, claims processing and other health-related services.

"As the only physician in the [Pennsylvania] General Assembly, I have seen the rapid growth of AI usage in health care in real time," said Democratic Pennsylvania state Rep. Arvind Venkat in a statement announcing the legislation. Venkat is an emergency medicine physician in Pittsburgh.

Venkat said that while AI has increased efficiency in administrative tasks for physicians like himself, he's concerned about its growing use in decision-making about patient care and whether health services are covered by insurance.

This year alone, more than a dozen states have passed laws regulating AI in health care, according to Manatt Health, a national health services firm.

Arizona, Maryland, Nebraska and Texas now ban insurance companies from using AI as the sole decision-maker in prior authorization or medical necessity denials. Nevada and Oregon prohibit AI from representing itself as a health care provider, while several other states—including Utah and New York— have regulated the use of AI-enabled chat bots in mental health.

State AI legislation has been sponsored by both Democrats and Republicans and has been approved with bipartisan support, a reflection of broad concerns over its expanding use in health care.

The Pennsylvania lawmakers say their proposed legislation would force insurers and health care providers to be transparent about how they use AI; require a human to make the final decision any time AI is used: and mandate that insurers and providers to show evidence of minimizing bias in their use of AI.

"As AI use has grown in the health care industry, we have already seen evidence that AI usage can reinforce bias and discrimination," Venkat said in a statement. "This will allow us to ensure that insurers, clinicians and hospitals use AI effectively and do not use it to perpetuate potentially harmful biases in the medical field."

More than half of American patients say AI in health needs more oversight, according to an August poll from the United States of Care, a nonprofit focused on affordable and accessible health care. Meanwhile, more than two-thirds of physicians have reported an increase in AI use for administrative tasks like recording and managing patient information, and nearly half reported using AI more frequently in patient care and clinical decision-making.

National groups, including the American Medical Association, have also called for more oversight of AI. Last year, physician use of AI more than doubled, according to the latest data from the association.

2025 States Newsroom. Distributed by Tribune Content Agency, LLC.