By Rose Hoban
As artificial intelligence becomes widely used in health care, researchers studying the technology warn that it’s a tool that requires constant monitoring and revision.
“You always need a human in the loop, particularly in health care,” Fay Cobb Payton, a NC State University professor emeritus, said last week.
Cobb Payton, whose area of expertise is information technology and business analytics, was one of two keynote speakers at the 46th annual Minority Health Conference last week at the Friday Conference Center at UNC Chapel Hill.
Even as artificial intelligence has the potential to transform health care into something more efficient and deliver the “personalized” health care long promised by technology, conference organizers noted flaws. “While technological innovations hold great promise for improving access to care, they often risk amplifying inequities when not designed with marginalized communities in mind,” the conference literature states.
That was the reality displayed by Cobb Payton, who argued that if no one ensures that collection and use of data is done with an eye toward equity, outcomes can be skewed in ways that harm patients.
“In health care, you’re not selling widgets, you’re not selling books, you’re not buying a piece of clothing,” she said. “You still need discernment.”
Her message was a poignant one to the capacity crowd, given recent moves by President Donald Trump’s new administration to erase programs focused on diversity, equity and inclusion from programs funded by federal dollars. The administration is pressuring business leaders to follow suit, and several bills have been introduced in the state Senate and state House to curtail such programs in public schools and state agencies.
That reality loomed over the conference, which has taken place almost every year since it was started in 1977 by the Minority Student Caucus at UNC. According to conference organizers, a student-run group like the Minority Student Caucus falls outside of the new constraints on the university policies that had addressed diversity, equity and inclusion.
The conference was packed to capacity, with more than 600 people at the Friday Center and another 200 tuning in virtually. The topics and exhibitions were wide-ranging — from helping underserved populations understand how to access reliable health resources online to using telehealth to better serve a transgender population that has been the subject of care-limiting executive orders from the Trump administration.
New topic, old biases
The topic of artificial intelligence spurred robust discussion.
Artificial intelligence has the potential to cut some treatment costs in half because of its ability to home in on a diagnosis more quickly and earlier in the disease process, Cobb Payton said, citing research out of Harvard University. She said researchers also believe early intervention can help lead to 40 percent better health outcomes.
Cobb Payton noted that AI innovation in health care could be a $200 billion market by 2030, offering providers strong incentives to adopt these new technological tools — and creators incentives to develop them.
But they need to be adopted carefully to avoid potential bias.
“Largely tech developers don’t have public health experts and clinical experts in the room,” she said. “There needs to be some transparency when it comes to how the technology is being developed, even when it’s developed.”
Cobb Payton shared examples of tools that were developed without the input of people of color that wound up having “algorithmic bias” that skewed results.
One such example was an algorithm developed to assign patients to a care management program run by an insurance company for 100 million patients. The idea was to find patients who had chronic conditions like high blood pressure and diabetes and give them extra resources to help them be healthier.
In a seminal paper published in 2019 in Science, a top-tier academic journal published by the American Association for the Advancement of Science, researchers studying the effectiveness of the tool noted that the algorithm used cost as a predictive variable, not the patients’ actual diagnoses.
The researchers showed that relying on that information skewed the needs assessment because Black patients who have the same needs as their white peers often spend less in the health care system because of wealth disparities and facing more barriers to accessing care.
“The algorithm falsely concluded that Black patients were healthier to equally sick white patients,” Cobb Payton said.
follow our coverage of new technologies in health care
As AI advances in NC health care systems, state leaders call for oversight
January 8, 2025January 7, 2025
In reality the Black patients had more illnesses and were actually more at risk of poor health outcomes without intervention. If cost had been taken out of the equation, or if race had been added to the algorithm, the study showed, the percentage of Black patients automatically assigned to the care management program would have risen from 17.7 percent to 47 percent.
When Cobb Payton talked about this finding, the crowd took a collective gasp.
“Here, the risk scores have become very critical because Black patients are considered sicker than white patients given the same risk score but receive fewer resource allocations,” she said.
Cobb Payton gave several other examples of how algorithmic bias had led to lower priority for Black kidney transplant candidates and more caesarean deliveries for Black mothers. “The technology is running wild while the health policy is still trying to figure it out,” she added.
Hard work, rejuvenation
Justin Wang, a UNC Chapel Hill senior majoring in public health and one of the two students who organized the event, noted the “booming” interest in artificial intelligence over the past year. “It’s important that we’re conscious of all these biases that are built into machine learning, AI algorithms that we deal with every single day,” he said.
Wang comes at the issues cropping up with the advancement of artificial intelligence with the perspective of a student in public health with a concentration in biostatistics.
“I kind of have been thinking a lot about how statistics can tie into minority health and how statistics can exacerbate the health disparities, but also how it can improve them,” Wang said. “And so that’s kind of where I started in thinking about all these things.”
Those thoughts led to further discussion with Mercy Adekola, who helped organize the conference — and thus the 2025 theme was born. “Minority health has always been something that I’ve been interested in, just my experiences within clinics and seeing how minorities might be treated differently just because of their background — or not even treated differently, but their experiences are different,” Wang said.
The two partnered with more than 30 other students to come up with a program for the conference, which has always been student-led.
Most of those attendees were members of minority student groups, and Wang and Adekola said that current events were on many people’s minds.
“Many people have fought to get here, to be able to use words like diversity, equity, inclusion, racism, bias, prejudice — and it all came from things like slavery, civil rights,” said Adekola, who was born in Nigeria but grew up in Connecticut.
Wang, whose grandparents emigrated from Taiwan, added: “There’s a lot of discouraging things going on in the world right now. But the important thing, I think, that I try to keep in mind, and I think this conference really tries to emphasize, is you can really only do what you can do. Regardless of what else is going on outside… there are always things that you can do in your own space, with the community around you, that you can help or improve the lives of.”
Despite the many hours the students spent developing the conference program, and the current political environment in which the president and others are trying to strip DEI programs of their funding and mission, Adekola said she felt rejuvenated after the conference.
“We are the future, you know? And I feel like tomorrow is now,” she said. “I feel like, as students, that sense of urgency is there to craft innovative solutions that help our tomorrow, because tomorrow is today, you know, and we are the tomorrow.
“We have to start being inspired to innovate solutions.”
KEEP UP WITH THE LATEST
As food aid requests rise, advocates work to help feed the 1 in 5 NC children who go hungry every day
March 4, 2025March 3, 2025
Frustrated by defendants languishing in their jails, three sheriffs bring mental health treatment behind bars
March 3, 2025March 4, 2025
Mission Hospital understaffing, preventable deaths reflect return to conditions that led to sanctions, advocates say
March 1, 2025March 1, 2025
The post Student conference focuses on AI and DEI in health care appeared first on North Carolina Health News.
