The AI industry is doing a great deal of handwaving at the moment. Our founder and CEO, Adam Paulisick, said as much when he opened a recent live conversation, pointing to the hyperbolic claims vendors make about what their products will accomplish on day zero.
The folks charged with separating those claims from substance are leaders like Dr. Shannah Tharp-Gilliam, Director of the Allegheny County Area Agency on Aging at DHS and Adam's guest for that conversation. Through her experience building Aria, the agency’s skilly guide, she takes a practitioner's view of a question every buyer should be asking: what should conversational AI have to prove before it is trusted to serve the public?
For the Area Agency on Aging, the stakes are concrete. Shannah's team answers to older adults, their caregivers and their families through the SeniorLine and a network she puts at no fewer than 80 partner organizations; a wrong answer in that setting leaves someone without assistance that can quickly snowball into an urgent gap. What follows is what we took from the conversation.
The questions that keep coming back
Early on, Adam put a number on the opportunity: a guide like Aria can absorb roughly 40% of an agency's question load, which he described as the questions no human being should be answering.
Shannah paused him there to add some useful nuance: "the reason why no human should be answering them is not because they're stupid questions, [and] not because the question is not of value." The reason, as she explained it, is simply that her team sees these same frontline questions over and over again. When conversational AI can handle what she called "the top 50 questions," the team member who would otherwise be answering them is free to concentrate on the cases that are more complex in their human elements.
https://youtube.com/shorts/Z6hm5ym1Hdw
Stale information has a human cost
Addresses, locations and phone numbers change constantly, and Shannah was specific about what that means for the people her agency serves. She described someone who relies on a walker or an electric wheelchair, who has already gotten up, gotten dressed and boarded an ACCESS van to reach a service, only to arrive and learn that it is scheduled for next week or, worse, that it took place last week. "The information needs to be fresh," she said. "It needs to be accurate because people are depending on it."
She and Adam brought the point down to a single example: where is the nearest food bank? Answering it well means knowing whether the site that was open last week will be open next week, and whether it is a pantry or a food bank in the first place. A guide connected to current sources can check, and so can the staff member on the phone, who with one click can tell a caller that the location is closed this Tuesday but open the next.
What matters to the AAA is that the person stays in the conversation, with staff working alongside skilly while "retaining that connection with our older adults." It is the same reason we have put so much recent work into dynamic data feeds, down to details as local as where the farmers markets are.
https://youtube.com/shorts/QoTY_UYpkaQ
Tone is part of the service
Shannah was candid that none of this arrives ready-made. Her team at AAA invested staff time in loading the agency's information, working through the questions, and answering them aloud so the platform could capture the nuance of those responses as the standard.
She also knows what the alternative sounds like. A few weeks before the conversation, while arranging help for her mother, she received a follow-up call from one company's AI caller, and its tone was rude enough that she told it to have its developers make an adjustment before taking her business elsewhere. "I did not want that to be our experience," she said. Her description of Aria, by contrast, sets the bar for any guide that serves the public: "She makes you feel like your question is a good question."
That care shows in how people use the guide. Adam mentioned one user who recently spent over an hour conversing with skilly, covering about 35 topics along the way. Shannah's reading of that was straightforward: "Sometimes people just need someone to talk to." A guide has no clock to watch, which matters a great deal with an aging population.
https://www.youtube.com/shorts/9nx32TBYhX8
Two fears and three red flags
Shannah named two fears she encountered when her agency started down this road. The first was that the agency was opening the door for generic AI to take over human connection, in effect telling everyone to go talk to a bot. Her experience has been the opposite: "We're enhancing the connection between people."
The second concerned personal information, and the worry that simply offering the tool would expose older adults to scams. Her answer here was about design. An agency with the right tooling can guard against that risk by choosing a product built not to accept, request or record personal information, and to direct people to a human whenever the answer can't be found in its approved sources.
https://www.youtube.com/shorts/4v5J-N_uWhM
She went on to describe what she listens for when a vendor answers her questions, and three red flags stood out:
- Data ownership. A vendor that glosses over who owns the data has raised what she calls "a flaming pink flag," because the only acceptable answer is that your data is your data.
- Personal information. Buyers should ask what guardrails govern the questions the AI puts to people, and where the information goes when someone shares personal details by accident.
- Currency. The information has to stay current because, in her words, "The worst thing you want is to have a tool that looks like it has the right information and actually doesn't."
These are the questions we hope every buyer will put to us, and to every other vendor they consider.
When Adam asked how many computer scientists she has on staff, her answer was "less than one," which is true of most agencies and explains why her next point carries weight. Easy access to AI through no-code interfaces does not necessarily make it intuitive to build, and she called the professional, intentional training her team received on how to work with it "invaluable."
https://www.youtube.com/shorts/Cx27g9osf28
Why now
Adam asked why this is the moment to act, given that federal budgets and staffing pressures are making a hard job harder. Shannah questioned the premise. "Worse is subjective," she said, noting that the accountability and transparency requirements she sees coming are not bad on their face. They do, however, set a standard that public service, nonprofits and social services will not meet with the approach they have relied on in the past. "We're going to need data," she said, in order to demonstrate the return on investment in these systems.
https://www.youtube.com/shorts/thhw6DKsNh0
She holds the guide to the same discipline. Her team reviews what Aria actually told people and asks whether each answer was as good as, or better than, the one a person would have given. Visitors can also rate their experience on a simple scale of faces, from frowning to smiling, which tells the agency whether they got an answer and whether they know what to do next. As Shannah put it, "No one wants these questions going into a black box."
What we took away
Adam closed by asking what Shannah would say to leaders in her seat everywhere, and her advice was to bring other people into the problems that keep you up at night. "We don't have all the answers," she said, "but I think collectively we do."
Watch the full vodcast with Dr. Shannah Tharp-Gilliam here.
