ICYMI: The Role of Humans in an Increasingly AI-Centric Service World

CX Summit panel discussion on the role of humans in AI-driven service delivery, defining “human in the loop” (active review in workflows) versus “human on the loop” (oversight and intervention when flagged). Panelist also discuss adoption approaches, emphasizing employee trust, governance, readiness, and change management.

Agile Product Discovery Course | ACT-IAC

Summary - A Hole in One with ACT-IAC

Subscribe on your favorite podcast platform to never miss an episode! For more from ACT-IAC, follow us on LinkedIn or visit http://www.actiac.org.

Learn more about membership at https://www.actiac.org/join.
Donate to ACT-IAC at https://actiac.org/donate.
 
Intro/Outro Music: See a Brighter Day/Gloria Tells
Courtesy of Epidemic Sound

(Episodes 1-159: Intro/Outro Music: Focal Point/Young Community
Courtesy of Epidemic Sound)

Transcript

Copy Transcript Copied

Mary Ann Monroe: [00:00:00] Okay, good morning. Good morning. I'm Maryanne Monroe. I'm really excited to be with some esteemed colleagues here today from industry and from government, former government, and we're gonna continue the conversation on our panel today. Um, we're gonna be talking about the role of humans in an AI increasingly world of, of service delivery and how, um.

Mary Ann Monroe: Our panelists and different, um, examples that they'll bring forward of lessons learned, how they're doing this in, in the real world, and how they're balance balancing this in, in their service delivery. So, um, I wanted to get started by, um, asking each panelist to introduce themselves and then I'm gonna level set and go right into questions.

Mary Ann Monroe: So why don't we start with you, Kathleen. 

Kathleen Featheringham: Sure. Hi, I am Kathleen Featheringham. I am actually the lead for AI at Maximus. So a large, uh, contact center, uh, [00:01:00] business process outsourcing, uh, federal provider. And so a lot of what we're doing is trying to get to those outcomes. So how can we use AI to drive, um, drive the outcomes for our, our customers and honestly for ourselves?

Crystal Sprague: Good morning. My name's Crystal Sprague. I'm the director of performance and innovation for a unified city, county government in the state of Kansas. So we actually call ourselves the unified government of Kansas City, Kansas, Wyandot County. And, uh, I am primarily responsible for our call center, but also driving, uh, technology advancements, including the use of AI across our organization.

Victor Udoewa: Good morning. Is it on? 

Mary Ann Monroe: Can you hear me? It's a little delayed. It'll, it'll. Push it up and then it'll 

Victor Udoewa: Morning. Good morning. Uh, my name is Victor Udoewa. I'm an engagement manager at Bloom Works. We're a civic digital consultancy that works with local state county governments across the us, um, as well as in Canada with provincial governments.

Victor Udoewa: And [00:02:00] prior to that, I've, I've served in about five different federal government agencies. Uh, most recently I was a service design lead at the Centers for Disease Control and a Chief Experience Officer for two large programs at nasa. 

Marcy Jacobs: Hi everybody. Good morning. It's nice to see so many familiar faces. I'm Marcy Jacobs.

Marcy Jacobs: I am a deputy secretary in Maryland's department of IT and Chief Digital Experience Officer. And before that I was with the federal government, uh, with the US Digital Service and Department of Veterans Affairs leading digital projects. Uh, and it's interesting, in Maryland we're doing a lot of experimentation and a lot of, uh, looking at what problems AI can help us accelerate.

Marcy Jacobs: So how do we improve outcomes for our residents? By leveraging all of the tools available, AI being one of many. 

Mary Ann Monroe: Excellent. I'm excited to dig in with all of you and, um, all of us learn from your experiences. So before we get started into q and a, um, I just wanted to level set on, on some terminology that I think is really important to [00:03:00] think about as we talk today with the panel human in the loop and human on the loop.

Mary Ann Monroe: What do we mean? When we, we speak about these processes, um, when we talk about human in the loop, this means humans are actively involved in an AI system. Um, so that the AI system isn't doing things completely by itself, but, um, for example, if AI makes a prediction or a recommendation, there's humans involved in that process to review and approve a process or a document, et cetera.

Mary Ann Monroe: So if you think about a chat bot that's drafting. Responses for us in, in customer service, there's a human in the loop that may be reviewing that before it actually gets sent. When we think about human on the loop, um, with human in the loop, humans are involved inside the workflow with that AI system.

Mary Ann Monroe: When we think about humans on the loop, it's um. They're involved in the AI system, but they're not, um, directly [00:04:00] involved in every, in every decision intervening. Um, so that there is still some process by humans, but that, um, the AI acts more autonomously and humans monitor performance and step in.

Mary Ann Monroe: Something is flagged. Maybe there's an audit that's flagged, maybe the confidence level is low, um, of what the ai, um, information that is being produced where a human would step in and, um, help make a review and a decision on that. So with human on the Loop, humans are involved outside the workflow and they provide oversight and control rather than constant.

Mary Ann Monroe: Involvement and interaction. And I think that's important because as we talk about how we integrate humans and, uh, with different AI systems, we have to think about, uh, the level of involvement and how that's carrying out. So, we'll, we'll talk a little bit more about those examples. So I wanted to begin, um, with you Crystal.

Mary Ann Monroe: We're all using [00:05:00] AI in our everyday life, in business and at home and in our personal lives. We'd love to hear more about how you've approached AI in, um, the city, in the local government of Kansas, and, uh, talk about how you've started integrating it and what have you learned by the balance of AI systems and humans.

Crystal Sprague: Big question. There you go. So I'm, I'm gonna take it, uh, a couple of steps at a, at a time there. So the first question was about how do we approach the use of AI at a local government level? AI can be pretty scary for frontline workers. We tend to be a little closer to the folks that have been doing the job, uh, as one of the speakers mentioned earlier.

Crystal Sprague: Uh, getting rewarded for that heroic effort, um, individualized. Uh, maybe at a local government level more frequently. So, as we start to approach ai, we, we soften the language around AI and we talk about how their product will feed the AI system, providing a better customer experience. And I really like to focus [00:06:00] on the concept that AI is there to help you.

Crystal Sprague: It's a tool. It's not there to replace you. Um, what we really need to understand is that. In our technology, AI is when we are looking at things like service level delivery on say, a website, that we are taking your good information and repackaging it so that folks can get to that information the best way that they can, the way they want to get to that information.

Crystal Sprague: But what that frees you up to do as a practi, as a a, a. Person on the, on the front line is to really focus on those really complex tasks that need your human interaction to get resolved. So it actually gets the civil servant back to what they, um, what they started their career journey to do, which is to help solve problems, not necessarily answer, uh, what application do I need and where do I.

Crystal Sprague: Submit that application and what is the address in your business hours. So we approach it that way first, uh, take some of the, the scariness out of the concept of what it [00:07:00] looks like for ai. And your second part of that question, Maryanne, 

Mary Ann Monroe: well, I wanted to, um, learn a little bit more about how has this.

Mary Ann Monroe: Affected your employees? Are they scared? Are they empowered? Um, you know, we often talk about AI is, is extremely helpful in helping us automate and complete tasks. And, um, how have you approached that with your, with your employees and what have been some of their responses? 

Crystal Sprague: So earlier today we heard, uh, about um, how we approach AI involvement and what that can, um, be interpreted as from a frontline employee.

Crystal Sprague: We also heard about the need for governance. Uh, so I think and readiness. So I think that really approaching those governance and readiness conversations and ownership upfront and changing. The conversation instead of around the technology to how are we going to govern the technology and who owns it, uh, really involves, uh, all the members cross-functionally in the output of the [00:08:00] technology and they begin to become not only more comfortable, but they own the you.

Crystal Sprague: Input to that technology so that they then own the output and it's an easier transition for them to understand how they start to monitor, uh, what is going out instead of due, um, maybe due, filling out the form, fixing the form. Now. How are you monitoring the interaction for the customer service?

Mary Ann Monroe: Excellent. Excellent. You, you teed? Um. My colleague Kathleen. Um, well, because Kathleen, you're involved so closely in setting the direction of the AI strategy for Maximus and, um, crystal, you mentioned the governance and the involvement of, of people in the process. Kathleen, what are we doing at our company at Maximus that you would like to talk about in terms of how we've really accelerated our strategy and how we're moving forward with our people?

Kathleen Featheringham: Sure. Um, so one of the things that we're doing, and, and this may say sound a little counter, is we're actually really leaning [00:09:00] into data. So not just, and I don't mean just the cleaning of the data. I mean, think of like behavioral science. So some parts of, and this is what we're doing actually for both our clients as well as internal, is not only just asking them what they want and what they do, but we're also observing behavior and where that as where it is from an adoption standpoint.

Kathleen Featheringham: So a lot of the times if you ask somebody in terms of what they want, they may not know exactly what to tell you, but if you're also being able to have the backend. Analytics of like, what are they doing in certain systems? Why, how, and things like that. So, uh, I'll give an example of it. So for an internal deployment, we put in, um, an ag agent, uh, bot to be able to start to be able to help people put in it, help desk tickets, HR help desk tickets.

Kathleen Featheringham: And I answer general questions about like different parts of the company or let's say benefits, anything you want. Parts of what we had to understand is where people were and their comfort levels and what they're doing. So, you know, [00:10:00] much to my surprise, like one of the number one, when we first, um, kicked off with a small group and one of the number one questions that kept being asked in the spot was, what is the corporate calendar like to me, I was shocked.

Kathleen Featheringham: 'cause I was like, I feel like everybody knows this. Like we get basically government holidays off and things. Then we got ones of. Can you tell me who my manager is? And so what we started to realize is if you start to think about like the behavioral science parts, they were literally asking questions to see if they could trust the system.

Kathleen Featheringham: Would it give answers that they already knew the answers to the questions that they were asking? Then you start building it up and you could start to see things like that. So a lot of what we're being able to figure out is where are people at in their maturity of understanding even how to ask for things or use tools or expectations and being able to add to it.

Kathleen Featheringham: As well as doing an analysis on what is it they're actually looking for. So a lot of times we're changing the content of what we think, but sometimes we're wholesale adding new content because there's content that they're asking that I'm not sure we [00:11:00] would've thought to even add to it, that they, or that they would've had the perception that this would've, that this types of tools would've been added.

Mary Ann Monroe: That's excellent. Thanks Kathleen. Marcy, how about you from your perspective in the state of Maryland, how are you thinking about adopting AI systems, um, in your work? 

Marcy Jacobs: It, as you would expect with a, a large, uh, government population, we're seeing a ton of variability. We have people who are very much leaning into experimentation.

Marcy Jacobs: I think we've run 140 pilots in the past year with different agencies. There's a lot of curiosity. There's also a lot of nervousness and a lot of, I don't know how to do this. I'm not sure what I'm supposed to do. We have deployed Google Gemini to every state employee so that they can experiment and they can work in a, in a safe, enclosed environment.

Marcy Jacobs: Uh, and we also have a community of practice every month where we are highlighting what are other agencies doing? How are they thinking about this? What are the problems that they are solving that maybe you are also experiencing [00:12:00] something similar within your agency that you can learn from. So there's a lot of, a lot of appetite.

Marcy Jacobs: I think we're trying to really approach that in a very responsible and measured way and make sure that we aren't building tons of, um, prototypes and pilots that kind of get built and then abandoned. And how we think about how do these things scale? What data are we using? How do we build off of what we're learning?

Marcy Jacobs: So taking a very measured approach in the state. 

Mary Ann Monroe: Excellent, excellent. Victor, from your perspective, um, I'd like to know a little bit more about, um, there's a lot of concern about bias, um, in using AI and, um, what perspective do you have on ensuring that we preserve, um. We preserve people's privacy and so forth, but that we don't introduce issues with bias.

Mary Ann Monroe: It's a big issue, it's a big conversation. What are your thoughts on that and how are you approaching that? 

Victor Udoewa: Yeah, I, I could say a lot [00:13:00] of things about it. So I'll, I'll start with, uh, one thing and then we can go from there. Uh, the difficulty I think, with the human in the loop, human on the loop framework is that if you only think about having a human.

Victor Udoewa: In the workflow loop, you've already missed entryway points of bias before that. Um, so, you know, you could do CX in many different ways, right? You can be more proactive, more reactive. You could have more focus on survey based CX or more holistic, et cetera. And so the, the type of CX work that I've primarily done is, um, design focus to design forward.

Victor Udoewa: And so when we do cx, we, we are working with, um, or as service designers. What, what I've experienced is that a lot of, a lot of service designers and UX designers and researchers have thought and focused on the look, the feel, the experience of a tool that has AI in it. However, the [00:14:00] experience of an, of an AI tool is also affected by how that tool is built.

Victor Udoewa: That's right. And so I've had to do a lot of retraining. I was doing this at the CDC, um, as late as last year. Retraining our designers and researchers on what does it mean to actually design an AI tool. And so when you look at the whole, um, AI pipeline or let, let's look specifically, let's say machine learning pipeline from.

Victor Udoewa: Choosing, like data availability, choosing a, a data set, um, the data engineering, the organization, organizing and cleaning, the, uh, choosing or selecting a model or ag algorithm training approach, training it, um, validating it, deploying it, monitoring it, right? Looking for model drift modeling, et cetera. At every single one of those steps, there are so many decisions that humans have been making and that can introduce bias.

Victor Udoewa: So the only point that you come in to, to check as a design or research ethicist, like trying to make sure it's ethical, is after it's [00:15:00] already created. It's way too late. And so we have our designers and researchers work alongside our data scientists and data engineers throughout the entire pipeline process.

Victor Udoewa: Identifying points of where bias can introduce. Where people are using what we call traumatized data sets, where we can then, um, address bias as it's going. Because once it's built, there's already bias already in in it. Um, so this is one of the ways that we think about it as well, is it's in the very process of building it.

Victor Udoewa: 'cause there's so many decisions that are made even before you get to the actual tool to be used, 

Mary Ann Monroe: you make such good points that the design ha has to be integrated from the beginning and the steps that. When it's too late. Um, and I think that really reframes a lot around the whole design services themselves and the, and the capabilities that people are bringing.

Mary Ann Monroe: You have really got to bring in those designers much sooner into the whole process. As, as, as you're designing the services and looking at the data and the responses. Um, [00:16:00] okay, let's, let's switch over. I wanna talk for a moment around, um, the future of work and, and human relevance. Um, there, there will be a panel right after this, so we're not gonna dive too, too deeply.

Mary Ann Monroe: Um, but I would like to see what, what any of you on the panel think about human in the loop systems. Um. Do you think that they'll become obsolete as AI matures or, um, is, is it the only way to preserve human agency? What are, what are the alternatives? Kathleen, let, let's start with you. I'd like to hear your thoughts on that.

Kathleen Featheringham: Sure. Um, I, I would add a little bit to what was Victor had already said too, and 'cause thinking about it is it's. Part of the design, but also data also has bias in it by who created the data set. So you have a whole bunch of things there. And so it's all parts of the process. So as we start going along in terms of where humans, humans actually sometimes in, um, actually introduce [00:17:00] the most risk.

Kathleen Featheringham: We have to kind of think about it. And parts of what we're seeing 'em doing is, it's not a right or wrong one tool or this all or nothing. It's about the workflows. So break down the workflows. Think about where are the appropriate parts of where you want to introduce, where it's necessary. What types of AI are you using?

Kathleen Featheringham: Are you using deterministic where it's more, um, you know, traditional software development that the input that goes in is the same as the input. That comes out every time, um, versus maybe probabilistic. So a lot of where we're looking at in terms of what human on the loop or where is really about the impact, you know, is it the impact that it's going to have on a human?

Kathleen Featheringham: And what part of the workflow? So not all parts necessary. If, let's say it's deciding and giving a recommendation on something that is very pivotal to a human on whether they could be able to get it, there's the decisions of how can you [00:18:00] audit and trace that back and understand what went into it. So it's not a black box.

Kathleen Featheringham: So. There's no one right answer here. It's very specific of understanding that full end-to-end flow, and then again, working from the design aspects all the way through, but really thinking about where is it appropriate and where actually it might cause hindrance or slow down the very nature. So for example, if you're trying to introduce, you know, intelligent voice assistance and you have to have a human review, every answer that the intelligent voice assistant gives, what's, what's the point, right?

Kathleen Featheringham: You have to be able to think about what is impact and where it's helpful, and then being transparent about what is the variability and what types of tools are being used so people can make their also their own decisions on what they think is the output of it. So having to be a lot more clear that, you know, issues could happen, that, um, errors could happen.

Kathleen Featheringham: So I think there's, there's parts of it in education, but there's also parts of. Uh, upskilling that has to happen [00:19:00] along the way to kind of be able to push back and not just necessarily be able to default to it. 

Crystal Sprague: And I'll take it, uh, from a practitioner's level. I think that humans, um. Are very vital to any kind of AI technology that you put in.

Crystal Sprague: Uh, they not only have to trust it themselves as a practitioner, but they have to be the voice to the end user that it is trustable. Uh, so knowing as a practitioner that I trust the product, I understand it's not mystical and magical. I understand how to be flexible with it. When administrations change, when requirements change, uh, then I can become an advocate.

Crystal Sprague: For ultimately what we're trying to do, which is to get the end user to use the, the interaction. Uh, so allowing that trust level at a practitioner level is super important. But I would also add that it's important that they understand that it is not magical and mystical and they know how to make adjustments when, uh, change happens.

Crystal Sprague: So, uh, [00:20:00] understanding their ownership and their role changing from. Moving a PDF from one pile on their desk to the other, to really looking at how people are interacting and making adjustments. Um, and being able to have confidence in knowing how to make those adjustments and advocate for it is critical as well.

Mary Ann Monroe: Yeah, I think you bring up a really good point, crystal. Um, the change management as part of this process, as we evolve in using new tools and, and new systems, um, what have you done specifically within your teams to, um, ensure that proper change management and communication is taking place? 

Crystal Sprague: Well, local government's favorite word, a governance board.

Crystal Sprague: It's not our favorite word. We tend to be very, uh, strapped in resources and not very mature in what it looks like to have a governance model. So being able to, uh, start those conversations early and often and really practice the discipline of what it looks like to, to really have a [00:21:00] change, um, management team that looks across the organization to see how things impact, um, each one of those parties and to ensure that all those parties are buying in.

Crystal Sprague: Uh, to whatever the change is. So I think really practicing the disciplined model of, uh, change governance or a governance model is where you start, um, not the afterthought for how you respond when there is an issue that needs to be escalated. 

Marcy Jacobs: Microphones. Um, I think the, the, uh, change model is interesting in thinking about governance in the way we are thinking about that in Maryland is it's.

Marcy Jacobs: Dependent on the risk of, of the decision being made, there are ways that you can use ai. And we're using AI right now in, in some very low risk ways that helps us get better understanding of data that is available. There are high risk decisions that we are, that we are learning about in a very human observed, human managed way.

Marcy Jacobs: So we're [00:22:00] certainly not gonna let AI make decisions on permits or benefits. Um, but can AI help us with. Looking across sentiment analysis for lots and lots of inputs, which then goes to a human to figure out what do we do with this? Now that we've seen the trends across a volume of information that maybe we couldn't understand previously.

Marcy Jacobs: We have a, a pilot right now with our, uh, fleet or our motor vehicle fleet. How do we help, uh, drivers find the cheapest place to get a tire repaired depending on where they are? These are lower risk types of scenarios. Require a different type of governance model than something that is going to determine can someone get their occupational license renewed.

Marcy Jacobs: So we are looking at governance across that spectrum and really looking at the risk model that informs how we engage. 

Mary Ann Monroe: That's excellent. That's excellent. Kathleen, would you like to talk a little bit about some of the practical applications, um, that we've applied in industry with our government [00:23:00] partners and within our own company?

Kathleen Featheringham: So a lot of it is when you talk to people about like what do they need or what do they do, right? Sometimes they're talking about, I have to do this part to do my real job. So we've really kind of embraced what's taking onto that. So a lot of things where really large intelligent document processing, where we have, uh, documents that are coming in that need things extracted and put into a certain format so that then somebody could use their actual critical expertise.

Kathleen Featheringham: To evaluate we're using on those types of things. So, um, call summarizations things of after the calls have happened, we just need to have it for historical records. Um, we are, as I said, for it service management. How can you get better responses through large contacts and, um, knowledge, information agent assist.

Kathleen Featheringham: Things like being able to pull information to somebody, say, Hey, they might be asking about this. These are probably areas at, uh, your fingertips of what you could do. Those are just [00:24:00] some of the things, but I, I, I'd be remiss in saying if we didn't say it this way, AI is not always the answer. It is not always the answer.

Kathleen Featheringham: Sometimes it's a combination of things and sometimes there is automations or things that already exist today that you can put in parallel. So it's looking across of how do you get to what is the outcome that you're trying to drive? So that part's the really critical part of, you know, not just doing it because it's cool, but what.

Kathleen Featheringham: What output are you trying to get to? And then designing what is the right path to get there of what tools, what things are there. So I get asked a lot in saying, Hey, we need ai. And my, typically, my first question is to do what? Right? Like that's, that is the bigger part. And sometimes the part that gets missing where, because everybody has a need and says, well, I was told I need the ai, 

Marcy Jacobs: can I just draw a line under that?

Marcy Jacobs: 'cause I totally wanna foot stomp that. AI is not always the answer and I think it's such an important point. We see a lot of conversations of I need AI to accelerate X. Do [00:25:00] you truly understand what the problem is? Do you understand where people are getting stuck? We have done pilots that have, that have taken a lot of energy and a lot of work that could have been resolved by just better form design.

Marcy Jacobs: So there are sometimes solutions that maybe are not as like cool right now that we've had for a long time. And I think that the hard work of understanding. What are the AI shaped problems? To find those, you actually need to understand what are the friction points, what are the blockers? What are you trying to resolve?

Marcy Jacobs: AI as a tool, service design is a tool. Policy is a tool. There could be lots of things that you uncover about why this policy or why this permit takes 150 days to process. Maybe we can accelerate that with ai. Maybe we can accelerate that with 10 other ways. Getting smart about the problem is I think sometimes a skipped first step.

Marcy Jacobs: Like, I just need things to be faster and I want this magic to come and do all the hard things. I think that's where people [00:26:00] kind of step in traps. 

Kathleen Featheringham: And I'll give you a very real example of that if you wanna put it in perspective. So we talked about large documents, right? Forms coming in. We had one where it was like, okay, we need these forms to account for, and one of the, it seems silly, but one of the minute problems is remember all the boxes that have like the letter that you put your letters in so that you'd space it out so somebody could actually see it, right?

Kathleen Featheringham: So in some of the, um, OCRing and AI capabilities, it would put like a, like, almost like a one or an I in between things. So. We had it where, you know, you pay per page of trying to do processing of some of these documents. And so that does add up. The more you go, let's say you're going like eight to 10 million pages a day, that can add up a lot.

Kathleen Featheringham: And so they were saying, okay, we need to fix this, so we have to add this other service to get that little like line to not show up or, or different things. Well, that's two or 3 cents per page that you added on it. Well, it's a two page form. Take the boxes out. Like that's a, that's a lot cheaper of a thing to say, take five minutes, change your form.

Kathleen Featheringham: So that's when we say is the outcomes really [00:27:00] matter of what you're trying to do, because you don't necessarily need to add something that's gonna be massive cost for it. 

Crystal Sprague: You've obviously hit a, uh. Love point. Keep going. Keep 

Mary Ann Monroe: going. 

Crystal Sprague: Well, and I would say also thinking about what happens to the practitioners or the, the, the, um, civil servants once the AI is deployed.

Crystal Sprague: So deploying the AI might solve the problem that has been identified, but then are there other pieces that need to come around the AI to support it? Do the other practitioners understand what the AI is doing and what their role is? In responding to it, um, you know, we, we put a uh, um. Digital agent chat bot on our website.

Crystal Sprague: Um, helping the content creators understand how to respond to data that we're handing back to them, saying this question is being asked and we're getting negative feedback so that they know how to then go in and make those adjustments and how important, important those adjustments are, I think is [00:28:00] also important.

Crystal Sprague: Slow down or speed up the time in getting a permit issued. Um, but what happens when the permit applicant who has the permit comes in and wants to make a change to that permit, or adjust that permit? So understanding all the change management that needs to happen around that is not necessarily ai, um, will make it be successful or help it be successful if the practitioners don't think that the AI implementation.

Crystal Sprague: What made their life easier because something happened downstream. They will find ways to work around it and to have the citizens work around it as well. 

Victor Udoewa: Yeah. I think your, your original question was about if we will always need human judgment in this, in this, uh, space. And I, I don't, I don't see so much as like, you know, we need human judgment or we just need the ai.

Victor Udoewa: They're just kind of. Entangled in a sense, you know that the AI is a reflection really, of us. And we make errors. We have bias. So the AI will make [00:29:00] errors and the AI will have bias. And the only reason that me, that I as a human can look at something that AI is producing and recognize the bias is.

Victor Udoewa: Probably because the bias that I recognize is not the bias I have, which speaks to the fact that we always need a diversity of people doing this. And we do not have a diversity of people creating the AI that we use, right? Usually same socioeconomic level, same education background, same race, same gender, et cetera.

Victor Udoewa: And so we really need, um, participatory forms of, um. Not just AI evaluation, but also AI creation, um, uh, even of governance boards and things like that. Um, we are not there yet. And so we definitely still need humans, not just in the loop, not just on the loop. Uh, Salesforce uses the term human at the helm.

Victor Udoewa: I don't actually know exactly what they mean, but it speaks more to what I'm talking about, that there are hundreds of decisions that can introduce ai. And so having humans as, uh, walking through that is super helpful as we begin to [00:30:00] build these things. 

Mary Ann Monroe: Such great points. And I, I love that each of you has, has brought the points out that, um, AI isn't for everything and it has to be evaluated very carefully.

Mary Ann Monroe: It's a tool in our toolbox. And, um, the other thing is the importance of the data and what are we learning and how are we adjusting as a result of what we're learning. Um, the final question I wanna ask before we go to q and a from the audience is. Is around personalization. We know that we like personalization in our own lives in terms of getting automated reminders to, um, order our, reorder our shampoo on Amazon or it's ready for our our benefit renewal time.

Mary Ann Monroe: Um, how are any of you thinking about or applying more personalization in service delivery, uh, within your organizations or in, in your experiences in government? 

Crystal Sprague: Think it was, uh, spoken about on the earlier panel today, but really [00:31:00] understanding if a, a citizen wants to interact in a personalized way, some do not, some do not wanna share all of their information, uh, so that they can be reminded others do.

Crystal Sprague: So how are we responding individually to those folks? Um, but you know, I'll take what was discussed. This morning in the panel, maybe one step further, uh, beyond personalization in local government particularly, and I would imagine at state and federal government, we have these issues as well. People don't know what they're eligible for.

Crystal Sprague: They would take advantage of a, uh, a service if they knew they were eligible or if they knew the service even existed. So we spend lots of energy and effort to try to promote. Promote, promote. You're eligible. You could be eligible, go to our web service and put in your information to see if you're eligible.

Crystal Sprague: But we, we don't often hit that target audience that doesn't even know to ask the question whether or not they're eligible. So one of the goals that we have for the next year is to let folks interact. [00:32:00] With our, our agent, our digital agent, and we tell them, we think you might be based on the information that you added, um, requesting information about maybe your vehicle tags, you know, your license plates for your vehicle.

Crystal Sprague: Based on that information, we think you might also be eligible for a senior citizen tax rebate. And they might not know that the senior citizen tax rebate was there or how to ask for it. Um, so instead of focusing our efforts in outreach, outreach, outreach. Kind of flip that outreach script around a little bit and say, you're already interacting with us in a digital way.

Crystal Sprague: Um, here's something based on your information we think you might be able to take advantage of. 

Mary Ann Monroe: That's excellent, Kathleen. 

Kathleen Featheringham: One thing I would add is, um, well, lemme start with a question. Have any of you guys used the new Alexa? Like, have you said where you can change it and stuff too? Alright. So you can change a voice and we're starting to see people wanting to do that personalization, but I'll take it even one step further.

Kathleen Featheringham: So like, I changed it to, I forget, it was a nice male voice with an accent. It was great. But then he [00:33:00] started describing that he was Zane, he was zen and laid back. And actually just my natural response that I didn't think he was gonna respond to was like, wait. Whoa, you're doing too much. 'cause it was like seven in the morning and it literally said, sorry, I'll pull it back.

Kathleen Featheringham: So parts of personalization is, and then it was like, do you wanna tell me more about how you prefer I answer and things. So that's where things are going towards and that's what we're also working on in personalization is, you know, not to say that interfaces are gonna be dead here soon. They're gonna take some differences because a lot of it's gonna be about your AI agents that are going out to acquire the things that you want to do, whether it's knowledge, whether it's take action, whether it's doing those things, and it's gonna be interacting with other agents and those parts to be able to pull back what you want in the preference of how you want it answered.

Kathleen Featheringham: And so that is the other parts that we're seeing very heavy. Is that there's an expectation coming, [00:34:00] and the expectation is that one size doesn't fit all. And it's that I want to be able to use these tools to get it in the manner that suits me best. And so for mine is I, I don't want a lot of hyperbole rounded.

Kathleen Featheringham: I just want the answer right, like, gimme what the parts are there. But there's others who, that doesn't work for them. They want to feel that different type of interaction. 

Victor Udoewa: Yeah, I'll just add that per personalization adds a lot of challenges. Um, AI has tons of challenges, um, from energy use and sustainability, uh, to accessibility, especially when you're personalizing, um, interfaces or personalizing, those kinds of things.

Victor Udoewa: We do know, I think one of the areas that personalization is super helpful for is in educational services. You know what it's like to be in a, in a group of people where you're taught almost based on the average of where we are. But if I can learn what I need and you learn what you're, you need, you're able to kind of move forward and, and do what you can.

Victor Udoewa: The, the, the principle I think that we've, we follow in our work with, uh, local [00:35:00] county and city governments, uh, state governments is, um. We want to balance the amount of data. 'cause often personal personalization requires data. The amount of data that we are requesting or need with the benefit that people receive, and sometimes we're requesting much more data in terms of how people value it.

Victor Udoewa: Then what they're actually getting as a, as a benefit. And then we also wanna use things like progressive disclosure, progressive consent, et cetera. Um, because maybe people are willing to take less personalization and give less data, and that's fine as well. So we think about balancing and following these principles, um, that are led by the people that we're trying to serve.

Marcy Jacobs: Personalization is such an interesting topic. Uh, I would say one area that, that we're thinking about in the permitting and licensing space in Maryland is how do we customize the journey based on what you are actually trying to do? Because opening a business is not opening a business. It's. Like a thousand different things.

Marcy Jacobs: How do we use AI so that you can have a, a [00:36:00] more navigable, curated path based on your situation and based on your needs and where you are in the journey. That also relies on having the right answers in a consumable way and like, it's the boring, hard, not sexy work, but like content matters and we have a lot of content that doesn't.

Marcy Jacobs: Maybe answer the questions as effectively as it could. So it's that balance of what we're putting in needs to be quality information so that people can get good answers out, and otherwise we're gonna have shinier bad answers. So how do we make sure that we are approaching this from multiple lenses?

Marcy Jacobs: There are still people who are gonna navigate websites. There are people who are gonna go straight to ai. How do we accommodate all of those needs by thinking about truly maintainable quality information that can be consumed in a variety of different methods. 

Mary Ann Monroe: Great points, great points. Well, we're gonna pivot over for the next few minutes here and, um, [00:37:00] entertain questions from the audience and, um, thinking about what this panel and the, the, the things that we've talked about today, are there questions or, um.

Mary Ann Monroe: Comments that you would like to bring up at this time? It's open forum.

Mary Ann Monroe: We see a couple hands up front too, 

AUDIENCE: Jamie from, uh, Datadog, Jamie Gonzalez from Datadog. Um, just wondering on some of your AI experiments and um, AI agents in production, what are some of the, uh, negatives or challenges you've had with some of the outcomes you're getting?

Kathleen Featheringham: Thank you. I'd say it, it's, it's a mixture. It's perception. It's what people are perceiving that it should be able to do, even though no one has ever told them it would be able to do that. Right. Um, so you're having to deal with a whole new level of it. Like even some [00:38:00] of our internal things like where you have very heavy technical people who are starting to use it.

Kathleen Featheringham: We'll tell them what data sets it's actually connected to and what it can respond to. And then when it doesn't answer, they'll literally tell us, no, it's bad. It doesn't answer this. And we'll say, but we told you that's not connected to those data sets. And they'll say things like, well, I would've expected it to know it.

Kathleen Featheringham: So you're dealing with a lot of expectations in addition to that. And then, um, a lot of it is. The parts of having to deal with, do you have a good baseline of understanding of how things are working today? Because if you're trying to put in an AI agent or something like that to improve it. How do you know if you're actually improving it?

Kathleen Featheringham: Because a lot of tendencies is to put that AI has to be, uh, 98% to a hundred percent accurate, and again, people aren't. So it's actually having to go back and get really, like lessons learned as well as having people understand that you're gonna have to go back and get a baseline. So let's say you're doing [00:39:00] it service management, and it's how fast, um, the tickets are triaged to go, what to what group, right.

Kathleen Featheringham: So if you're using AI to now triage it, and let's say you have 75, it's 75% accurate, um, some parts of people are saying, oh, well that's not good. It's 75%. But they, what they don't realize is that the humans were at 55%. So that's actually a pretty big jump, but that's not how it's seen. So you have the perception of what it should do and the perception of what human accuracy is right now.

Crystal Sprague: And I would add, um, administration, uh, release of their common understanding. And, and the example that I think of is we have a, um, our treasury department is in charge of our tagging and our, our, um. License plates, uh, and we have a queuing system so people can get in line online and, and be able to start their day in the queuing system.

Crystal Sprague: Um, the website is written very specifically about how to get in line in the queuing [00:40:00] system, and, uh, we kept going back to the administrator and saying that the chat bot is not returning an answer that is satisfactory for the customer. And the administrator would say, but this is how you get in line.

Crystal Sprague: They're asking how do you get in line? Um, and helping them release that control of this is how you get in line and understand that the question really was, yeah, I'm trying to, and I can't, so that was the second question. We see your directions, we understand it. The digital agent is giving us a summary of those age, of those, uh, directions based on how I'm asking the question.

Crystal Sprague: But my second question is, yeah, that's cool. But I can't. And helping the administrator, see, you have a content issue here. It's not about how you get in line. You've done a great job spelling that out. But now what happens when they can't and they run into that roadblock? What should they do next? And when we made that change and the administrator kind of let go of their common terminology and the way they [00:41:00] explain the question, um, all of a sudden we started getting a lot of thumbs up in response.

Crystal Sprague: Like, yeah, this is good. Now I know and. The answer to the question, by the way, was not, you do this to get in line when you can't. It was, we're sorry we're full. Here is another thing that you can do, um, to be more successful with compliance the next day. 

Marcy Jacobs: I wanna go to the pilot part of your question. 'cause we have a lot of pilots that are going on across the state, and I think, uh, with some of these, we run into the challenges that you would expect in a bureaucracy.

Marcy Jacobs: There is a. A maybe lower bar for a pilot because it's a pilot and we can do something in a few months. And then when we want that pilot to scale, we run into who actually is gonna own this? How do we sustain this in a longer term way? We had money for the initial pilot. We had a team for the initial pilot.

Marcy Jacobs: Now what? And even when things are successful, and we have some of these right now that we're saying. Who do we give this to, who [00:42:00] will maintain this and, and drive this forward and continue to iterate and improve? Uh, so I think we're still kind of navigating that piece of how do we go from a place of learning to a place of, of scaling and expanding.

Victor Udoewa: Um, I'll just, I'll focus on the outcomes part. I, I was thinking about, um, over reliance. There's a ton of different challenges we face, but over reliance on, on the AI that causes inaccuracies errors. So, uh, I'll use an example from the CDC. We were, uh, looking at, uh, human in the loop. Having radiologists come in and look at, um, AI assessed x-rays of people's chest to say, oh, there's this problem, there's not this problem.

Victor Udoewa: And it was beautiful, the accuracy and. Proved, uh, up to like 90 something percent of radiologists being able to, um, correctly diagnose it as some, someone has something. The second part of this research project, we then made the AI purposely make mistakes. To see if, oh, the radiologist was obviously gonna, this is wrong.

Victor Udoewa: Come [00:43:00] on. This person has this pulmonary condition, et cetera. And, um, the accuracy of the radiologist's, um, diagnoses dropped to something like in the fifties. Like, it, it was, it, it, it's a problem of learning. What does it mean to be a human in the loop? Um, if implicitly, I'm just still reinforcing whatever it tells me.

Mary Ann Monroe: Really good example. We have time for one more question. I see one here in the middle.

Speaker 7: Thank you. Uh, my name's Tyler. I work at hud, um, with cx. And, uh, my question is about plain language content on websites. Um, so right now, you know, on our agency website as many websites, agency websites are, there's a lot of kind of long-winded, really bureaucratic sounding content. Um. I've had the thought that like, oh, you could run it through a plain language engine, AI engine, and it immediately improve a lot of that language.

Speaker 7: But, um, I'm just curious, like, what would you recommend, uh, [00:44:00] what would, what approach would you recommend to solving this huge challenge that we face? 

Crystal Sprague: Yeah, and I think that, uh, Tyler speaks to the question we were just talking about in our, um, environment. Man, those administrators, they like to hold on to their very.

Crystal Sprague: Uh, professional descriptions of how to do things. And, uh, we got them over the hump in that understanding by putting them through some strenuous UAT testing on our agent and saying, well, here's your content. Looks really pretty, looks great. 1, 2, 3, do these three things and you can be compliant. Uh, use the word motor vehicle licenses, which no one in the state of Kansas uses that language.

Crystal Sprague: We say, we gotta go get our cars tagged. Uh, so let's run it through some real life testing. Uh, we are going to actually take our bot and we're gonna let our three one one call center and our treasury call center test side by side. So every time the phone rings and you pick up 70,000 times a year, every time the phone rings, answer the question [00:45:00] over here and answer the question over here and write down exactly what the person is asking you and see the negative results.

Crystal Sprague: And that really helped the administrator say. Wait a minute, we've got a problem here. I don't always need to say motor vehicle license. Sometimes I can say tag and then we will get a better answer and results. So, UAT testing with frontline staff when there's a problem, it's, or a complaint, it's always a request to fix it.

Crystal Sprague: So define it that way, test it that way, and your administrator will see those results and, and understand that. Um, sometimes what's written in non plain language on the website needs to be consumable in a different way. 

Kathleen Featheringham: I would just add to that really quickly. Um, there's no common definition of what plain language means, so that's the inherent.

Kathleen Featheringham: Fallacy in that parts of, Hey, who is it playing to? Right? So we're doing very similar of user testing, but we're also doing what I was talking about earlier, that behavioral science parts of what are people actually putting in and what is the outcomes of it, and like which knowledge articles. Get [00:46:00] drawn and which ones come up and what's missing?

Kathleen Featheringham: Gap areas. So we're actually using AI to do almost trending of what is our gap areas of knowledge that's being missed based of what it thinks it's being missed, that it didn't have a knowledge article to go to based off the question. So we're trying to use a combination of AI data science in the background, plus the testing of it because I, I can tell you like we've spent a lot of time, there's no plain language of what everyone's gonna prefer.

Kathleen Featheringham: And what exactly and in what language and contextually in what language. 

Marcy Jacobs: I, I love your example about we write for ourselves. We are the experts. We are writing from the expert perspective and somebody who is trying to answer a question has no idea what we're talking about. We have done some pilots around, I'm trying to transform several hundred websites right now, and they are like an archeological dig of content.

Marcy Jacobs: That just get piled on and piled on, and we're going through the painful, like what should we throw away? Hopefully more than half of [00:47:00] this. Uh, what should we keep? And then with what we keep, does it answer the question that someone is coming to our site to answer that perspective of the outside in who's coming and what are they coming to do?

Marcy Jacobs: And are we answering their question in a digestible way is where we are starting. And then we can look at the content and say. We have a sense of our audience. We know what they're trying to do, and they are calling us because this makes no sense to them. So how do we use ai? How do we use content strategists?

Marcy Jacobs: How do we use various tools to consolidate multiple pages on the same topic, uh, to speak in a way that makes sense to our audience? And then to test and validate that actually we are answering people's questions in a way that they can understand it and they're not still gonna call if they don't choose to.

Mary Ann Monroe: Excellent. Excellent way to wrap. Um, thank you all for your expertise and for sharing your examples and lessons learned. We're going to wrap up our panel now, but let me give you some instructions about what's next. [00:48:00] We're gonna go into breakout sessions. So in this room, um, the breakout session is empowering the workforce for the digital future.

Mary Ann Monroe: So if that interests you, stick around here in this room. And on the other side, um, past the elevator, same floor is unifying data to build customer profiles on the east side. So thanks again, panelists, and uh, we'll see you in the next breakout session.