ICYMI: Closing Keynote - AI Acquisition Forum 2025

In this final keynote, Jeff Sage, Director of Enterprise Service and Analysis Division at NASA's Office of Procurement, discusses the integration of AI in NASA's procurement processes. Introduced by Andrew McAllister, Jeff elaborates on NASA's historical and current use of AI, particularly focusing on its role in optimizing procurement operations amid workforce challenges. He shares practical AI applications like the statement of work generator that significantly enhances efficiency. Jeff also talks about the cultural and operational shifts needed for successful AI implementation, including training in prompt engineering and maintaining human oversight to mitigate biases. The session emphasizes that while AI is a powerful tool, it requires human expertise to fully leverage its potential.

Subscribe on your favorite podcast platform to never miss an episode! For more from ACT-IAC, follow us on LinkedIn or visit http://www.actiac.org.

Learn more about membership at https://www.actiac.org/join.
Donate to ACT-IAC at https://actiac.org/donate.

Intro/Outro Music: See a Brighter Day/Gloria Tells
Courtesy of Epidemic Sound

(Episodes 1-159: Intro/Outro Music: Focal Point/Young Community
Courtesy of Epidemic Sound)

Chapters

• 1:49 Keynote Speaker Introduction
• 3:21 NASA's Use of AI in Procurement
• 4:47 AI Implementation and Challenges
• 7:38 Cultural and Operational Changes
• 11:00 Ensuring Fair and Effective AI Use
• 18:15 Q&A and Closing Remarks

Transcript

SORAYA CORREA: [00:00:00] All right, so last but not least, our final keynote, uh, is gonna be Jeff Sage talking about some of the stuff he's doing over at nasa. And to introduce Jeff is Andrew McAllister, our Senior Director of Business Development at Commercial and Civil Space at SAIC. So, Andrew, come join us.
SORAYA CORREA: Welcome. Thank you. 
ANDREW MCALLISTER: Good afternoon everyone. Today I have the, uh, great pleasure of, uh, introducing, um, our keynote speaker, Mr. Jeff Sage. Uh, Mr. Sage is the director of enterprise service and analysis division at the, uh, NASA Office of Procurement. Um, in this role, he, uh, provides senior executive leadership and oversight for key operational functions including enterprise E-business systems, driving digital [00:01:00] transformations, and ensuring fair and, um.
ANDREW MCALLISTER: Fair pricing and maintaining accountability and compliance. Um, Mr. Sage, good to see you. Um, so, so Mr. Sage has also served, um, as a, uh, the NASA Federal Acquisition Regulation Supplement Manager and the performance management lead. So he has got a great deal of experience in principle, um, contract for procurement policy and guidance, and managing the creation and maintenance of procurement regulations and overseeing enterprise wide e-business, uh, systems.
ANDREW MCALLISTER: So, uh, Jeff, uh, you know, in, um, we're, we're kind of opposite sides of the same coin, um, as a, as a contractor and, uh, as the, uh, procurement authority, um, we use ai, uh, every day, uh, to, uh, enhance our, uh, interactions with the government, um, [00:02:00] and in, and, uh, increase the effectiveness of our interactions with, uh, with the government.
ANDREW MCALLISTER: Um. So, uh, how do, as, uh, can you share a little bit with us, uh, what some NASA's objectives are, uh, for the use of AI in the procurement process? 
JEFF SAGE: Sure. 
ANDREW MCALLISTER: So, 
JEFF SAGE: AI is a long history at nasa. We've been using AI for literally decades, right? So we have AI on Mars rovers. We have AI on the space station. We have ai, you name it in space, there's ai.
JEFF SAGE: It's been there forever. We're now recently with Jet GPT, right? An explosion everywhere. Starting to look at it in the procurement space and really more the, I call it the earth-based usage for ai, right? So the institutional sort of mission enabling support office functions, uh, you know, to, to enable what we do in that space.
JEFF SAGE: Um, so with procurement, we have been getting in, we're in our infancy. I'll, I'll just. Be truth to be told. Um, I think that we've been [00:03:00] dipping our toes in, in places. I think from my perspective, the challenge that I've had is that I've not seen commercial solutions that meet our immediate problems to be just frank about it.
JEFF SAGE: Um, we see a lot of solutions. We see a lot of things labeled as ai, but when you actually step back and look and what our mission set is and what our needs are. They're not meeting it quite yet. So we've been, you know, working with our chief AI office to, to develop sandboxes practical use cases that we can start building around and, and coming up with, you know, how we can leverage AI best for us.
ANDREW MCALLISTER: So in what areas are you using AI to aid the decision process of, of the procurement organization? 
JEFF SAGE: Sure. So, um. I can give one use case that we have done recently and then I can talk, and I think it may probably come up in a future question, come up with, you know, how we've used AI most recently. So, um, one thing that we've been looking at across the agency is our, uh, acquisition portfolio, right?
JEFF SAGE: The number of contracts that we have, how much. [00:04:00] Workforce we have. So we've been declining in workforce for the past two years. It's been more than just the past six months, uh, with DRPs, et cetera. So we've had to become more efficient and more effective at what we do. And, and folks said at earlier acquisitions of team sport.
JEFF SAGE: So it's not just procurement losing people, it's also our technical expertise that's being lost as well. Um, so looking, we look for innovative ways to bring in AI to help early in the acquisition phase, um, and use it in a way that we were comfortable with. The, the results and when and where we were using them.
JEFF SAGE: Um, so the example I, that I have is with, we created a, a, a statement of work generator, I'll call it that. Um, NASA has 10 centers located across the country, um, all have the similar NASA mission, but they're all unique in their own special ways. Uh, so as we looked at combining contracts and coming up with enterprise contracts.
JEFF SAGE: We had a big challenge. We had a lot of folks that, you know, one center needed this to launch rockets and this center needed this to [00:05:00] run wind tunnels, right? But at the end of the day, they were all doing logistical services. They all had maintenance contracts. So we used chat, GBT, or not chat G, excuse me.
JEFF SAGE: We used an ai, um, to take those 10 disparate statements of work and put them into one enterprise statement of work. And so. We used it after the fact, to be honest with everybody, just because we wanted to test and essentially dipping their toes into the, into the pool. So what we did was, um, we put those 10 individual statements working, we generated one common statement of work for the agency.
JEFF SAGE: Um, and then we took that statement work and compared it to what humans did a year earlier. Right? Because we're following, sort of tailing this process, uh, as a use case. We found that in the, the 30 seconds that it took the AI to generate an enterprise statement to work, it hit 85% of the requirements that the humans did.
JEFF SAGE: That took them a year. Right? So now being able to leverage that moving forward, refine the model a little bit more, do some tweaks here and there. We're actually at [00:06:00] 90, 95% when we do it now. So it's been a huge change for us to. To leverage that for speed. Um, I think the other thing that it does, you can get analysis paralysis, right?
JEFF SAGE: Folks Spend too much time looking at something. It's a lot easier to polish a diamond a rock than try to create one from, from scratch, right? Um, so we've used the tool to help us in that space and really speed up the acquisition process. So 
ANDREW MCALLISTER: as part of the, uh, uh, the development of AI at, uh, nasa, how are you preparing your team for the cultural and operational changes, uh, that are required to implement, uh, at large scale?
JEFF SAGE: So it's a great question. Uh, and I think a lot of panelists talked about it before. Um, you know, so for me again, the action workforce is more than just procurement. So I have to worry about procurement folks being on board with ai, I have to worry about legal being on board with ai. Um, I have to worry about our technical folks being on board with ai.
JEFF SAGE: Uh, we have some ludite that don't want to use AI [00:07:00] as bad. They don't want to touch it to the other end of the spectrum where people want to use AI for literally. If it could, you know, if it could be rosy and the Jetsons or that referenced earlier and cook their meals for it, that's what they would do.
JEFF SAGE: Right. Um, so for us, you know, over the past summer, uh, we partnered with, with our chief acquisition officer, uh, excuse me, chief AI Officer, uh, to have a summer of AI at nasa. So we had over 4,000 people come, um, from the agency, learn about use cases, play in sandboxes, shared lessons learned, uh, across the agency.
JEFF SAGE: It's been a fantastic way for people to dip their toes in. Right? 
ANDREW MCALLISTER: Yeah. And, uh, this brings us to how this will change, um, you know, the operating model and rhythm of, of procurements. Uh, I mean, I know from a contractor side, uh, that that decisions can often take, uh, a year or more to, uh, to make the final award.
ANDREW MCALLISTER: Uh, is this speeding the, uh, the process of, of award for, for NASA? [00:08:00] 
JEFF SAGE: Absolutely. So the, the example I used earlier about the, the AI to do the, um, do the statement of work analysis, um, has sped us up when we are starting to look at our data now and, and we're being a whole lot more intentional and folks talk about this earlier, intentional with our data to ensure that we're enabling it to use AI effectively.
JEFF SAGE: Um, that has proven and, you know, infinite results for us before we even get to the eye stage of having good data. Um, so we're looking forward to that. 
ANDREW MCALLISTER: Uh, what skills and capabilities, um, uh, are needed, uh, within the workforce to drive the successful implementation of, uh, an AI enabled process? Sure. So, um, 
JEFF SAGE: patience I think is the big one for our folks.
JEFF SAGE: Um, so we've had, and, and folks have mentioned before, we've had use cases where I've given an AI tool and said, Hey, go play with this. And when the person plays with it, something goes wrong, right? We saw it right here on the stage with experts doing it. I have some people that immediately shut down, I don't wanna [00:09:00] use ai, it's bad.
JEFF SAGE: Leave me alone. And then I have other people that are like, good. Now I figured out the way not to do it again, and I'm gonna try a different way. Right? So having that conversation with my folks about being patient, allowing for this cultural change to happen. Uh, other panels talked about AI is not replacing anybody's job, right?
JEFF SAGE: Mm-hmm. We view AI as an enabler to enable the skills that these folks have. Um. AI's not gonna negotiate and sit across the table from someone. You have to look at somebody in the eyes to do that. They're not gonna be able to have those sort of nuanced conversations to get to win-win. You know, negotiations, AI's not gonna do that.
JEFF SAGE: Right? And you're not getting that subtlety through the ai. That's where you need that human to, to still have those skills. 
ANDREW MCALLISTER: So as far as the human, uh, interaction with the a i, um, what role does prompt engineering play in ensuring a fair and equitable procurement? 
JEFF SAGE: Yeah, so Prompt Eng engines, engineering's a real [00:10:00] thing.
JEFF SAGE: Um, you know, when my daughter Googles something, it's a paragraph and she puts every single word in there and I said, listen, you can do it with four or five words, not, you know, three or four sentences. Um, so for prompt engineering, my folks need to be trained on how to do good prompt engineering, and it's really about consistency and the results that you get out of it.
JEFF SAGE: Bad prompts will give you bad output. Even good prompts will give you bad output, but that's where the skill of recognizing it, you know, I've more than one time have threatened my AI's life with getting me the right answer. It's very effective if folks haven't tried it. Um, but you know, there are times where AI doesn't work and you know, that prompt engineering and that piece of, of bringing consistency of responses is, is absolutely critical.
ANDREW MCALLISTER: And, uh, we all know that, uh, even the best, uh, large language models these days, uh, can exhibit some bias. Um, and how, how will, uh, the office of procurement kind of, uh, mitigate the potential for bias [00:11:00] in decision making or, or market research? 
JEFF SAGE: Sure. So I think everybody else, right? It's, we're taking a risk based approach to add it.
JEFF SAGE: Um, you know, I think for us dipping our toes in now we are doing sort of after the fact fact checking of the humans, right? So we allow the humans to do their piece. Then we apply the AI and say, okay, well how good is the AI doing? That gives us objectionable objective, excuse mean objective data to look and say, Hey, this really is working, or it's not.
JEFF SAGE: You know, I have a fear of, you know, someone getting a good response back from AI and hitting the easy button, right? And then they just coast and they're not having the skillset that they need to still do that critical analysis. Still be that human in the. 
ANDREW MCALLISTER: Yeah. As, as a contractor, I have seen AI generated, uh, uh, RFPs in the past, and, uh, it was easy to tell because there was some nonsense inside the, uh, the RFPs.
ANDREW MCALLISTER: So how, how are you, um, enforcing that? Um, that human in the loop, uh, portion [00:12:00] of the decision making process. 
JEFF SAGE: Sure. So, um, nail, we talked about that earlier and that's, you know, Sariah started something for the federal government in that innovation space that is spread, right? That's a legacy that she has. We built upon that with a nail, uh, and leveraging that space to do some of this innovative work and give a safe space, give an intentional space for folks to do work AI at least out of the gate until we get past these sort of risk.
JEFF SAGE: Places where we're risk to more risk tolerant. It's extra work. Like the human and loop piece of it is no doubt extra work, right? So when we look at it and say. Is it worth the effort to do that extra work and then is it gonna save us time on the back end? Um, so we've been really intentional in that space.
JEFF SAGE: We've been really intentional in keeping it in things that have already been completed until we get comfortable, then we move forward. Um, you know, I think for us, as we do that, we owe the taxpayers, we owe our contractor base. The, the knowledge that we are using ai, how we're using [00:13:00] AI and, you know, same back from them to make sure we understand.
ANDREW MCALLISTER: And is that being codified in policy, uh, at this time or is that something in the future? Uh, 
JEFF SAGE: yep. So they're working through, um, we have agency framework out there for AI usage. Um, AI or NASA's been extremely intentional about doing it from a global perspective. Um, we wrote the framework in 2023, I think it was the last time we updated it.
JEFF SAGE: They're in the process now, uh, with recent OMB, the S that have come out. It was, that was, that framework was even before generative ai, right? Or chat bt So it's due for a refresh. Um, it's not a wholesale refresh. We had a lot of good foundation principles that went into it that were tweaking the edges to make sure that we're staying up to speed.
ANDREW MCALLISTER: With the significant efforts that you've made implementing AI at, uh, NASA are, what are the most significant lessons learned that could benefit other organizations? 
JEFF SAGE: AI is not a one size fits all tool. I think that that's really important to understand. Um, you know, we've used [00:14:00] AI for very simple tasks and it does things very well.
JEFF SAGE: We've tried to use it for more complex tasks and it struggles a bit. Um, and some of that's the training. So that's the model, like we need to work through that. But recognizing where the tool fits, and while I talked about it earlier. A is not gonna solve every problem, right? Like, that's not the goal of ai.
JEFF SAGE: I think there's an over application of it. Um, so look at it for what it's worth, look at it for what can bring to the organization is the juice worth the squeeze? And, and another misconception that I talk a lot about and, and the CIO folks know AI is not free despite what chat GPT wants to tell you, right?
JEFF SAGE: Every time you prompt ai. It costs money and someone's paying that bill, right? So we need to be effective. My wife has a very bad habit of saying thank you to ai. She talks to it like it's a real human being. And folks probably saw the, the, you know, chat. GPT President said it was tens of millions of dollars in just saying thank you, right?
JEFF SAGE: Like, it's a huge problem. So for us, [00:15:00] right? We need to make sure we're, we're doing the right thing. Um, 
ANDREW MCALLISTER: as, as, um, then the agency heads into the future, um, what do you think are the, uh, most significant future trends and innovations, uh, that will have, uh, a significant impact on NASA procurement? 
JEFF SAGE: Sure. So I think that for us, a lot of the pre, you know, if you look at the, the, I call it the circle of life with acquisition lifecycle, right?
JEFF SAGE: The pre-award side has a lot of. Um, opportunities. I'll say it that way. So I think you've seen demonstrations. The market research is a huge opportunity statement of work requirements, generation process is a huge opportunity. When you put the timeline of the acquisition life cycle and the PA time that we're all tracked to now, it's everything that happens before Paul, that takes a long time, right?
JEFF SAGE: And so that's where we're looking to really, you leverage AI as an enabler for that piece and get the biggest return on our investor. 
ANDREW MCALLISTER: And, uh, as, uh, AI evolves, uh, how is NASA staying at the forefront of [00:16:00] technology, uh, with, uh, well-trained models and so forth? 
JEFF SAGE: Sure. So, um, our Chief AI officer, it's a close collaboration with them.
JEFF SAGE: It's a close collaboration with our general counsel office as well, um, to make sure that we're bringing in tools that allow not only our researchers that are doing AI and laboratories for space-based stuff. Giving our, you know, mission enabling workforce, the ability to have sandbox tools, the ability to have training, the ability to have a safe place to, to try out ai.
JEFF SAGE: I learned a bunch of things today that I'm gonna go back. I'm excited to go back and talk to my chat, GPT now. Uh, I didn't think about that, right? I was sitting next to, to our, uh, chief acquisition, uh, excuse me, chief AI officer to many cas. Um, she was telling me every morning for five minutes, she gets a, a down brief of AI in the news, right?
JEFF SAGE: Simple, like how great is that? Right? How can we leverage that type of stuff? So Yep. You know, those are the types of things that we're doing to make sure our workforce and giving them, you know, sort of those ideas of how they [00:17:00] can leverage it. 
ANDREW MCALLISTER: Yep. And I think we may have time for one, one or two questions here.
SORAYA CORREA: Right. Come, so I didn't get any questions. I certainly have a question. Oh, excellent. 'cause I, you know, this is my topic. I'm big on this, so. NASA's always kind of been out there on the forefront and that kind of stuff. What do you think about when you think about the H helping employees really understand and grasp the value and how they can be enabled on ai?
SORAYA CORREA: Because everybody talks about the fear of losing jobs and all that stuff. I don't think it's that bad. I think people get that now it's, but how do we help them learn how to use the AI more effectively? 
JEFF SAGE: Yeah, so there's one thing that I think of I've found, um. This worked really well in the organization is building AI into our normal work stream.
JEFF SAGE: Not having AI as a standalone tool where they have to, you know, sort of get out of their normal workload pivot, go do this, and then come back. Um, so for us doing it as a part of, [00:18:00] hey, this is the normal workload as we go through our contract writing system, this is part of our normal workflow to get an RFP off the street.
JEFF SAGE: How do we embed the AI in those critical places that it's not a. It doesn't seem as a big of a lift, right? It's a, Hey, I'm just working in my normal process. This thing's here. Hey gave me a cool answer that works for me. I can move forward. Right? So that's been, I think a big part for us and a big part of how I've been trying to shape the AI adoption at the at within procurement is to say, this is another tool in your tool belt.
JEFF SAGE: Mm-hmm. That's in line with what you've been doing already. 
SORAYA CORREA: Yeah, that's outstanding. Integrated into your regular tool set. Well, I know that we're out of town. Look at that. Perfect. Huh? Perfect. I wanna thank you Andrew, and especially Jeff. Thank you so much both, both of you, for being here. Absolutely.