ABERDEEN PROVING GROUND, Md.- Deciding on a name is never easy. From newborn babies to neighborhood softball leagues, the pressure attached to a name choice can be overwhelming. Namers face decisions like whether to stay safe, perhaps go with something tried and true, or to be daring, and possibly doom your team—or child, to a life a ridicule. Frank Frisby faced this very issue, and after vacillating between options, he turned to a trusted confidant that was in the midst of a meteoric rise.

It was May 2023, and Frisby, a data scientist with the U.S. Army Communications-Electronics Command Software Engineering Center, was entrusted with bestowing a name upon his team as they prepared to participate in an Army-led artificial intelligence challenge that would span the next four months. Frisby had been working with AI for years at this point, so the decision to turn to ChatGPT, the chatbot developed by OpenAI, seemed almost obvious.

Frisby typed in the parameters, anticipating the benefit of specificity with the software, “We are a team of computer scientists preparing to compete against other teams. We are tasked with providing a model for a ground vehicle that will improve visibility and autonomous function in an off-road environment. We are a military organization…”

The Chatbot began to spit out potential names. As the list grew, Frisby read through the options until, finally, he saw it.

Frisby brought it back to his teammates, Maluki Montgomery, a computer scientist with CECOM SEC, and Roy Trieu, a computer scientist assigned to U.S. Army Combat Capabilities Development Command but attached to CECOM SEC. They all agreed, this was it, and that’s when the AI Avengers first assembled.

“Maneuver in Realistic Environments” – Driving Autonomy in Ground Vehicles

Roughly five months later, in October 2023, the AI Avengers were being called as the winner of the second phase of the Deep Green Challenge. The CECOM team, with Frisby and Trieu working from Aberdeen Proving Ground, Maryland, and Montgomery from Fort Gregg-Adams, Virginia, had outpaced every other team with their model.

The Deep Green Challenge is an annual event managed by the Army Office of Business Transformation. For the last two challenges, parameters and applications have been supported through a partnership between OBT and the DEVCOM Army Research Laboratory. This year’s competition was broken into two parts, the latter of which, won by the AI Avengers, focused on the development of an AI model to support the vision of autonomous ground vehicles in a realistic environment. Specifically, OBT and ARL challenged the teams to facilitate a vehicle’s ability to identify and react given an outdoor combat environment with a particular focus on depth perception.

Phillip Osteen, a researcher with DEVCOM ARL, served as the lead for management of the challenge parameters. As a team lead within the Army’s Artificial Intelligence of Maneuver and Mobility Essential Research Project, Osteen was well suited to develop a problem statement that would challenge the teams and provide benefits via AI breakthroughs for the Army. The winning model, developed by the AI Avengers, would be applied to preexisting systems and within a project by AIMM and the Army as autonomous vehicle research and development continues.

“One of the key challenges for this data is that these systems have to be able to perform in different types of environments because the Army needs to operate anywhere,” Osteen said. “We’re focusing on off-road data for autonomous maneuver in realistic environments.”

“A Couple of Breakthroughs” – Securing a Win

The road ahead would be arduous, and the AI Avengers knew it.

In addition to the work on the competition, the team members still had normal work duties that needed attention. During a major portion of the summer, from late June to early August, the team dropped the competition all together.

“We almost gave up because of time constraints,” Frisby said. “We actually stopped working on it for a couple months.”

Frisby’s passion for AI is contagious. Radiating excitement about the field, he often grins as he gets excited about a topic. He is fastidious when he talks shop, keenly aware of his audience; especially when he speaks with those outside his field. He pulls back on the throttle when he explains more technical aspects of his work, like a pilot taking a civilian on a familiarization flight—best to slow down and avoid a mess in the cockpit.

As summer came to an end, the team learned that new tools were available to the competitors. Specifically, the teams could test their models more easily, and most importantly, they would now have results faster than before. With these changes, the team refocused and approached the challenge in creative ways.

Aside from exploiting new opportunities, another advantage came from a decision to not use an assistive tool.

Light detection and ranging, commonly known as Lidar, is a remote sensing method that uses lasers to measure depth perception and to generate three dimensional images. For the Deep Green Challenge, teams were permitted to use Lidar in building their models, but the AI Avengers decided against using the technology.

“Our team realized that we don’t need Lidar,” Frisby said. “As humans, we don’t have additional tooling like Lidar to see how far something is away from us.”

Aside from supporting the team’s overall project, forgoing the use of Lidar provided practical utility.

Osteen, the competition manager with DEVCOM ARL, said that Lidar usually emits a noticeable signal. He added that although it was allowed in this competition, eventually the Army’s autonomous vehicles will require a stealthier profile. A model not reliant on Lidar would, theoretically, support a more secure vehicle.

Into early autumn, the team’s testing results, and their confidence, steadily improved.

Montgomery, the AI Avenger from Fort Gregg-Adams, recalled a mood shift as the competition passed the halfway point.

“We kind of got rejuvenated, being able to get the immediate feedback in terms of how well the model was performing,” Montgomery said. “It was kind of addictive; you get that immediate feedback and then you can tweak things and improve them.”

Montgomery emits a tranquility when he speaks. He articulates clearly and concisely while remaining consistently calm as he explains nuanced concepts. A computer scientist with SEC for nearly 20 years, it’s difficult to discern whether his placid demeanor is the result of two decades of experience or his upbringing on the West Coast. More than likely, it’s both.

Heading into the end of the challenge, the team watched as the leaderboard looked more and more friendly.

“A week before the competition was coming to an end, we came across a couple of breakthroughs that let us overtake the leader,” Montgomery said.

After their breakthroughs, with only a few days left, the AI Avengers grabbed hold of their spot and refused to let go. The dance for first place was over.

“Leaders in This Space” – AI’s Impact on the Joint Force

After most competitions, even winners don’t have much to show for it. Outside of professional athletes, a better record, and a bit of pride is the most a team can hope for.

For Frisby, Montgomery, and Trieu, this win was more than a win.

“For the Army, I’m very happy that we can help move technology forward,” Trieu said. “That’s one of the dreams I have…to help move the Army a lot faster than we are right now.”

Trieu is noticeably enthusiastic about moving the Army forward. Though less experienced with AI than his teammates, his expertise in computer science is palpable. Trieu’s confidence is nearly as reassuring as his humility; an expert quick to acknowledge he may not know something but is excited to learn. With prior experience on Wall Street and in the private sector, his desire to advance the Army is assertive, yet caring.

“Every single second counts on Wall Street. When I joined the Army [workforce], I had the same thought. If the Solider can get information faster, it saves lives.” Trieu said.

It may seem like hyperbole, saving lives through software, but the model built by the AI Avengers is already showing tangible results. Frisby said that the model, which is still in a testing phase, has shown a 95% success rate. With continued success, the model will forge a path into new Army autonomous systems and in doing so, directly benefit the soldiers that will use them.

Osteen said that the Deep Green Challenge has been an exciting pipeline for autonomous vehicle innovation. According to him, the winning model of the Deep Green Challenge Phase I was applied to official Army autonomy programs within a matter of months. For Phase II a similar trajectory is expected after initial testing.

“That is how these things will impact not just Solider-machine interaction…,” said Osteen. “…but really just improve autonomy performance overall because perception is really critical.”

The impact is not lost on the AI Avengers.

Montgomery notes that AI has captured the world’s attention, and supporting the Army’s efforts in the field is incredibly meaningful.

“There’s a definite need for the Army, and the Department of Defense at large, to really be leaders in this space,” he said. “I firmly believe it’s a game changer.”

At some point in the near future, assuming testing continues using the team’s winning model, a U.S. Army autonomous vehicle system will be better able to operate in the harsh and austere reality that is combat.

Three CECOM teammates, bound by a commitment to the Army and innovation, went from uncertainty about their team’s name to assembling a model that could potentially change the Army’s AI landscape.

“This model gets to be used out there in the real world,” Frisby said. “Where it helps against adversaries, it helps make sure we protect U.S. interests. It makes me happy that I was able to participate in something like that, and to provide something like that to the Army.”