New method helps coordinate complex missions in changing environments
Researchers used digital models and reinforcement learning to improve how mission assets work together.
A new methodology integrates high-fidelity digital mission models with Reinforcement Learning (RL) for adaptive task allocation. This approach helps coordinate missions in uncertain and dynamic environments.
The method uses a Digital Engineering (DE) infrastructure. This infrastructure includes a digital mission model and agent-based simulation. Mission tactics management is formulated as a Markov Decision Process.
Reinforcement Learning (RL) is used to train a coordinator with Proximal Policy Optimization. This maps mission states to tactical actions from realized outcomes. An aerial firefighting mission with three scenarios was used to test the approach.
Single-scenario coordinators can be brittle. A coordinator trained on a forgiving scenario performs like random tactics in more demanding scenarios. A coordinator trained on both scenarios concurrently matches scenario-specific coordinators in their own scenarios.
Sensitivity analyses show that variability during training shapes performance after deployment. This includes variability in ignition location, weather, scenario, and training curricula. The methodology can be extended to fleet design and selection.
Key facts
- Coordinators trained on both scenarios concurrently outperform other policies.
- A coordinator trained on a forgiving scenario degrades to random tactics in demanding scenarios.
- The approach uses a Digital Engineering (DE) infrastructure with agent-based simulation.