Quantifying the Sim-To-Real Gap in UAV Disturbance Rejection
Диагностика непериодических аномалий при выполнении политик обучения с подкреплением (краткое сообщение)
2024-01-01
SCID: 54.1/fgvf583w
Discuss with AI
Jensen-Shannon divergencemeasurement autoencoderoctorotor flight controlreinforcement learning controllersim-to-real transfer
Figures from the paper
Abstract (AI)
Due to the safety risks and training sample inefficiency, it is often preferred to develop controllers in simulation. However, minor differences between the simulation and the real world can cause a significant sim-to-real gap. This gap can reduce the effectiveness of the developed controller. In this paper, we examine a case study of transferring an octorotor reinforcement learning controller from simulation to the real world. First, we quantify the effectiveness of the real-world transfer by examining safety metrics. We find that although there is a noticeable (around 100%) increase in deviation in real flights, this deviation may not be considered unsafe, as it will be within > 2m safety corridors. Then, we estimate the densities of the measurement distributions and compare the Jensen-Shannon divergences of simulated and real measurements. From this, we show that the vehicle’s orientation is significantly different between simulated and real flights. We attribute this to a different flight mode in real flights where the vehicle turns to face the next waypoint. We also find that the reinforcement learning controller actions appear to correctly counteract disturbance forces. Then, we analyze the errors of a measurement autoencoder and state transition model neural network applied to real data. We find that these models further reinforce the difference between the simulated and real attitude control, showing the errors directly on the flight paths. Finally, we discuss important lessons learned in the sim-to-real transfer of our controller.
Key Findings
1
An octorotor reinforcement-learning controller transferred from simulation to real flights with approximately 100% increased deviation, yet remained within >2 m safety corridors.
2
Jensen–Shannon divergence analysis showed that vehicle orientation distributions differ significantly between simulated and real flights.
3
Measurement autoencoder and state-transition-model errors on real data reinforced and visibly localized the simulated-versus-real attitude-control discrepancy along flight paths.
4
The real vehicle adopted a distinct flight mode, turning toward the next waypoint, which likely caused the observed attitude-control mismatch.
5
The reinforcement-learning controller’s actions appeared to counteract disturbance forces correctly despite the simulation-to-reality differences.
Research Object
Octorotor reinforcement learning flight controller transferred from simulation to real-world flights
Research Subject
the sim-to-real transfer effectiveness and non-intermittent anomalies in the controller’s flight behavior, including safety deviations, orientation differences, and attitude-control errors
Publication Details
Publication Date
2024-01-01
Journal
Publisher
ISSN
Cited by
4
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest