Projecte llegit
Títol: Towards Autonomous Vertiport Operations: A Deep Reinforcement Learning Perspective
Estudiants que han llegit aquest projecte:
MARTÍN GOUSHESH, NAYIB (data lectura: 12-09-2024)- Cerca aquest projecte a Bibliotècnica
MARTÍN GOUSHESH, NAYIB (data lectura: 12-09-2024)Director/a: BARRADO MUXÍ, CRISTINA
Departament: DAC
Títol: Towards Autonomous Vertiport Operations: A Deep Reinforcement Learning Perspective
Data inici oferta: 12-01-2024 Data finalització oferta: 12-09-2024
Estudis d'assignació del projecte:
- GR ENG SIST AEROESP
| Tipus: Individual | |
| Lloc de realització: Fora UPC | |
| Supervisor/a extern: Raquel García Lasheras | |
| Institució/Empresa: CRIDA AIE | |
| Paraules clau: | |
| Deep Reinforcement Learning, Deconfliction, U-space, AirSim, Last-Mile Delivery, Logistics, Vertiport, Advanced Air Mobility, Unmanned Traffic Management | |
| Descripció del contingut i pla d'activitats: | |
| En este trabajo se propone la realización de un servicio de separación táctica del U-space basado en el aprendizaje por refuerzo y entrenado mediante un entorno virtual realista. El aprendizaje por refuerzo es una rama de la AI que '
Nombrar algún proyecto de RL de SESAR En concreto se proponen los siguientes objetivos específicos que marcan las pautas del plan de trabajo: Obj1 ' Crear un entorno de simulación urbano sobre AirSim en el cual el tráfico UAM pueda incrementarse de forma que se llegue a provocar pérdidas de separación entre vehículos con una frecuencia suficientemente elevada Obj2 ' Desarrollar un servicio de gestión de tráfico que interactúe con AirSim y proponga la priorización necesaria entre vehículos que vayan a entrar en conflicto para evitarlo. Obj3 ' Definir las métricas necesarias que permitan medir la seguridad, la eficiencia y la equidad de la política de separación aplicada Y Obj4 ' Usar las métricas definidas y los casos de pérdida de separación generados para entrenar un sistema de aprendizaje por refuerzo alternativo al del Obj2 que pueda resultar en unos valores de calidad más elevados. La planificación es dedicar 1 mes a cada objectivo mas 1-2 meses adicionales a las pruebas y documentación. |
|
| Overview (resum en anglès): | |
| This project explored the feasibility of using Deep Reinforcement Learning (DRL) to manage drone traffic at a Vertiport dedicated to automated package delivery. A DRL agent, utilizing the Proximal Policy Optimization (PPO) algorithm, was trained to coordinate multiple drones concurrently, aiming to achieve safe and efficient operations within a simulated urban environment.
A realistic simulation environment was developed by integrating the AirSim simulator with a custom server, 'DroneManagerServer', built using the FastAPI framework. This environment enabled the creation of complex scenarios and provided a platform for training and evaluating the DRL agent's performance. The agent's learning process was guided by a meticulously designed reward function that penalized collisions and inefficient actions while incentivizing safe inter-drone separation and efficient completion of delivery missions. While demonstrating the potential of DRL for Vertiport management, the project also revealed challenges associated with applying DRL to real-world control problems. Limitations in simulator performance, the complexity of hyperparameter tuning, and the intricacies of reward function design influenced the agent's overall performance and limited the scope of evaluation to a single challenging scenario. The project's key contributions include: successfully demonstrating DRL's capability to learn complex drone coordination strategies; identifying specific challenges in applying DRL to drone traffic management (UTM); and developing a flexible DRL framework that can serve as a foundation for future research. Future research directions include: enhancing the simulation environment to accommodate higher drone densities and more realistic scenarios; exploring advanced DRL techniques to improve performance and scalability; and investigating methods to enhance the agent's robustness and generalization capabilities. Furthermore, integrating U-space regulations and exploring hybrid approaches that combine DRL with other AI techniques are identified as promising avenues for advancing DRL-based UTM. By addressing these future research directions, this project aims to contribute to the development of robust and reliable autonomous systems for managing drone traffic, ultimately facilitating the safe and efficient integration of drones into urban airspace and unlocking the full potential of Urban Air Mobility. |
|