# High-performance aerial platforms

Autonomous functioning via real-time monitoring and information management is an attractive ingredient in the design of any complex system. The inevitable presence of uncertainties due to malfunctions, environmental variations, ageing, and modeling errors, requires this management to be adaptive. Several projects are ongoing in the area of high performance aerial platforms , and are listed below. Recent publications are marked with [HA] in the Publications page.Check out this podcast', 'on a historical overview of adaptive control and learning.

Check out this quadrotor video that shows how adaptive control copes with real-time uncertainties. The uncertainties were emulated in the form of a break in one of the propeller blades mid-flight, that in turn introduced an asymmetric reduction in thrust due to a loss of control-moment.

## A high-order tuner for accelerated performance and learning

Features in machine learning problems are often time varying and may be related to outputs in an algebraic or dynamical manner. The dynamic nature of these machine learning problems renders current accelerated gradient descent methods unstable or weakens their convergence guarantees. This project pertains to higher-order learning concepts that attempts to improve performance in the presence of time-varying regressors. Elements of convex optimization in the presence of constraints, high-order parameter learning, exponential convergence, and trade-off between performance and learning are being explored.

## Adaptive Control and Intersections with Reinforcement Learning

Adaptive control and reinforcement learning are two different methods that are both commonly employed for the control of uncertain systems. Historically, adaptive control has excelled at real-time control of systems with specific model structures through adaptive rules that learn the underlying parameters while providing strict guarantees on stability, asymptotic performance, and learning. Reinforcement learning methods are applicable to a broad class of systems and are able to produce near-optimal policies for highly complex control tasks. This is often enabled by significant offline training via simulation or the collection of large input-state datasets. This article attempts to compare adaptive control and reinforcement learning using a common framework. The problem statement in each field and highlights of their results are outlined. Two specific examples of dynamic systems are used to illustrate the details of the two methods, their advantages, and their deficiencies. The need for real-time control methods that leverage tools from both approaches is motivated through the lens of this common framework.

## Adaptive Control in the Presence of Rate Saturation

Actuator rate saturation nonlinearities are not often explicitly accounted for in the design of flight control systems. Rate saturated actuators pose the risk of failing, rendering a control system unstable and creating pilot induced oscillations (PIO). Current work is focused on how adaptation can occur in the presence of rate limits for general flight platform models. A filter placed in the control path accommodates the presence of rate limits, but introduces other challenges in analytical tractability. These challenges are overcome using an output feedback based adaptive controller. Analytical guarantees of bounded solutions and satisfactory tracking as well as numerical validations using a high-fidelity model have been provided.

## Shared Control and Cyber-Physical & Human Systems

As aerial vehicles become more autonomous, and guidance and navigation systems become increasingly network-centric, there is a need to consider a swift response to the growing forms of anomalies that may occur during operation. An on-going project in our lab is the development of a shared control architecture that includes the actions of both a human pilot and an autopilot to ensure resilient tracking performance in the presence of anomalies. Autonomous model-based controllers, including model reference adaptive control, rely on model-structures, specified performance goals, and assumptions on structured uncertainties. Trained human pilots, on the other hand, are able to detect anomalous vehicle behavior which differs from their internal model but are found to have limits when attempting to rapidly learn unfamiliar and anomalous vehicle dynamics. This problem is exacerbated when the human pilot is operating the vehicle from a remote ground station. The goal is to therefore examine shared control architectures where the pilot is tasked with higher-level decision making tasks such as anomaly detection, estimation and command regulation and the autopilot is assigned lower-level tasks such as command following. A general goal here is to understand how such cyber-physical & human systems can be designed for safe and efficient performance.

## Adaptation and Optimization

There is a concerted effort ongoing in understanding the fundamental relationships between adaptation, learning, and optimization. While adaptation is necessarily a concept that is based on the past and the present, optimization is focused on the future; learning is a link between these two foundational concepts. Several directions are being pursued to gain this understanding. The following are some of the highlights:### An Integral Approach that Combines Adaptive Control and Reinforcement Learning

We address the problem of real-time control and learning in dynamic systems subjected to parametric uncertainties through a combination of Adaptive Control (AC) in the inner loop and a Reinforcement Learning (RL) based policy in the outer loop. This AC-RL combination allows the inner-loop AC contracts the closed-loop dynamics towards a reference system, and as the contraction takes hold, the RL in the outerloop directs the overall system towards optimal performance. Two classes of nonlinear dynamic systems are considered, both of which are control-affine. The first class of dynamic systems utilizes equilibrium points with expansion forms around these points and employs a Lyapunov approach while second class of nonlinear systems uses contraction theory. AC-RL controllers are proposed for both classes of systems and shown to lead to online policies that guarantee stability using a high-order tuner and accommodate parametric uncertainties and magnitude limits on the input. In addition to establishing a stability guarantee with real-time control, the AC-RL controller is also shown to lead to parameter learning with persistent excitation for the first class of systems. Numerical validations of all algorithms are carried out using a quadrotor landing task on a moving platform. These results point out the clear advantage of the proposed integrative AC-RL approach.The videos below show the AC-RL approach applied to a quadrotor landing on a moving platform. For a 25% parametric uncertainty, we observed that the AC-RL succeeded (2nd video below) to complete the landing task 80% of the time in a time interval of 3.5seconds. In contrast, RL (1st video below) completed the task 48% of the time with an average time of about 7.5 seconds. Success here is defined not only in terms of landing on the platform but also making sure that the altitude never goes below that of the platform, and in terms of severe restrictions on lateral variables. In the video shown, for the same initial condition, the RL crashed (shown via **) while the AC-RL did not.

### Algorithms for Accelerated Learning

We address the problem of real-time control and learning in dynamic systems subjected to parametric uncertainties through a combination of Adaptive Control (AC) in the inner loop and a Reinforcement Learning (RL) based policy in the outer loop. This AC-RL combination allows the inner-loop AC contracts the closed-loop dynamics towards a reference system, and as the contraction takes hold, the RL in the outerloop directs the overall system towards optimal performance. Two classes of nonlinear dynamic systems are considered, both of which are control-affine. The first class of dynamic systems utilizes equilibrium points with expansion forms around these points and employs a Lyapunov approach while second class of nonlinear systems uses contraction theory. AC-RL controllers are proposed for both classes of systems and shown to lead to online policies that guarantee stability using a high-order tuner and accommodate parametric uncertainties and magnitude limits on the input. In addition to establishing a stability guarantee with real-time control, the AC-RL controller is also shown to lead to parameter learning with persistent excitation for the first class of systems. Numerical validations of all algorithms are carried out using a quadrotor landing task on a moving platform. These results point out the clear advantage of the proposed integrative AC-RL approach.### Indirect Adaptive Optimal Control In the Presence of Input Saturation

We have developed a combined Magnitude Saturated Adaptive Control (MSAC)-Model Predictive Control (MPC) approach for linear quadratic optimal control problems with parametric uncertainties and input saturation. The proposed MSAC-MPC approach first focuses on a stable solution and parameter estimation, and switches to MPC when parameter learning is accomplished. We show that the MSAC, based on a high-order tuner, leads to parameter convergence to true values while providing stability guarantees. We also show that after switching to MPC, the optimality gap is well defined and proportional to the parameter estimation error. The approach is validated using a linear two-input unstable plant.### A high-order tuner for discrete-time adaptive control

The problem is discrete-time direct adaptive control of a class of nonlinear systems in the presence of noisy disturbances. We apply the high-order tuner developed in the AAC laboratory to the problem of direct adaptive control in the presence of noisy disturbances. We explore how high-order tuners accommodate noisy inputs.