# My Publications

- Feedback Linearization and PID Control of Aero Thrust Pendulum using FPGA.
- Reinforcement learning for optimal energy management of a solar microgrid.
- Distributed Optimization of Solar Micro-grid Using Multi Agent Reinforcement Learning.

## Feedback Linearization and PID Control of Aero Thrust Pendulum using FPGA.

This paper presents a model based design and implementation of a closed loop PID
controller for `Aero Thrust Pendulum`

. It was done using `MATLAB - System Generator Toolbox`

, to
demonstrate the Direct Real Time Simulation and Implementation `(DRTSI)`

scheme.
A method for live tuning of proportional, integral, derivative and system gain is validated and implemented to facilitate robust
tuning. This serves to highlight the effect of each component of the PID controller on
the response of the system for `educational`

purposes. Also, `Serial`

communication is established between FPGA and
MATLAB/Simulink to facilitate extensive logging of the system’s output and provides
visual representation of the system’s response for various gains.

Keywords - MATLAB – System Generator; FPGA-Serial; Aero Thrust pendulum; FPGA-ADC

## Reinforcement learning for optimal energy management of a solar microgrid.

In an optimization based control approach for solar microgrid energy management, consumer as an agent continuously interacts with the environment and learns to take optimal actions autonomously to reduce the power consumption from grid. A model-free Reinforcement Learning algorithm, namely three-step-ahead Q-learning, is used to optimize the battery scheduling in dynamic environment of load and available solar power. Solar power and the load feed the reinforcement learning algorithm. Simulation results using real numerical data are presented for a reliability test of the system. The uncertainties in the solar power and the load are taken into account in the proposed control framework.

Keywords - Optimization; Q-learning; Reinforcement learning; Solar microgrid.

## Distributed Optimization of Solar Micro-grid Using Multi Agent Reinforcement Learning.

In the distributed optimization of micro-grid, we consider grid connected solar micro-grid system which contains a local consumer, a solar photovoltaic system and a battery. The consumer as an agent continuously interacts with the environment and learns to take optimal actions. Each agent uses a model-free reinforcement learning algorithm, namely Q Learning, to optimize the battery scheduling in dynamic environment of load and available solar power. Multiple agents sense the states of the environment components and make collective decisions about how to respond to randomness in load, intermittent solar power using a Multi-Agent Reinforcement Learning algorithm, called Coordinated Q Learning (CQL). The goals of each agent are to increase the utility of the battery and solar power in order to achieve the long term objective of reducing the power consumption from grid.

Keywords - Solar micro-grid; Multi-agent Reinforcement Learning; CQ-learning; battery scheduling; optimization