Environment, policy, reward and value function defined for six tasks in financial markets
|
Deep Reinforcement Learning (pdf)
|
2.18 M |
Reinforcement learning has four elements, an environment with a state, a policy as the rule for acting in each state, a reward and a value function, the cumulative reward under a policy. In financial markets, these elements are defined for six tasks, optimal trade execution, portfolio optimisation, option pricing and hedging, electronic market making, robo-advising and smart order routing. For each task, state variables, actions, benchmarks and rewards are listed, for example the TWAP and VWAP benchmarks in trade execution. Market making has three sources of risk, inventory, execution and adverse selection. The setting is limit order books, over-the-counter trading and the stylised facts of financial time series. Students identify the states, actions and rewards of a use case in recap exercises. Open research directions are risk-sensitive learning, offline learning, sample efficiency and transfer to new assets.