Second-Order Differential Machine Learning

Abstract

High-dimensional stochastic models are indispensable in many real-world applications, such as biology, material science and quantitative finance. Stochastic differential equations have played a crucial role in advancing these domains, as noisy environments are integral to understanding complex phenomena beyond the scope of standard dynamical system theory. However, in practice, the simulation of stochastic models requires expensive Monte Carlo methods. The recent advancements in Machine Learning provide a promising avenue for creating much faster yet accurate surrogate models, as illustrated in Differential Machine Learning. It augments the typical supervised learning process with differential data labels obtained via automatic differentiation. This additional loss factor results in an effective, unbiased form of regularization.

We extend the learning process of neural network-based surrogate models with second-order derivative information. Using forward-over-reverse mode automatic differentiation and dimensionality reduction techniques, it is feasible to find relevant second-order hessian-vector products. If the three loss terms (payoff, differential payoff, and second-order differential payoff) are balanced correctly, the additional second-order information significantly enhances the final accuracy of the model. In a Bachelier model of a basket option with n correlated assets, we observe a doubling in the final model accuracy. An improvement that cannot be obtained through the prolonged execution of the original (differential) training setup. A further case study around the Heston model is considered. The ultimate goal is to create accurate surrogate models faster to bring effective pricing and online risk assessment of complex models closer to reality. Beyond finance, Second-Order Differential Machine Learning is a generally applicable tool for finding highly efficient, accurate surrogate models of existing stochastic numerical models.