Data Driven Stabilization Schemes for Singularly Perturbed Differential Equations

Abstract

This thesis presents a novel way of leveraging Artificial Neural Network (ANN) to aid conventional numerical techniques for solving Singularly Perturbed Differential Equation (SPDE). SPDEs are challenging to solve with conventional numerical techniques such as Finite Element Methods (FEM) due to the presence of boundary and interior layers. Often the standard numerical solution shows spurious oscillations in the vicinity of these layers. Stabilization techniques are often employed to eliminate these spurious oscillations in the numerical solution. The accuracy of the stabilization technique depends on a user-chosen stabilization parameter whose optimal value is challenging to find. A few formulas for the stabilization parameter exist in the literature, but none extends well for high-dimensional and complex problems. In order to solve this challenge, we have developed the following ANN-based techniques for predicting this stabilization parameter: 1) SPDE-Net: As a proof of concept, we have developed an ANN called SPDE-Net for one-dimensional SPDEs. In the proposed method, we predict the stabilization parameter for the Streamline Upwind Petrov Galerkin (SUPG) stabilization technique. The prediction task is modelled as a regression problem using equation coefficients and domain parameters as inputs to the neural network. Three training strategies have been proposed, i.e. supervised learning, L 2-Error minimization (global) and L2-Error minimization (local). The proposed method outperforms existing state-of-the-art ANN-based partial differential equations (PDE) solvers, such as Physics Informed Neural Networks (PINNs).

2) AI-stab FEM With an aim for extending SPDE-Net for two-dimensional problems, we have also developed an optimization scheme using another Neural Network called AI-stab FEM and showed its utility in solving higher-dimensional problems. Unlike SPDE-Net, it minimizes the equation residual along with the crosswind derivative term and can be classified as an unsupervised method. We have shown that the proposed approach yields stable solutions for several two-dimensional benchmark problems while being more accurate than other contemporary ANN-based PDE solvers such as PINNs and Variational Neural Networks for the Solution of Partial Differential Equations (VarNet)

3) SPDE-ConvNet In the last phase of the thesis, we attempt to predict a cell-wise stabilization parameter to treat the interior/boundary layer regions adequately by developing an oscillations-aware neural network. We present SPDE-ConvNet, Convolutional Neural Network (CNN), for predicting the local (cell-wise) stabilization parameter. For the network training, we feed the gradient of the Galerkin solution, which is an indirect metric for representing oscillations in the numerical solution, along with the equation coefficients, to the network. It obtains a cell-wise stabilization parameter while sharing the network parameters among all the cells for an equation. Similar to AI-stab FEM, this technique outperforms PINNs and VarNet. We conclude the thesis with suggestions for future work that can leverage our current understanding of data-driven stabilization schemes for SPDEs to develop and improve the next-generation neural network-based numerical solvers for SPDEs.