submitted articles & preprints

1. Self-concordant Smoothing for Convex Composite Optimization

   A. D. Adeoye, and A. Bemporad

   Submitted 2023

   Abs PDF Code

   We introduce the notion of self-concordant smoothing for minimizing the sum of two convex functions: the first is smooth and the second may be nonsmooth. Our framework results naturally from the smoothing approximation technique referred to as partial smoothing in which only a part of the nonsmooth function is smoothed. The key highlight of our approach is in a natural property of the resulting problem’s structure which provides us with a variable-metric selection method and a step-length selection rule particularly suitable for proximal Newton-type algorithms. In addition, we efficiently handle specific structures promoted by the nonsmooth function, such as l1-regularization and group-lasso penalties. We prove local quadratic convergence rates for two resulting algorithms: Prox-N-SCORE, a proximal Newton algorithm and Prox-GGN-SCORE, a proximal generalized Gauss-Newton (GGN) algorithm. The Prox-GGN-SCORE algorithm highlights an important approximation procedure which helps to significantly reduce most of the computational overhead associated with the inverse Hessian. This approximation is essentially useful for overparameterized machine learning models and in the mini-batch settings. Numerical examples on both synthetic and real datasets demonstrate the efficiency of our approach and its superiority over existing approaches.

2. An Inexact Sequential Quadratic Programming Method for Learning and Control of Recurrent Neural Networks

   A. D. Adeoye, and A. Bemporad

   Submitted 2022

journal articles

2023

1. 

   SCORE: approximating curvature information under self-concordant regularization

   A. D. Adeoye, and A. Bemporad

   Computational Optimization and Applications 2023

   Abs Bib HTML PDF

   Optimization problems that include regularization functions in their objectives are regularly solved in many applications. When one seeks second-order methods for such problems, it may be desirable to exploit specific properties of some of these regularization functions when accounting for curvature information in the solution steps to speed up convergence. In this paper, we propose the SCORE (self-concordant regularization) framework for unconstrained minimization problems which incorporates second-order information in the Newton-decrement framework for convex optimization. We propose the generalized Gauss-Newton with Self-Concordant Regularization (GGN-SCORE) algorithm that updates the minimization variables each time it receives a new input batch. The proposed algorithm exploits the structure of the second-order information in the Hessian matrix, thereby reducing computational overhead. GGN-SCORE demonstrates how to speed up convergence while also improving model generalization for problems that involve regularized minimization under the proposed SCORE framework. Numerical experiments show the efficiency of our method and its fast convergence, which compare favorably against baseline first-order and quasi-Newton methods. Additional experiments involving non-convex (overparameterized) neural network training problems show that the proposed method is promising for non-convex optimization.

   @article{adeoye2023score,
     title = {SCORE: approximating curvature information under self-concordant regularization},
     author = {Adeoye, A. D. and Bemporad, A.},
     journal = {Computational Optimization and Applications},
     pages = {1--28},
     year = {2023},
     publisher = {Springer},
   }

2020

1. 

   Effects of radiative heat and magnetic field on blood flow in an inclined tapered stenosed porous artery

   J. U. Abubakar, and A. D. Adeoye

   Journal of Taibah University for Science 2020

   Abs Bib HTML

   A porous tapered inclined stenosed artery under the influence of magnetic field with radiation was considered. The momentum and energy equations with thin radiation governing the blood flow in the inclined artery were obtained taking the flow to be Newtonian. These equations were simplified under assumptions of mild stenosis, non-dimensionalized and solved using Differential Transform Method (DTM). The DTM were coded on Mathematica software to obtain expressions for velocity, temperature and the volumetric flow rate of the blood. The results presented graphically show that the velocity of the blood flow and the blood temperature decreases as the radiation parameter (N) increases.

   @article{abubakar2020effects,
     title = {Effects of radiative heat and magnetic field on blood flow in an inclined tapered stenosed porous artery},
     author = {Abubakar, J. U. and Adeoye, A. D.},
     journal = {Journal of Taibah University for Science},
     volume = {14},
     number = {1},
     pages = {77--86},
     year = {2020},
     publisher = {Taylor \& Francis},
   }

technical reports & theses

1. arXiv

   SC-Reg: Training Overparameterized Neural Networks under Self-Concordant Regularization

   A. D. Adeoye, and A. Bemporad

   2021

   Abs Bib PDF

   In this paper we propose the SC-Reg (self-concordant regularization) framework for learning overparameterized feedforward neural networks by incorporating second-order information in the Newton decrement framework for convex problems. We propose the generalized Gauss-Newton with Self-Concordant Regularization (SCoRe-GGN) algorithm that updates the network parameters each time it receives a new input batch. The proposed algorithm exploits the structure of the second-order information in the Hessian matrix, thereby reducing the training computational overhead. Although our current analysis considers only the convex case, numerical experiments show the efficiency of our method and its fast convergence under both convex and non-convex settings, which compare favorably against baseline first-order methods and a quasi-Newton method.

   @techreport{adeoye2021sc,
     title = {SC-Reg: Training Overparameterized Neural Networks under Self-Concordant Regularization},
     author = {Adeoye, A. D. and Bemporad, A.},
     institution = {IMT School for Advanced Studies Lucca},
     year = {2021},
   }

2. MSc

   A Deep Neural Network Optimization Method Via A Traffic Flow Model

   A. D. Adeoye, and P. Petersen

   2021

   Abs Bib PDF

   We present, via the solution of nonlinear parabolic partial differential equations (PDEs), a continuous-time formulation for stochastic optimization algorithms used for training deep neural networks. Using continuous-time formulation of stochastic differential equations (SDEs), relaxation approaches like the stochastic gradient descent (SGD) method are interpreted as the solution of nonlinear PDEs that arise from modeling physical problems. We reinterpret, through homogenization of SDEs, the modified SGD algorithm as the solution of the viscous Burgers’ equation that models a highway traffic flow.

   @techreport{adeoye2021dnn,
     title = {A Deep Neural Network Optimization Method Via A Traffic Flow Model},
     author = {Adeoye, A. D. and Petersen, P.},
     institution = {African Institue for Mathematical Sciences, Rwanda},
     year = {2021},
   }

3. MSc

   Blood Flow in an Inclined Tapered Stenosed Porous Artery under the Influence of Magnetic Field and Heat Transfer

   A. D. Adeoye

   2018

   Abs Bib PDF

   A tapered inclined porous artery with stenosis was considered under the influence of magnetic field and heat transfer. The mathematical formulation for the momentum and energy equations of the blood flow considered to be Newtonian were obtained. The energy equation which was obtained by taking an extra factor of heat source and the nonlinear momentum equation were simplified under the assumption of mild stenosis. These equations were non-dimensionalized and solved using Differential Transform Method (DTM) to obtain expressions for velocity, temperature and volumetric flow rate. The graphs of the expressions were plotted against radius of the artery to simulate the effects of magnetic field, heat transfer and other fluid parameters on the velocity, temperature and the volumetric flow rate of the blood. It was observed that as the magnetic field parameter (M) increases, the velocity, temperature and the volumetric flow rate of the blood increase but wall shear stress decreases at the stenosis throat. It was further observed that the effects of heat transfer and magnetic field resulted into a greater variation in the volumetric flow of an inclined artery in the converging region than in the diverging region.

   @mastersthesis{adeoye2018thesis,
     title = {Blood Flow in an Inclined Tapered Stenosed Porous Artery under the Influence of Magnetic Field and Heat Transfer},
     author = {Adeoye, A. D.},
     institution = {African Institue for Mathematical Sciences, Cameroon},
     year = {2018},
   }

4. BSc

   ON SOME FINITE DIFFERENCE METHODS FOR SOLVING PARTIAL DIFFERENTIAL EQUATIONS

   A. D. Adeoye

   2016