About DML-CZ | FAQ | Conditions of Use | Math Archives | Contact Us
  Previous |  Up |  Next

Article

Anh, Nguyen Thi Van ; Heldt, Adrian ; Khan, Akhtar Ali ; Tammer, Christiane
First- and second-order adjoint methods for stochastic identification problems. (English). Applications of Mathematics, vol. 70 (2025), issue 6, pp. 763-795
MSC: 35R30, 49N45, 65J20, 65J22, 65M30 | DOI: 10.21136/AM.2025.0151-25
Full entry | Fulltext not available (moving wall 24 months)      Feedback
Keywords:
stochastic inverse problem; partial differential equations with random data; stochastic Galerkin method; regularization; first-order adjoint method; second-order adjoint method
Summary:
We present a unified framework for estimating stochastic parameters in general variational problems. This nonlinear inverse problem is formulated as a stochastic optimization problem using the output least-squares (OLS) objective, which minimizes the discrepancy between observed data and the computed solution. A key challenge in OLS-based formulations is the efficient computation of first- and second-order derivatives of the OLS functional, which depend on the corresponding derivatives of the parameter-to-solution map often costly and difficult to evaluate, especially in stochastic settings. To address this, we develop a rigorous computational approach based on first- and second-order adjoint methods for inverse problems governed by stochastic variational problems. Specifically, we propose a new first-order adjoint method for computing the gradient of the OLS objective and introduce two novel second-order adjoint methods for Hessian evaluation. A stochastic Galerkin discretization framework is employed, enabling efficient implementation of the adjoint-based derivative computations. Numerical experiments demonstrate the accuracy and efficiency of the proposed computational framework.
References:
[1] Alekseev, A. K., Navon, I. M.: The analysis of an ill-posed problem using multi-scale resolution and second-order adjoint techniques. Comput. Methods Appl. Mech. Eng. 190 (2001), 1937-1953. DOI 10.1016/S0045-7825(00)00299-1 | MR 1813303 | Zbl 1031.76040
[2] Alekseev, A. K., Navon, I. M., Steward, J. L.: Comparison of advanced large-scale minimization algorithms for the solution of inverse ill-posed problems. Optim. Methods Softw. 24 (2009), 63-87. DOI 10.1080/10556780802370746 | MR 2489430 | Zbl 1189.90221
[3] Alexanderian, A., Petra, N., Stadler, G., Ghattas, O.: Mean-variance risk-averse optimal control of systems governed by PDEs with random parameter fields using quadratic approximations. SIAM/ASA J. Uncertain. Quantif. 5 (2017), 1166-1192. DOI 10.1137/16M106306X | MR 3725286 | Zbl 1391.93289
[4] Altaf, M. U., Gharamti, M. El, Heemink, A. W., Hoteit, I.: A reduced adjoint approach to variational data assimilation. Comput. Methods Appl. Mech. Eng. 254 (2013), 1-13. DOI 10.1016/j.cma.2012.10.003 | MR 3002758 | Zbl 1297.93053
[5] Arens, K., Rentrop, P., Stoll, S. O., Wever, U.: An adjoint approach to optimal design of turbine blades. Appl. Numer. Math. 53 (2005), 93-105. DOI 10.1016/j.apnum.2004.11.003 | MR 2128516 | Zbl 1116.49326
[6] Narayanan, V. A. Badri, Zabaras, N.: Stochastic inverse heat conduction using a spectral approach. Int. J. Numer. Methods Eng. 60 (2004), 1569-1593. DOI 10.1002/nme.1015 | MR 2068927 | Zbl 1098.80008
[7] Benner, P., Onwunta, A., Stoll, M.: Block-diagonal preconditioning for optimal control problems constrained by PDEs with uncertain inputs. SIAM J. Matrix Anal. Appl. 37 (2016), 491-518. DOI 10.1137/15M1018502 | MR 3483160 | Zbl 1382.65074
[8] Boger, D. A., Paterson, E. G.: A continuous adjoint approach to design optimization in cavitating flow using a barotropic model. Comput. Fluids 101 (2014), 155-169. DOI 10.1016/j.compfluid.2014.06.014 | MR 3233426 | Zbl 1391.76062
[9] Borggaard, J., Wyk, H.-W. van: Gradient-based estimation of uncertain parameters for elliptic partial differential equations. Inverse Probl. 31 (2015), Article ID 065008, 33 pages. DOI 10.1088/0266-5611/31/6/065008 | MR 3350627 | Zbl 1321.35028
[10] Borzì, A., Winckel, G. von: Multigrid methods and sparse-grid collocation techniques for parabolic optimal control problems with random coefficients. SIAM J. Sci. Comput. 31 (2009), 2172-2192. DOI 10.1137/070711311 | MR 2516148 | Zbl 1196.35029
[11] Cacuci, D. G.: Second-order adjoint sensitivity analysis methodology (2nd-ASAM) for computing exactly and efficiently first- and second-order sensitivities in large-scale linear systems. I. Computational methodology. J. Comput. Phys. 284 (2015), 687-699. DOI 10.1016/j.jcp.2014.12.042 | MR 3303639 | Zbl 1352.65130
[12] Cacuci, D. G.: Second-order adjoint sensitivity analysis methodology (2nd-ASAM) for computing exactly and efficiently first- and second-order sensitivities in large-scale linear systems. II. Illustrative application to a paradigm particle diffusion problem. J. Comput. Phys. 284 (2015), 700-717. DOI 10.1016/j.jcp.2014.11.030 | MR 3303640 | Zbl 1352.65131
[13] Cahill, N. D., Jadamba, B., Khan, A. A., Sama, M., Winkler, B. C.: A first-order adjoint and a second-order hybrid method for an energy output least-squares elastography inverse problem of identifying tumor location. Bound. Value Probl. 2013 (2013), Article ID 263, 19 pages. DOI 10.1186/1687-2770-2013-263 | MR 3341371 | Zbl 1292.35323
[14] Chen, P., Quarteroni, A., Rozza, G.: Stochastic optimal Robin boundary control problems of advection-dominated elliptic equations. SIAM J. Numer. Anal. 51 (2013), 2700-2722. DOI 10.1137/120884158 | MR 3115461 | Zbl 1281.49015
[15] Chen, P., Quarteroni, A., Rozza, G.: Multilevel and weighted reduced basis method for stochastic optimal control problems constrained by Stokes equations. Numer. Math. 133 (2016), 67-102. DOI 10.1007/s00211-015-0743-4 | MR 3475656 | Zbl 1344.93109
[16] Chen, Z., Zhang, W., Zou, J.: Stochastic convergence of regularized solutions and their finite element approximations to inverse source problems. SIAM J. Numer. Anal. 60 (2022), 751-780. DOI 10.1137/21M1409779 | MR 4403550 | Zbl 07506907
[17] Cho, M., Jadamba, B., Kahler, R., Khan, A. A., Sama, M.: First-order and second-order adjoint methods for the inverse problem of identifying non-linear parameters in PDEs. Industrial Mathematics and Complex Systems Springer, Singapore (2017), 147-163. DOI 10.1007/978-981-10-3758-0_9 | MR 3728376 | Zbl 1396.49028
[18] Cho, M., Jadamba, B., Khan, A. A., Oberai, A. A., Sama, M.: Identification in mixed variational problems by adjoint methods with applications. Modeling and Optimization: Theory and Applications Springer Proceedings in Mathematics & Statistics 213. Springer, Cham (2017), 65-84. DOI 10.1007/978-3-319-66616-7_5 | MR 3746179 | Zbl 1384.49009
[19] Choi, Y., Lee, H.-C.: Error analysis of finite element approximations of the optimal control problem for stochastic Stokes equations with additive white noise. Appl. Numer. Math. 133 (2018), 144-160. DOI 10.1016/j.apnum.2018.03.002 | MR 3843759 | Zbl 1397.65255
[20] Cioaca, A., Alexe, M., Sandu, A.: Second-order adjoints for solving PDE-constrained optimization problems. Optim. Methods Softw. 27 (2012), 625-653. DOI 10.1080/10556788.2011.610455 | MR 2946050 | Zbl 1260.49060
[21] Cioaca, A., Sandu, A.: An optimization framework to improve 4D-var data assimilation system performance. J. Comput. Phys. 275 (2014), 377-389. DOI 10.1016/j.jcp.2014.07.005 | MR 3250338 | Zbl 1349.90887
[22] Daescu, D. N., Navon, I. M.: Efficiency of a POD-based reduced second-order adjoint model in 4D-var data assimilation. Int. J. Numer. Methods Fluids 53 (2007), 985-1004. DOI 10.1002/fld.1316 | MR 2289211 | Zbl 1370.76122
[23] Dambrine, M., Khan, A. A., Sama, M.: A stochastic regularized second-order iterative scheme for optimal control and inverse problems in stochastic partial differential equations. Philos. Trans. R. Soc. Lond., A, Math. Phys. Eng. Sci. 380 (2022), Article ID 20210352, 13 pages. DOI 10.1098/rsta.2021.0352 | MR 4505744
[24] Dambrine, M., Khan, A. A., Sama, M., Starkloff, H.-J.: Stochastic elliptic inverse problems: Solvability, convergence rates, discretization, and applications. J. Convex Anal. 30 (2023), 851-885. MR 4650286 | Zbl 1522.35577
[25] Dippon, J., Gwinner, J., Khan, A. A., Sama, M.: A new regularized stochastic approximation framework for stochastic inverse problems. Nonlinear Anal., Real World Appl. 73 (2023), Article ID 103869, 29 pages. DOI 10.1016/j.nonrwa.2023.103869 | MR 4563581 | Zbl 1519.35367
[26] Dominguez, N., Gibiat, V., Esquerre, Y.: Time domain topological gradient and time reversal analogy: An inverse method for ultrasonic target detection. Wave Motion 42 (2005), 31-52. DOI 10.1016/j.wavemoti.2004.09.005 | MR 2133087 | Zbl 1189.74070
[27] Faverjon, B., Puig, B., Baranger, T. N.: Identification of boundary conditions by solving Cauchy problem in linear elasticity with material uncertainties. Comput. Math. Appl. 73 (2017), 494-504. DOI 10.1016/j.camwa.2016.12.011 | MR 3600206 | Zbl 1370.35275
[28] Gong, B., Liu, W., Tang, T., Zhao, W., Zhou, T.: An efficient gradient projection method for stochastic optimal control problems. SIAM J. Numer. Anal. 55 (2017), 2982-3005. DOI 10.1137/17M1123559 | MR 3730543 | Zbl 1386.60239
[29] Gunzburger, M. D., Lee, H.-C., Lee, J.: Error estimates of stochastic optimal Neumann boundary control problems. SIAM J. Numer. Anal. 49 (2011), 1532-1552. DOI 10.1137/100801731 | MR 2831060 | Zbl 1243.65008
[30] Guth, P. A., Kaarnioja, V., Kuo, F. Y., Schillings, C., Sloan, I. H.: A quasi-Monte Carlo method for optimal control under uncertainty. SIAM/ASA J. Uncertain. Quantif. 9 (2021), 354-383. DOI 10.1137/19M1294952 | MR 4241521 | Zbl 1475.49004
[31] Hawks, R., Jadamba, B., Khan, A. A., Sama, M., Yang, Y.: A variational inequality based stochastic approximation for inverse problems in stochastic partial differential equations. Nonlinear Analysis and Global Optimization Springer Optimization and Its Applications 167. Springer, Cham (2021), 207-226. DOI 10.1007/978-3-030-61732-5_9 | MR 4238578 | Zbl 1472.35451
[32] Huang, F., Chen, Y., Chen, Y., Sun, H.: Stochastic collocation for optimal control problems with stochastic PDE constraints by meshless techniques. J. Math. Anal. Appl. 530 (2024), Article ID 127634, 18 pages. DOI 10.1016/j.jmaa.2023.127634 | MR 4623961 | Zbl 07759006
[33] Jadamba, B., Khan, A. A., Oberai, A. A., Sama, M.: First-order and second-order adjoint methods for parameter identification problems with an application to the elasticity imaging inverse problem. Inverse Probl. Sci. Eng. 25 (2017), 1768-1787. DOI 10.1080/17415977.2017.1289195 | MR 3698480 | Zbl 1387.92057
[34] Jadamba, B., Khan, A. A., Raciti, F., Sama, M.: A variational inequality based stochastic approximation for estimating the flexural rigidity in random fourth-order models. Commun. Nonlinear Sci. Numer. Simul. 111 (2022), Article ID 106406, 11 pages. DOI 10.1016/j.cnsns.2022.106406 | MR 4402577 | Zbl 1489.35312
[35] Jadamba, B., Khan, A. A., Sama, M.: A regularized stochastic subgradient projection method for an optimal control problem in a stochastic partial differential equation. Mathematical Analysis in Interdisciplinary Research Springer Optimization and Its Applications 179. Springer, Cham (2021), 417-429. DOI 10.1007/978-3-030-84721-0_19 | MR 4404289 | Zbl 1500.93146
[36] Jadamba, B., Khan, A. A., Sama, M., Starkloff, H.-J., Tammer, C.: A convex optimization framework for the inverse problem of identifying a random parameter in a stochastic partial differential equation. SIAM/ASA J. Uncertain. Quantif. 9 (2021), 922-952. DOI 10.1137/20M1323953 | MR 4279154 | Zbl 1471.35332
[37] Jadamba, B., Khan, A. A., Sama, M., Yang, Y.: An iteratively regularized stochastic gradient method for estimating a random parameter in a stochastic PDEs: A variational inequality approach. J. Nonlinear Var. Anal. 5 (2021), 865-880. DOI 10.23952/jnva.5.2021.6.02 | Zbl 07557729
[38] Jensen, J. S., Nakshatrala, P. B., Tortorelli, D. A.: On the consistency of adjoint sensitivity analysis for structural optimization of linear dynamic problems. Struct. Multidiscip. Optim. 49 (2014), 831-837. DOI 10.1007/s00158-013-1024-4 | MR 3198981
[39] Jin, B., Zou, J.: Inversion of Robin coefficient by a spectral stochastic finite element approach. J. Comput. Phys. 227 (2008), 3282-3306. DOI 10.1016/j.jcp.2007.11.042 | MR 2392734 | Zbl 1141.65005
[40] Kennedy, G. J., Hansen, J. S.: The hybrid-adjoint method: A semi-analytic gradient evaluation technique applied to composite cure cycle optimization. Optim. Eng. 11 (2010), 23-43. DOI 10.1007/s11081-008-9068-9 | MR 2601731 | Zbl 1273.74389
[41] Keyanpour, M., Nehrani, A. M.: Optimal thickness of a cylindrical shell subject to stochastic forces. J. Optim. Theory Appl. 167 (2015), 1032-1050. DOI 10.1007/s10957-014-0663-y | MR 3424702 | Zbl 1332.74043
[42] Knopoff, D. A., Fernández, D. R., Torres, G. A., Turner, C. V.: Adjoint method for a tumor growth PDE-constrained optimization problem. Comput. Math. Appl. 66 (2013), 1104-1119. DOI 10.1016/j.camwa.2013.05.028 | MR 3093483 | Zbl 07863342
[43] Kourounis, D., Durlofsky, L. J., Jansen, J. D., Aziz, K.: Adjoint formulation and constraint handling for gradient-based optimization of compositional reservoir flow. Comput. Geosci. 18 (2014), 117-137. DOI 10.1007/s10596-013-9385-8 | MR 3208745 | Zbl 1393.76122
[44] Kubínová, M., Pultarová, I.: Block preconditioning of stochastic Galerkin problems: New two-sided guaranteed spectral bounds. SIAM/ASA J. Uncertain. Quantif. 8 (2020), 88-113. DOI 10.1137/19M125902X | MR 4051982 | Zbl 1447.65151
[45] Dimet, F.-X. Le, Navon, I. M., Daescu, D. N.: Second-order information in data assimilation. Mon. Weather Rev. 130 (2002), 629-648. DOI 10.1175/1520-0493(2002)130<0629:SOIIDA>2.0.CO;2
[46] Lewis, R. M.: Numerical computation of sensitivities and the adjoint approach. Computational Methods for Optimal Design and Control Progress in Systems and Control Theory 24. Birkhäuser Boston (1998), 285-302. DOI 10.1007/978-1-4612-1780-0_16 | MR 1646782 | Zbl 0924.65054
[47] Liu, G., Geier, M., Liu, Z., Krafczyk, M., Chen, T.: Discrete adjoint sensitivity analysis for fluid flow topology optimization based on the generalized lattice Boltzmann method. Comput. Math. Appl. 68 (2014), 1374-1392. DOI 10.1016/j.camwa.2014.09.002 | MR 3272547 | Zbl 1367.76044
[48] Lord, G. J., Powell, C. E., Shardlow, T.: An Introduction to Computational Stochastic PDEs. Cambridge Texts in Applied Mathematics. Cambridge University Press, New York (2014). DOI 10.1017/CBO9781139017329 | MR 3308418 | Zbl 1327.60011
[49] Lozano, C.: Discrete surprises in the computation of sensitivities from boundary integrals in the continuous adjoint approach to inviscid aerodynamic shape optimization. Comput. Fluids 56 (2012), 118-127. DOI 10.1016/j.compfluid.2011.12.001 | MR 2978800 | Zbl 1365.76100
[50] Zanganeh, M. Namdar, Kraaijevanger, J. F. B. M., Buurman, H. W., Jansen, J. D., Rossen, W. R.: Challenges in adjoint-based optimization of a foam EOR process. Comput. Geosci. 18 (2014), 563-577. DOI 10.1007/s10596-014-9412-4 | MR 3253767
[51] Nouy, A., Soize, C.: Random field representations for stochastic elliptic boundary value problems and statistical inverse problems. Eur. J. Appl. Math. 25 (2014), 339-373. DOI 10.1017/S0956792514000072 | MR 3197997 | Zbl 1298.60056
[52] Oberai, A. A., Gokhale, N. H., Feijóo, G. R.: Solution of inverse problems in elasticity imaging using the adjoint method. Inverse Probl. 19 (2003), 297-313. DOI 10.1088/0266-5611/19/2/304 | MR 1991790 | Zbl 1171.35490
[53] Papadimitriou, D. I., Giannakoglou, K. C.: Aerodynamic shape optimization using first and second order adjoint and direct approaches. Arch. Comput. Methods Eng. 15 (2008), 447-488. DOI 10.1007/s11831-008-9025-y | MR 2453310 | Zbl 1170.76348
[54] Pingen, G., Evgrafov, A., Maute, K.: Adjoint parameter sensitivity analysis for the hydrodynamic lattice Boltzmann method with applications to design optimization. Comput. Fluids 38 (2009), 910-923. DOI 10.1016/j.compfluid.2008.10.002 | MR 2645691 | Zbl 1242.76254
[55] Reséndiz, E., Pinnau, R.: Adjoint-based optimization of particle trajectories in laminar flows. Appl. Math. Comput. 248 (2014), 567-583. DOI 10.1016/j.amc.2014.10.007 | MR 3276711 | Zbl 1338.76027
[56] Shen, W., Ge, L., Liu, W.: Stochastic Galerkin method for optimal control problem governed by random elliptic PDE with state constraints. J. Sci. Comput. 78 (2019), 1571-1600. DOI 10.1007/s10915-018-0823-6 | MR 3934678 | Zbl 1428.65095
[57] Tottorelli, D. A., Michaleris, P.: Design sensitivity analysis: Overview and review. Inverse Probl. Eng. 1 (1994), 71-105. DOI 10.1080/174159794088027573
[58] Touzeau, C., Magnain, B., Lubineau, G., Florentin, E.: Robust method for identifying material parameters based on virtual fields in elastodynamics. Comput. Math. Appl. 77 (2019), 3021-3042. DOI 10.1016/j.camwa.2018.08.005 | MR 3958649 | Zbl 1442.74077
[59] Uy, W. I. T., Grigoriu, M.: Specification of additional information for solving stochastic inverse problems. SIAM J. Sci. Comput. 41 (2019), A2880--A2910. DOI 10.1137/18M120155X | MR 4008643 | Zbl 1439.35571
[60] Volkov, O., Protas, B., Liao, W., Glander, D. W.: Adjoint-based optimization of thermo-fluid phenomena in welding processes. J. Eng. Math. 65 (2009), 201-220. DOI 10.1007/s10665-009-9292-0 | MR 2546438 | Zbl 1180.76050
[61] Wang, S.-L., Yang, Y.-F., Zeng, Y.-H.: The adjoint method for the inverse problem of option pricing. Math. Probl. Eng. 2014 (2014), Article ID 314104, 7 pages. DOI 10.1155/2014/314104 | MR 3191166 | Zbl 1407.91275
[62] Wang, Y., Chen, P., Li, W.: Projected Wasserstein gradient descent for high-dimensional Bayesian inference. SIAM/ASA J. Uncertain. Quantif. 10 (2022), 1513-1532. DOI 10.1137/21M1454018 | MR 4512980 | Zbl 1506.62270
[63] Wang, Z., Navon, I. M., Dimet, F. X. Le, Zou, X.: The second order adjoint analysis: Theory and applications. Meteorology Atmospheric Phys. 50 (1992), 3-20. DOI 10.1007/BF01025501
[64] Ye, X., Li, P., Liu, F. Y.: Exact time-domain second-order adjoint-sensitivity computation for linear circuit analysis and optimization. IEEE Trans. Circuits Syst. I, Regul. Pap. 57 (2010), 236-248. DOI 10.1109/TCSI.2009.2015720 | MR 2729824 | Zbl 1469.94232
Partner of
EuDML logo