Article
Full entry | Fulltext not available
(moving wall
24 months)
Feedback
Keywords:
unconstrained optimization; gradient descent method; line search; simple quadratic model
unconstrained optimization; gradient descent method; line search; simple quadratic model
Summary:
This study introduces an accelerated gradient descent method based on a non-monotone backtracking line search scheme. A simple adaptive quadratic model is enhanced by utilizing a real, positive definite scalar matrix derived from the Taylor expansion of the objective function, rather than relying on the exact Hessian. The global and superlinear convergence of the defined model is established under appropriate conditions. Numerical experiments on a set of standard unconstrained optimization problems and image restoration problems show that the new algorithm outperforms other comparable methods in terms of efficiency and robustness.
References:
[1] Ahookhosh, M., Amini, K.: An efficient nonmonotone trust-region method for unconstrained optimization. Numer. Algorithms 59 (2012), 523-540. DOI 10.1007/s11075-011-9502-5 | MR 2892563 | Zbl 1243.65066
[2] Andrei, N.: An acceleration of gradient descent algorithm with backtracing for unconstrained optimization. Numer. Algorithms 42 (2006), 63-73. DOI 10.1007/s11075-006-9023-9 | MR 2249567 | Zbl 1101.65058
[3] Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10 (2008), 147-161. MR 2424936 | Zbl 1161.90486
[4] Andrei, N.: Open problems in nonlinear conjugate gradient algorithms for unconstrained optimization. Bull. Malays. Math. Sci. Soc. (2) 34 (2011), 319-330. MR 2788405 | Zbl 1225.49030
[5] Armijo, L.: Minimization of functions having Lipschitz continuous first partial derivatives. Pac. J. Math. 16 (1966), 1-3. DOI 10.2140/pjm.1966.16.1 | MR 0191071 | Zbl 0202.46105
[6] Conn, A. R., Gould, N. I. M., Toint, P. L.: Trust-Region Methods. MPS/SIAM Series on Optimization 1. SIAM, Philadelphia (2000). DOI 10.1137/1.9780898719857 | MR 1774899 | Zbl 0958.65071
[7] Dolan, E. D., Moré, J. J.: Benchmarking optimization software with performance profiles. Math. Program. 91 (2002), 201-213. DOI 10.1007/s101070100263 | MR 1875515 | Zbl 1049.90004
[8] Goldstein, A. A.: On steepest descent. J. Soc. Ind. Appl. Math., Ser. A, Control 3 (1965), 147-151. DOI 10.1137/0303013 | MR 0184777 | Zbl 0221.65094
[9] Gonzales, R. C., Woods, R. E.: Digital Image Processing. Prentice Hall, Upper Saddle River (2008).
[10] Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton's method. SIAM. J. Numer. Anal. 23 (1986), 707-716. DOI 10.1137/0723046 | MR 0849278 | Zbl 0616.65067
[11] Hager, W., Zhang, H.: A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2 (2006), 35-58. MR 2548208 | Zbl 1117.90048
[12] Hestenes, M. R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49 (1952), 409-436. DOI 10.6028/jres.049.044 | MR 0050378 | Zbl 0048.09901
[13] Ivanov, B., Stanimirović, P. S., Milovanović, G. V., Djordjević, S., Brajević, I.: Accelerated multiple step-size methods for solving unconstrained optimization problems. Optim. Methods Softw. 36 (2021), 998-1029. DOI 10.1080/10556788.2019.1653868 | MR 4432929 | Zbl 1489.65085
[14] Li, D.-H., Fukushima, M., Qi, L., Yamashita, N.: Regularized Newton methods for convex minimization problems with singular solutions. Comput. Optim. Appl. 28 (2004), 131-147. DOI 10.1023/B:COAP.0000026881.96694.32 | MR 2072529 | Zbl 1056.90111
[15] Li, Q., Zheng, B., Zheng, Y.: An efficient nonmonotone adaptive cubic regularization method with line search for unconstrained optimization problem. Appl. Math. Lett. 98 (2019), 74-80. DOI 10.1016/j.aml.2019.05.040 | MR 3959825 | Zbl 1423.90141
[16] Mirzaei, S. H., Ashrafi, A.: Correction of nonmonotone trust region algorithm based on a modified diagonal regularized quasi-Newton method. J. Inequal. Appl. 2024 (2024), Article ID 90, 22 pages. DOI 10.1186/s13660-024-03161-x | MR 4768662 | Zbl 1546.49058
[17] Niri, T. D., Amini, K.: Effective nonmonotone trust region method based on a simple cubic model for unconstrained optimization problems. J. Appl. Math. Comput. 71 (2025), 707-723. DOI 10.1007/s12190-024-02241-x | MR 4854953 | Zbl 08078311
[18] Nocedal, J., Wright, S. J.: Numerical Optimization. Springer Series in Operations Research and Financial Engineering. Springer, New York (2006). DOI 10.1007/978-0-387-40065-5 | MR 2244940 | Zbl 1104.65059
[19] Petrović, M. J.: An accelerated double step size model in unconstrained optimization. Appl. Math. Comput. 250 (2015), 309-319. DOI 10.1016/j.amc.2014.10.104 | MR 3285540 | Zbl 1328.90115
[20] Petrović, M. J., Stanimirović, P. S.: Accelerated double direction method for solving unconstrained optimization problems. Math. Probl. Eng. 2014 (2014), Article ID 965104, 8 pages. DOI 10.1155/2014/965104 | MR 3193716 | Zbl 1407.90262
[21] Polyak, R. A.: Regularized Newton method for unconstrained convex optimization. Math. Program. 120 (2009), 125-145. DOI 10.1007/s10107-007-0143-3 | MR 2496429 | Zbl 1189.90121
[22] Stanimirović, P. S., Ivanov, B., Ma, H., Mosić, D.: A survey of gradient methods for solving nonlinear optimization. Electron. Res. Arch. 28 (2020), 1573-1624. DOI 10.3934/era.2020115 | MR 4182341 | Zbl 1458.65067
[23] Stanimirović, P. S., Miladinović, M. B.: Accelerated gradient descent methods with line search. Numer. Algorithms 54 (2010), 503-520. DOI 10.1007/s11075-009-9350-8 | MR 2670666 | Zbl 1198.65104
[24] Toint, P. L.: An assessment of nonmonotone line search techniques for unconstrained optimization. SIAM. J. Sci. Comput. 17 (1996), 725-739. DOI 10.1137/S106482759427021X | MR 1384262 | Zbl 0849.90113
[25] Ueda, K., Yamashita, N.: A regularized Newton method without line search for unconstrained optimization. Comput. Optim. Appl. 59 (2014), 321-351. DOI 10.1007/s10589-014-9656-x | MR 3253745 | Zbl 1302.90218
[26] Wolfe, P.: Convergence conditions for ascent methods. SIAM Rev. 11 (1969), 226-235. DOI 10.1137/1011036 | MR 0250453 | Zbl 0177.20603
[27] Yu, G., Huang, J., Zhou, Y.: A descent spectral conjugate gradient method for impulse noise removal. Appl. Math. Lett. 23 (2010), 555-560. DOI 10.1016/j.aml.2010.01.010 | MR 2602408 | Zbl 1186.94017
[28] Zhang, H., Hager, W. W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM. J. Optim. 14 (2004), 1043-1056. DOI 10.1137/S1052623403428208 | MR 2112963 | Zbl 1073.90024
[29] Zhang, H., Ni, Q.: A new regularized quasi-Newton algorithm for unconstrained optimization. Appl. Math. Comput. 259 (2015), 460-469. DOI 10.1016/j.amc.2015.02.032 | MR 3338382 | Zbl 1390.90527
[30] Zhang, H., Ni, Q.: A new regularized quasi-Newton method for unconstrained optimization. Optim. Lett. 12 (2018), 1639-1658. DOI 10.1007/s11590-018-1236-z | MR 3857025 | Zbl 1407.90313
[31] Zheng, Y., Zheng, B.: A modified adaptive cubic regularization method for large-scale unconstrained optimization problem. Available at https://arxiv.org/abs/1904.07440 (2019), 21 pages. DOI 10.48550/arXiv.1904.07440
Search
Partner of
