Article
Full entry |
PDF
(0.3 MB)
Feedback
PDF
(0.3 MB)
Feedback
Keywords:
dynamic programming method; optimal stochastic control
dynamic programming method; optimal stochastic control
Summary:
This work analyzes a discrete-time Markov Control Model (MCM) on Borel spaces when the performance index is the expected total discounted cost. This criterion admits unbounded costs. It is assumed that the discount rate in any period is obtained by using recursive functions and a known initial discount rate. The classic dynamic programming method for finite-horizon case is verified. Under slight conditions, the existence of deterministic non-stationary optimal policies for infinite-horizon case is proven. Also, to find deterministic non-stationary $\epsilon-$optimal policies, the value-iteration method is used. To illustrate an example of recursive functions that generate discount rates, we consider the expected values of stochastic processes, which are solutions of certain class of Stochastic Differential Equations (SDE) between consecutive periods, when the initial condition is the previous discount rate. Finally, the consumption-investment problem and the discount linear-quadratic problem are presented as examples; in both cases, the discount rates are obtained using a SDE, similar to the Vasicek short-rate model.
References:
[1] Arnold, L.: Stochastic Differential Equations: Theory and Applications. John Wiley and Sons, New York 1973. DOI 10.1002/zamm.19770570413 | MR 0443083 | Zbl 0278.60039
[2] Ash, R., Doléans-Dade, C.: Probability and Measure Theory. Academic Press, San Diego 2000. MR 1810041 | Zbl 0944.60004
[3] Bellman, R.: Dynamic Programming. Princeton Univ. Press, New Jersey 1957. MR 0090477 | Zbl 1205.90002
[4] Bertsekas, D., Shreve, S.: Stochastic Optimal Control: The Discrete Time Case. Athena Scientific, Massachusetts 1996. MR 0511544 | Zbl 0633.93001
[5] Brigo, D., Mercurio, F.: Interest Rate Models Theory and Practice. Springer-Verlag, New York 2001. DOI 10.1007/978-3-662-04553-4 | MR 1846525 | Zbl 1109.91023
[6] Black, F., Karasinski, P.: Bond and option pricing when short rates are lognormal. Financ. Anal. J. 47 (1991), 4, 52-59. DOI 10.2469/faj.v47.n4.52
[7] Carmon, Y., Shwartz, A.: Markov decision processes with exponentially representable discounting. Oper. Res. Lett. 37 (2009), 51-55. DOI 10.1016/j.orl.2008.10.005 | MR 2488083 | Zbl 1154.90610
[8] Vecchia, E. Della, Marco, S. Di, Vidal, F.: Dynamic programming for variable discounted Markov decision problems. In: Jornadas Argentinas de Informática e Investigación Operativa (43JAIIO) - XII Simposio Argentino de Investigación Operativa (SIO), Buenos Aires, 2014, pp. 50-62.
[9] Cox, J., Ingersoll, J., Ross, S.: A theory of the term structure of interest rates. Econometrica 53 (1985), 385-407. DOI 10.2307/1911242 | MR 0785475 | Zbl 1274.91447
[10] Dothan, U.: On the term structure interest rates. J. Financ. Econ. 6 (1978), 59-69. DOI 10.1016/0304-405x(78)90020-x
[11] Feinberg, E., Shwartz, A.: Markov decision models with weighted discounted criteria. J. Finan. Econ. 19 (1994), 152-168. DOI 10.1287/moor.19.1.152 | MR 1290017 | Zbl 0803.90123
[12] González-Hernández, J., López-Martínez, R., Pérez-Hernández, J.: Markov control processes with randomized discounted cost. Math. Method Oper. Res. 65 (2006), 1, 27-44. DOI 10.1007/s00186-006-0092-2 | MR 2302022 | Zbl 1126.90075
[13] González-Hernández, J., López-Martínez, R., Minjarez-Sosa, A.: Adaptive policies for stochastic systems under a randomized discounted cost criterion. Bol. Soc. Mat. Mex. 14 (2008), 3, 149-163. MR 2667162 | Zbl 1201.93130
[14] González-Hernández, J., López-Martínez, R., Minjarez-Sosa, A.: Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion. Kybernetika 45 (2008), 5, 737-754. MR 2599109 | Zbl 1190.93105
[15] Guo, X., Hernández-Del-Valle, A., Hernández-Lerma, O.: First passage problems for a non-stationary discrette-time stochastic control systems. Eur. J. Control 15 (2012), 7, 528-538. DOI 10.3166/ejc.18.528-538 | MR 3086896
[16] Hernández-Lerma, O., Lasserre, J. B.: Discrete-Time Markov Control Processes. Basic Optimality Criteria. Springer-Verlag, New York 1996. DOI 10.1007/978-1-4612-0729-0 | MR 1363487 | Zbl 0840.93001
[17] Minjarez-Sosa, J.: Markov control models with unknown random sate-action-dependent discount factors. TOP 23 (2015), 3, 743-772. DOI 10.1007/s11750-015-0360-5 | MR 3407674
[18] Hinderer, K.: Foundations of non-stationary dynamical programming with discrete time parameter. In: Lecture Notes Operations Research (M. Bechmann and H. Künzi, eds.), Springer-Verlag 33, Zürich 1970. DOI 10.1007/978-3-642-46229-0 | MR 0267890
[19] Ho, T., Lee, S.: Term structure movements and pricing interest rate claims. J. Financ. 41 (1986), 1011-1029. DOI 10.1111/j.1540-6261.1986.tb02528.x
[20] Hull, J.: Options, Futures and other Derivatives. Sixth edition. Prentice Hall, New Jersey 2006.
[21] Hull, J., White, A.: Pricing interest rate derivative securities. Rev. Financ. Stud. 3 (1990), 573-592. DOI 10.1093/rfs/3.4.573
[22] Mercurio, F., Moraleda, J.: A family of humped volatility models. Eur. J. Finance 7 (2001), 93-116. DOI 10.1080/13518470151141440
[23] Rendleman, R., Bartter, B.: The pricing of options on debt securities. J. Financ. Quant. Anal. 15 (1980), 11-24. DOI 10.2307/2979016
[24] Rieder, U.: Measurable selection theorems for optimization problems. Manuscripta Math. 24 (1978), 115-131. DOI 10.1007/bf01168566 | MR 0493590 | Zbl 0385.28005
[25] Schäl, M.: Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Z. Wahrscheinlichkeit 32 (1975), 179-196. DOI 10.1007/bf00532612 | MR 0378841 | Zbl 0316.90080
[26] Vasicek, O.: An equilibrium characterization of the term structure. J. Financ. Econ. 5 (1977), 177-188. DOI 10.1016/0304-405x(77)90016-2
[27] Wei, Q., X, X. Guo: Markov decision processes with state-dependent discount factors and unbounded rewards costs. Oper. Res. Lett. 39 (2011), 369-374. DOI 10.1016/j.orl.2011.06.014 | MR 2835530 | Zbl 1235.90178
[28] Ye, L., Guo, X.: Continuous-time Markov decision processes with state-dependent discount factors. Acta Appl. Math. 121 (2012), 1, 5-27. DOI 10.1007/s10440-012-9669-3 | MR 2966962 | Zbl 1281.90082
[29] Zhang, Y.: Convex analytic approach to constrained discounted Markov decision processes with non-constant discount factors. TOP 21 (2013), 2, 378-408. DOI 10.1007/s11750-011-0186-8 | MR 3068494 | Zbl 1273.90235
Search
Partner of
