Previous |  Up |  Next


Title: Discounted Markov control processes induced by deterministic systems (English)
Author: Cruz-Suárez, Hugo
Author: Montes-de-Oca, Raúl
Language: English
Journal: Kybernetika
ISSN: 0023-5954
Volume: 42
Issue: 6
Year: 2006
Pages: 647-664
Summary lang: English
Category: math
Summary: This paper deals with Markov Control Processes (MCPs) on Euclidean spaces with an infinite horizon and a discounted total cost. Firstly, MCPs which result from the deterministic controlled systems will be analyzed. For such MCPs, conditions that permit to establish the equation known in the literature of Economy as Euler’s Equation (EE) will be given. There will be also presented an example of a Markov Control Process with deterministic controlled system where, to obtain the optimal value function, EE applied to the value iteration algorithm will be used. Secondly, the MCPs which result from the perturbation of deterministic controlled systems with a random noise will be dealt with. There will be also provided the conditions which allow to obtain the optimal value function and the optimal policy of a perturbed controlled system, in terms of the optimal value function and the optimal policy of deterministic controlled system corresponding. Finally, several examples to illustrate the last case mentioned will be presented. (English)
Keyword: discounted Markov control process
Keyword: deterministic control system
Keyword: Euler equation
Keyword: deterministic control system perturbed by a random noise
MSC: 90C40
MSC: 93E20
idZBL: Zbl 1249.90312
idMR: MR2296506
Date available: 2009-09-24T20:19:46Z
Last updated: 2015-03-29
Stable URL:
Reference: [1] Benveniste L. M., Scheinkman J. A.: On the differentiability of the value function in dynamic models of economics.Econometrica 47 (1979), 727–732 Zbl 0435.90031, MR 0533081, 10.2307/1910417
Reference: [2] Bertsekas D. P.: Dynamic Programming: Deterministic and Stochastic Models.Prentice-Hall, Englewood Cliffs, New Jersey 1987 Zbl 0649.93001, MR 0896902
Reference: [3] Cruz-Suárez D., Montes-de-Oca, R., Salem-Silva F.: Conditions for the uniqueness of optimal policies of discounted Markov decision processes.Math. Methods Oper. Res. 60 (2004), 415–436 Zbl 1104.90053, MR 2106092, 10.1007/s001860400372
Reference: [4] Fuente A. De la: Mathematical Methods and Models for Economists.Cambridge University Press, New York 2000 Zbl 0943.91001, MR 1735968
Reference: [5] Duffie D.: Security Markets.Academic Press, Boston 1988 Zbl 0861.90019, MR 0955269
Reference: [6] Durán J.: On dynamic programming with unbounded returns.J. Econom. Theory 15 (2000), 339–352 Zbl 1101.91339, MR 1789235, 10.1007/s001990050016
Reference: [7] Heer B., Maußner A.: Dynamic General Equilibrium Modelling: Computational Method and Application.Springer-Verlag, Berlin 2005 MR 2378171
Reference: [8] Hernández-Lerma O.: Adaptive Markov Control Processes.Springer-Verlag, New York 1989 MR 0995463
Reference: [9] Hernández-Lerma O., Lasserre J. B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria.Springer-Verlag, New York 1996 Zbl 0840.93001, MR 1363487
Reference: [10] Van C. Le, Morhaim L.: Optimal growth models with bounded or unbounded returns: a unifying approach.J. Econom. Theory 105 (2002), 158–187 Zbl 1013.91079, MR 1912663, 10.1006/jeth.2001.2880
Reference: [11] Levhari D., Srinivasan T. N.: Optimal savings under uncertainty.Rev. Econom. Stud. 36 (1969), 153–164 10.2307/2296834
Reference: [12] Mirman L. J.: Dynamic models of fishing: a heuristic approach.In: Control Theory in Mathematical Economics (Pan-Tai Liu and J. G. Sutinen, eds.), Marcel Dekker, New York 1979, pp. 39–73 Zbl 0432.90024
Reference: [13] Rincón-Zapatero J. L., Rodríguez-Palmero C.: Existence and uniqueness of solutions to the Bellman equation in the unbounded case.Econometrica 71 (2003), 1519–1555 Zbl 1160.49304, MR 2000255, 10.1111/1468-0262.00457
Reference: [14] Santos M. S.: Numerical solution of dynamic economic models.In: Handbook of Macroeconomic, Volume I (J. B. Taylor and M. Woodford, eds.), North Holland, Amsterdam 1999, pp. 311–386
Reference: [15] Stokey N. L., Lucas R. E.: Recursive Methods in Economic Dynamics.Harvard University Press, Cambridge, Mass. 1989 Zbl 0774.90018, MR 1105087


Files Size Format View
Kybernetika_42-2006-6_2.pdf 765.9Kb application/pdf View/Open
Back to standard record
Partner of
EuDML logo