Probability and Stochastic Processes

Code: 6193

Studies where it is offered:     1st. curse of  Statistical Science and Techniques (mandatory)

                                               5th. curse of Matematics (optative)

Length: 4.5 theoretical credits  + 1.5 practical credits

Calendar: spring semester 2001-2003

Teacher: Juan Ferrándiz

Program for theoretical lessons

Topic 1.- Probabilistic space: Sigma-field of events. Probability measures. Interpreting probability. Kolmogorov’s axioms. Conditional probability and the learning process. Law of total probability. Bayes’ rule.  Independence.

Topic 2.- Conditional probability and expectation: univariate and multivariate random variables. Borel sigma-field in the n-dimensional Euclidean space. Joint and marginal distributions. Computing probabilities by conditioning. Conditional expectation. Computing expectations by conditioning.

Topic 3.-  A summary introduction to stochastic processes: Sequences of random variables. Borel sigma-field of sequences of real numbers. Finite dimensional distributions. Daniell-Kolmogorov theorem. Convergence in distribution, convergence in probability, almost sure convergence and convergence in mean square. Random walks. The laws of large numbers.

Topic 4.- Discrete-time Markov chains: Motivating examples. Markov property. Transition probabilities. Chapman-Kolmogorov equations. Finite-dimensional distributions. Classification of states. Transition graphs. Periodicity.

Topic 5.- Limiting behaviour: Times between successive visits. Recurrent and transient states in terms of  times between successive visits. Occupancy times. Recurrent and transient states in terms of occupancy times.  Limit distribution. Stationary distribution. Occupancy distribution. Conditions for existence and uniqueness. Reducible chains. First-passage times.

Topic 6.- The Poisson Process: Times between system transitions. Counting processes. The exponential distribution as a model for waiting times. Memoryless properties. Hazard rate. Minimum, maximum and sum of independent exponential random variables. The Poisson process as a counting process. The Poisson process as a process with stationary and independent increments. Conditioning to the total number of events. Compound Poisson processes.

Topic 7.- Continuous-time Markov chains: Times between transitions and jump probabilities. Time dependent transition probabilities. Transient analysis. Auxiliary discrete-time Markov chain. Occupancy times. Limit distribution. Stationary distribution. Occupancy distribution. Conditions for existence and uniqueness.

Topic 8.- Generalized Markov Models: Motivation. Renewal processes. Cumulative processes. Semi-Markov processes. Long-term analysis. First-passage times. Occupancy distribution.

Program of practical sessions

Practice 1.- Conditional probability: Statistical classification problems. Prior and posterior distributions. Solving decision trees sequentially. Playing the ‘siete y medio’ cards game.

Practice 2.- Simulating the process: Simulating random variables. Random paths generation by conditioning to the past. Estimating the characteristics of the stochastic process. Quality of the estimates.

Practice 3.- n-step transition probabilities: Computing n-step transition probabilities. Spectral decomposition of transition probabilities matrices. Interpreting transient and long-run behaviour in terms of the preceding decompositions.

Practice 4.- Analysis of a case-study: Context perception. Elaborating a suitable discrete-time Markov chain model. Simplicity versus realism. Long-run and transient analysis. Criticism.

Practice 5.- The Poisson process: Simulating Poisson processes inspired on real case-studies. Estimating the intensity of the process. Verifying consequences of the memoriyless properties. Comparing with simulations based on waiting time distributions other than exponential.

 

BIBLIOGRAPHY

Hoel, P. G., Port, S. C. and Stone, C. J. (1972) Introduction to Stochastic Processes. Boston: Houghton Mifflin Co.

Karlin, S. and Taylor, H. M. (1975) A first course in Stochastic Processes. New York: Academic Press.

Kulkarni, V. G. (1995) Modeling and analysis of stochastic systems. London: Chapman and Hall.

Kulkarni, V. G. (1999) Modeling, analysis, design and control of stochastic systems. New York: Springer-Berlag.

Resnick, S. I. (1992) Adventures in Stochastic Processes.  Boston: Birkhäuser.

Stirzaker, D.(1994) Elementary probability. Cambridge: Cambridge University Press.

OBJECTIVES

The purpose of this module is to provide the student with a basic probabilistic tool in order to model systems with random evolution. Motivating realistic examples are always used to introduce significant concepts. A basic mathematical level is kept all along the course, although it is enough to get a solid knowledge of the main properties of stochastic processes. We want the student to model simple real case-studies and to become ready to understand further developments without being disoriented by mathematical sophistications.

Practical sessions, focused on important topics, are planned to motivate student’s reflection. Computers and adequate software (mainly matlab and R language) are used as an auxiliary tool. Simulation is a crucial instrument and brings the surprising behaviour of random systems close to the empirical student’s perception.

EVALUATION

The final exam comprises a theoretical part (a short writing of a topic) and a practical part (three problems, some conceptual, some computational). The report made along the practical sessions of the course is mandatory and has to be delivered before the final exam.