Code: 6193
Studies where it is offered: 1st.
curse of Statistical Science and
Techniques (mandatory)
5th.
curse of Matematics (optative)
Length: 4.5 theoretical credits + 1.5 practical credits
Calendar: spring semester 2001-2003
Teacher: Juan Ferrándiz
Topic 1.- Probabilistic space: Sigma-field
of events. Probability measures. Interpreting probability. Kolmogorov’s axioms.
Conditional probability and the learning process. Law of total probability.
Bayes’ rule. Independence.
Topic 2.- Conditional probability and
expectation: univariate and multivariate random variables. Borel
sigma-field in the n-dimensional Euclidean space. Joint and marginal
distributions. Computing probabilities by conditioning. Conditional
expectation. Computing expectations by conditioning.
Topic 3.- A
summary introduction to stochastic processes: Sequences of random
variables. Borel sigma-field of sequences of real numbers. Finite dimensional
distributions. Daniell-Kolmogorov theorem. Convergence in distribution,
convergence in probability, almost sure convergence and convergence in mean
square. Random walks. The laws of large numbers.
Topic 4.- Discrete-time Markov chains: Motivating examples. Markov
property. Transition probabilities. Chapman-Kolmogorov equations.
Finite-dimensional distributions. Classification of states. Transition graphs.
Periodicity.
Topic 5.- Limiting behaviour: Times between successive visits.
Recurrent and transient states in terms of
times between successive visits. Occupancy times. Recurrent and
transient states in terms of occupancy times.
Limit distribution. Stationary distribution. Occupancy distribution.
Conditions for existence and uniqueness. Reducible chains. First-passage times.
Topic 6.- The Poisson Process: Times between system transitions.
Counting processes. The exponential distribution as a model for waiting times.
Memoryless properties. Hazard rate. Minimum, maximum and sum of independent
exponential random variables. The Poisson process as a counting process. The
Poisson process as a process with stationary and independent increments.
Conditioning to the total number of events. Compound Poisson processes.
Topic 7.- Continuous-time Markov chains: Times between transitions and jump
probabilities. Time dependent transition probabilities. Transient analysis.
Auxiliary discrete-time Markov chain. Occupancy times. Limit distribution.
Stationary distribution. Occupancy distribution. Conditions for existence and
uniqueness.
Topic 8.- Generalized Markov Models: Motivation. Renewal processes.
Cumulative processes. Semi-Markov processes. Long-term analysis. First-passage
times. Occupancy distribution.
Practice 1.- Conditional probability: Statistical classification problems.
Prior and posterior distributions. Solving decision trees sequentially. Playing
the ‘siete y medio’ cards game.
Practice 2.- Simulating the process: Simulating random variables. Random
paths generation by conditioning to the past. Estimating the characteristics of
the stochastic process. Quality of the estimates.
Practice 3.- n-step transition probabilities: Computing n-step transition
probabilities. Spectral decomposition of transition probabilities matrices.
Interpreting transient and long-run behaviour in terms of the preceding
decompositions.
Practice 4.- Analysis of a case-study: Context perception. Elaborating a
suitable discrete-time Markov chain model. Simplicity versus realism. Long-run
and transient analysis. Criticism.
Practice 5.- The Poisson process: Simulating Poisson processes
inspired on real case-studies. Estimating the intensity of the process.
Verifying consequences of the memoriyless properties. Comparing with
simulations based on waiting time distributions other than exponential.
Hoel, P. G., Port, S. C. and Stone, C. J. (1972) Introduction to Stochastic Processes. Boston: Houghton Mifflin Co.
Karlin, S. and Taylor, H. M. (1975) A
first course in Stochastic Processes. New York: Academic Press.
Kulkarni, V. G. (1995) Modeling
and analysis of stochastic systems. London: Chapman and Hall.
Kulkarni, V. G. (1999) Modeling,
analysis, design and control of stochastic systems. New York:
Springer-Berlag.
Resnick, S. I. (1992) Adventures in
Stochastic Processes. Boston:
Birkhäuser.
Stirzaker, D.(1994) Elementary probability. Cambridge: Cambridge
University Press.
The purpose
of this module is to provide the student with a basic probabilistic tool in
order to model systems with random evolution. Motivating realistic examples are
always used to introduce significant concepts. A basic mathematical level is
kept all along the course, although it is enough to get a solid knowledge of
the main properties of stochastic processes. We want the student to model
simple real case-studies and to become ready to understand further developments
without being disoriented by mathematical sophistications.
Practical
sessions, focused on important topics, are planned to motivate student’s
reflection. Computers and adequate software (mainly matlab and R language) are
used as an auxiliary tool. Simulation is a crucial instrument and brings the
surprising behaviour of random systems close to the empirical student’s
perception.
The final
exam comprises a theoretical part (a short writing of a topic) and a practical
part (three problems, some conceptual, some computational). The report made
along the practical sessions of the course is mandatory and has to be delivered
before the final exam.