Lecturer: Giuseppe De Nicolao  

Course name: Model Identification and Data Analysis
Course code: 500543
Degree course: Ingegneria Elettronica e Informatica
Disciplinary field of science: ING-INF/04
L'insegnamento è caratterizzante per: Ingegneria Elettronica e Informatica
University credits: ECTS 12
Course website: n.d.

Specific course objectives

Knowledge of basic notions of: estimation theory (maximum likelihood estimation, a-posteriori estimation); neural-based model identification; stochastic processes (mean, autocovariance, spectral density, optimal prediction); identification of ARMAX models. Ability to solve identification and prediction problems ranging from model formulation to the use of computer tools (Matlab) for parameter estimation and model simulation.

Course programme

System Identification deals with methodologies that enable the construction of mathematical models of systems and signals based on experimental data. In presence of complex systems whose behavior can be hardly reduced to known "laws of nature", the use of identification techniques is often the only way to obtain models to be used in the context of forecasting, simulation, and control. The methods presented in the course are widely used in heterogeneous fields such as automation, biomedical engineering, econometry, hydrology, geophysics and telecommunications. Some basic notions of probability, estimation theory and stochastic processes are recalled. The main properties (stability, input-output description in the time and frequenct domains) of linear discrete-time systems are introduced. In the context of parametric estimation, the issues of model validation and model complexity are extensively discussed. Neural based identification is also illustrated and discussed, pointing out pros and cons with respect to standard approaches. The study of dynamic systems addresses three main topics: the optimal prediction of stationary stochastic processes (Wiener filtering), the identification of linear discrete-time systems, and spectral estimation (both nonparametric and maximum-entropy).

Probability: basic notions

• probability notion;
• independence, conditional probability, total probability and Bayes theorems;
• Bernoulli trials, Poisson events;
• the notion of random variable (R.V.), cumulative distribution function, probability density function, functions on one R.V.;
• mode, median, moments of a R.V.;
• joint random variables: distribution, density, moments, independence, incorrelation, functions of random variables;
• Law of Lrge Numbers, Gaussian R.V., Central Limit Theorem.

Statistics: basic notions

• notion of estimator; properties of estimators;
• sample moments and their main properties;
• confidence interval for the sample mean, Student's t.

Identification of linear-in-parameter models:

• the least squares method, normal equations, identifiability;
• Best Linear Unbiased Estimator: estimator, variance of parameters;
• validation and choice of complexity: chi-square test, F-test, FPE, AIC, and MDL criteria.

Estimation theory:

• maximum likelihood estimation: properties and examples;
• a-posteriori estimation, Bayes estimator;
• cross-validation, model complexity and the bias-variance dilemma;
• identification of nonlinear-in-parameter models.

Neural identification:

• Radial basis function neural networks;
• Multi-layer perceptron networks;
• generalization, overfitting, selection of network size.

Stochastic processes and optimal prediction:

• mean, autocorrelation, autocovariance, independence, incorrelation;
• white noise, random walk, MA, AR, and ARMA processes, Yule-Walker equations;
• stationarity, power spectral density, nonparametric spectral estimatiom;
• spectral factorization, optimal prediction.

Identification of dynamic systems:

• classes of dynamic models: output error, ARX, ARMAX;
• prediction-error methods for system identification;
• least-squares identification of ARX models: probabilistic analysis and persistent excitation.

Course entry requirements

Basic notions of set theory, logic, calculus, function maximization.

Course structure and teaching

Lectures (hours/year in lecture theatre): 75
Practical class (hours/year in lecture theatre): 25
Practicals / Workshops (hours/year in lecture theatre): 8

Suggested reading materials

A. Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw-Hill.

T. Söderstrom, P. Stoica. System identification. Prentice-Hall.

Testing and exams

Written Exam Closed-book, closed-notes, 2 hour written exam consisting of 4 sections assessing knwoledge and understanding of the course topics and ability to apply them in a problem solving context. Each section will be independently graded Threshold to pass is 18/30 an maximum mark is 30/30 cum laude. The final mark is obtained as the weighted mean of marks given to each section of the written exam.