Understanding Smoothing, Correlation, and Spectra in Environmental Data Analysis
Explore the interrelationships between smoothing, correlation, and power spectral density in environmental data analysis through topics like autocorrelation, cross-correlation, Fourier series, and more. Learn how to apply these concepts using MatLab for effective data analysis.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Environmental Data Analysis with MatLab 2ndEdition Lecture 19: Smoothing, Correlation and Spectra
SYLLABUS Lecture 01 Lecture 02 Lecture 03 Lecture 04 Lecture 05 Lecture 06 Lecture 07 Lecture 08 Lecture 09 Lecture 10 Lecture 11 Lecture 12 Lecture 13 Lecture 14 Lecture 15 Lecture 16 Lecture 17 Lecture 18 Lecture 19 Lecture 20 Lecture 21 Lecture 22 Lecture 23 Lecture 24 Lecture 25 Lecture 26 Using MatLab Looking At Data Probability and Measurement Error Multivariate Distributions Linear Models The Principle of Least Squares Prior Information Solving Generalized Least Squares Problems Fourier Series Complex Fourier Series Lessons Learned from the Fourier Transform Power Spectra Filter Theory Applications of Filters Factor Analysis Orthogonal functions Covariance and Autocorrelation Cross-correlation Smoothing, Correlation and Spectra Coherence; Tapering and Spectral Analysis Interpolation Linear Approximations and Non Linear Least Squares Adaptable Approximations with Neural Networks Hypothesis testing Hypothesis Testing continued; F-Tests Confidence Limits of Spectra, Bootstraps
Goals of the lecture examine interrelationships between smoothing, correlation and power spectral density
Autocorrelation and Cross-correlation
Autocorrelation Measure of correlation in time series at different lags u(t) t
Autocorrelation Measure of correlation in time series at different lags t t lag, multiply and sum area no lag a(t) 0 lag, t
Autocorrelation Measure of correlation in time series at different lags t t lag, multiply and sum area small lag a(t) 0 lag, t
Autocorrelation Measure of correlation in time series at different lags t t lag, multiply and sum area large lag a(t) 0 lag, t
Autocorrelation Measure of correlation in time series at different lags t t lag, multiply and sum area a(t) 0 lag, t
Autocorrelation Measure of correlation in time series at different lags t t lag, multiply and sum area a(t) 0 lag, t a(t)=u(t) u(t)
crooss-correlation Measure of correlation between two time series at different lags u(t) t v(t) t
crooss-correlation Measure of correlation between two time series at different lags u(t) t v(t) t c(t)=u(t) v(t)
important relationships c(t) = u(t) v(t) = u(-t)*v(t) c( ) = u*( ) v( ) a( )= |u( )|2
rough time series u(t) t sharp autocorrelation a(t) 0 lag, t wide spectrum |u( )|2 frequency, 0
smooth time series u(t) t wide autocorrelation a(t) 0 lag, t narrow spectrum |u( )|2 frequency, 0
v(t) t rough timeseries a(t) a(t) 0 lag, t 0 lag, t v(t) t a(t) 0 lag, t
Part 1 Smoothing a Time Series
smoothing as filtering (example of 3-point smoothing)
smoothing as filtering (example of 3-point smoothing) non-causal
fix-up allow for a delay
fix-up allow for a delay d dsmoothed and delayed = s * d s * dobs causal filter, s s
triangular smoothing filters 3 points si index, i 21 points si index, i
smoothing if Neuse River Hydrograph 4 x 10 2 d-obs 1 0 50 100 150 200 250 300 350 400 450 500 time, days 4 x 10 2 3-point 1 0 50 100 150 200 250 300 350 400 450 500 time, days 4 x 10 2 21-point 1 0 50 100 150 200 250 300 350 400 450 500 time, days
question how does smoothing effect the the autocorrelation of d
answer the autocorrelation of s acts as a smoothing filter on the autocorrelation of d
effect of smoothing on autocorrelation autocorrelation of smoothed time series
effect of smoothing on autocorrelation autocorrelation of smoothed time series everything written as convolution
effect of smoothing on autocorrelation autocorrelation of smoothed time series everything written as convolution regrouped
effect of smoothing on autocorrelation autocorrelation of smoothing filter autocorrelation of time series convolved with *
answer the autocorrelation of s acts as a smoothing filter on the autocorrelation of d
Part 2 What Makes a Good Smoothing Filter?
dsmoothed(t) = s(t) * dobs(t) then by the convolution theorem
dsmoothed(t) = s(t) * dobs(t) then by the convolution theorem so what s this look like?
example of a uniform or boxcar smoothing filter s(t) 1/T time, t 0 T
take Fourier Transform where sinc(x) = sin( x) / ( x)
A) T=3 1 |s| for L=3 0.5 0 0 5 10 15 20 25 30 35 40 45 50 frequency, Hz B) T=21 1 |s| for L=21 0.5 0 0 5 10 15 20 25 30 35 40 45 50 frequency, Hz
1 |s| for L=3 0.5 0 0 5 10 15 20 25 30 35 40 45 50 frequency, Hz B) T=21 1 |s| for L=21 falls off with frequency (good) 0.5 0 0 5 10 15 20 25 30 35 40 45 50 frequency, Hz
1 |s| for L=3 0.5 0 0 5 10 15 20 25 30 35 40 45 50 frequency, Hz B) T=21 1 bumpy side lobes (bad) |s| for L=21 0.5 0 0 5 10 15 20 25 30 35 40 45 50 frequency, Hz
a box car filter does not suppress high frequencies evenly the challenge find a filter that suppresses high frequencies evenly
Normal Function Fourier Transform of a Normal Function is a Normal Function (which has no side lobes)
B) T=3 1 A) L=3 |s| for L=3 0.5 0 0 5 10 15 20 25 30 35 40 45 50 frequency, Hz B) T=21 1 |s| for L=21 0.5 0 0 5 10 15 20 25 30 35 40 45 50 frequency, Hz
but a Normal Function is non-causal (unless you truncate it, in which case it is not exactly a Normal Function)
Box Car sidelobes simplicity Triangle Normal Function
Part 3 Designing a Filter that Suppresses Specific Frequencies
General form of the IIR Filter z-transform
General form of the IIR Filter ratio of polynomials z-transform
