Exponential Moving Average in RL

Description:

• Moving Average
• Traditional average: $Avg(x)=\frac{1}{N}\sum _n{x}_n$
  • Need to have all samples at once
• Exponential MA:
  • The running interpolation update: ${\bar{x}}_n=(1-\alpha ).{\bar{x}}_{n-1}+\alpha .x_n$
  • Makes recent samples have more weights: ${\bar{x}}_n=\frac{x_n+(1-\alpha ).x_{n-1}+(1-\alpha {)}^2.x_{n-2}+...}{1+(1-\alpha )+(1-\alpha {)}^2+...}$
• Decreasing (alpha) can give converging averages