Solutions to Technical Exercises

Author

Andrew Pua

Published

October 5, 2022

Properties of summation notation

\[ \sum_{t=1}^n b = nb \tag{1}\]

\[ \sum_{t=1}^n aX_t = a\sum_{t=1}^n X_t \tag{2}\]

\[ \sum_{t=1}^n \left(X_t+Y_t\right) = \sum_{t=1}^n X_t + \sum_{t=1}^n Y_t \tag{3}\]

Technical Exercise 1

  1. Prove that \[ \sum_{t=1}^n \left(aX_t+bY_t\right) = a\sum_{t=1}^n X_t + b\sum_{t=1}^n Y_t \]

    • Solution: \[\begin{eqnarray}\sum_{t=1}^n \left(aX_t+bY_t\right) &=& \sum_{t=1}^n aX_t+\sum_{t=1}^n bY_t \\ &=& a\sum_{t=1}^n X_t+b\sum_{t=1}^n Y_t \end{eqnarray}\] The first equality follows from Equation 3. The second equality follows from Equation 2.

  2. Prove that you have three ways of expressing the following sum: \[ \begin{eqnarray} \sum_{t=1}^n \left(X_t-\overline{X}\right)\left(Y_t-\overline{Y}\right) &=& \sum_{t=1}^n \left(X_t-\overline{X}\right)Y_t \\ &=& \sum_{t=1}^n X_t\left(Y_t-\overline{Y}\right) \\ &=& \sum_{t=1}^n X_tY_t -n\overline{X}\cdot\overline{Y} \end{eqnarray} \]

    • First expression: \[\begin{eqnarray} \sum_{t=1}^n \left(X_t-\overline{X}\right)\left(Y_t-\overline{Y}\right) &=& \sum_{t=1}^n \left(X_t-\overline{X}\right)Y_t - \sum_{t=1}^n \left(X_t-\overline{X}\right)\overline{Y} \\ &=& \sum_{t=1}^n \left(X_t-\overline{X}\right)Y_t - \overline{Y} \sum_{t=1}^n \left(X_t-\overline{X}\right) \\ &=& \sum_{t=1}^n \left(X_t-\overline{X}\right)Y_t \end{eqnarray}\] The first equality follows from distributing the product and Equation 3. The second equality uses Equation 1. The third equality follows from the property of means (the average of deviations from the mean should be zero, refer to our board notes for more).
    • Third expression: \[\begin{eqnarray} \sum_{t=1}^n \left(X_t-\overline{X}\right)\left(Y_t-\overline{Y}\right) &=& \sum_{t=1}^n \left(X_tY_t-\overline{X}Y_t-X_t\overline{Y}+\overline{X}\cdot \overline{Y}\right) \\ &=& \sum_{t=1}^n X_tY_t - \sum_{t=1}^n \overline{X}Y_t - \sum_{t=1}^n X_t\overline{Y} +\sum_{t=1}^n\overline{X}\cdot \overline{Y} \\ &=& \sum_{t=1}^n X_tY_t - \overline{X} \sum_{t=1}^n Y_t - \overline{Y}\sum_{t=1}^n X_t +n \overline{X}\cdot \overline{Y} \\ &=& \sum_{t=1}^n X_tY_t - \overline{X}\cdot n\overline{Y} - \overline{Y}\cdot n\overline{X}+n\overline{X}\cdot \overline{Y} \\ &=& \sum_{t=1}^n X_tY_t - n\overline{X}\cdot \overline{Y} \end{eqnarray}\] The first equality follows from distributing the product. The second equality uses Equation 3. The third equality uses Equation 2 and Equation 1. The fourth equality rewrites the mean as as total. The fifth equality is just simplification.

Technical Exercise 2

  1. Is \(Y_t\) a linear transformation of \(X_t\)? Explain.

    • Yes. It is a combination of scaling (multiplying a constant \(a\)) and a translation (adding a constant \(b\)).

  2. Show that, in general, we must have \[\begin{eqnarray}\overline{Y} &=& a\cdot \overline{X} + b \\ \frac{1}{n}\sum_{t=1}^n \left(Y_t-\overline{Y}\right)^2 &=& a^2\cdot \frac{1}{n} \sum_{t=1}^n \left(X_t-\overline{X}\right)^2\end{eqnarray}\]

    • First expression: \[\overline{Y} = \frac{1}{n}\sum_{t=1}^n Y_t = \frac{1}{n}\sum_{t=1}^n\left(aX_t+b\right) = a\frac{1}{n}\sum_{t=1}^n X_t + \frac{1}{n}\sum_{t=1}^n b = a\cdot \overline{X} + b \] Here we apply the result of Technical Exercise 1 (or if you wish, Equation 3 and Equation 2) and the definition of the mean.

    • Second expression: \[\begin{eqnarray}\frac{1}{n}\sum_{t=1}^n \left(Y_t-\overline{Y}\right)^2 &=& \frac{1}{n}\sum_{t=1}^n \left(aX_t+b-a\cdot \overline{X} - b\right)^2 \\ &=& \frac{1}{n} \sum_{t=1}^n a^2\left(X_t-\overline{X}\right)^2 \\ &=& a^2\cdot \frac{1}{n} \sum_{t=1}^n \left(X_t-\overline{X}\right)^2\end{eqnarray}\] Here we apply the previous result \(\overline{Y}=a\cdot \overline{X} + b\) for the first equality. We then use Equation 2 for the third equality.

  3. If \(Y_t\) is on a standardized scale, what would \(a\) and \(b\) be equal to?

    • \(a\) would be equal to the reciprocal of the standard deviation of \(X\). In other words, letting \(s_x\) be the standard deviation of \(X\), we have \(a=1/s_x\). Next, \(b=-\overline{X}/s_x\).

  4. If \(Y_t\) is on a standardized scale, use some of your previous answers to find the mean and the standard deviation of the \(Y_t\)’s.

    • Since we know that \(\overline{Y}=a\cdot \overline{X} + b\), substituting the values for \(a\) and \(b\) in the previous item gives us \[\overline{Y}=\frac{\overline{X}}{s_x}-\frac{\overline{X}}{s_x}=0\]
    • Similarly for the standard deviation of \(Y\), we can apply the result from the previous item to have \[s_y^2 = \frac{1}{n}\sum_{t=1}^n \left(Y_t-\overline{Y}\right)^2= a^2\cdot \frac{1}{n} \sum_{t=1}^n \left(X_t-\overline{X}\right)^2= \left(\frac{1}{s_x}\right)^2 s_x^2=1\] Therefore, \(s_y=1\).1

Technical Exercise 3

\[\sum_{t=1}^n \left(Y_t-\widehat{Y}_t\right)^2=\sum_{t=1}^n \left(Y_t-\widehat{\beta}_0\right)^2=\sum_{t=1}^n Y_t^2-2\widehat{\beta}_0\sum_{t=1}^n Y_t+n\widehat{\beta}_0^2 \tag{4}\]

  1. Simplifying: \[\begin{eqnarray} \sum_{t=1}^n \left(Y_t-\widehat{\beta}_0\right)^2 &=& \sum_{t=1}^n \left(Y_t^2-2\widehat{\beta}_0 Y_t+ \widehat{\beta}_0^2\right) \\ &=& \sum_{t=1}^n Y_t^2-\sum_{t=1}^n 2\widehat{\beta}_0 Y_t+\sum_{t=1}^n\widehat{\beta}_0^2 \\ &=& \sum_{t=1}^n Y_t^2-2\widehat{\beta}_0\sum_{t=1}^n Y_t+n\widehat{\beta}_0^2 \end{eqnarray}\] The first equality follows from expanding the square. The second equality follows from Equation 3. The third equality follows from Equation 2 and Equation 1.

  2. Find the optimal value of \(\widehat{\beta}_0\). Does it agree with our example from the slides?

    • We calculate the first-order condition of the minimization problem. \[\begin{eqnarray}\frac{d}{d\widehat{\beta}_0} \left[\sum_{t=1}^n Y_t^2-2\widehat{\beta}_0\sum_{t=1}^n Y_t+n\widehat{\beta}_0^2\right] &=& 0 \\ -2 \sum_{t=1}^n Y_t+ 2n\widehat{\beta}_0&=& 0 \end{eqnarray}\] Solving for \(\widehat{\beta}_0\), we have \[\widehat{\beta}_0=\frac{1}{n}\sum_{t=1}^n Y_t\]
    • To check if this produces a minimum, we check the second-order conditions: \[\begin{eqnarray}\frac{d^2}{d\widehat{\beta}_0^2} \left[\sum_{t=1}^n Y_t^2-2\widehat{\beta}_0\sum_{t=1}^n Y_t+n\widehat{\beta}_0^2\right] &=& \frac{d}{d\widehat{\beta}_0} \left[-2 \sum_{t=1}^n Y_t+ 2n\widehat{\beta}_0 \right] \\ &=& 2n \end{eqnarray}\] The second derivative of the objective function is \(2n\) and it is greater than zero for our situation. Thus, \(\widehat{\beta}_0=\overline{Y}\) is a minimizer of Equation 4.
    • Yes, the optimal value of \(\widehat{\beta}_0\) is equal to the mean of \(Y\). This will agree with the slides. Of course, we cover the general case and the slides cover a specific example.

  3. Given #2, what is the minimized value of Equation 4? Is there a known name for this quantity?

    • The minimized value of Equation 4 is \[\sum_{t=1}^n \left(Y_t-\widehat{\beta}_0\right)^2= \sum_{t=1}^n \left(Y_t-\overline{Y}\right)^2\] We know this to be \(n\) (or \(n-1\)) times the variance of \(Y\) (or if you wish, proportional to the variance of \(Y\)).

  4. What will be the fitted value for each observation in this case?

    • The fitted value for each observation will just be \(\widehat{Y}_t = \overline{Y}\).

  5. What will be the residuals for each observation in this case?

    • The residual for each observation will just be \(Y_t-\widehat{Y}_t = Y_t-\overline{Y}\).

  6. What will be the average of the fitted values? Prove your finding.

    • The average of the fitted values will just be the mean of \(Y\) itself: \[\frac{1}{n}\sum_{t=1}^n \widehat{Y}_t = \frac{1}{n}\sum_{t=1}^n \overline{Y} = \frac{1}{n}\cdot n \overline{Y}=\overline{Y}\] The first equality follows from item #4. The second equality follows from Equation 1.

  7. What will be the average of the residuals? Prove your finding.

    • The average of the residuals is equal to zero: \[\frac{1}{n}\sum_{t=1}^n \left(Y_t-\widehat{Y}_t \right)= \frac{1}{n}\sum_{t=1}^n \left(Y_t-\overline{Y} \right) = 0\] The first equality follows from item #5. The second equality follows from the properties of the mean.

Footnotes

  1. Note that for the general case where \(Y\) is not necessarily on a standardized scale but was linearly transformed from \(X\), the result will then be \[s_y=|a|s_x\] Most will forget to put absolute value signs for \(a\). The square root of \(a^2\) is \(|a|\).↩︎