⧼exchistory⧽
5 exercise(s) shown, 0 hidden
BBy Bot
May 21'24
1. Prove the statement of Example.
2. Let $P_\theta$ denote the distribution of $X \sim \Bern(\theta)$. Show that
[$] \KL(P_\theta, P_{\theta'})\ge C(\theta-\theta')^2\,. [$]
BBy Bot
May 21'24

Let $\p_0$ and $\p_1$ be two probability measures. Prove that for any test $\psi$, it holds

[$] \max_{j=0,1}\p_j(\psi\neq j) \ge \frac14e^{-\KL(\p_0,\p_1)}\,. [$]

BBy Bot
May 21'24

For any $R \gt 0$, $\theta \in \R^d$, denote by $\cB_2(\theta, R)$ the (Euclidean) ball of radius $R$ and centered at $\theta$. For any $\eps \gt 0$ let $N=N(\eps)$ be the largest integer such that there exist $\theta_1, \ldots, \theta_N \in \cB_2(0,1)$ for which

[$] |\theta_i-\theta_j| \ge 2\eps [$]

for all $i \neq j$. We call the set $\{\theta_1, \ldots, \theta_N\}$ an $\eps$-packing of $\cB_2(0, 1)$.

• Show that there exists a constant $C \gt 0$ such that $N \le C_d/\eps^d$.
• Show that for any $x \in \cB_2(0,1)$, there exists $i=1, \ldots, N$ such that $|x-\theta_i|_2\le 2\eps$.
• Use (b) to conclude that there exists a constant $C' \gt 0$ such that $N \ge C'_d/\eps^d$.
BBy Bot
May 21'24

Show that the rate $\phi=\sigma^2d/n$ is the minimax rate of estimation over:

1. The Euclidean Ball of $\R^d$ with radius $\sqrt{\sigma^2d/n}$.
2. The unit $\ell_\infty$ ball of $\R^d$ defined by
[$] \cB_\infty(1)=\{\theta \in \R^d\,:\, \max_j|\theta_j|\le 1\} [$]
as long as $\sigma^2 \le n$.
3. The set of nonnegative vectors of $\R^d$.
4. The discrete hypercube $\frac{\sigma}{16\sqrt{n}}\{0,1\}^d$.
BBy Bot
May 21'24

Fix $\beta \ge 5/3, Q \gt 0$ and prove that the minimax rate of estimation over $\Theta(\beta, Q)$ with the $\|\cdot\|_{L_2([0,1])}$-norm is given by $n^{-\frac{2\beta}{2\beta+1}}$.

Hint: Consider functions of the form

[$] f_j=\frac{C}{\sqrt{n}}\sum_{i=1}^N \omega_{ji} \varphi_i [$]

where $C$ is a constant, $\omega_j \in \{0,1\}^N$ for some appropriately chosen $N$ and $\{\varphi_j\}_{j\ge 1}$ is the trigonometric basis.