Policies.klUCBloglog_forGLR module¶

The generic kl-UCB policy for one-parameter exponential distributions with restarted round count t_k. By default, it assumes Bernoulli arms. Note: using log(t) + c log(log(t)) for the KL-UCB index of just log(t) - It is designed to be used with the wrapper GLR_UCB. - By default, it assumes Bernoulli arms. - Reference: [Garivier & Cappé - COLT, 2011](https://arxiv.org/pdf/1102.2490.pdf).

Policies.klUCBloglog_forGLR.c = 3¶: Default value when using \(f(t) = \log(t) + c \log(\log(t))\), as klUCB_forGLR is inherited from klUCBloglog.

Policies.klUCBloglog_forGLR.TOLERANCE = 0.0001¶: Default value for the tolerance for computing numerical approximations of the kl-UCB indexes.

class Policies.klUCBloglog_forGLR.klUCBloglog_forGLR(nbArms, tolerance=0.0001, klucb=<function klucbBern>, c=2, lower=0.0, amplitude=1.0)[source]¶

Bases: Policies.klUCB_forGLR.klUCB_forGLR

The generic KL-UCB policy for one-parameter exponential distributions, using a different exploration time step for each arm (\(\log(t_k) + c \log(\log(t_k))\) instead of \(\log(t) + c \log(\log(t))\)).

It is designed to be used with the wrapper GLR_UCB.
- By default, it assumes Bernoulli arms.
- Reference: [Garivier & Cappé - COLT, 2011](https://arxiv.org/pdf/1102.2490.pdf).

__init__(nbArms, tolerance=0.0001, klucb=<function klucbBern>, c=2, lower=0.0, amplitude=1.0)[source]¶

New generic index policy.

nbArms: the number of arms,
lower, amplitude: lower value and known amplitude of the rewards.

computeIndex(arm)[source]¶

Compute the current index, at time t and after \(N_k(t)\) pulls of arm k:

\[\begin{split}\hat{\mu}_k(t) &= \frac{X_k(t)}{N_k(t)}, \\ U_k(t) &= \sup\limits_{q \in [a, b]} \left\{ q : \mathrm{kl}(\hat{\mu}_k(t), q) \leq \frac{\log(t) + c \log(\max(1, \log(t)))}{N_k(t)} \right\},\\ I_k(t) &= U_k(t).\end{split}\]

If rewards are in \([a, b]\) (default to \([0, 1]\)) and \(\mathrm{kl}(x, y)\) is the Kullback-Leibler divergence between two distributions of means x and y (see Arms.kullback), and c is the parameter (default to 1).

computeAllIndex()[source]¶: Compute the current indexes for all arms, in a vectorized manner.

__module__ = 'Policies.klUCBloglog_forGLR'¶