Jupyter Notebooks :notebook: by Naereen @ GitHub¶
This folder hosts some Jupyter Notebooks, to present in a nice format some numerical experiments for my SMPyBandits project.
The wonderful Jupyter tools is awesome to write interactive and nicely presented :snake: Python simulations!
1. List of experiments presented with notebooks¶
MAB problems¶
- Easily creating various Multi-Armed Bandit problems, explains the interface of the
Environment.MAB
module.
Single-Player simulations¶
- A simple example of Single-Player simulation, comparing
UCB1
(for two values of $\alpha$, 1 and 1/2),Thompson Sampling
,BayesUCB
andkl-UCB
. - Do we even need UCB? demonstrates the need for an algorithm smarter than the naive
EmpiricalMeans
. - Lai-Robbins lower-bound for doubling-tricks algorithms with full restart.
Active research on Single-Player MAB¶
Multi-Player simulations¶
- A simple example of Multi-Player simulation with 4 Centralized Algorithms, comparing
CentralizedMultiplePlay
andCentralizedIMP
withUCB
andThompson Sampling
. - A simple example of Multi-Player simulation with 2 Decentralized Algorithms, comparing
rhoRand
andSelfish
(for the “collision avoidance” part) combined withUCB
andThompson Sampling
for learning the arms. Spoiler:Selfish
beatsrhoRand
!
(Old) Experiments¶
2. Question: How to read these documents?¶
2.a. View the notebooks statically :memo:¶
- Either directly in GitHub: see the list of notebooks;
- Or on nbviewer.jupiter.org: list of notebooks.
2.b. Play with the notebooks dynamically (on MyBinder) :boom:¶
Anyone can use the mybinder.org website (by clicking on the icon above) to run the notebook in her/his web-browser. You can then play with it as long as you like, for instance by modifying the values or experimenting with the code.
- Do_we_even_need_UCB.ipynb Binder
- Easily_creating_MAB_problems.ipynb Binder
- Example_of_a_small_Single-Player_Simulation.ipynb Binder
- Example_of_a_small_Multi-Player_Simulation__with_Centralized_Algorithms.ipynb Binder
- Example_of_a_small_Multi-Player_Simulation__with_rhoRand_and_Selfish_Algorithms.ipynb Binder
- Lai_Robbins_Lower_Bound_for_Doubling_Trick_with_Restarting_Algorithms.ipynb Binder
- Exploring different doubling tricks for different kinds of regret bounds.ipynb Binder
- Experiments of statistical tests for piecewise stationary bandits.ipynb Binder
- Demonstrations of Single-Player Simulations for Non-Stationary-Bandits.ipynb Binder
2.c. Play with the notebooks dynamically (on Google Colab) :boom:¶
Anyone can use the colab.research.google.com/notebook website (by clicking on the icon above) to run the notebook in her/his web-browser. You can then play with it as long as you like, for instance by modifying the values or experimenting with the code.
- Do_we_even_need_UCB.ipynb Google Colab
- Easily_creating_MAB_problems.ipynb Google Colab
- Example_of_a_small_Single-Player_Simulation.ipynb Google Colab
- Example_of_a_small_Multi-Player_Simulation__with_Centralized_Algorithms.ipynb Google Colab
- Example_of_a_small_Multi-Player_Simulation__with_rhoRand_and_Selfish_Algorithms.ipynb Google Colab
- Lai_Robbins_Lower_Bound_for_Doubling_Trick_with_Restarting_Algorithms.ipynb Google Colab
- Exploring different doubling tricks for different kinds of regret bounds.ipynb Google Colab
- Experiments of statistical tests for piecewise stationary bandits.ipynb Google Colab
- Demonstrations of Single-Player Simulations for Non-Stationary-Bandits.ipynb Google Colab
3. Question: Requirements to run the notebooks locally?¶
All the requirements can be installed with pip
.
3.a. Jupyter Notebook and IPython¶
sudo pip install jupyter ipython
It will also install all the dependencies, afterward you should have a jupyter-notebook
command (or a jupyter
command, to be ran as jupyter notebook
) available in your PATH
:
$ whereis jupyter-notebook
jupyter-notebook: /usr/local/bin/jupyter-notebook
$ jupyter-notebook --version # version >= 4 is recommended
4.4.1
3.b. My numerical environment, SMPyBandits
¶
- First, install its dependencies (
pip install -r requirements
). - Then, either install it (not yet), or be sure to work in the main folder.
Note: it’s probably better to use virtualenv, if you like it. I never really understood how and why virtualenv are useful, but if you know why, you should know how to use it.
:information_desk_person: More information?¶
- More information about notebooks (on the documentation of IPython) or on the FAQ on Jupyter’s website.
- More information about mybinder.org: on this example repository.
:scroll: License ? GitHub license¶
All the notebooks in this folder are published under the terms of the MIT License (file LICENSE.txt). © Lilian Besson, 2016-18.
Maintenance Ask Me Anything ! Analytics made-with-jupyter made-with-python ForTheBadge uses-badges ForTheBadge uses-git