A/B test with python (Bandit)

In this article, I am going to write a Bayesian Bandit algorithm.

import matplotlib.pyplot as plt
import numpy as np
from scipy.stats import beta

NUM_TRIALS = 2000
BANDIT_PROBABILITIES = [0.2, 0.5, 0.75]

Bandit probabilities are divided into three value just in case. Quick favorite.
Trial number is 2000.

So, I am going to define a class called Bandit.

class Bandit:
    def __init__(self, p):
        self.p = p
        self.a = 1
        self.b = 1
        
    def pull(self): # arm of slot machine
        return np.random.random() < self.p
    
    def sample(self): # sample from current beta distribution
        return np.random.beta(self.a, self.b)
    
    def update(self, x):
        self.a += x
        self.b += 1 - x

This works like slot machine.
a, b are the beta parameters defined a uniform distribution in the beginning and p is the probability of winning.

This class should have a ability to pull the slot machine defined definition pull() returns the random number and we can get also sample from its current distribution.

Lastly, we have also update function. x is either 0 or 1.

def plot(bandits, trial):
    x = np.linspace(0, 1, 200)
    for b in bandits:
        y = beta.pdf(x, b.a, b.b) 
        plt.plot(x, y, label="real p: %.4f" % b.p)
    plt.title("Bandit distributions after %s trials" % trial)
    plt.legend()
    plt.show()

This is just going to plot PDF of each bandit and we can compare it on the same chart.

OK. Then, we need to run a actual experiment.

def experiment():
    bandits = [Bandit(p) for p in BANDIT_PROBABILITIES]
    
    sample_points = [5, 10, 20, 50, 100, 200, 500, 1000, 1500, 1999]
    
    for i in range(NUM_TRIALS):
        bestb = None
        maxsample = -1
        allsamples = []
        for b in bandits:
            sample = b.sample()
            allsamples.append("%.4f" % sample)
            if sample > maxsample:
                maxsample = sample
                bestb = b
        if i in sample_points:
            print("Current samples: %s" % allsamples)
            plot(bandits, i)
            
        x = bestb.pull()
        bestb.update(x)

if __name__ == '__main__':
    experiment()

Bandit should be initialize at first.
Sample points are where we want to reasonably show a plot. Any values are ok.
We also keep track of the maximum sample and best will be.

A/B test with python

This is just going to make our generated datas as well as is able to emulate a real data collection surface.

class DataGenerator:
    def __init__(self, p1, p2):
        self.p1 = p1
        self.p2 = p2
        
    def next(self):
        if np.random.random() < self.p1:
            click1 = 1
        else:
            click1 = 0
        
        if np.random.random() < self.p2:
            click2 = 1
        else:
            click2 = 0
        return click1, click2

p1 and p2 are probability of click for group 1 and group 2.

Next, I will write a function for obtaining the p-value.

def get_p_value(T):
        det = T[0,0]*T[1,1] - T[0,1]*T[1,0]
        c2 = float(det) / T[0].sum() * det / T[1].sum() * T.sum() / T[:,0].sum() / T[:,1].sum()
        p = 1 - chi2.cdf(x=c2, df=1)
        return p

I am going to explain p-value later in the other article.

Next, I will write a function for running a experiment.
That is going to include the probability of click for group1 and group2 and the number of samples.
In this case, I am going to take 2500 samples.

def run_experiment(p1, p2, N):
    data = DataGenerator(p1, p2)
    p_values = np.empty(N)
    T = np.zeros((2,2)).astype(np.float32)
    for i in range(N):
        c1, c2 = data.next()
        T[0,c1] += 1
        T[1,c2] += 1
        if i < 10:
            p_values[i] = None
        else:
            p_values[i] = get_p_value(T)
    plt.plot(p_values)
    plt.plot(np.ones(N)*0.05)
    plt.show()
    
run_experiment(0.1, 0.11, 2500)

    data = DataGenerator(p1, p2)

The data written above is to create an instance of data generator.

     if i < 10:
            p_values[i] = None
        else:
            p_values[i] = get_p_value(T)

We have to ignore the first a few datas in terms of taking into account p-value.
Because if we try to calculate the p-value too early, the formula might be broken.

      c2 = float(det) / T[0].sum() * det / T[1].sum() * T.sum() / T[:,0].sum() / T[:,1].sum()

I divided by row sums and column sums, so if any of those are 0, I cannot calculate it.

はてなブログにソースコードを貼る方法（メモ）

適当に実装したものもメモとしてこのブログに残しておきたいと思ったのだが、

そもそもはてなブログへのソースコードの貼り方、埋め方を知らなかったので、とても簡単に調べて、簡単にまとめておこう。

まず、前提として

はてなブログでソースコードをそれっぽく表示するには、

htmlで<pre><code>と</pre></code>の間に表示させたいソースコードを入れるのが一番簡単な方法だと思う。

もっと見やすい出し方とかもあるが、僕はそこまでエナジーを注ぎたくはないので、

<pre><code>ソースコード</pre></code>

の一強でしばらくいきたい。

画像の赤丸で囲んである部分からHTML編集に飛んで、

と打ち込めば、例えばCSSなら下のように簡易的な表示が可能。なるほど。

.entry-content h3{ 
padding: 5px 10px; 
border-left: 10px solid #fc0;
}

色をつけたりする場合は、別の方法の方が良いかも？！