Writing (another) Game

During this shut in, I've just finished writing my second multiplayer turn-based game using simple tools. (You can read about the first one Space Miner, and my motivations). The new game is called Multi Armed Bandit, and you can play it now.

As before, I used Notepad++ as my sole development environment, and coded it in PHP on the server side (I’m a masochist), MySQL for the database, and vanilla HTML (with a little JavaScript) for the client.

Multi Armed Bandit

Idea

Multi Armed Bandit was written with the same core mechanic as Space Miner such that players can asynchronously login to make their moves as often, or rarely, as they like, but still be entered into playing each round. This game is based around the classic Multi Armed Bandit problem in which you attempt to maximize the money you earn from a collection of slot machines which have an unspecified expected payout schedule (see below).

You can read the final game rules here:

Rules and Instructions

Multi Armed Bandit is a casino simulator game.

There is a virtual casino, inside which are a dozen slots machines. Your goal is to make the most money from playing these machines. The issue is that you don’t know the payout chances of any of the machines.

Every hour you are given free pulls that you can use on these machines. You start off with five free pulls per hour, and at midnight (PST) everyday, you get an additional pull per hour; up until you reach a maximum of twenty-five pulls per hour.

You decide which machines to play, and can distribute your pulls between the twelve machines (but there is a limit of ten pulls on any machine per hour). You don’t even need to be present to play, your preferences will be saved. If you don’t login to change the distribution of your pulls, the machines will be played each hour in the same distribution as the previous hour (with the exception that any bonus pulls you get at midnight are not allocated, so it’s in your interest to login at least once a day to claim that new pull and allocate it!)

You can let your pull distribution ride by not logging in frequently (gathering detailed data on the same machines), or login every hour and micro-manage to tweak and get information on a wider range of machines; reacting dynamically to new information. You can change your pull distribution as often as you like, without consequence, but, whatever your configuration is at five minutes past the hour is the pull distribution that is used for that hour.

The dilemma is, what is the optimal strategy? This is a classic Data Science problem called the Multi Armed Bandit problem.

Obviously, your optimal strategy is to invest all your pulls on the machines with the highest expected return, but how do you determine which these machines are? You could spread your money wide, testing each machine for a long time, to get an accurate estimate of their expected returns, but with this strategy you are wasting money on machines that are poor performers. How much time (resources) do you invest in research before leveraging this information?

Alternatively, you might find a machine that is ‘good enough’ and invest heavily in this machine from the outset, but is the grass greener? Could there be better paying machines out there that you are ignoring?

Of course, expected return is calculated based on the luxury of having an infinite number of pulls. In reality, you have a finite number. You might ignore a machine as being a ‘poor payer’ if you have too small a sample, but if you had waited just a bit longer could have experienced a monster payout.

In this game each machine has a fixed ‘personality’, it’s rules and expected return are fixed (there is no dynamic remapping of payouts). Some machines payout with high frequency, some with low frequency, some give huge payouts occasionally, some never, some mix it up. Some give medium sized payouts a medium amount of time. You just don’t know (until you start playing!). The longer you play the better model you get of the machines.

As there is a cap of a maximum of ten pulls per machine per hour, you can’t just put all your eggs in one basket (even if you think you have the best basket). Over time, you could max out a couple of machines, and still have a few pulls left to sow wild oats looking for a better deal.

Instructions

After logging in, the first thing you’ll want to do is select the top option to ‘Place Bets’. It is on this screen that you will configure the distribution of your pulls. Use the combo boxes to select the number of pulls for each machine. Pay close attention to the number of pulls you have; the game will not allow you to cheat and lock in more pulls than you are allowed! Click update when you are happy with your configuration.

Also on this page are buttons to cancel (will leave the configuration as it was and ignore any changes), and a button to quickly clear all configurations to allow you to start afresh.

To help you decide where to place your bets this page also gives information about past pulls. The page shows your distribution of pulls from the previous hour (a green arrow will appear by the side of this number if there a change between the last configuration, and the configuration to come).

The rubric also shows the number of all people who played each machine the last hour (giving you an indication of what machines others are playing), and also a count of the number of times that machine has ever been played by all players since the casino opened for business. Do you use this information to follow a trend? Do you shadow where others are playing (do you think they have more information than you do, or is it mass hysteria?)

There is a high score table that shows the current top twenty players. Scoring is based simply on total cumulative earnings. Clicking on a player in this list gives access to a graph showing that last few days of their performance. Graphs show curves of a players winnings over time and also a curve showing the threshold for what score is needed to get you into the top twenty. If you are not ranked in the top twenty, your stats will appear, along with your rank, at the bottom of the table.

The statistics button gives access to tabular and graphical data for your own account for the last 48 hours.

To allow you to optimize your strategy, two tools are provided. The first is a table showing a summary of your results for all machines. This table shows the results of the last set of pulls (along with a sum of payouts and the average payout), as well as global totals for all your pulls along with global averages. The highest value is highlighted in green for each of the two average columns. Note: This data is based purely on information you have learned from the machines. You don’t have any information for how anyone else has performed on these machines (the only hint you might get for this is if you believe that how popular a machine is on the betting screen is an indication of how ‘hot’ it is).

The second tool is the history tool and this breaks down your earnings for every machine and every pull. You can see the distribution of earnings and how each machine is paying out. Navigation buttons on the top allow you to move backwards and forwards through each hour to see how your winnings were accumulated.

Disclaimer

If it’s not obvious, this game is for fun. It’s not a real casino. The money is virtual, and your pulls are free. You can’t take money out of the game, nor put any money into the game. It’s free to play and for entertainment purposes only.

Good Luck!

What are you waiting for? Try out the game now!