Playing a Video Game with Your Mind

Image for post
Image for post
Image from Chrome Dino

Not having Wifi sucks…but at least there’s this little dino that makes it all 10x better:

Image for post
Image for post

If you’ve been blessed with having good WiFi for a while, you might not remember it. But basically, you press the space bar to make it jump over cacti and pterodactyls until it runs into something and the game ends (it’s actually kind of addicting: chrome://dino/).

But instead of having to click the spacebar to jump, let’s try it using our brain waves 🧠. With a Muse headset and a little code, you can make it too…

How it Works

We’ll be playing this game by monitoring our brain signals and using them as game commands. The method we’ll use for monitoring is called electroencephalography (EEG), where electrodes are placed on the skull and pick up on activity when groups of neurons fire.

When we do involuntary tasks such as blinking, a lot of electrical activity is produced. Typically, this can be a disadvantage when trying to monitor brain signals since eye blinks generate such strong disruptions. But in some cases, like this project, we can use that to our advantage and use eye blinks as the desired input.

Image for post
Image for post
Image from Science Direct

The above image demonstrates this pretty well; the circled blue parts show prominent data disruptions when someone blinks.

Now that we have a bit of background, let’s get started!

What you’ll need: Muse 2

Muse is a company that specializes in consumer products that use EEG reading to help users optimize their mental state and performance. The Muse S headset focuses on tracking sleep and provides insights that help users understand their sleep patterns and how to get a better night’s rest.

Image for post
Image for post

For this project, we’ll be using a Muse 2 headset, their flagship product that enhances a user’s meditation experience tenfold. If you’ve ever sat down to meditate, you’ll know how much of a challenge it can be to stop your mind from racing and come to a grounded state. Sometimes I won’t even realize my mind is wandering until ten minutes later when my ‘meditation session’ has turned into an overthinking marathon🙁. The Muse 2 solves this by monitoring brain waves in real-time and alerting you when your mind begins to drift, so you know it’s time to refocus.

Since the device uses electrodes to pick up on brain activity, it’s also handy for this project where we’ll use it to monitor signals from when a person blinks 👁. While there are other EEG based headsets out there, Muse is the most intuitive and easy to operate.

So now that we have our Muse headset, let’s talk about how to actually set up the game to respond to brain activity!

1. How to Set Up the Game 🦖

There are essentially two parts to being able to control the game with our blinks. The first is adding to the extracted HTML repository for this game so that it knows we want it to connect with Muse. The second part is creating a file containing commands that tell the dino to jump when we blink.

To start, we open up our terminal and clone the extracted game’s repository by typing this in:

Then we’ll open up our directory and add the following commands to initialize a new NPM project and install muse-js:

After that, we’ll also need to install SystemJS, a module loader that we’ll set up to be able to load the muse-js library. Enter this to install it:

Now, we can add code to the index.html page which will load SystemJS. It also loads brain.js, which we’ll create later to add new controls for the game.

2. Blink Detection 👁

With the initial set up complete, we can now work on the code that’ll actually enable the game to pick up on blink artifacts and use it as a command.

So we’ll create a file called brain.js and add the following code in the window:

That might look a bit confusing, so let’s breakdown what the above code is actually doing. The beginning initializes the Muse client and filters through the detected blinks. The point of line 15 is to set a noise threshold; the dino will only jump if the noise created by the blink exceeds this threshold we set. So for that reason, this line may need to be adjusted depending on the environment — sometimes lower numbers (100–150) may work best if the environment isn’t as noisy.

Now to the same brain.js file, we need to add the following code:

In a normal game, the dino jumps when we hit the space bar; but we want it to jump when we blink instead. So what the above code does is basically replaces the spacebar command with a keydown event that’ll make the game think we hit the spacebar when we’re actually blinking.

3. Connect

We’re nearly done! To finish up the code, we’ll need to add a connect button to the end of index.html that will allow us to connect to Muse directly from the game page.

4. How to Play:

To actually play, we’ll need a web server to serve the HTML. To do this, we can download Visual Studio Code and open up our trex-runner file there. Once on the explorer tab, be sure to open index.html with Live Server — which will allow it to run.

Image for post
Image for post

…And then you should finally see this:

Image for post
Image for post

Pretty satisfying right? Now you can play the dino game whenever you want — with your brain data 👁 🧠 👁 .

Thanks for reading! I’m Sarah Naghmi, an innovator, aspiring entrepreneur, and neurotech researcher. Feel free to connect on Linkedin and Twitter. You can also sign up for my newsletter where I share exciting project updates, insights, and helpful resources. This post was adapted from Uri Shaked’s project.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store