I like to mix technology with manual skills, the virtual with the physical. As soon as the Code Connection app was released, I wondered what could have been the service I wanted to see integrated. Maybe Microbit and Minecraft? Maybe Arduino and Minecraft? Definitely Microbit and Minecraft!
I would have really liked to have the possibility to operate something on some arduino-compatible board and see a reaction in Minecraft and vice versa of course.
The three coding services available to interface with Minecraft: Education Edition at that time were Make Code, Tynker and Scratch (only after Code.org was added).
Giving a thorough look at the website MakeCode I noticed that other hardware solutions also adopted that coding environment: those hardware solutions that I liked so much like Arduino and Micro: bit.
For about two months I have been waiting for the idea of creating the extension needed to allow their hardware to interface with Minecraft from the Microbit developers. Everything is ready ... you only need to implement the appropriate commands as already done by the four available services (Make Code, Tynker, Scratch and Code.org).
One day I finally decided to try alone and see to what extent this integration was achievable.
Microbit and Minecraft: sniffing network traffic
It is clear that MakeCode can communicate with Minecraft in some way. I just have to understand how. I did the first experiment with what seemed to be the simplest and fastest experiment.
When you select MakeCode from Code Connection it does not open a web browser, but everything "runs" in your machine locally: the idea then is to simply listen between the Code Connection application and Minecraft Education Edition (MEE). Apparently it seems that MEE reacts to indications coming from outside (MakeCode in this case started by Code Connection).
I therefore started to sniff the traffic between the two applications residing on my PC, but I could not intercept something readable. Suspecting the use of an encrypted communication I put myself in the best condition to analyze everything again.
Unlike MakeCode, if you select Tynker or Scratch from Code Connection, a classic web browser is started from which the commands to MEE appear to start. And as soon as I insert a line of code in Scratch via the web, this line of code starts in plain text in the upper "layer" of the browser.
Then if the browser knows how to encrypt everything good for him, but, having access to the browser development tools, I can see in clear what is about to be encrypted by the browser and then sent to Minecraft Education Edition.
Microbit with Minecraft: network tools
Once I started Scratch via Code Connection and opened the browser, I started the developer's tools, focusing on the Network tab.
Why I had not just tried to immediately check the listening ports used by my PC?
I started as an administrator the Dos Commands Prompt and typed: netstat -ab
Code Connection is listening on two ports: one is 8080 port as indicated by the browser and the other is the same port that I used to connect with MEE. You know when you start Code Connection before MEE? What does it show to you? See the following screen and compare the listening port with the output of the netstat -ab command
It seems that Code Connection is working as a bridge. Code Connection for Minecraft.exe is listening locally on port 8080 and on port 19131
Microbit with Minecraft: the analysis proceeds
Scratch talks to Code Connection who talks to MEE who performs the action. I like it.
I insert two simple commands in Scratch and execute them, always monitoring the browser side traffic. The Scratch command Move FORWARD is translated into an http call of the type http://localhost:8080/move?direction=forward A clear correspondence between the Scratch block executed and the intercepted command, isn't it? And then the Scratch command Move RIGHT? It is transformed into the call http://localhost:8080/turn?direction=right
And changing the commands the result does not change: it is even clearer. It is sufficient for someone to enter the IP address of my machine, the 8080 port and an appropriate sequence of commands interpretable by Minecraft. Someone like a browser for example. I do not even need Scratch anymore. I type in my browser http://localhost:8080/turn?direction=right and I get: from browser side the result of the command sent and from CodeConnection / MEE side the Agent that turned to his right.
If a browser makes those calls and the Agent reacts accordingly, I can not help but try to replace the browser with something that makes me more comfortable. An app for example.
I developed the app quickly with AppInventor and created buttons that when pressed they make appropriate HTTP GET calls. The app must be connected to the same network on which the PC running MEE is connected. Having my app call Code Connection running on the PC I need to know what is the IP address of the PC and be able to configure it in my app. And from the app - as you can see from the first red rectangle - you can specify it on time.
With the buttons under MOVE and DESTROY you can instead move the robot in different directions and destroy the blocks.
Now everything is ready to connect the micro: bit with Minecraft Education Edition and use the app as a bridge.
Doing it via serial would be much simpler. It was enough to create a small server in python listening on the serial, but I liked the idea of "wireless" and therefore taking advantage of the bluetooth connectivity present in Micro: bit.
Micro: bit connects to the app via bluetooth and the app receives the variations of the micro: bit. For each tilt variation the app makes the appropriate HTTP GET calls to the PC on which Code Connection and MEE are running.
To put the micro: bit in contact with the app it is sufficient to "pair" them according to this guide.
From the app using the buttons at the top - Scan, Stop Scan, Connect, Disconnect - you can then connect and disconnect it easily.
Below are some images of the code inserted in the app, while both the apk (if you only want to install it) and the source code of the app (and according to the Creative Commons BY-SA license) are available here