Autonomy 101: Crafting a Minecraft bot with ROS 2
November 1th 2022
In just two weeks, the Autonomy team has developed a Minecraft bot with autonomous behaviors to chop down trees! The crews had a blast testing their bot in Minecraft while learning more about artificial intelligence/autonomous design in ROS 2.
The Autonomy team has always been at the heart of Ascend’s drone activities using artificial intelligence. At Ascend, we aim high and work hard, but that doesn’t make us experts at the start. When Dag Økland joined Ascend in 2021 as an Autonomy member, getting familiar with ROS and AI was a little overwhelming. However, he always loved playing Minecraft and aspired to bring something fun and creative to Ascend. That is why as the new leader of the Autonomy team, Dag introduced Minecraft as the startup project for the team to get started in learning AI and programming with ROS 2. For Bendik, Christopher, and Bård, this is also one of their first software projects at Ascend.
Who Are New Autonomy guys?
Bendik (20) is currently pursuing a 5-year master’s program in Computer Science. For him, aerial robotics is only one of his technical interests. Bendik has also been developing an indie game project and working part-time at NTNU’s IT department. By joining the Autonomy team, he seeks to broaden his skills and experience in more than just artificial intelligence.
"I am interested and want to work with a wide variety of software.” - Bendik Norli
Christopher (19) just started his first semester in the cybernetics and robotics program. His charismatic personality and positive mindset bring an equal amount of joy and technical contribution to the team. As the brain that finalized the finite-state machine with ROS 2, he displays immeasurable talents in autonomous design. And don’t let the picture deceive you. Christopher is jacked underneath.
“Autonomy is the brain, the most critical part of Ascend.” - Christopher Yuxiang Chen
Born and raised in Dubai, UAE, Bård is still technically a Norwegian. However, unlike his sun tan and Arabic accent, his passion for video games never fades. He grew up playing Minecraft, which fostered his interest in making his own AI after dying to the ender dragon. He joined the Autonomy team to learn the art of programming in drone AI so he could one day take his revenge. Bård is equipped with python skills and ready to tackle any challenges except the ender dragon.
“I don’t like C++, and I prefer python.” - Bård Raddum Wikmark
Getting Started in ROS 2
The general motivation for the Minecraft bot project was for the new Autonomy team to get started with ROS 2. According to Wikipedia and the official documentation, ROS (Robot Operating System) is essentially a set of software frameworks, libraries, and tools for building robot applications and services. Some of its core features are hardware abstraction, device control, message-passing, and real-time visualization. ROS 2 is the brain for robots on a computer, that allows the creation of robotics software programs without modifying the hardwares.
With that being said, artificial intelligence is not always rocket science at NASA or SpaceX. After brainstorming a few options for what their bot should do in Minecraft, the Autonomy team decided to make an autonomous lumberjack. One of the easiest ways to do this is to avoid perceiving trees in the game. Then the question is, how can it be a tree-chopping bot if it can not detect any trees? Well, as it turns out, the trick here is to simply pass the in-game tree’s location data as messages through ROS 2’s node processes.
A ROS 2 node is a single computational process running in real-time. For example, one node can be responsible for publishing and calculating the path's direction. Another node then receives the path message and processes it as the robot’s wheel movement. A robotic system can have multiple nodes with different names running simultaneously; other nodes would still function if one node failed to run the process. In aerial robotics development, the node feature is the most dominating factor. Drones are well-optimized and easily maintained by minimizing code complexity and maximizing fault tolerance with the help of ROS 2 nodes.
Connecting ROS 2 and Minecraft
Back to Minecraft, ideally, ROS 2 can tell the Minecraft bot in-game, “here is the tree's location,” and the bot would move forth and perform the chopping action autonomously. But since the sole purpose of developing the bot is to start programming and testing nodes in ROS 2, all the location data doesn’t need to be randomized.
Here’s a list of steps for implementing the Minecraft bot:
- Get the player and tree’s position.
- Create ROS 2 nodes.
- Publish information to the server.
- A script to send that information.
- PID controller.
- A lot of math.
Several Autonomy members worked on getting the tree's position and orientation of the player. This also includes emulating the mouse and keyboard inputs to control the bot.
Here’s the tricky part though. The system that dictates the bot’s behavior in Minecraft is not the collection of ROS 2 nodes. Because most of the ROS 2 nodes are only receiving and subscribing to the message-passing processes from Minecraft. For example, one node contains the message for Minecraft’s tree location data, and this information only. But like other subscriber nodes, all the information is processed in a publisher node called finite-state machine.
A finite-state machine or FSM directs every state and action with the Minecraft bot. It runs within ROS 2 and changes the Minecraft bot’s actions based on its current state. For instance, after sending and “calculating” the message for the closest tree, the state changes from “tree_position” to “hug” and the bot performs the action to chop down tree blocks (Minecraft’s universal interact mechanics).
Running the Minecraft Bot!
Finally, after merging everyone’s python scripts, Christopher posted the result of Autonomy’s startup project:
Cool! Nicely done, Autonomy team. Yes, it’s a rather simple bot, and yes a lot of it is hard-coded. But for the Autonomy team this is a successful first step in learning something new, fun and exciting, and everybody feels a lot more confident in accomplishing the next steps!
“You know, there was a really steep learning curve during the course period, but now it’s super interesting.” - Christopher Yuxiang Chen
“I didn't get a deeper understanding. But hey, I got to know the essential thing (ROS 2) if I need to check with something later.” - Bendik Norli
“I would be too scared to start on the drone immediately. It's nice to have something to practice instead of boring theories in the first three weeks. But I just love coding, so it's fun to just write more code.” - Bård Raddum Wikmark
For the Autonomy team 2023, the Minecraft bot is their first startup project. But it is also nothing like doing school assignments. Christopher thought it was amusing that the team get to choose the tasks themselves. Since there were no concrete plans or consequences at the beginning, everybody was free to learn and do whatever they wanted to become more useful for the team.
In the next couple of months, the autonomy team will get back on the main track to developing autonomous features for our upcoming drone in the SUAS (Student Unmanned Aerial System) competition next spring. Their next steps are to implement an efficient Airdrop design with an accurate path planning system. Best of luck, the Autonomy team!