From Air to Ground – Building a realistic ground control framework
Written by: TheDevBird
At this point in development, Airwave only modeled the operations of an approach controller. Aircraft would enter the airspace and the players would guide them to land. But, what happens after an aircraft lands? In the real world, they need to taxi from the runway and to their gate. Thus, we begin our journey of following a flight from beginning to end.
Planning the data model
First of all, we needed to model of what an airport looks like. We chose a simple set of objects:
- Runways
- Taxiways
- Terminals
- Gates
With this, we can treat the majority of the navigation model as lines. Lines are great, because they can intersect. Intersections provide us with waypoints, the exact positions for an aircraft to travel to. For an aircraft to go from point A to point B, however, it needs to know the path.
A simple color-coded sketch of what an airport looks like to Airwave.
JavaScript canvas sucks
We introduced a basic method of drawing the ground view on the frontend and a switch to swap between ground and air(space) view. Our first hurdle, ensuring that the translation and zoom settings behaved nicely when switching between the views. To get over this hurdle, we had to build a scaling system to project the taxiways and runways as the viewport moved and zoomed.
JavaScript canvas uses weird coordinates where 0,0 is the top left and the y axis is flipped (positive = down). We had to remap coordinates we received from the backend to fit the JS model, which was a headache. Canvas couldn't help us much as the built-in translation and scaling methods caused issues with positioning and pixelation (due to extreme scaling for ground view).
Building our own projection & scaling system was an incredibly useful add-on to our canvas pipeline. Once we added that, among other helper functions, drawing to the canvas became significantly easier, with less headaches.
Finding our path
Luckily, real-world pathfinding is a very manual process. Navigation instructions are given directly by the ground controller, the pilots not usually following paths on their own. With that in mind, It was going to be the player's responsibility to route aircraft from the runway to their gate or vice versa. Since we weren't going to need a smart algorithm, taxiing consisted of implementing taxi waypoints and the logic for aircraft to move between them.
The initial algorithm worked as follows:
- Receive taxi instructions from ATC (such as, “Taxi to Gate Alpha 1 via Alpha, Bravo, Charlie”)
- Parse those instructions into waypoints and hold onto that list
- Know our current location (such as, “Runway 27”)
- Take the line of our current location and the line of our next waypoint
- Find the intersection between those two lines
- Turn the aircraft towards the intersection and start moving if we aren't already
- Repeat from #3 until we run out of queued waypoints
This was very simple in terms of compute, relying solely on the players ability to provide the aircraft with a route to the destination. We included some simple checks to ensure that the aircraft wouldn't turn 180 degrees, because that's not how aircraft work. However, one of the problems with this setup was that the loop ran lazily, only generating a new waypoint once the current one was reached.
Having the algorithm run like this meant that the aircraft couldn't determine if it would actually get to its destination until it got there (if it ever did). Aircraft would also taxi in odd ways causing unexpected behavior. So, shortly after we play-tested this version of the pathfinder, we decided to build new iteration of the entire taxi system that solved these problems.
A polished, smarter algorithm
We developed the new taxiing and pathfinding system from the ground-up. This time, it would be eager, calculating all waypoints before the aircraft even begins to taxi. Furthermore, we would optimize the calculation process by precalculating all of the possible intersections when the game starts. That would allow the pathfinder to simply find its current location in the waypoint graph and search for the best path that fit the instructions given by the player.
Graphs are awesome. With a graph, we could loosen the responsibility of the player needing to provide all of the pathfinding logic. Instead, you could simply tell an aircraft to, “Taxi to Gate Alhpa 1”, and it would find a path on its own. Though, not very comparable to a real-world command, it lowered the barrier to entry.
Pathfinding is hard. Even with the smarter algorithm, it all came down to “interpretation”. How should the aircraft behave when it hits a runway? What kind of paths should it favor? Does a generated path line up with what the user expected? Most of the time, that last question was the main problem (and still pops up once in a while). The problem lies with implementing the core pathfinder and filtering algorithms that determine what the end result will be, which needs to work exactly as a user would expect it to. Not an easy task.
That said, at this point of development, we were happy with the way it was working then, so we continued on. Next was teaching then improving how the LLM interprets taxi commands.
Smarter algorithm, smarter LLM
Ground control is a very complex system. It includes long-winded commands that vary slightly and rely heavily on context. Air control, on the other hand, is almost stateless. Telling an aircraft to “Turn left, heading 220” is simple compared to “Taxi to runway 27 via Alpha, Bravo, hold at Charlie” (an example of a ground command). This is because ground control requires stateful operations. An aircraft needs to know where it is, the expectation of a controller's instructions, and how to follow them.
A real-world controller could tell an aircraft to hold short of any point, meaning it should wait until a go-ahead is received. Implementing this took some time. We had to provide almost double of the original examples and commands. The LLM not only had to understand how to operate the aircraft in the air and the ground, It needed to construct the waypoint instructions from the player perfectly, or else the pathfinder could fail downstream.
In order to build up resiliency with the LLM, we had to provide many detailed examples to ensure that it would only hold short and take paths when it was told to; parsing a controller's commands verbatim. Despite the now long-winded prompt, more words = more better, in our experience. The LLM finally understood almost all of our taxi instructions, despite a few issues with holding short (took a while for us to get it to stop doing that).
Bigger airports
As the new, sophisticated taxiing algorithm was being finalized, we wanted to build out a new airport to properly test Airwave end-to-end. We kept the same design of intersecting runways, but placed taxiways everywhere. We also used two terminals instead of just one. Terminals are defined as a set of named waypoints, so we could theoretically have as many as we wanted.
Developing and using our new airport was fairly straightforward, highlighting the ease of use of our new system. The whole game was built without hard-coding most of the underlying data, so it supported customization out-of-the-box (that is, if you knew Rust). We also weren't limited to two runways, as long as each runway was named uniquely (just like real-world airports).
With great ground comes great responsibility
We copied over most of the code for landings, extending the behavior of the control flow to switch an aircraft into taxi mode once they landed on a runway. Then, the player could taxi the aircraft to any gate. In order to keep the game flowing, we set up an intention system where incoming aircraft would intend to land and departing aircraft would intend to takeoff. When an aircraft arrived at a gate, its intention would be flipped to become a departure.
End to end
At this milestone, the player could guide aircraft to land, taxi to their gate, wait, taxi to a runway for takeoff, then takeoff out of the airspace. With ground operations being fully supported, the player was never out of tasks to do during their gameplay.
There was still more for us to do, however. We envisioned this to become larger, scaling to multiple airports and airspaces. Thus, we could build large, detailed worlds such as the entire United States, with each airspace being controllable. And surprisingly, we weren't that far away from that next milestone.
Airwave is open-source and available on GitHub.
For questions or feedback, feel free to reach out on Bluesky, on our Discord, or via email: [email protected].