Anti-DDoS Game : Powerful protection specifically designed to counter attacks to game servers
It’s now easier than ever to launch a denial of service (DDoS) attack. You no longer need advanced technical skills to disrupt a service or make it unavailable. Anti-DDoS protection, like the one that OVH developed in 2013, can limit the scope of such malicious attacks, which are growing in frequency and severity (in 2016, one-terabit-per-second attacks were recorded for the first time). The gaming and e-sports industries are particularly prone to DDoS attacks and the protective measures implemented by service providers show their limitations when faced with the intensity and frequency of these attacks. Especially those which exploit UDP, a connectionless networking protocol used by most game and voice servers. OVH therefore felt it necessary to develop anti-DDoS protection specifically adapted to game servers. Clément Sciascia, an OVH developer, provides more details.
Why are attacks on game servers a specific problem?
First, attacks more frequently target game servers. An unhappy gamer, a teenager who sees themselves as a hacker, a wannabe blackmailer, a dishonest competitor… the motivations for attacks are numerous and often pointless. If attacks are increasing, it is because they are becoming easier than ever to carry out. Tutorials and scripts are readily available online, and for a few pounds, anyone can rent an army of cloud servers to bombard a server with requests. Some websites are offering what is akin to “DDoS” as a service. They claim to provide “load testing” services, but it is obvious how such a service could be misused.
Besides, game and voice servers are especially sensitive: the slightest lag caused by server overload or bandwidth saturation affects players. This results in slowed game play or in the case of voice servers, conversations that are choppy or simply lost.
VAC technology is used to counter “traditional” anti-DDoS attacks, following a three-step process: 1. packet analysis, 2. “vacuuming” of incoming traffic in the event of an attack, 3. mitigation (filtering of non-legitimate packets). The “vacuum” is therefore activated slightly after an attack begins.
We are only talking about a few seconds’ delay, which is not noticeable when running ordinary applications, but for game servers, the disruption is significant. In addition, numerous game and voice communication services, like TeamSpeak or Mumble (services that players in the same team use to speak to one another during game play), use the UDP protocol for communication. UDP, also used by streaming services, is ideal because it transmits small data packets quickly and efficiently. On the other hand, UDP, unlike TCP, sends data packets without negotiation; there is no handshake, prior to data transmission. When players join a game, connection authorisation is managed at the application layer (L7). This makes it difficult to distinguish between packets sent by an authorised IP and those sent by a spoofed IP.
In other words, mitigation is complicated because illegitimate packets have the same characteristics as legitimate packets, on the surface. For example, in “Source Engine Query” attacks, which target Counter Strike servers, the attack consists of overloading the server with queries, to retrieve information about the server. If these types of packets are filtered indiscriminately, players (the genuine ones!) would not be able to view the server or access information about it.
The VAC, which effectively handles a large variety of attacks, has more difficulty dealing with this kind of attack. Our only choice was to innovate. For example, by also intervening with the outgoing server traffic. In the case of the Source Engine Query attack, the solution is to store the server’s response in cache. In the event of a flood, the cache can respond if the response is already in the cache, which prevents the server from getting overloaded by illegitimate queries.
What exactly is the difference between anti-DDoS Game protection and traditional anti-DDoS protection? Which mechanisms have you put in place and how did you get there?
The first stage of the process, which lasted a total of more than six months, was to establish a list of games and voice communication services based on two criteria: commercial success and their susceptibility to DDoS attacks. VeryGames, one of our customers specialising in hosting services related to video games, explained to us that there are very popular games that are rarely attacked. One such example is Farming Simulator; whose players are on average older than Minecraft players. Within our lab, we installed a selection of games on laptops and connected them to servers to analyse network packets. This allowed us to foresee the different possible attack strategies for each game. Initially, it was easier for us to use reverse engineering than to contact the software developers of each game. For a passionate online gamer, like myself, it was a bit frustrating. The idea was not to enter big gaming contests in the name of R&D. In contrast, we were only interested in the connection phase between the player and the server because this is where attacks should be detected and countered.
Next, we imagined building an infrastructure to complement the “traditional” anti-DDoS (the VAC), an infrastructure that would enable us to analyse both incoming and outgoing traffic (which is not the case with the VAC). This creates two-way mitigation. The filter analyses both incoming and outgoing traffic. Another difference is that it is constantly active, meaning the system reacts to the first packets of an attack. The goal was to ensure the server remained “playable” throughout the duration of DDoS attacks and even better, to make sure that the players were unaware of any malicious activity.
As the diagram shows, a Tilera box, situated close to the server, inspects TCP/IP and UDP packets, initiates mitigation and can act as a cache to lighten the load on the machine under attack when it is difficult to filter illegitimate packets from legitimate packets. In the event of a “traditional” attack, i.e. when the VAC knows how to mitigate, the Tilera device guarantees protection until the VAC is activated and takes over. In addition, as the Tilera is placed as close to the server as possible (at the same level as the switches), the protection works even when the attack comes from within the OVH network itself. In these cases, the mitigation filters the attack until the machines located in OVH that caused the attack are identified and suspended.
The Tilera hardware was chosen for its computing power. Several thousands of packets per second are screened using particularly complex algorithms, all at very high speed. Unlike the Arbor solution, Tilera hardware is delivered without software: software development on Tilera is done in-house.
The mitigation code (the algorithms) is implemented based on information collected during the reverse engineering phase. It was not possible to develop a universal mitigation code. For each large family of games (Counter Strike, Minecraft…), we instead developed a “profile”, or a set of predefined rules that users can deploy in one click on the Tilera box, (via the customer control panel) to filter, with the greatest possible accuracy, illegitimate incoming and outgoing server traffic.
Is the protection that you put in place unique? What results did you achieve?
Once the solution was deployed, we tested it. First, it was tested internally then in the form of a beta test open to the public, during which fifty machines could be rented for a maximum duration of fifteen days. The goal was to increase the number of games tested to prove the effectiveness of the anti-DDoS protection, fix any weaknesses and correct algorithms to eradicate false positives. We learned that Counter Strike and Global Offensive have a connection protocol which varies in function depending on the method of connection used by players (via a browser, joining a friend by direct connection…). That was quite a headache!
In June 2015, we achieved satisfactory results, which meant we could seriously consider putting a range of game servers on the market with this protection included.
However, our work did not stop there. We always have an eye fixed on attacks and very carefully study the ones which we have not listed yet. Some are a result of an administrator’s incorrect server configuration, which are false positives. Others are real attacks which allow us to continue to optimise the protection offered by integrating algorithms to counter such activity. We mustn’t get ahead of ourselves however, we’re playing cat and mouse with those who launch attacks or those, luckily fewer in number, who attempt to pass through our protection, by successively trying to develop new methods of attack. We will never find a universal solution to counter all attacks but the important thing is that we stay far enough ahead to anticipate the attacks of tomorrow. For this reason, it would be counter-productive to reveal any more about our algorithms. Moreover, games are regularly updated, so we must also update our “profiles” accordingly.
Are we the only ones to offer effective gaming anti-DDoS protection? Today, few providers provide such protection. Nevertheless, it is not impossible to copy: it cost money, equipment and man hours. This requires a certain know-how and likely some recognition as a credible player on the gaming server hosting market, which will enable further cooperation with game developers in the future. OVH has something else that sets it apart: a global network capacity of over 11 Tbps enabling it to withstand intense attacks. Even the most advanced protection can never guarantee server availability if attacks saturate the mitigation equipment’s upstream network. And this is the reason why developers operating their own platforms will run into more difficulties in the future. The maximum intensity of attacks is increasing (OVH withstood an unprecedented 1 Tbps DDoS attack in September 2016) .
This results in two consequences for operators that do not have the network capacity to absorb the volume of such traffic: backbone saturation and service disruption for all their customers, and/or non-negligible financial consequences (transit fees to be paid for the excess traffic).
Will research and development of anti-DDoS Game benefit other OVH services or other business sectors that are online? Is this going to drive improvements in the “traditional” (VAC) anti-DDoS
It makes no sense to observe the outgoing traffic of the 270,000 servers hosted by OVH because the VAC functions very well for most attacks.
But you could imagine other applications for the protection we specifically deployed for game servers. For example, to improve protection for VoIP servers, which also use the UDP protocol and are exposed to the same risks that a lot of game severs are. Or to protect SQL servers, some of which use the connectionless protocol (notably MSSQL). Similarly, if we imagine a two-way anti-DDoS option, some services like video or music streaming could benefit.
The long-term plan is to combine anti-DDoS and routing within vRouter (an ongoing project) to simplify the architecture of the OVH network, ensuring better control and complete traceability. This advance would require a change in technology as it must be compatible with x86 architecture, which Tilera is not.