Some background info:
In the early years of high school, I was a part of a community that produced game mods (which I’ll now call AAA). There was another community who also developed similar mods (which I’ll now call BBB). Not surprisingly, there was a lot of animosity between the two communities and it quickly escalated to more than competing by building better products.
While both communities didn’t condone it, members of both eventually started launching DDoS attacks against other. The impact of these attacks was more than just taking down the community forums. Since both communities developed game mods, downtime of the servers meant downtime for their thousands of users.
The Problem:
Members of both communities launched DDoS using boaters. Booters, or “network stress testers”, are DDoS services usually comprised of compromised dedicated servers that send massive amounts of traffic. Booters are relatively cheap and extremely easy to use – the perfect option for script kiddies. At the time, an effective booter would cost just a few dollars for an hour of DDoS’ing. If someone wants to take down a server, he goes onto the booter, types in the IP and port, and the attack is sent.
The downtime had a significant impact. Too much downtime would cause users to leave. Losing customers meant a loss in revenue. At the time, both communities had peaks of 10k simultaneous users each – not a small number.
The admin of AAA was considering paying for DDoS protected servers, which were very, very expensive at the time. The profit margins from the community could not justify renting such a server.
As an active member of the community, I decided to take a look. From my initial research I, this was my initial understanding of the network.
The problem, I figured, was that AAA.com’s main server IP was publicly visible, so that anyone with half a brain could just type it into a booter and launch DDoS attacks.
The Proposed Solution
The solution, in my mind, was to find a way to prevent the true server’s IP from being publicly visible.
A reverse proxy sits between a client and the server. Instead of the client directly querying the server, the client would instead query the reverse proxy. The reverse proxy then queries the server and returns the result, acting as a middleman. Reverse proxies have many benefits, mainly improving bandwidth consumption by caching requests and static content.
How could this prevent DDoS you may ask. Couldn’t an attack just flood the reverse proxy? Another good question.
Reverse Proxies for DDoS Protection
A domain, such as AAA.com, has DNS records known as “A records” – the main DNS record that maps a domain to an IP. For example, AAA.com would resolve to it’s server’s IP, 123.123.123.123. This allows attackers to take the IP and DDoS it.
What’s lesser known is that a domain can have multiple A records. A browser connecting to AAA.com will automatically choose one of the A records (usually the fastest to respond). If the IP does not connect successfully, the browser will automatically choose another.
The intuition, therefore, was to set up multiple reverse proxies. That way, if someone chooses to DDoS AAA.com, only one of the reverse proxies goes down.
The architecture for this system was fairly simple. The AAA main server sat behind multiple reverse proxy servers. These reverse proxy servers were just cheap VPS servers running squid. In the initial test run, we rented 10 budget VPS’s, all from different providers and datacenters. The total cost was less than $100/month.
The Results
After deploying the reverse proxy servers, downtime literally fell to 0%. When the attackers from BBB tried to launch booter DDoS attacks, they could only enter 1 IP at a time, effectively taking down just one reverse proxy server. It is, however, very easy to see that they could’ve launched 10 simultaneous DDoS attacks on all the reverse proxy servers. I planned for this by buying a couple redundant VPS servers that I could quickly add in case all 10 main proxies fell. I never had to resort to them.
Whether it was BBB’s lack of knowledge or resources, the 10 reverse proxies stood solid. I imagine it’s not hard paying for 10 concurrent DDoS attacks, but I suppose it would cost significantly more money. I don’t think there was ever a time where more than 3 reverse proxy servers were down. Since the real server sat behind the reverse proxies, an attacker couldn’t directly attack it.
Using the cheap VPS servers as DDoS fodder was great for us – but not so much for the providers. Certain ethical judgements were passed when all of this was set up. It was known that a DDoS attack on a VPS may also take down its neighbors. As a result, we were kicked out of a few providers.
Nevertheless, I would say it was a general success. The DDoS attacks from BBB eventually almost stopped. No downtime on AAA meant that AAA members were less likely to retaliate, so both communities ended up winning.
Conclusions
While this was a good solution a few years ago, a lot has changed. This method is still very viable and I see a few companies actually offering DDoS-protected reverse proxy servers. But with the introduction of Cloudflare and Incapsula, which are both cheaper and easier to set up, I wouldn’t recommend using reverse proxies anymore for DDoS protection.
Of course, it’s best to just be on everyone’s good side and not get DDoS’ed.
Well, since many people and webmasters do not want to get their traffic sniffed by third parties like CloudFlare and Incapsula or Sucuri etc.etc. this is just the perfect solution and it is exactly what i was looking for. Brilliant idea, thanks for sharing this pearl of wisdom.
Iptables doesn’t work well with CloudFlare. Have you try install vDDoS Protection Reverse Proxy from https://sourceforge.net/p/vddos-protection Layer 7 Filter Mitigate DOS, DDOS, SYN Floods, or HTTP Floods attack?
[…] Reverse Proxy DDoS Protection / Eric … – December 19, 2013 Reverse Proxy DDoS Protection. Skip to the good part. Some background info: In the early years of high school, I was a part of a community that … […]