Post to webappsec mailing-list on WAF and pen-test: dead again
There is no doubt in my mind that some very strong experts out there have put WAF or WAF-like technology to good use. However, WAF is dead and dying regardless.
I think that very large-installation, Internet-facing web applications require Anti-DDoS technology in the form of an appliance, preferably one that does rate-based behavior detection. I often feel that those same organizations also require SLB appliances, although I prefer to see these integrated with a switch fabric in a chassis-based, large backplane network switch. In a year’s time, SLB Layer-2 technology could be replaced by VMware DRS clustering and/or an equivalent like Microsoft PRO. I was always a fan of Anycast to replace SLB at Layer-3. I continue to suggest these models/architectures today.
Can whitelist WAF technology be used by those same devices in the short-term (Anti-DDoS or SLB appliance)? Absolutely, as long as it’s done by an expert and tuned to the applications. Should these devices sometimes be separated out of a traditional operational role, due to auditability and for compliance scoping purposes? Probably not. Should they perform monitoring, debugging capability, or solving hard production problems? Probably not.
The reason that the first question is a yes, and the others are a no is because Anti-DDoS and SLB devices are already performance-ready-capable of providing WAF whitelisting functionality (note: not in all cases, but this works especially well for devices that provide rate-based behavior detection before mitigation). Monitoring does NOT require an inline device. All it requires is network taps (or potentially port-mirroring, but most professionals recommend taps over SPAN ports). Also, infrastructure is changing rapidly, so it’s not wise to invest in a dying model.
Additionally, I know that companies like Sourcefire and Reflex Systems are integrating at the VMsafe API layer, which is a hypervisor introspection layer much like XenAccess. This is really where much of the AV/IDS/IPS/HIMS/DLP/WAF/blacklisting-whitelisting technology belongs. VNET will also change the introspection layer (in addition to almost completely eliminating the physical network layer and SIM/SEM/SIEM/NMS/EMS moves & changes), as it simply adds to introspection functionality. I have already alluded to Cisco AXG becoming a VNET “module”, but what if Reflex Systems or StoneSoft start integrating WAF not only as a VNSS (Virtual Network Security System), but also at the hypervisor introspection layer?
Fortunately, for application security, server virtualization and the evolutions it’s bringing with it e.g VNET and VMsafe, are going to dominate traditional networks and cut their existing budgets. Unfortunately for application security, the new virtualization evolution also brings with it tons of object reuse (there are at least two new controls channels available to adversaries), and new ways of establishing covert channels.
This means a few things. First of all, the word “firewall” is dead, and therefore, the word “web application firewall” and the associated acronym, WAF, are also dead. Imagine today if there existed a control channel that, when taken over by adversaries, it became a covert channel that had unlimited object reuse control of every physical RAM on every computer in existence all at once. This is cloud computing, but virtualized.
Not only that, but we are saying that adversaries have already bypassed traditional firewalls by using the application layer i.e. Hacking Intranets from Jeremiah Grossman. Thus, this master, covert control channel is already on its way to being built (at least as man-in-the-browser). Imagine for a second that you don’t use NoScript with Firefox and additionally implement the features of Chrome by using multiple Firefox profiles. Imagine for a second that you are a regular user, with all of those Clickjacking and modern application attacks available to anyone who wants to get to you.
Like many of us used the words “brick-and-mortar” to describe backwards-companies during the dotCom bubble, I think “fire-and-wall” well-describes organizations that continue to cling to traditional networks and network security as answers to Internet, Enterprise IT, and any operational risk.
Do I intend to sell you on the idea that we should all instead jump to Fortify RTA or Microsoft SRE? No. There are potential consequences to any of this. This is only the functionality required to reduce risk to applications, not the assurance that risks have been removed.
TCSEC says that we need to balance functionality and assurance. But nobody ever bothered to do any assurance. Assurance is the Microsoft SDL, SDL Pro, and SDL-IT. @Stake and Foundstone are gone and have split into tons of fractured security evaluation and risk assessment boutiques that have 1-300 developer-security-tester guru’s that mix SAST and DAST with expert review.
But the SAST+DAST market is less than $100M, while WAF is at least 20% more than that (although probably inflated).
I hate to be the bearer of bad news, but you don’t just say “DO BOTH” because nobody will do the SAST+DAST work. We tried that last time, when tcp_wrappers and the DEC firewall came about The underground that wanted to keep their covert control channels alive started dumping rootkits on pre-pwned Unix machines. Then Dildog and others made it possible to easily access Windows machines, and after that – botnets and the like have reigned. There are already backdoors in our web applications. OWASP Scrubbr is not going to save us all by itself.
Who did the work back then? OpenBSD? Certainly not Microsoft, and even today their SDL appears to be failing by some, but imagine if it did not exist at all. We obviously have to do better with assurance practices.
Can functionality-based controls work easier, better, and faster than assurance ones? Are they that less complex and easier to train? Or is there just more written about them because it’s easier to SELL them by baking them into products rather than customizing them to an ISV organization or an Enterprise development team?
If you are part of the group that is spending $120M on WAF technology, then you are hurting the SAST+DAST market because you’re taking away that spending. Clearly, risk analysis is not taking place and people are spending based on familiarity in addition to PCI-DSS requirement 6.6, which all but forces the inequality to happen.
Look at the best in exploitation-countermeasure functionality-based controls that work on object reuse problems e.g. DEP, ASLR, SafeSEH, SafeInt, et al. Are adversaries still bypassing these? Security researchers in the offensive-research space are. These countermeasures are closer to the code (even HIPS is closer than network-based IPS), like many WAF suggestions. Is is true that we still require assurance even after 15 years of exploitation-countermeasure optimization? I remember when stack-guard protections were first coming out – they were seen as a huge joke (i.e. toy/researcher technology), much like Fortify RTA, CORE GRASP, Microsoft AntiXSS-SRE/AntiCSRF, GDS Security GPF, and HDIV are seen now.
I know to many of you out there, this looks like a rant, and I really could go on forever about this topic. So, go to the datacenter, give your WAFs a hug, and continue to buy into the “functionality is better than assurance” argument. You’ll feel better in the morning, right after you forget that you just opened up your database to any talented people who want to make money from the data in it.
Also, pen-testing is dead. We no longer need to prove that applications are insecure. We know they’re insecure – no matter how many functionality controls you layer on top of them. Unless YOU prove that the applications that YOU are responsible for ARE secure, you are working against the rights of users, consumers, cardholder data, personally identifiable information tied to healthcare and financial records, trade secrets, and the ability to control our critical infrastructure. Enjoy.

I think you make false assumption – pentest is not about proving that software is vulnerable, pentest shows you (good pentest should) exactly what are the vulnerabilities and what are the risks connected with it. To make software more secure first you have to know, what’s wrong in it.
I always find these types of post entertaining. So here’s my problem — think of (the concept of) firewalls and pen-testing as a component of a process. It’s not a silo as alluded to in this post — stating because one can get past a “firewall” that it is (killing the word kills the technology) now “dead”. A layered defense is still useful and needed, if not just for correlation using SIEM solutions (”Monitoring does NOT require an inline device.” — right but having something that can do something about it is advantageous). At that point the firewall can just be another point of inspection, but with broader capabilities.
The clincher for me is the hypocrisy stated here: “Certainly not Microsoft, and even today their SDL appears to be failing by some, but imagine if it did not exist at all.” Read it again: “…imagine if it did not exist at all”. Sure, a firewall at a perimeter may not be able to address a SQL injection attack. But, like you said all you need to do is monitor, so XYZ WAF sees it coming across the wire (not inline, but off of one of your taps), and tells said SIEM solution of which tells the firewall to drop the state and block that IP until it’s reapproved by a knowledgeable admin. Sure, the attacker can source elsewhere, but it may be just frustrating enough and slow them down that much, or pushes them to try a different approach / path that leads them to expose him/herself.
I guess I don’t see how moving everything to an API in VM and “eliminating the physical network layer” buys you anything but a flawed trust model… I think it is a component of an overarching process when designing new architectures, but I don’t think it’s an end-all-be-all solution as portrayed.
“OWASP Scrubbr is not going to save us all by itself.” — no, but it helps. In the end this article reads like there’s a holy grail that has yet to be found. When really it’s just a smart culmination of current and future technologies. VMsafe API: just another same solution to a newer technology, nothing new.
ME proving that MY applications are secure? It’s impossible (the wording of it). Unless you have and understand every piece of information that everyone else knows there’s no way to qualify that. The first person to do this will abolish any need for any additional security because if one could actually prove that then the “proven one” wouldn’t need any external security measures.
We don’t live in a vacuum. We can’t predict the future. And it’s all f’in software. The flaw is in and of itself.
@ carstein:
I look to the MITRE CWE to determine what’s wrong with software. If a pen-tester wants to update that list with new techniques or weaknesses, then I’m ok with that. Keep it in the lab.
@ windexh8er:
Layered defenses are only good up until the point where they become redundant. I hate to get in another separate argument about routers vs. firewalls, but I am a firm believer that routers can do more with receive-only ACLs and null routes to help security than any firewall ever could.
Monitoring is great. Intrusion detection is great. I would just rather see these things baked into an application than layered into the network or on top, or through a hypervisor introspection layer. Agility in app development is much more easier to execute than in operations, even with standardized products and straightforward instructional resources. While this is not true in all organizations today, it can be true with the correct organizational change and organizational development.
I already knew that “Certainly not Microsoft, and even today their SDL appears to be failing by some, but imagine if it did not exist at all” was my weakest argument, and would probably be a source of controversy in any comment thread. I build a lot of strawman arguments up to see how people will knock them down. There are a few problems with your reasoning. An operator can surely do many things to temporarily prevent adversaries, even with automation — and some are more obvious to the adversary than others. I prefer subtlety.
Almost every organization is facing intelligent, organized, unstoppable, dedicated adversaries. Blocking their IP is akin to telling them that you know that they are there, but that you have no idea how to stop them. It just steps up the game. It’s best to simply let them get away with cracking a few passwords or trying a few SQL statements, while at the same time tracking them down, striking back at their applications and infrastructure (two can play at the SQLi game… they probably have their own database of sorts… and if they want passwords or customer data, just feed their database a list from http://fakenamegenerator.com – who knew that their attacker tools worked so well!), and then working with a take-down service or law enforcement (or military in some cases) to remove the adversaries’ capability to attack permanently. It’s called “Intelligence” and “War Strategy” for a reason. Know your enemy.
It doesn’t even have to be that obvious even. You could simply gather a list of successfully cracked accounts and then flag those accounts to require an extra step when logging in. Something that the real user would know that the new-owner of a user/pass pair would not. However, “lock-down” capabilities must usually be written into the application. I consider this a good thing.
“I guess I don’t see how moving everything to an API in VM and “eliminating the physical network layer” buys you anything but a flawed trust model… I think it is a component of an overarching process when designing new architectures, but I don’t think it’s an end-all-be-all solution as portrayed”.
I’m actually going to have to agree with you on this point. I’m just trying to explain where the industry is headed. Also see: cloud security: the biggest oxymoron the technology industry has ever heard. If security is the removal of all assets from all threats, and cloud computing brings all assets to all threats, then we’re talking about a major problem we’re going to have to address. Maybe application security can help?
I agree that OWASP Scrubbr is one of the best projects out there today for getting to the temporary source of attackers’ new agenda: to take over the web application layer. Certainly we can do even better than this. SpyBye and Scrubbr technology should be merged and automated.. this would make for an interesting project.
You said, “In the end this article reads like there’s a holy grail that has yet to be found. When really it’s just a smart culmination of current and future technologies”. I don’t know about a holy grail, but I like DAST+SAST+Expert-Review. I like the concept of a Secure Software Group that creates a self-service Application Assessment Factory, which indexes findings based on real risk analysis such as OCTAVE, NSA IAM, OSSTMM 3.0, Certified Secure Web, NIST SP 800-30/39 and FAIR.
You also said, “VMsafe API: just another same solution to a newer technology, nothing new”. Again, agreed, and I explicitly stated that XenAcess has had this for quite some time.
As a final point, you stated, “ME proving that MY applications are secure? It’s impossible”. It is not impossible. Pete Herzog clearly defines “Perfect security” is attainable, and he attempts to prove this scientifically through his work with his fellows at ISECOM. Perfect security is “just enough security”. The Microsoft SDL is probably 4SIGMA to getting to 6SIGMA, and while the SDL Pro Network and the SDL-IT are in 2SIGMA, they will also quickly gain highly quality and operational excellence in process and by design.
If you read the Microsoft SDL approach to the SANS/CWE Top 25 programming errors of all time, you will see a very well thought-out execution of “perfect security” where the application security program attempts to stay one or two steps ahead of any adversary. When you put the SDL up against people like Charlie Miller, Jacob Honoroff, Sean Fay, and Geoff Morrison and/or Mark Daniel, Shane Macaulay, Derek Callaway, and Alexander Sotirov, you actually have what looks to be a fairly solid foundation of prevention.
Application security is like Hallmark: when you care enough to prevent the very best.
>>
It’s best to simply let them get away with cracking a few passwords or trying a few SQL statements, while at the same time tracking them down, striking back at their applications and infrastructure