So I would suggest to add Issue five (you could argue this is included in Issue four, but it is too much important not to be exposed separately): Perform thorough security testing and fixing after that. No matter what, do it at least in beta phase and definitely after introducing major functional changes. This should be done by security experts, people skilled well to find security errors, with proven success record in the area and with drive to follow latest attacking trends. Architects and developers are not skilled enough to do that, and they generally do not have time and appropriate attitude to do the job of being the attacker. They are meant to build things, not break them and frankly - they do not have time to gain this special, but complex expertise. It is too expensive. The similar is with testers - usually they have “breaking” mentality, but not technical knowledge and appropriate reputation in the teams.

I would like to see that testing actual security would be a simple task, but it is not. I would also like to see that in development we are preventing vulnerabilities in early phases, but this is almost impossible to do. There are a lot of possibilities to make security mistakes in coding or fixing and almost every line of code could be a source of vulnerability.

I strongly support the use of fuzzing tools, but in general for reliability purposes and finding well known vulnerabilities. If you are running particular fuzz tools, you need to be able to know their limits - they are not silver bullet for every problem, sometimes they are hard to use and if you want to really gain from them, they are not very easy to configure. Even the most specialized ones are able to detect only particular patterns, not all of the problems - so I am a little bit skeptic about their coverage strength. I sometimes see people doing better in this area, because they have imagination and intelligence and they are learning. But for simple patterns they are nice to use and they are good for hygiene. At the end you still need security expert to dive into the mass of (mainly) false positives and to target real issues.

With the respect to application security what we need is also engineering attitude to see how good we are doing at the end. I did not see a lot of systematic approach in measuring actual application security at the end, not even very simple ones, such as: functional vs. security errors, security defect arrival rate (sDAR?), security defect density (sDD?), percentage of security testing efforts in project development effort, no. of “low hanging fruit vulnerabilities”, no. of “easy to fix” ones and similar. I do not favour any vendor, but I liked the SDL suggestion of the expected bug number formula in M. Howard presentations, I would like to see things like that from other vendors, too. This kind of information is something we have to collect and after that make quality decisions for future. It is not a rocket science :).