Posted by Pedram Amini
Recently I presented a talk titled "Mostrame la Guita!" (regional Spanish for "show me the money") at Ekoparty in Buenos Aires, Argentina. The purpose of the talk was to provide transparency into the world of vulnerability markets and for the first time expose the inner workings, statistics and some anecdotal stories behind the TippingPoint Zero Day Initiative (ZDI) to the public. As far as I know, the only other available resource on the matter of markets and pricing is Charlie Miller's 2007 paper "The Legitimate Vulnerability Market". The information presented was garnered through public resources, numerous interviews and my own personal experiences in the marketplace from 2002 to present day.

The talk was well received and there was a lot of interest from various parties in getting their hands on the slides. Along side the release of my presentation I prepared this blog entry as the slides are terse and contain some eye charts that, without having heard the talk, could be easily misinterpreted. The slides were presented through Google docs and are now available at:

Mostrame la Guita! Adventures in Buying Vulnerabilities

Please read on for notes on specific slides of interest. If you have any questions or comments feel free to post them as a comment here or contact me directly via e-mail.

Market PricesFor the purposes of my talk I broke the market down into four categories:

• Vendor market: bug bounties offered by Mozilla and Ghostscript for example.
• "White" market: purchase programs where the researcher knows where the information they are selling is going to end up. This includes the TippingPoint ZDI and iDEFENSE VCP who both disclose all purchases to the affected vendor.
  
• "Grey" market: programs where the final destination of the information sold is unclear. This includes direct / indirect sales to the government as well as some word of mouth programs.
• "Black" market: sales to the "bad" guys.

Vendor bug bounties generally fetch less than $1,000 USD per security exposure, for the rest of the markets:


[full slide]

The wide range of pricing has led many to claim that there is no fair market value for vulnerability research. I disagree and stated so in my talk. The reasoning behind my logic is that you get what you are willing to give. The individual markets offer different pricing but also demand different requirements.

On the white market, and specifically at the ZDI for example we don't need nor do we ask for exploit code. If you can point out a bug we'll take it from there. In fact, frequently we receive very limited crash reports through the ZDI that require significant effort to dissect thoroughly. The white market also offers credit for your discovery and the guarantee that the reported issue will be forwarded to the affected vendor and disclosed in a responsible fashion.

On the grey and black markets credit is completely out of the question. You are selling the rights to your information and expected to never talk about it again. The vendor will not be contacted. Crash reports will not be entertained and nor will buggy or basic proof-of-concept exploit code. Exploits must be polished and weaponized. This is a drastic departure from the white market and many researchers are not comfortable operating in it.

Acquired Research Statistics
Since the launch of the Zero Day Initiative in August of 2005 we have signed up over 1,000 researchers and received almost 1,900 vulnerability reports between them. We only accept critical issues affecting top enterprise software and out of the the bugs we have seen we have purchased over 500 (about 30%) at an average of about 10 a month. That's a lot of bugs. The table presented in the following slide outlines purchase statistics for our most prominent vendors:


[full slide]

The first numeric column shows how many vulnerability purchases we have made which affects the captioned vendor. Microsoft accounts for the largest number, 133, that's 33 Microsoft critical issues we are responsible for disclosing on average per year. The next column shows what our accept ratio is for that specific vendor, recall that our overall is roughly 30%. The final column shows how much of our total ZDI budget was spent securing that specific vendor. As Microsoft accounts for most of our purchases it is no surprise that they account for most of our expenditure as well, 30%. Apple comes in at a distance second accounting for 8% of our total budget.

Vendor Patch Time StatisticsFor each of our most prominent vendors we queried and charted their average time to patch per reported year. Let me explain this more clearly with an example. Between all the bugs we reported to Apple in 2006 their averaged time to patch was 166 days. If we reported a bug to Apple in December of 2006 and it wasn't patched until December of 2008 that 2-year patch time was averaged into 2006. For each year the longest and shortest patch times are highlighted in red and green appropriately. Through this measurement Mozilla comes out on top pretty much across the board:


[ full slide ]

We averaged the individual per year columns into the overall column which places Mozilla at the top with the quickest average patch time of 48 days and Symantec trailing the pack at 307 days. One thing to keep in mind however is that these numbers do not include the currently outstanding issues, which brings us to the next and final column.

The upcoming column displays the average number of days that all issues reported to the vendor have been outstanding for. Up to date listings of all our outstanding TippingPoint and ZDI discoveries are available at:

• http://dvlabs.tippingpoint.com/advisories/upcoming/
• http://www.zerodayinitiative.com/advisories/upcoming/
  

This column requires a bit of a caveat as well as the results get skewed with recent bundles of disclosures. For example, we recently disclosed a number of issues to Hewlett-Packard and this brings down their average outstanding patch time.

Longest Patch TimesWe have sliced our vendor data subjectively, by patch time and by days outstanding. The final view I offered the audience is a listing of the top 10 most outstanding bugs:


[ full slide ]

Entries with a + (plus) sign indicate that the issue is still outstanding, otherwise it has been patched. Note that although our recent disclosures to Hewlett-Packard improved their average in the previous slides upcoming column, they hold the title for the top two longest patch times... and counting.

Holding the 3rd and 4th place positions is Microsoft with two issues that affect their Office Web Components (OWC) and along with another OWC issue have all been recently patched in MS09-043 (ZDI-09-054, ZDI-09-055, ZDI-09-056). The two and a half year time to patch triggered some media attention but as I mentioned earlier, patching is not a trivial process in some cases. I verbally covered this matter in my "Behind the Hype" slide stating that the cause for delay on Microsoft's behalf was legitimate and not due to any underlying disorganization.

The presented data was our first unveiling of a vendor "report card". Within the next month or so we intend on creating a permanent home on the ZDI website with all these statistics and more. So check back at www.zerodayinitiative.com and let us know if there is anything in specific that you would like to see.

-pedram

Posted by Pedram Amini
Ekoparty 2009 is all wrapped and everyone had a great time. The venue was spectacular. An open split level warehouse which comfortably held the 500 researchers who attended this boutique con in Buenos Aires, Argentina. The talks were held in the theater upstairs in both English and Spanish via real-time translation. The downstairs area housed the various sponsors and a slew of interesting competitions.

CORE Security created a really fun 3-level simulated hardware reverse engineering challenge via the Ruckingenur Editor. Immunity had their NOP certification test. TOOOL had a lock picking competition. There was a fun CTF where teams had to hack into faux bank websites and steal money from each others accounts. Finally, my team had the DRINC challenge (see the previous blog announcement) where we intentionally exposed 17 bugs across various components for contestants to discover in exchange for drink tickets and a grand prize.

We had over 40 entrants participate in our challenge and over the course of the 2-days we ran the contest a handful of them discovered almost all of the exposed issues. At the end of the competition we were pleased to announce Gera from CORE Security as the grand prize winner and recipient of a our TippingPoint "Kick-Ass" trophy, a Zero Day Initiative laptop messenger bag and a bottle of Dom Perignon champagne. Here he is accepting his reward with the TippingPoint team:


[full size]

The following is a list of the various DRINC components and the discoverers of each of the exposed bugs.

AwesomeX.ocx

1. Peter Vreugdenhil, Fermin Serna
2. Peter Vreugdenhil, Fermin Serna

DRINCryptionSuite.zip

1. Gera CORE Security
2. Gera CORE Security

EkoFriendlyServer.exe

1. Esteban-Hernan, Costantino Leandro
2. Costantino Leandro, Esteban-Hernan

LogAnalyzer.exe

1. Charlie Miller, Victor from Hauttech Group
2. Charlie Miller, Victor from Hauttech Group

NetworkScriptingEngine.py

1. Esteban-Facundo, Agustin, Costantino Leandro, Gera CORE Security, Jean Sigwald

SecureLoggingServer.exe

1. Gera CORE Security
2. Gera CORE Security
3. No entries
4. No entries

TSRTVideoCodec.dll

1. Sergio Alvarez Recurity Labs
2. Sergio Alvarez Recurity Labs

Web30Server.exe

1. Costantino Leandro, Jean Sigwald
2. Gera CORE Security
3. Gera CORE Security, Esteban (this bug was not part of the contest!)
   

The third bug discovered by both Gera and Esteban was not actually among the list of purposefully planted bugs but rather a directory traversal issue in the underlying mongoose webserver that we modified for the purposes of the contest. A bug report has been opened with the mongoose developers.

The TippingPoint DRINC contest is now available for download as both a Windows MSI installer, which will properly install the various components, and a standalone archive. We are going to hold off on posting the solutions for now. However, if you want to see them simply drop one of us an e-mail and we'll shoot it over to you. If you e-mail us a find before we post the solutions we will add your name to the above list of discoverers. Here are some hints we shared with contestants that should help you get started:

• Don't bother fuzzing the AwesomeX ActiveX control, there is a mechanism to prevent it.
• On the LogAnalyzer the values 0x3 and 0x10 should save you some time.
• Be sure to look at the sample AVI provided when you are working on the video codec.
• Here is an IDAPython script for Web30Server that will add symbols to your IDB.
  

Here are some not-so-great-quality cell phone camera pictures we took at the event (click for a larger version):

The DRINC contest grand prize:

[full size]

Zoom up on the "Kick-Ass" trophy

[full size]

The audience during my talk

[full size]

2nd place DRINC team (Facundo, Emiliano, Hernan, Esteban)

[full size]

Cody and Cameron working with Charlie Miller on the DRINC contest

[full size]

Gera and I catching up before my talk

[full size]

The WOPR (yes from War Games) from the speaker stage

[full size]

There was a professional photographer at the event as well, we look forward to seeing those pictures when they are released. All in all everyone from my team had a great time at Ekoparty and we look forward to attending again next year.

-pedram