An Analysis of Google’s Project Zero and Alleged Vendor Bias

[Jake <jake () riskbasedsecurity com>]

https://www.riskbasedsecurity.com/2015/01/an-analysis-of-googles-project-zero-and-alleged-vendor-bias/

Google’s Project Zero has certainly been in the news the last couple
weeks.  But for many organizations, it was the first time they have
heard of the project. In July of 2014, Google announced a new
initiative called Project Zero.The basic premise of the project was
that Google invests heavily in their own security and had for quite
some time been also tasking their researchers part time work on
improving the security of other high-profile products.  Project Zero
is Google’s investment into a well-staffed team that they hoped to get
the ball rolling as they felt that people “should be able to use the
web without fear that a criminal or state-sponsored actor is
exploiting software bugs to infect your computer, steal secrets or
monitor your communications.”

Recently, there have been questions that have been raised wondering if
Google’s Project Zero is really unfair towards and targeting
Microsoft?  From reading several news articles as well as the blog
from Microsoft’s Chris Betz, who runs their Product Security Response,
you would definitely think that this is the case.

Since Project Zero was announced, we at Risk Based Security have been
following it closely as we were interested in how it might influence
vulnerability research. More specifically, we were curious if the
project would help improve the code that many rely on such as
third-party libraries and if it might ultimately impact the Bug Bounty
economy. With all of the recent conversations about Project Zero, we
decided to publish some of our own analysis of with an attempt at
using data to see if there is a bias towards targeting Microsoft, or
any other company.

One of the great things about Google and Project Zero is that they are
generally pretty transparent with their research and try to clearly
communicate their intentions about disclosure as well. But have they
really been clear with their disclosure policy? We know there is a
60-day Google company-wide intention for security disclosures that was
published back in July, 2010. But what about the policy for Project
Zero? We aren’t aware of a clear policy that has been published for
the project or any sort of blog post on the requirements (nothing on
their main blog, or the wiki in their tracker). In a Wired article
last year, it mentions that Google staff stated:

“… they’ll alert the company responsible for a fix and give it between
60 and 90 days to issue a patch before publicly revealing the flaw on
the Google Project Zero blog. In cases where the bug is being actively
exploited by hackers, Google says it will move much faster, pressuring
the vulnerable software’s creator to fix the problem or find a
workaround in as little as seven days.”

Based on reading the individual tickets you can see that they have
been communicating the 90-day policy since ID 44, opened on Jul 9,
2014 and since ID 89 they have been very consistent about telling all
vendors about the 90 day deadline and started automatically including
the boilerplate text:

“This bug is subject to a 90 day disclosure deadline. If 90 days
elapse without a broadly available patch, then the bug report will
automatically become visible to the public.”

In the initial announcement of Project Zero, there was no mention of a
hard line of 90 days for a disclosure timeline, but it does mention
they want to get things fixed in a “reasonable time”:

“Once the bug report becomes public (typically once a patch is
available), you’ll be able to monitor vendor time-to-fix performance,
see any discussion about exploitability, and view historical exploits
and crash traces. We also commit to sending bug reports to vendors in
as close to real-time as possible, and to working with them to get
fixes to users in a reasonable time.“

Chris posted a blog update in November 2014, where he highlighted many
bugs that were fixed well within Project Zero’s 90-day disclosure
deadline and that it represented a solid security response by the
teams responsible for Adobe Flash, Microsoft Office, Internet
Explorer, and Microsoft Windows. A more important point to mention is
the following:

We calibrate our deadline to balance two important factors: the right
of end users to know about a risk versus giving vendors a fair chance
to cleanly fix a bug under a reasonable embargo. Given the ratio of
deadline hits to misses, we think we’ve calibrated correctly for the
current state of the industry but we do hope to tighten the deadline
in the future.”

It seems that they could benefit from posting a formal policy at some
point in the future to clarify their disclosure process, but
regardless, they have been making vendors well aware of their expected
timelines. What we can say with certainty is that by publishing
details about their research, it allows us to get a better insight
into their work, their focus, as well as a limited view of their
interaction with vendors.

Show Me The Data

Project Zero has a tracker that they show their work in, and in some
cases publish very detailed information for us to see their process.
As of January 25, they have assigned 206 IDs in their tracker. One
aspect that was not clear is the reason that so many IDs, especially
earlier assignments, were restricted, meaning they were not open for
viewing to the public. We speculated that these restricted IDs were
ones that Google was not sharing for some specific reason, as they
could have just been IDs that have been abandoned, or they were still
being resolved with the affected vendors. Considering they still have
“Issue 1: This is a test” public, it didn’t seem that they were
cleaning up older entries that are not relevant.

We decided to reach out to Chris Evans who runs the Project Zero team
asking for clarification on why so many entries were restricted, and
he responded very quickly offering some great insight on the
situation. Chris shared that most of the early IDs that are shown as
restricted were due to a bunch of spam reports that were received when
the project was officially launched.  He mentioned that there are a
few other causes of early bugs not being made public.

The first few IDs were not filed as well as the project team would
have liked, so they filed a new ID again and marked the earlier issues
as invalid.
A couple more bugs were marked as duplicates because they
unfortunately filed the same report twice.
There were a few more issues that he shared were valid, but just
needed to be made public. In fact, our mail prompted him to open up
(ID #125) which he described as “a very interesting Flash bug that has
been fixed several days ago”. We thank Chris for opening it up
tonight, even though it made us redo our statistics!
And finally, as we expected, every ID currently from 144 onwards is
still less than 90 days old which aligns with their stated 90 day
disclosure timeline.

When looking at the 206 Project Zero IDs, the breakdown of tickets is
as follows:

ID Status            # of Vulnerabilities
Total Fixed                 83
Total WontFix             4
Total Restricted          73
Total Invalid                26

The “WontFix” designation means that the vendor does not consider the
reported issue to be a security problem, or does not cross privilege
boundaries.  In terms of the severity and impact to organizations of
the vulnerabilities that they have discovered, we looked quickly at
the VulnDB assigned CVSSv2 score for each ID.

For those that don’t like seeing just a CVSSv2 average (as there are
currently still a lot of issues with this version, and a new version
hopefully around the corner), we pulled more data from VulnDB to help
us understand Exploit availability:

Note that the high percentage of Proof of Concept (PoC) is largely due
to Google researchers developing them to demonstrate the
vulnerability, which typically makes it easier for the vendor to
diagnose and understand the problem.

Microsoft

As mentioned, there are theories being voiced that Google is
specifically targeting Microsoft and not practicingCoordinated
Vulnerability Disclosure, putting customers at risk.  A post from
ErrataRob reminds us that Microsoft seems to be enjoying a double
standard with their views on vulnerability disclosure. Additionally,
it has been noted that Microsoft’s latest plea for CVD is as much
propaganda as sincere, in a response to Microsoft’s Betz’s
articlepublished on the OSVDB blog.

It is important to understand where this perception comes from.
Articles posted like the one from PC World that talk about a “third
unpatched Windows zero-day” being released within a month is
polarizing in the context of the vulnerability disclosure debate. It
also makes it sound like this is a straight rivalry with one party
intent on hurting the other. From what we can see by analyzing the
data, this doesn’t appear to be the case. The disclosures are a
byproduct of Google’s stated policy of disclosing after 90 days, fix
or no fix. The use of “0day” denotes that Google published full
details of the vulnerability (after ~90 days) without a
vendor-supplied fix available.

Statistics                                    Metric
Total 0Day                                 19 Vulnerabilities
Average Disclosure Time          59.68 Days

It is critical to remember that not all vulnerabilities are created
equal, even within the same organization. While Microsoft may be able
to patch a particular product such as Office or Internet Explorer
quickly, they may not be able to do the same for Windows depending on
the numbers of platforms and versions affected. The life cycle of a
privately disclosed vulnerability varies wildly as it has to be
confirmed, patched, and then heavily tested. Organizations want and
really need Microsoft to test those patches adequately before
releasing, or you run into updates that can do more harm than good. So
this supposed “vulnerability disclosure war” as some are calling it
between two companies is also mixing in one Google employee’s spare
time activity with his day job.  No matter how many times someone may
say that “their thoughts and tweets do not represent their employers”,
it can be a very fine line specifically when you are a researcher
working for a company where a strong focus is on discovering
vulnerabilities in other company’s products. It can lead to questions
as to what constituents their employer’s versus their own time, and is
it decided who has ownership when it is more convenient. This is
especially true since that researcher’s day job and hobby are heavily
intertwined.

In doing further analysis on the Google Project Zero tracker, we find
that they have disclosed 20 vulnerabilities in Microsoft products as
of this blog. Of those 20, Microsoft labeled 4 as ‘WontFix’ (they
don’t consider them to be a security issue, and Google actually agrees
with most), 13 were fixed within 90 days, and 3 were released without
a fix due to the 90 day deadline being exceeded (the dreaded 0day!).
On the surface, considering ~81% have been handled in a coordinated
fashion that actually makes it seems Microsoft is doing pretty well.
Not only have they gone through the process with Google many times
already, they are obviously well aware of the 90 day disclosure
deadline.

While it is still debatable that Google should provide extra time for
companies when requested, it has become clear that this is not an
option in the eyes of Project Zero. Since Microsoft now knows that
Google is basically not flexible on that deadline, the onus is solely
placed on them moving forward. If Microsoft does not want to have a
0day disclosed by Project Zero they are going to have to determine how
to improve their response timeline and potentially dedicate more
resources to a given vulnerability than they have for the three that
were not fixed in time. It would be hard for anyone at Microsoft to
argue that producing secure products isn’t critical and that they are
short on resources to correct known vulnerabilities that may impact
their customers.

But the real question that begs to be asked; is Microsoft all alone in
this supposed ‘war’ and being unfairly picked on by Google? Again lets
take a quick look at the data to help us answer the question:

Vendors              # of Vulnerabilities
Apple                       39
Adobe                      36
Microsoft                  20
Linux Kernel             7
LibreSSL                  2
glibc                         1
PHP                         1
Red Hat                    1

The data suggests that there is a simple answer; of course Microsoft
is not being solely targeted. In fact, just a few days ago, Project
Zero hit the 90 day deadline on a trio of Apple Mac OS X issues that
went public without a fix.Ars Technica covered it in the same style as
other articles covered the “0day” dropping on Microsoft.  We found it
very interesting that there is no mention in this article that these
aren’t the first Google Project Zero “0day” dropped against Apple. In
fact, between July 4, 2014 and September 30, 2014, Google released
eleven vulnerabilities in Apple products without a patch.  While Apple
and Microsoft have each had “0day” issue due to missing the timeline.
We are also able to see that Project Zero has sent a substantial
amount of  vulnerability reports to the Adobe Flash team and they’ve
hit all the deadlines, which is very impressive.

That was the initial reason that prompted Brian Martin, our Director
of Vulnerability Intelligence to really dig even further than we had
previously into all of the public Project Zero vulnerabilities. The
analysis focused on the vendor, vendor notification date, disclosure
date (when the vulnerability was disclosed, which was by Google for
some, and by the vendor for others), CVSSv2 score and exploit
availability.

Conclusion

From our analysis of available information, we believe it removes some
of the emotional arguments and allows one to make their own educated
statements which appear to support that Google is not singling out
Microsoft with their Project Zero efforts. Furthermore, it shows that
Apple has received the most attention with Adobe close behind, both of
whom have received considerably more attention than Microsoft. It just
appears at this point, Microsoft is the only vendor that is failing to
meet the 90 day disclosure deadlines sometimes, and then publicly
criticizing Google for their work (or as some may say, providing free
security work to improve their products and ultimately ensure their
customers are better protected).

We sincerely hope that cooler heads will prevail as we all work
together to better secure our organizations and protect consumer
confidential information. Google’s work at Project Zero is impressive,
and clearly making a positive impact on the security of high-profile
products. We hope that companies will continue to work with Google and
understand the importance of fixing issues as soon as possible to
reduce the time of exposure customer’s face. But as with all policies,
there are times when some level of flexibility does make sense when
the purpose of the project is to improve security. If you look at
companies that Google has recently acquired, such as Dropcam, even
their disclosure policy states, “We ask that you give us a reasonable
amount of time to respond to your report before making any information
public”.

Of course, “reasonable amount of time” has been the cornerstone of the
coordinated vulnerability disclosure debate for decades and doesn’t
appear to be going away anytime soon.


--
Chief Information Security Officer
Risk Based Security
_______________________________________________
Dataloss Mailing List (dataloss () datalossdb org)
Archived at http://seclists.org/dataloss/
Unsubscribe at http://lists.osvdb.org/mailman/listinfo/dataloss
For inquiries regarding use or licensing of data, e-mail
        sales () riskbasedsecurity com 

Supporters:

Risk Based Security (http://www.riskbasedsecurity.com/)
YourCISO is an affordable SaaS solution that provides a comprehensive information security program that ensures focus 
on the right security.  If you need security help or want to provide real risk reduction for your clients contact us!