Bugtraq mailing list archives

Possible vulnerability in F5 BIG-IP LTM - Improper input validation of the HTTP version number of the HTTP reqest allows any payload size and conent to pass through


From: Eitan Caspi <eitanc () yahoo com>
Date: Thu, 7 Jan 2016 18:40:51 +0000 (UTC)

Initial note: The vendor has graded this issue as a vulnerability graded as "High" in my email exchange with it, but 
eventually posted the issue as a "Know Issue", so some of this issue's characteristic that follows can be treated as 
initial ones, as I ask the IS community to look into this issue and give a "second opinion" about it. Thank you.


Suggested severity level: High (per the vendor's initial response)

Local / Remote activated: Remote

Exploit Code: No need for experiencing the issue, but may be needed to realize an exploit via this issue

Assumed cause for the issue: Improper input validation

Assumed types of Possible Risks/Abuses (all assumed as possible directions, since I did not have the time nor resources 
to look into them):
Denial of Service
Code Injection
(See more at "CWE-20: Improper Input Validation" - https://cwe.mitre.org/data/definitions/20.html)

Affected Software:
Vendor: F5 Networks
Product: BIG-IP
Module: LTM (Local Traffic Manager)
Versions: 11.6.0, 11.5.3, 11.5.2, 11.5.1, 11.5.0, 11.4.1, 11.4.0, 11.3.0, 11.2.1, 11.2.0, 11.1.0, 11.0.0, 10.2.4, 
10.2.3, 10.2.2, 10.2.1, 10.2.0, 10.1.0

Summary:
Input data accepted for the HTTP version number section of the HTTP request is not enforced to be in the correct 
format, hence any payload content/format and size is getting through without being blocked immediately by an error or 
security reply, and only when the underlying TCP timeout is reached – only then the base TCP connection is ending by 
the server side (i.e. no HTTP response is accepted from the remote server side)

Description, reproduction instructions and communication with the vendor (it is taken from the blog post I published 
about this issue, http://fudie.net/possible-vulnerability-in-f5-big-ip-ltm/, so it is somewhat written as a kind of a 
"story"):
"
About a year ago, while I was performing a web site penetration test for a customer, I run a manual fuzzing phase, 
where I like to “question” even the most basic networking and application conventions, and this time it paid off more 
than the usual…

The site was behind a “F5 Networks” BIG-IP device, running the modules of LTM (load-balancer, 
https://f5.com/products/modules/local-traffic-manager) and ASM (WAF (Web Application Firewall, 
https://f5.com/products/modules/application-security-manager)).

Generally, each HTTP request made by the client towards a web site (normally using a web browser) should have its first 
line in the following format:
<Method> <URL> <HTTP Version>

– The <Method> part is replaced with one of a fixed set of words that describe the action the client wish to perform on 
the target site (usually GET for viewing a web page) – The <URL> is the site address the client wishes to access – The 
<HTTP Version> must be in the format of HTTP/<version number> (usually 0.9 or 1.0 or 1.1 which is the most common these 
days, e.g. HTTP/1.1)

For example:
GET http://www.somesite.com/ HTTP/1.1

I decided to attempt and disregard this format, so first I carefully tried messing with the HTTP version number – first 
I tried 0.1 (a non-existing HTTP version number) and sent something like:
GET http://somesite.com/ HTTP/0.1

To my surprise, instead of getting a reply of either an error message from the back-end web server or a security 
blocking message from ASM, for trying to be naughty – simply NOTHING came back, no reply arrived from the server side 
and the HTTP session ended only when its underlying TCP session timed out.

So, I dug dipper, to try and find what caused this phenomena, to see if there is some logic for this glitch – so… – I 
tried different numbers (e.g. 1000, 0.5, -4, etc.) and the result was the same.
– I tried without any number (e.g. HTTP/ ) – the result was the same.
– I tried omitting the forward slash – the result was the same.
– I changed HTTP to be helloworld (GET http://somesite.com/ helloworld) – the result was still the same.

So, it looked like there is no format verification and enforcement being made at the BIG-IP end regarding the HTTP 
version part of the request, a situation that looked risky to me, as the content/payload being pushed from the client 
to the server is not sanitized properly, possibly being accumulated at some memory, either on any relevant F5 BIG-IP 
module or at the back-end application server, which may lead to server resources being exhausted or possibly 
overflowing the memory, which can be much more dangerous, as it may allow the attacker to run there his/her own 
malicious code at the remote server side.

I tried to run some basic low-power DoS (Denial-of-Service) attacks using this issue, to see at what level, if at all, 
this issue can harm the target system – but since I didn’t have the sufficient resources to perform a decent large 
scale DoS attack, I wasn’t able to spot anything more than a minor traffic delay, but not something that I can be sure 
that this issue was the cause of it.
So, I had to stop my research at this point and turn to the vendor, “F5 Networks”.



First of all, I found the web page in which F5 states about its security vulnerability response policy (“sol4602: 
Overview of the F5 security vulnerability response 
policy“,https://support.f5.com/kb/en-us/solutions/public/4000/600/sol4602.html). Unfortunately this page did not 
include any public key (using PGP) or S/MIME certificate to encrypt the email exchange between me and F5 nor any SSL 
based web form.
So, since I wanted to report my finding securely, I sent on the 8-Feb-2015 the following email to the email address 
mentioned at the above web page as the one for reporting security issues – security-reporting () f5 com. I signed this 
email with my S/MIME certificate, so F5 will have a way to securely reply to me.
1. I wrote:
”
Hello,

I wish to report something to F5, but at 
https://support.f5.com/kb/en-us/solutions/public/4000/600/sol4602.html?sr=12234302 there is no mention of any way to 
security encrypt the email sent to F5, either PGP or s/mime – do you have something in place for this goal? (s/mime is 
preferred)

Thanks.
”

2. The next day, 9-Feb-2015, I was replied by F5 Support Reply <SupportOne () F5 com>:
”
Hello Eitan,

I have forwarded your request to our IT department, as I spoke with one of the security people and he does know what 
you are asking for, someone will be back in contact with you soon in regards to your request.

Respectfully,

<employee-name> | Technical Support Coordinator II ”

3. I promptly replied on the same day:
”
Cool, thanks.

I will be waiting for your move.
”

4. As no one approached me by the 14-Feb-2015, I sent the following email to the support email address I was replied 
from initially:
”
Hello F5,

Any news on this?
”

5. I was replied on the same day with:
”
Hello, Eitan:

I spoke with our Duty Manager about your security concerns.  He followed up with our IT Security Department.  I was 
told that they do not have an answer at this moment, but they are working on the problem.

Meanwhile, I am leaving a note for <employee-name>, the woman with whom you communicated last week.  She will be in 
tomorrow, and can get in touch with the team originally tasked with this.

I apologize for the length of time it is taking to resolve this, but I assure you that we are taking measures.

Regards,

<employee-name>

F5 Networks Support | www.f5.com
”

As no one replied to me by the 21-Feb-2015 I sent the following email (which included all of the above correspondence) 
to Mr. Manny Rivelo, who at the time was the company’s senior vice president in charge of security (and now he is the 
former-CEO, 
https://f5.com/about-us/news/press-releases/f5-networks-announces-appointment-of-long-time-f5-executive-john-mcadam-as-president-and-ceo),
 and I was in touch with him a few times during the time I worked at F5:
”
Guys, are you serious?! How long does it take?

You don’t treat security seriously.
”

On the same day he replied from his phone:
”
Eitan, I will reach out to our CIO to see what the issue is.
”

I replied on the same day:
”
Cool, it will help my will to securely hand F5 via email a possible security issue and also for others in the future.
Consider also adding a secured web form for this purpose, as a parallel and quick way to submit security issues.

In security response time is crucial.
”

Two days later, on the 23-Feb-2015 I was replied by M., a senior security supporter, who from now on will be my point 
of contact, and this was the first time I received an official support case number, to track this case:
”
Eitan,

First, apologies for the delay in responding to you.  I wasn’t involved, but I’ve been told there was some confusion 
with our first tier of support which caused a delay in getting this into the proper channels.

I’ll email you directly in a moment.  I don’t have S/MIME, but we do have two options for secure communication.  The 
first, and easiest, is <a service F5 uses. I will omit it as it is not relevant here and it is a quite long text. Eitan>

The second method we can use is PGP.  My public key is listed on MIT’s keyserver: <link>

I’ll email you directly twice – once with each method – to make things easier for you.  Then you can respond using your 
preferred method.

Thank you.

-M.
”

I replied to M. on the same day:
”
Hello M.,

Thank you for jumping in to save the day… 😉

Got your key and added to my keyring (btw, consider giving it an expiration date).
I don’t publish my PK on the net, but I added it here for you.

I will re-verify the issue, write my report about it and send it your way in the coming days, now that I am sure we 
have a secure (relatively, these days…) communication channel.

Will be in touch soon.

Cheers!
”

I sent M. the report using an encrypted email and on the 2-March-2015 he emailed me:
”
Eitan,
I received and successfully decrypted your email from Saturday.  I’ll attempt to produce the issue in our lab and I 
will let you know the status.
Thank you
”

On 8-May-2015 M. wrote to me:
”
Eitan,

We worked on this with our product development group and we’ve opened ID518020 for this issue.  We are classifying it 
as a Vulnerability, with a Severity of High under our policy – covered in SOL4602.  We will be fixing this in our next 
major release, 12.0.0, due out later this year.  We’re also planning on pulling the fix back to future Hotfix Rollups 
on our current supported releases.  That work is already underway.

From our testing, overall it appears that the behavior is basically similar to what we do if we get a connection that 
doesn’t send a complete request; we’ll eventually disconnect the client via the TCP profile timeout.  The reason these 
requests aren’t rejected outright is that the profile currently looks for a valid HTTP/1.0 or 1.1 request format – 
basically <method> <uri> HTTP1.[0|1] – but if it isn’t one of those it presumes it must be HTTP/0.9, which was much 
more forgiving with respect to the headers.  (HTTP/2 is experimental in the latest 11.6.0 releases, but that’s a 
different beast entirely.)  So the current profile errs on the side of trying to service the request and falls back to 
0.9; a server which strictly enforces 1.0 or 1.1 is likely to reject them outright.

We try to process this HTTP/0.9 request and look for a response from the server, but things break down when we try to 
proxy the bad request.  The short version is that we send a request to a pool member, but it sees it as HTTP/1.0 and 
since it isn’t a complete 1.0 header it is waiting for more.  We end up sitting in the middle until our timer fires and 
we close the connection or one of the endpoints times out and closes it, whichever comes first.  So this all stems from 
trying to be too accommodating with support for HTTP/0.9 and the fix is to get stricter about what we’ll accept.

The good news is that we’ve conducted volumetric automated testing and we haven’t been able to produce any noticeable 
issue with the BIG-IP; it has absorbed the queries as intended.  In theory a very high number of requests, from a 
larger number of attacking systems, could create a DoS situation, but the behavior appears to be the same as other 
known DoS methods in being related to the TCP profile timeout.  Even with 65,000 simultaneous connections (using all 
available source ports, in other words) from one host it barely moved the needle on the BIG-IPs resource utilization, 
so it would need to be a DDoS type of attack from a large number of attack hosts.  The actual cost per connection to 
the BIG-IP is negligible, it isn’t consuming much in the way of resources.

Overall we do see a few ways we could enhance the BIG-IP’s internal filters to more strictly enforce the request format 
and reject this kind of exploit in general more readily, and we’re looking into what improvements can be made in future 
releases.   Based on our testing, this attack has about the same impact on the BIG-IP as opening the TCP connection and 
sending nothing, until the TCP profile closes the connection.  It isn’t as resource intensive as Slowloris for example, 
though that’s obviously true on the client side as well, making this attack lower cost.  We tested a number of 
variations with different types of malformed headers, looking for any different characteristics.

We do appreciate your bringing this to our attention as it has given us a few ideas on ways we can further improve the 
product.  I’ll be working with our AskF5 team to prepare a Solution article for our site, and we can coordinate the 
publication with your timing as well.

I hope this explanation addresses the concerns you’ve raised.  Do you have any questions I might address?

Thank you.
”

Following this email we had some more emails exchange regarding various possible risk and mitigations about this issue, 
and I was asked to supply the title I wished to be presented in the acknowledgment part of the vulnerability support 
post.

At the 27-May-2015 I received an email from M. that eventually they published the issue at the support site, titled 
“SOL16672: An improperly formatted HTTP request-line may cause connections to hang and eventually 
timeout“,https://support.f5.com/kb/en-us/solutions/public/16000/600/sol16672.html.

According to this KB this issue is affecting versions ranging from version 10.1.0 (from December 2009) until a very 
recent version – 11.6.0 (August 2015) (Click the “Show Version” part on the upper right side of the KB post).

To my surprise the issue was not marked as a vulnerability but as a “Known Issue”, although during my discussions with 
“F5 Networks” they classified it as a “High” grade vulnerability (see above), and no one updated me, before publishing 
the support post – that they decided to demote it from a vulnerability.
Even the “Acknowledgments” section was deviated from the normal text pattern they use for vulnerabilities posts (see 
here an example here - https://support.f5.com/kb/en-us/solutions/public/16000/700/sol16728.html)

I asked M. why, after they admitted it is a “High” grade vulnerability by their own policy – they eventually  played it 
down to a “Known Issue”?

He replied:
”
Eitan,

There was a lot of discussion about this internally, but in the end the developers and management decided this is not a 
vulnerability in BIG-IP, but a defect.  The ID was re-categorized as Defect and is being addressed as such.  Hence 
there will be no CVE requested and the SOL format used was the Known Issue template used for product defects.

Through the testing that was done they were never able to produce a real DoS/DDoS effect on the BIG-IP and they didn’t 
feel that it rose to the level to be classified as a vulnerability.  But they did feel that it was not the proper 
handling of these requests, so it is considered a defect to be resolved in upcoming HF releases.  It was a very 
involved debate, with a lot of arguments made for both sides.  In the end the ‘Defect’ arguments carried the day.

The central point was really trying to rectify the test results, wherein little impact was observed on the BIG-IP, with 
calling it a Vulnerability and then trying to explain the minimal impact in a SOL.  There was concern that labeling it 
as such would cause undue concern for customers in light of the negligible impact observed, as there is something of a 
strong reaction to any Vulnerability SOL independent of the content.  It was agreed however that customers should be 
made aware of the issue in any case, and the Known Issue SOL was expedited for that purpose.

I had initially argued to handle it as a Vulnerability, but in the end I agreed with the consensus to handle it as a 
Defect after all of the arguments had been presented.
”

I publish this post because I guess most people are not aware of this issue as it was not flagged as a vulnerability 
hence folks who collect and react to vulnerabilities report could not possibly know about it.

I hope that with this post more “F5 Networks” customers will be aware of this issue and patch their systems with the 
fixes mentioned at the KB post (or instead use the suggested mitigation steps offered there as well); and that the 
security researchers community will learn about this issue, thus researchers with better tools, knowledge and 
experience than I have – will look deeper into this issue and give this issue a “second opinion”, whether it can be 
exploited as a vulnerability or not.
"

Direct binary based resolution: Download and upgrade to the latest main version of the LTM module, begging from version 
12.0.0 or install a hotfix for matching earlier versions, per the vendor's support article of "sol16672: An improperly 
formatted HTTP request-line may cause connections to hang and eventually timeout" at 
https://support.f5.com/kb/en-us/solutions/public/16000/600/sol16672.html

Workaround (in case you cannot apply the above binary code or do not wish to do so):
1.    Create an iRule (an internal BIG-IP script) to enforce correct HTTP version input format validation 2.    In 
addition it is possible to lower the TCP timeout value Exact details about these workarounds can be found at the 
vendor's support article of "sol16672: An improperly formatted HTTP request-line may cause connections to hang and 
eventually timeout" at https://support.f5.com/kb/en-us/solutions/public/16000/600/sol16672.html

Credit: Eitan Caspi, Israel

Past security advisories:

1. CVE-2002-0049 - Exchange 2000 System Attendant Incorrectly Sets Remote Registry Permissions 
http://www.microsoft.com/technet/security/bulletin/MS02-003.mspx
http://support.microsoft.com/kb/315085/en-us
http://online.securityfocus.com/bid/4053
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2002-0049

2. CVE-2002-1932 - Defined Actions for Administrative Alerts Do Not Occur When the Security Log Is Full
http://support.microsoft.com/?kbid=329350
http://online.securityfocus.com/bid/5972
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2002-1932

3. User downgraded from Administrator to User retains the ability to list other user's running tasks
http://www.securityfocus.com/archive/1/301624
http://online.securityfocus.com/bid/6280

4. "Compaq Web Agent" management session can be re-used without the need to perform authentication
http://online.securityfocus.com/archive/1/309442
http://online.securityfocus.com/bid/6736

5. Windows XP "welcome screen" exposes the names of all the members of the local administrators group
http://www.securityfocus.com/archive/1/314361
http://www.securityfocus.com/bid/7046

6. Symantec Antivirus client locally created scheduled scan is not running if the local console is logged off
http://www.securityfocus.com/archive/1/393800

7. CVE-2006-2612 - Novell Client login form enables reading and writing from and to the clipboard of the logged-in user 
http://www.securityfocus.com/archive/1/archive/1/434704/100/0/
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2006-2612

8. CVE-2006-4886 - McAfee VirusScan Enterprise - disabling the client side "On-Access Scan"
http://www.securityfocus.com/archive/1/archive/1/446220/100/0/
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2006-4886

9. CVE-2007-0833 - VMware workstation guest isolation weaknesses (clipboard transfer) 
http://www.securityfocus.com/archive/1/459140/30/90/
http://www.securityfocus.com/bid/22413
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-0833

10. CVE-2007-1056 - VMware Workstation multiple denial of service and isolation manipulation vulnerabilities 
http://www.securityfocus.com/archive/1/460664/30/60/
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-1056

11. "run as" local denial-of-service enables administrative account processes to be killed 
www.securityfocus.com/archive/1/472216/100/0/

Email: eitancaspi (at) yahoo (dot) com or via the "Contact" form at the following blogs

LinkedIn Profie: https://www.linkedin.com/in/eitancaspi

Information Security blogs:
FUD for thought (English) - http://fudie.net Not Safe/Sure (Hebrew) - http://security.caspi.org.il

Articles: You can find several IT, business and security articles I wrote some time ago at 
http://www.themarker.com/misc/search-results?searchType=textSearch&text=eitan+caspi&simpleSearch=simpleSearch

"Technology is like sex. No hands on - No fun." (Eitan Caspi)


Current thread: