Security Metrics - Fail!

PCI testing stupidity - false positives and the battle to get them reversed.

Safe and Sorry?

So you want to take payments on your website?
But you don’t want to hand over payment collection to PayPal or Stripe?
You really want the responsibility of handling people’s credit card details?

I’d strongly advise against it, but if you really must, read on…

PCI Compliance

All people handling card payments mus be PCI compliant. The full term is Payment Card Industry Data Security Standard (PCI DSS). PCI DSS is an information security standard for organizations that handle branded credit cards. There’s more about it here on WikiPedia and I’m sure a few searches will turn up some useful information should you be inclined to find out more.

I’m pretty sure that I have made my point of view regarding taking credit card details over the web clear many times before. Simply, don’t! Let somebody else do it. Let them take the responsibility for the security. If you haven’t had the credit card number in the first place, you can’t lose it or leak it. PayPal, Stripe, Sage et al may charge what seems to be an extortionate amount, but that pales into nothing compared to the potential liabilities should a ne’er-do-well breach your defences and make away with your customers’ credit card details.

Just because you engage a payment processor, that doesn’t mean you have no responsibilities under PCI DSS, just that the great majority of the most onerous and frightening areas of compliance are taken care of for you.

PCI Compliant Hosting

Supposing you are doing e-commerce, a good place to start might be to get yourself some PCI Compliant hosting. Should you be in the market for some, I can highly recommend these people Guru. (Full disclosure: that’s an affiliate link and should you buy from them following a click on that link, I benefit. Thank you)

Trust. What’s That?

If you go to your payment processor and tell them you are taking credit card details over the web and that you are using a PCI compliant host, they are not going to take your (or the host’s) word for it. They are going to insist that your website is tested and certified by some sort of accredited independent testing outfit.

Security Metrics is one such organisation. There are more available as your search engine of choice will tell you and I’m sure that they are all wonderful and that Security Metrics in particular represents the very pinnacle of Best Industry Practice™.

Automated Testing - What Can Possibly Go Wrong?

The problem is that the testing ticks all the right boxes but not necessarily in the right way.

Examples. Warning: Technical Content Ahead

Security Metrics claims: SMTP Service Cleartext Login Permitted.
This is rubbish - Guru’s SMTP server works over port 465 which is encrypted by default. The SMTP server will not accept unencrypted connections. If this test were at all competently devised and operated then this would not flag up.

Security Metrics response: Result set to false positive.
While technically “failing” this test, Security Metrics accepts that this is a false positive and allows the site through. Personally, I would have thought that it would be a good idea to fix the test so it no longer reported a fail where there is none.


Security Metrics claims: Banner Based Vulnerabilities for OpenSSH 7.4, CVE-2017-15906
On the face of it, this is plausible. Except, in the real world sometimes fixes are back-ported to older versions of software. That is there may be good reasons not to update to the latest version, but where a security vulnerability exists that may be fixed without updating the version number reported by the software. In this case, the OpenSSH server has been fixed. Unfortunately this test seems to be using the brain dead approach of equating reported version numbers with vulnerability.

(CAR ANALOGY: Suppose there’s a problem with 2014 Volvo S60 models. Volvo recalls all affected cars and fixes the problem. It is no longer valid to say that all 2014 S60s have that fault.)

Security Metrics response: None
Though reported and logged through the ticketing system with a complete (and correct) analysis of the situation, no response is forthcoming, neither rebuttal nor admission.


In the End

All this shenanigans takes time. Time to investigate. Time to report. Time spent on hold.

Before you decide to process credit card payments manually, please think twice. For many things in life, there is a price. If you don’t pay it in one place you’ll have to pay it somewhere else instead.

Take Care™