Digital forensics

The Case of the Great Router Robbery

NEWSFLASH: AnyTown Local News reports this Monday morning that the recent spate of office break-ins has continued with a weekend raid on the downtown branch office of HugeMegaCorp. In a statement, HugeMegaCorp said that "when staff arrived at the office on Monday morning, two laptops and a router were discovered stolen after a break-in; the laptops were encrypted, so no data was lost".

NEWSFLASH: BigBusinessNews was informed on Monday afternoon that HugeMegaCorp has suffered a massive data breach. Despite repeated requests, the company has yet to comment.

Learn Digital Forensics

Build your skills with hands-on forensics training for computers, mobile devices, networks and more.

Start Learning

Learn Digital Forensics

Build your skills with hands-on forensics training for computers, mobile devices, networks and more.

Start Learning

Theft of IT equipment is an obvious risk, especially where portable equipment is concerned or where high levels of physical security are not available. Often, such thefts are carried out for nothing more than the resale value of the device itself. Sometimes however, the raid may merely be a means to an end, part of a much grander plan.

In the hypothetical case of HugeMegaCorp, the data breach was not related to the loss of the encrypted laptops but to the loss of the branch office's router. Whereas a home or SOHO router is unlikely to have much more value than what can be had on eBay, it's a very different story when a corporate router is concerned.

This article will look at what information can be gleaned from a stolen Cisco router, the mechanism available to reduce your exposure in these circumstances, and an evaluation of its effectiveness.

Breaking In

Let's start by asking why we are interested in stealing a router in the first place. These days, large enterprises are generally very well connected regardless of how geographically dispersed they are. Site-to-site IPSec VPNs are common, allowing branch offices to seamlessly and securely access resources back at HQ via an encrypted tunnel between the branch office's router and one at HQ:

When a VPN like this is in play the branch office network is an extension of the HQ network, providing access to HQ resources without having to apply any special configuration or software to the branch office computers. Just plug them all into the branch office router, and you're all set.

The HQ router (hopefully!) isn't going to let just any router connect to it and build a VPN – access control will be tight. The branch office router is therefore physically in possession of the "keys to the kingdom" – or at least some of them – making it a high-value target:

If we can steal or clone a branch router perhaps we can convince HQ to let us build a VPN link, allowing us to roam around the HQ network from a distance.
If we're not going to actually use the router to attack like this, it still has great reconnaissance value. We can learn a lot about HugeMegaCorp by looking at the router's config and use this knowledge later on.
If we can bribe the branch office cleaners to steal the router on a Friday night, there's an excellent chance it won't be noticed missing until Monday morning…

Breaking In, again

After we've broken in to the branch office of HugeMegaCorp we need to break in to the router. The old adage that "if you can touch it, you can probably 0wn it" is sometimes true for Cisco kit – every CCNA knows how to perform password recovery given physical access to a router and a console cable.

The term "password recovery" is something of a misnomer – you can't always recover lost passwords in the sense that you get them back. Instead, it could be better called "recovery from a lost password situation" in the sense that you end up with full access to the router and the ability to modify its configuration (to reset or remove passwords, for example).

If we're going to attempt password recovery, we're best doing it with the router totally disconnected from the network. We don't want it accidentally phoning home to HQ and betraying our activities.

Breaking In, yet again

OK, so we've stolen a router from a branch office, we've cracked it open with password recovery, and we're sitting at an enable prompt in our terminal emulator. After typing "show running-config", we can start to look for tasty bits and bobs. Clearly no two networks are alike, but here are a few interesting things to look out for (this is not intended to be an exhaustive list!):

Device access policing

Looking at how an organisation controls administrative access to their network devices tells you a lot about their security posture and general management diligence. There are several levels of authentication for things like telnet, SSH and HTTP device access, including no authentication(!), local authentication, and remote authentication via RADIUS or TACACS+.

Telnet and SSH connections are handled by virtual terminal interfaces called vtys. Let's look at a few examples from the cracked config of our stolen router:

! No authentication; only telnet supported

line vty 0 4

no login

transport input telnet

! How serious is the victim org about security if they're using telnet?

! Password-only authentication; ssh and telnet supported

line vty 0 4

password verySecure

transport input ssh telnet

! Local username/password authentication
username admin password adminadmin

line vty 0 4

! Remote authentication via RADIUS; use local username/password

! if RADIUS unavailable. This is a reasonably well implemented

! access policy

radius-server host 10.9.8.7 auth-port 1645 acct-port 1646 key mySecret

aaa new-model

aaa group server radius myRadiusServers

server 10.9.8.7

auth-port 1645 acct-port 1646

aaa authentication login default group myRadiusServers local

username emergency password keepmesecret

line vty 0 4

transport input ssh

In the last example above, a remote authentication server (10.9.8.7, probably residing at HQ and reachable over the VPN) is used to police SSH access. The bolded line is interesting – it says that the default method for checking credentials is RADIUS, but if the RADIUS server is unavailable use a local account instead. I wonder if the password for the "emergency" account is the same for all of HugeMegaCorp's routers? If it is, all we need to do to log in is disconnect the router's internet connection (thus taking out the ability to contact the RADIUS server) and log in as "emergency". That might come in handy if we break into other branch offices…

If HTTP access to the router is in place, you might see lines like these:

! Use "enable" password and no username for HTTP/S login
ip http server

ip http secure-server

! Local username/password authentication
username admin password adminadmin

ip http server

ip http authentication local

! Use RADIUS to police authentication

! aaa/RADIUS config is omitted as it is the same as for the SSH example above

ip http server

ip http authentication aaa

The above access policies can also be applied to the router's console interface and the auxiliary interface if one is present. One would hope that the authentication schemes in use for different access methods are applied consistently - it's no good having decent authentication set up on the vty interfaces if there's none in place for HTTP, for example.

SNMP access is policed by a set of community strings, some read-only, some read-write:

snmp-server community mySecretString RO

snmp-server community myVeryVerySecretString RW

Do you think the community strings are the same for all of HugeMegaCorp's routers…?

Finally, there's one other nugget of useful information our router can give us here – the location of some of HugeMegaCorp's IT staff. In-band access (via Telnet/SSH/HTTP/HTTPS/SNMP) can be policed by source IP address. By knowing permitted IP addresses, we know where the people responsible for maintaining the router live – if we can penetrate HugeMegaCorp at a later date, these machines might be good targets as they are likely to have privileged access to other systems:

! Define permitted source IP addresses

access-list 1 permit 10.9.8.0 0.0.0.255

! Permit HTTP access only from 10.9.8.0/24

ip http access-class 1

line vty 0 4

! Permit telnet access only from 10.9.8.0/24

access-class 1

! Permit SNMP access only from 10.9.8.0/24

snmp-server community mySecretString RO 1

Are we being watched?

A Cisco router can be a very verbose device if configured to be that way – syslog output, netflow exports and SNMP traps can all be sent to various collectors. If we're going to attempt an attack on HugeMegaCorp later on, it's helpful to know if we're being watched:

! Netflow exports sent to a remote machine

ip flow-export destination 10.9.8.123 2055

! Syslog messages sent to a remote machine

logging trap debugging

logging 10.9.8.124

! SNMP traps sent to a remote machine

snmp-server host

10.9.8.125 traps myOtherSecretString

Wireless

For a small branch office, the router may pull double duty as a wireless access point. Looking at the wireless config may open other doors for us:

! WPA with pre-shared key

! What's the betting each office's PSK is the same to allow

! roaming executives to connect everywhere they go?

dot11 ssid HugeMegaCorp

authentication open

authentication key-management wpa

wpa-psk ascii mySecretPSK

! EAP authentication via remote RADIUS server

! Knowing the RADIUS server's secret can be very useful!

radius-server host 10.9.8.7 auth-port 1645 acct-port 1646 key mySecret

aaa new-model

aaa group server radius myRadiusServers

server 10.9.8.7

auth-port 1645 acct-port 1646

aaa authentication login eap_wifi group myRadiusServers

dot11 ssid HugeMegaCorp

authentication open eap eap_wifi

authentication key-management wpa

! Broadcast SSID

dot11 ssid HugeMegaCorp

! If "guest-mode" is missing, the SSID is not broadcast

guest-mode

VPN config

If our branch office router does indeed build a VPN connection to HQ, working out how it does it may help us to build our own VPN connections. The VPN config on a Cisco can be very complicated indeed, but things to look out for include:

! This router uses a pre-shared key to build the VPN
crypto isakmp key HugeMegaCorpPSK address 10.20.30.40 255.255.255.255 no-xauth

! The HQ router may have a different key for each branch office…

! …or maybe it won't!

! The isakmp policy below will use this pre-shared key

crypto isakmp policy 10

encr 3des

authentication pre-share
group 2

! This is a crypto trustpoint

! It defines the location of a certificate authority (CA) plus

! the information to put into a certificate signing request.

crypto pki trustpoint HugeMegaCorp

enrollment url http://10.9.8.130:80

serial-number

fqdn brance-office1.huge.mega.corp

ip-address none

subject-name O=Huge Mega Corp, OU=Network Security, CN=branch-office1.huge.mega.corp, C=US, ST=CA, E=pki@huge.mega.corp

! This isakmp policy will use a certificate for authentication

! Certificate auth is the default, so there's no "authentication" line

! Certificates are the strongest and most scalable means of authenticating

! VPN connections.

crypto isakmp policy 10

encr aes 256

group 2

If the router is using certificates as above, you're unlikely to be able to extract them for use elsewhere. The certificates themselves are portable, but the private keys that go along with them are held in a part of the router's nvram: filesystem that is inaccessible to users.

Defending the treasure trove

As we've seen, a router's config can tell us a lot about an organisation; maximally, the information disclosed may give us the knowledge needed to actually break in to HugeMegaCorp via SSH, Wifi or IPSec. Other portable devices like laptops can be encrypted to decrease the impact of loss due to theft; what's the equivalent for a router?

The bulk of the value derived from our stolen router was a by-product of the password recovery process. Fortunately, this feature can be disabled with the "no service password-recovery" command:

rtr# configure terminal

rtr(config)# no service password-recovery

WARNING:

Executing this command will disable the password recovery mechanism.

Do not execute this command without another plan for password recovery.

Are you sure you want to continue? [yes/no]: yes

Attempting password recovery on a router so protected will result, some say, in the startup-config being erased; others say the entire nvram: filesystem is wiped. Neither is quite accurate – whilst the protection seems only to apply to the nvram: filesystem, the truth is somewhere in between these extremes.

Let's take a look at the nvram: on a router with password recovery disabled, prior to a password recovery attempt:

rtr# dir nvram:

Directory of nvram:/

109 -rw- 15457 <no date> startup-config

110 ---- 2754 <no date> private-config

111 -rw- 15457 <no date> underlying-config

1 ---- 62 <no date> persistent-data

2 -rw- 695 <no date> ifIndex-table

3 -rw- 595 <no date> IOS-Self-Sig#1.cer

4 -rw- 595 <no date> IOS-Self-Sig#2.cer

5 -rw- 906 <no date> my-ca#7F.cer

6 -rw- 839 <no date> my-ca#1CA.cer

7 -rw- 984 <no date> my-ca#0.cer

8 -rw- 2945 <no date> cwmp_inventory

131072 bytes total (102569 bytes free)

Interesting stuff here obviously includes the startup-config, but also:

private-config – this one has an intriguing name. I wonder what's in it?

rtr# more nvram:private-config

%Error opening nvram:private-config (Permission denied)

Curiouser and curiouser. It turns out that this is probably the most sensitive file on the box – it's where IOS stores RSA private keys:

rtr# show crypto key storage

Default keypair storage device has not been set

Keys will be stored in NVRAM private config
Files ending in .cer. These are certificates, either those of a trusted Certificate Authority, or those that have been issued to a device itself. They're largely useless without the accompanying private keys, but they may contain interesting information nonetheless.

OK; let's reload the router and try password recovery. As described in the Cisco documentation the startup-config is erased – what about private-config and the certificates?

rtr# dir nvram:

Directory of nvram:/

126 -rw- 0 <no date> startup-config

127 ---- 0 <no date> private-config

128 -rw- 0 <no date> underlying-config

1 ---- 62 <no date> persistent-data

2 -rw- 695 <no date> ifIndex-table

3 -rw- 595 <no date> IOS-Self-Sig#1.cer

4 -rw- 595 <no date> IOS-Self-Sig#2.cer

5 -rw- 906 <no date> my-ca#7F.cer

6 -rw- 839 <no date> my-ca#1CA.cer

7 -rw- 984 <no date> my-ca#0.cer

8 -rw- 2945 <no date> cwmp_inventory

131072 bytes total (120780 bytes free)

startup-config is zero bytes long – that means that most of the information this article has highlighted is not available. Also private-config is zero bytes long – it looks like our RSA keys are toast, too:

rtr# show crypto key mypubkey rsa

rtr#

However, the certificates themselves are still present. Let's get one and take a look:

rtr# copy nvram:my-ca#7F.cer tftp://10.11.12.13/my-ca#7F.cer

Having extracted the file, let's see what's in it:

C:>openssl x509 -inform der -in c:my-ca#7F.cer -text -noout

Certificate:

Data:

Version: 3 (0x2)

Serial Number: 1 (0x1)

Signature Algorithm: md5WithRSAEncryption

Issuer: CN=HugeMegaCorp-CA

Validity

Not Before: Mar 1 00:21:09 2002 GMT

Not After : Jan 1 00:00:00 2020 GMT

Subject: CN=branch-office1.huge.mega.corp

Subject Public Key Info:

Public Key Algorithm: rsaEncryption

Public-Key: (1024 bit)

Modulus:

00:ba:2e:a7:12:a8:59:de:3d:31:b5:32:bd:5d:87:

ee:3d:00:9e:c7:3e:01:c2:c7:b1:be:0e:40:f3:73:

<snip>

40:d7:c2:22:9c:47:f5:7c:33

Exponent: 65537 (0x10001)

X509v3 extensions:

X509v3 Key Usage: critical

Digital Signature, Key Encipherment

X509v3 Authority Key Identifier:

keyid:F1:B3:16:<snip>

X509v3 Subject Key Identifier:

9B:53:90:<snip>

Signature Algorithm: md5WithRSAEncryption

a1:1f:93:5a:03:d4:9b:08:47:e4:e8:73:02:f6:ea:39:ee:c1:

48:6b:5c:cc:0a:24:e2:12:16:3e:aa:d2:db:02:75:80:f8:f7:

<snip>

1f:ce

It's completely useless as a certificate without the accompanying private key (which was in the wiped nvram:private-config file), but we can at least still see the name of the Certificate Authority that issued the cert (so we can tell if they are using a private CA or a public one like Verisign) and the name of the device that the cert was issued to. It's a small haul, but better than nothing.

An important point…

…is that only the nvram: filesystem seems to be touched when password recovery is disabled. All other filesystems (including flash: ) are unscathed. It's probably worth taking a look around, especially if the router was using the configuration archive feature in recent IOS versions:

Defending against the first Break In

The first break in was the physical theft of the router. A branch office probably won't have the level of physical security lavished on HQ, but simple steps like keeping the router in a locked room or cabinet might help.

Defending against the second Break In

As discussed, "no service password-recovery" is an effective way of stopping someone from gaining access to a stolen router. Make certain, however, that access to the device was well secured – disabling password recovery is pointless if you've not got any authentication on the console port.

Learn Digital Forensics

Build your skills with hands-on forensics training for computers, mobile devices, networks and more.

Start Learning

Learn Digital Forensics

Build your skills with hands-on forensics training for computers, mobile devices, networks and more.

Start Learning

Defending against the third Break In

If a properly secured device with password recovery disabled goes walkabout, you still need to take urgent action – the attackers may attempt to plug the router into an Internet connection in the hope that a VPN connection comes up. The true value of the router needs to be assessed in terms of what it contains and what it is configured to do, and a suitably tailored policy needs to be in place to cater for loss:

What to do if a router is stolen

Hope that someone notifies you in a timely fashion that a theft has taken place. Finding out three days after the event is three days that the attackers have had to play around. Make sure that people realise that notifying HQ of a theft is at least as important as notifying the police. Make sure they have the contact details of the right person at HQ to call.
If the router has a certificate, consult your documentation to determine the cert's serial number and revoke it at the CA. You do have a list like this, right?
Hope that password recovery was disabled. If not, you'll need to determine everything that all your branch routers have in common (local user accounts, SNMP community strings, RADIUS secrets, wireless keys, static ISAKMP keys, etc.) and change them everywhere.
Even if password recovery is disabled, hope that there are no copies of the config on flash: or other filesystems.
Even if password recovery is disabled, if you have a VPN that uses the same pre-shared key for all remote devices you need to change the key on all of them as soon as possible. Without breaking the VPN. And without trashing your SSH connection whilst you're at it. If you don't do this, the attackers could use the stolen router to connect to HQ.
Look very carefully for signs that the router has been used between now and the last time it was physically accounted for. Consult netflow (looking for activity to and from the IP subnets at the remote site) and your central syslog collector looking for any crypto logs indicating that the stolen router has brought up a VPN connection (having "crypto logging session" turned on at the HQ router is a real boon here).

Posted: May 24, 2011

Alec Waters

View Profile

Alec Waters is a security researcher for InfoSec Institute and a network security specialist working for Dataline Software in the UK. Working with defense and healthcare clients, his focus is on incident response, network forensics and secure infrastructure design.