For an incident responder, verbose and plentiful security logs are like an archiaologist finding a new dig site. The problem is all that dirt in the way, and logs are no exception.
If you look in most logs, they are full of extra entries that are on the “I should clean that up/look into why that is always doing that” pile. (–mark– is one of my favorites…but it has value on really quiet devices.)
Stop wondering and fix it. I once worked for a VP that had more IT experience based on sheer number of hours worked than many of us will ever achieve. He had a little saying, “fix one thing a day”, meaning all the little annoyances we put up with on enterprise computer systems. At the end of the year 200 things would be fixed and real problems would be easier to see.
What was fixed today? Vyatta firewall/router and SSH. I think it’s great that Vyatta has a way to disable root login on it’s SSH daemon, why don’t they have a way to choose which interface it listens on?
Who knows, but it’s a promising product for a Cisco replacement, it has commandline and gui, VPN, routing, natting and you can put a pretty cranky stateful rule set in place, completed with rule by rule logging. Not bad fr free, but the world trying to log in to the firewall as root was filling up the logs.
Under the covers Vyatta is Linux so the fix was pretty simple. the sad thing is this Vyatta box is proof once again that lab gear will eventually find it’s way into production and little things will be missed in that slow migration.
Rule one for internet facing systems, if you don’t need it, turn it off … nuff said.
“But I need SSH to manage it …”
From the internet or just inside? For most companies internal lan is the only place it needs to be open. Most Linux services can be told to “listen” on specific interfaces, by default they listen on all. ( Why should microsoft have all the default service glory? 🙂
————- SSH listening on all interfaces —————-
skinner:/var/log# netstat -an | grep 22
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
—————-edit the ssh daemon config file ———–
# Package generated configuration file
# See the sshd(8) manpage for details
# What ports, IPs and protocols we listen for
# Use these options to restrict which interfaces/protocols sshd will bind to
# changed to listen on internal nic only, Feb 24th/2011 DEL
Protocol 2 …..
——————– Save the file and restart ssh. ——————
/etc/init.d/ssh stop; /etc/init.d/ssh start
———– confirm it’s now listening on just one interface ———
skinner:/etc/init.d# netstat -an | grep 22
tcp 0 0 192.168.99.1:22 0.0.0.0:* LISTEN
And the log file is no longer being filled up with the world trying to log in as root.
Feb 23 20:13:04 192.168.99.1 sshd: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=22.214.171.124 user=root
Feb 23 20:13:05 192.168.99.1 sshd: Failed password for root from 126.96.36.199 port 46411 ssh2
Feb 23 20:13:08 192.168.99.1 sshd: Address 188.8.131.52 maps to localhost, but this does not map back to the address – POSSIBLE BREAK-IN ATTEMPT!
Feb 23 20:13:08 192.168.99.1 sshd: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=184.108.40.206 user=root
Feb 23 20:13:10 192.168.99.1 sshd: Failed password for root from 220.127.116.11 port 48692 ssh2
Feb 23 20:13:12 192.168.99.1 sshd: Received signal 15; terminating.
Feb 23 20:13:12 192.168.99.1 sshd: Server listening on 192.168.99.1 port 22.
Feb 23 20:13:13 192.168.99.1 sshd: Address 18.104.22.168 maps to localhost, but this does not map back to the address – POSSIBLE BREAK-IN ATTEMPT!
Feb 23 20:13:13 192.168.99.1 sshd: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=22.214.171.124 user=root
Feb 23 20:13:15 192.168.99.1 sshd: Failed password for root from 126.96.36.199 port 51282 ssh2
So those very estute readers should now be wondering why I didn’t write a firewall rule to block all external addresses accessing SSH? Two reasons, this was a quick and dirty lab setup that go moved, and SSH was needed on the “outside” originally, just never got cleaned up. The second reason is this device is now production, and mistakes happen. No need to worry about a firewall rule bloking SSH getting turned off if the service isn’t even on.