When a good hacker enters your network he can also erase all traces he leaves in your logs. By forwarding all messages sent to rsyslog to a remote host over network, you can prevent the bastard to clean his tracks.
As I was unable to find a good up-to-date technical howto on this topic, I decided to write my own to share with you.
By default and on historical grounds syslog messages were sent as UDP fragments over the network. Despite also TCP is possible and much more reliable, it will not cover network hickups completely. See the blog article "On the (un)reliability of plain tcp syslog..." for a very good explanation.
RELP is designed to overcome these shortcomings of TCP and UDP in application layer. Whenever possible, it is advised to use RELP for reliability.
Please check with your Linux distribution what kind of syslog daemon it is shipped with. Recent Debian runs rsyslog, earlier versions than Lenny were shipped with sysklogd. This article applies to the use of rsyslog only, both server and client. Please check with your Linux distribution on how to move to rsyslog before proceeding here.
First, make sure you have installed the RELP extensions for rsyslog.
For Debian and -derivatives you'll need to install the package
That package includes support for RELP for both input and output.
Add the following two lines to the configuration of your rsyslog configuration.
Preferably, add it as a separate file in
$ModLoad imrelp # Load the input module ('im') 'relp' $InputRELPServerRun 20514 # Set the port to 20514
This will enable rsyslog to listen on TCP port 20514 for RELP syslog messages. Change the port to whatever you want it to listen on.
Don't forget to reload rsyslog to activate the change:
/etc/init.d/rsyslog restart or
service rsyslog restart for Ubuntu and confirm it is running on the specified port with for example
netstat -ntlp | grep rsyslog.
Now you're all set for the server.
First, install the same RELP support as on the server.
Add the following to the rsyslog configuration (like we did on the server) to forward all messages to the server:
Explanation from left to right: match all, output module 'relp', to the host
loghost.example.com on port 20514.
RSYSLOG_ForwardFormat is used to maintain some high-precision timestamp.
See also the statement on the omrelp module documentation:
Rsyslog's high-precision timestamp format is used, thus the special "RSYSLOG_ForwardFormat" (case sensitive!) template is used.
Reload rsyslog—you will now see the messages from the client appearing in the log files on the server.
While messages are forwarded over the network, it is still logged in local files. I should cover this in a later post.
Apart from security you can think of other reasons to enable remote logging:
- Embedded devices with insufficient storage (e.g. network devices). However, most embedded devices will not support RELP. Configuring rsyslog to listen also for regular syslog messages on the network is not covered in this article.
- When you have lots of logging from lots of different virtual machines it is usually a lot of random I/O on the backend storage. With remote logging that becomes sequential, more efficient writes.