Why remote logging?
When a good hacker enters your network he can also erase all traces he leaves in your logs. By forwarding all messages sent to rsyslog to a remote host over network, you can prevent the bastard to clean his tracks.
As I was unable to find a good up-to-date technical howto on this topic, I decided to write my own to share with you.
Background
About RELP
By default and on historical grounds syslog messages were sent as UDP fragments over the network. Despite also TCP is possible and much more reliable, it will not cover network hickups completely. See the blog article "On the (un)reliability of plain tcp syslog..." for a very good explanation.
RELP is designed to overcome these shortcomings of TCP and UDP in application layer. Whenever possible, it is advised to use RELP for reliability.
other syslog daemons
Please check with your Linux distribution what kind of syslog daemon it is shipped with. Recent Debian runs rsyslog, earlier versions than Lenny were shipped with sysklogd. This article applies to the use of rsyslog only, both server and client. Please check with your Linux distribution on how to move to rsyslog before proceeding here.
Configure the server
First, make sure you have installed the RELP extensions for rsyslog.
For Debian and -derivatives you'll need to install the package rsyslog-relp.
That package includes support for RELP for both input and output.
Add the following two lines to the configuration of your rsyslog configuration.
Preferably, add it as a separate file in /etc/rsyslog.d:
$ModLoad imrelp # Load the input module ('im') 'relp'
$InputRELPServerRun 20514 # Set the port to 20514
This will enable rsyslog to listen on TCP port 20514 for RELP syslog messages. Change the port to whatever you want it to listen on.
Don't forget to reload rsyslog to activate the change: /etc/init.d/rsyslog restart or service rsyslog restart for Ubuntu and confirm it is running on the specified port with for example netstat -ntlp | grep rsyslog.
Now you're all set for the server.
Configure the client
First, install the same RELP support as on the server.
Add the following to the rsyslog configuration (like we did on the server) to forward all messages to the server:
*.* :omrelp:loghost.example.com:20514;RSYSLOG_ForwardFormat
Explanation from left to right: match all, output module 'relp', to the host loghost.example.com on port 20514.
The template RSYSLOG_ForwardFormat is used to maintain some high-precision timestamp.
See also the statement on the omrelp module documentation:
Rsyslog's high-precision timestamp format is used, thus the special "RSYSLOG_ForwardFormat" (case sensitive!) template is used.
Reload rsyslog—you will now see the messages from the client appearing in the log files on the server.
Disable local logging
While messages are forwarded over the network, it is still logged in local files. I should cover this in a later post.
Other valid use cases
Apart from security you can think of other reasons to enable remote logging:
- Embedded devices with insufficient storage (e.g. network devices). However, most embedded devices will not support RELP. Configuring rsyslog to listen also for regular syslog messages on the network is not covered in this article.
- When you have lots of logging from lots of different virtual machines it is usually a lot of random I/O on the backend storage. With remote logging that becomes sequential, more efficient writes.