CentOS Question posted in
The official documentation can be found here.

Rsyslog – logs not displayed on server – CentOS

Trying to configure Rsyslog Client to Send Logs to Rsyslog Server.

Both machines run on Centos7 with Vagrant.

See configuration of each machine below.

When I log inside the Client machine – It is not reflected on the server’s logs.

For example:

logger "Some message..."

But when I add the server’s IP to the command:

logger -n 192.168.11.11 "Some message..."

I can see that a directory with the Client’s IP is being created on the Server machine:

[root@server log]# ls -la
total 200
drwxr-xr-x. 11 root   root     4096 Aug  1 12:02 .
drwxr-xr-x. 18 root   root      254 Jul 31 21:39 ..
drwx------.  2 root   root       25 Jul 31 22:18 192.168.11.22 <--- HERE
drwxr-xr-x.  2 root   root      191 Feb 28 20:54 anaconda
drwx------.  2 root   root       23 Jul 31 21:39 audit
-rw-------.  1 root   utmp        0 Aug  1 03:14 btmp
drwxr-xr-x.  2 chrony chrony      6 Apr 12  2018 chrony
-rw-------.  1 root   root      188 Jul 31 21:39 cron
-rw-r--r--.  1 root   root    26911 Aug  1 12:02 dmesg
-rw-r--r--.  1 root   root    26742 Jul 31 21:39 dmesg.old
-rw-r--r--.  1 root   root      374 Jul 31 21:47 firewalld
-rw-r--r--.  1 root   root   292292 Aug  1 12:12 lastlog
-rw-------.  1 root   root      198 Jul 31 21:39 maillog
.......
.......

And inside of it there is a vagrant.log file where I can see the log message for the Client:

2019-08-01T13:00:06+00:00 192.168.11.22 vagrant: Some message...

Question: Any Idea why I can’t see the Client local logs?

For this discussion:

  1. SELINUX is disabled.
  2. This didn’t helped.

Rsyslog Configuration

Server IP: 192.168.11.11.
Client IP: 192.168.11.22.

Both machines have rsyslog installed & enabled and have the following common setup in the /etc/rsyslog.conf file:

#### MODULES ####    
$ModLoad imuxsock 
$ModLoad imjournal

#Open port 514 For UDP    
$ModLoad imudp
$UDPServerRun 514

#### GLOBAL DIRECTIVES ####
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
$OmitLocalLogging on
$IMJournalStateFile imjournal.state

#### RULES ####
kern.*                                       /dev/console    
*.info;mail.none;authpriv.none;cron.none     /var/log/messages
authpriv.*                                   /var/log/secure    
local7.*                                     /var/log/boot.log

Additions for each machine on the /etc/rsyslog.conf file.

On the Server machine:

## Rules for processing remote logs 
 $template RemoteLogs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log" 
 *.* ?RemoteLogs 
 & ~

On the Client machine I Setup UDP Forwarding to the Server’s IP:

# ### begin forwarding rule ###
#UDP Forwarding
*. * @192.168.11.11:514 
# ### end of the forwarding rule ###

Firewall Configuration

On the Server machine – Opening port 514 on firewall:

sudo firewall-cmd --permanent --add-port=514/udp
sudo firewall-cmd --reload

And Verifying:

[root@server /]#  sudo ss -tulnp | grep "rsyslog"

udp    UNCONN     0   0     *:514  *:*   users:(("rsyslogd",pid=2711,fd=3))
udp    UNCONN     0   0    :::514  :::*  users:(("rsyslogd",pid=2711,fd=4))
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Credit to @Alexey that helped me in two critical points.

    I'll also add a few more useful points that might be helpful for others:

    1: First of all, check the connectivity of the two machines. For example, if using vagrant networking, place the two machines in a network mode that they can see (ping) each other. For example Host-only mode is OK, but internal mode where each machine is on its isolated network - will not work.

    2: Uncommenting the following code is required only in the syslog server:

    $ModLoad imudp
$UDPServerRun 514

    3: On the client machine - you can comment the all #### Rules #### section and place the forwarding rule at the end of file OR you can specify the forwarding inline (with the client IP or hostname, if configured in /etc/hosts):

    #### RULES ####
kern.*                                       /dev/console    
*.info;mail.none;authpriv.none;cron.none     @192.168.11.11:514 # <--- Like this
auth,authpriv.*                              /var/log/secure    
local7.*                                     /var/log/boot.log

    4: A quick check to see if the UDP connection succeed:

    Forward auth logs to server (like this auth,authpriv.* @192.168.11.11:514).

    ssh into the client machine, switch between users and view the following files under /var/log/ directory on server:

    sshd.log  su.log  sudo.log  systemd-logind.log

    5: Be careful with the rsyslog syntax - For example, If you're using sed as part of a provision script in order to replace and edit lines in the config file - check the output:

    Specifying the $template section in one line:

    $template RemoteLogs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log" *.* ?RemoteLogs

    Will lead to an error:

    Aug 03 13:50:04 server rsyslogd[5539]: error: extra characters in config line ignored: '. ?RemoteLogs' [v8.24.0-34.el7]

    This is the correct syntax - break rule into a new line (if generated with sed - just add /n):

    $template RemoteLogs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log" 
*.* ?RemoteLogs

    6: Don't forget the systemctl restart rsyslog after each change in /etc/rsyslog.conf. Because of cases like #5 always check the rsyslog status: systemctl status rsyslog.

    7: Newer versions of rsyslog (I think 5+) support a much cleaner syntax - it comes now as default on newer versions of Ubuntu for example, but for some reason not on Centos7.

    8: On the syslog-server - Make sure your firewall is enabled and configured to accept UDP on relevant port (see configuration script in Question):

    Chain IN_public_allow (1 references)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh ctstate NEW
ACCEPT     udp  --  anywhere             anywhere             udp dpt:syslog ctstate NEW

  2. 0

    Try to remove one blank after *. on the client’s host:
    *. * @192.168.11.11:514

    Default format for sending logs having any level on remote host looks like:
    *.* @remote-host:514

    Then restart rsyslog service on client and server with systemctl restart rsyslog and check again.

    EDIT:

    Pasted From the comments – 2 more steps that where done on the syslog server that solved the issue:

    1. Commenting the #### Rules #### section on server.

    2. The Removal of & ~ from the ## Rules for processing remote logs section.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search