Arch Linux

I second this switch.

FYI - rsyslog is up-to-date and in [community] http://aur.archlinux.org/packages.php?ID=20186

Has anybody extensively tested this on Arch?
What are the cons?

The only con that i can find just by browsing the web and reading about rsyslog, mostly vs. the other logging systems is that someone has to implement it. :)

when I tried to install it a year or two ago, there was no package. otherwise I'd be using it now.

Might this fix  FS#9061  ?

I've been using rsyslog since July without any issues. I haven't played with any advanced configuration yet, just left the standard log file in place that came with syslogd during installation.

Wasn't this move affected in a bad way by the very recent upgrade to syslog ng 3.0.x ?
http://www.archlinux.org/pipermail/arch-dev-public/2009-March/010532.html

Apparently the config syntax was changed, which means that rsyslog is no longer a drop in replacement or what?

well i bumped it to new major version, i don't think this will affect this feature request.

I totally overlooked something. The comment in the original report mentions syslog and not syslog-ng :
"rsyslog is a drop in replacement for former syslog (If you have a custom syslog.conf, you can copy it to /etc/rsyslog.conf)"
So this advantage doesn't really apply for Arch. So bumping syslog-ng probably did not harm in any way, sorry.

is this feature request still valid for syslog-ng? Or this task can be closed?

fedora preferred rsyslog over syslog-ng for several reasons :
http://fedoraproject.org/wiki/Releases/FeatureRsyslog#Why_Not_Syslog-ng.3F

If you think all these reasons are invalid, just close.

(I have no opinion on this, I don't know how the logging system works and never configured it, I just look at /var/log/ :))

This looks like another one that needs a decision made by Aaron or someone who feels the features of rsyslog warrant a switch.

I'm in the camp of currently happy with syslog-ng, but I don't object to switching if others need features of rsyslog.

Aaron?

Looking at this some more, it seems to me like there are good reasons to switch to rsyslog, but no pressing *need* to do so.

Who would actually reap benefits of switching if we switch? What features/benefits would those be for you? Please write in and share.

If there aren't some answers here to these questions by February bug day, I recommend we close as "won't implement".

For me, the biggest thing is that while both syslog-ng and rsyslog are fairly feature compatible, the syntax of syslog-ng is quite different from standard syslogd syntax whereas rsyslog is a drop-in replacement. Anyone experienced with a unix prior to coming over to linux will be familiar with syslogd syntax. Same goes for people with experience with enterprise linux experience (with the exception of newer SLES, though they're switching away from syslog-ng to rsyslog too now if they haven't already). So, let's use rsyslog instead of syslog-ng as a benefit to so many people who already know the older syntax but also give them the option of using advanced logging features. Additionally, most Linux certifications (if not all that I've seen) require knowledge of syslogd syntax and _not_ syslog-ng syntax for what it's worth.

I'm generally in favor of switching, based on the arguments here and the ones offered in the Fedora thread(s).

If there are no objections, I'll take rsyslog into [extra] next month, once I've had a chance to finish tying up the loose ends from having brought the new dcron into production.

I'll tackle making it the default and bringing it into [core] as a second step in the following month.

From a quick read, it looks like the advantages of rsyslog *over syslog-ng* are (1) some licensing advantages, and (2) syntax will be more familiar to users coming to Arch from other distros that don't use syslog-ng.

Maybe on balance this warrants a switch, but the case doesn't look that compelling to me. (2) is counterbalanced by (3) *existing* Arch users who've modified their syslog-ng.conf will now themselves have to rewrite them to a new syntax. (Predictably, I'm one such.)

The Fedora thread also says https://www.redhat.com/archives/fedora-devel-list/2007-June/msg02401.html.

Thanks, Jim, for the counterpoints here. Well timed to factor into an informal vote currently in progress on arch-dev-public.

@Profjim
I don't think that local customizations to a config warrant keeping us from evolving for the better. If I make local customizations to a config file on my arch boxes, I certainly expect to have to convert those customizations into a different syntax at some time in the future, it's just the nature of *nix administration. I teach Linux administration courses covering the mainline server distros (RHEL/SLES/Ubuntu Server) and after talking with literally hundreds of administrators from companies small and massive with diverse backgrounds in UNIX/Linux/BSD the overwhelming consensus that I've received when teaching logging is that A) syslog-ng syntax is ugly in comparison to (r)syslog, B) the majority of them are converting from syslog-->rsyslog _or_ syslog-ng-->rsyslog.
I see two likely solutions, announce the switch and make it seemless for those who don't have local customizations while informing others that they do need to cope with a syntax change, or else just do nothing for existing installations and make rsyslog the default logging daemon for new installations (while still keeping syslog-ng around for those who want it).
Just my 2 cents.

> announce the switch and make it seemless for those who don't have local
> customizations while informing others that they do need to cope with a
> syntax change

Sure, or they can stick with syslog-ng even if the Arch default moves to rsyslog. I fully agree that users who customize should not hold back needed changes.

I was just responding to one of the cited advantages to rsyslog: its syntax will be more familiar to some newcomers to the distro. Yes, and also its syntax will be unfamiliar to some existing users of the distro. (Even if one knows standard sysklogd syntax, the rsyslog syntax extends that for the features it adds.) How this balances out is a judgment call.

Choosing between these systems from scratch is a somewhat different decision than choosing whether to move an installed base of one of them to the other. Perhaps there are good reasons for making such a move, but I wish I had a clearer view of what they were. I've followed the links here and googled a bit, and read Fedora and Debian discussions of why they chose rsyslog over syslog-ng. But in the end, boiling it all down, I'm still not myself seeing compelling reasons for it.

I don't have so much experience with the two systems; sounds like you do or have been talking to people who do, so it's helpful to hear your report. I'd like to know what their reasons were for preferring rsyslog to syslog-ng.

Just reading around, I've so far seen 3 people complain about the ugliness of rsyslog's syntax, and now 1 (you) report opposite complaints. So I guess this is a judgment call too.

I just want to state that "because Redhat and Debian do it" is never a valid reason to make a change.
So think about the motivations for the change before actually implementing it. If we're stuck in
that mindset we might as well give up pacman.

I don't have anything against rsyslog or syslog-ng in particular. I'll only be slightly inconvenienced
if there's a switch. Syslog-ng is working fine for me, so I see no need to change.

@louipc - Nobody has said we should do it just because they have. We've mentioned discussions they have had for going with rsyslog so that we don't have to revisit the same topics over and over again and can learn from what others have done already.
@Profjim - I appreciate your request for more information and I'll gladly oblige.
Reason #1 that I, and those whom I've had the chance to teach Linux prefer rsyslog to syslog-ng: It's very similar to what they already know. Almost all of those that I teach are transitioning to Linux from either Windows, a UNIX variant (mostly AIX/HP-UX/Solaris, but also *bsd as well), or desktop Linux. Those with a Windows background usually prefer rsyslog syntax to syslog-ng syntax as it's quicker to pick up the basics of how to configure logging, though since it's all new, they usually don't care as strongly as others. Those coming from AIX/HP-UX/Solaris prefer rsyslog because it's just an extension of the knowledge they already know from configuring logging on their UNIX variant of choice (all three UNIX variants use syslogd). Same thing goes for *bsd users who I've taught, {free,open}bsd use syslogd by default. Those that come from desktop Linux are usually a toss-up, though I see that changing as days go by. Most of them come from a background of Ubuntu (most popular), Fedora (next most popular), or openSUSE (third in popularity). Ubuntu (and debian as we've seen by the links in this bug report) defaults to rsyslog, as does Fedora. As a result, those students have a preference to rsyslog. It used to be that openSUSE (and SLES/SLED) used syslog-ng and so a fair amount of them would have the opinion that it's better. Now that openSUSE defaults to rsyslog, they have been changing their minds.
Reason #2 - Certification. We teach a fair number of classes that prepare people for Linux certification. (mostly Red Hat Certified Technician, Red Hat Certified Engineer, and the Linux Professional Institute's LPIC-1 and LPIC-2). Both organizations require knowledge of syslogd syntax (which again is very similar to rsyslog -- rsyslog adds extensions to the syntax). Admittedly, Novell's SUSE certification does require syslog-ng knowledge but it's not a certification that comes close to the popularity of even the LPI certs which are a lot less known than the Red Hat certs.

I don't mean to argue solely on the idea that we should do like everyone else except in a way I do. What I mean by this is that while I do have a personal preference to rsyslog and I've explained why, I feel that we ought to make using Arch as easy as possible for those coming from other flavors of Linux and from UNIX backgrounds and let's be honest, do we really want them to have to relearn logging when it's not necessary and syslog-ng doesn't give us any strong reason to use it over rsyslog? I don't argue against pacman because it's one of the defining characteristics of Arch and it has very good things about it that help it stand apart from other package managers -- syslog-ng vs rsyslog isn't that type of issue.

As for making the current install base switch -- I don't see any reason why it ought to be required. Like I said, I think the default should be rsyslog for those new to the distro. I'm not out to convert every current Arch user to rsyslog.

Sorry for the long response. Hopefully it helps.

We should revisit the topics when they arise and decide ourselves whether the change is right for Arch.
Redhat and Debian can't properly make those decisions for Arch. That's what I'm saying.
There may be great valid reasons to switch, but other distros should not influence it too much.

I'll add my two cents worth, although I can't say its right on point.

The basic issue surrounding these has been that the original (syslog?) was working fine when first Arch was installed. Then switching to the supposedly better syslog-ng gave me no benefits I could find, and was way too confusing to use. So I removed syslog-ng. Now I noticed that the logs are out of date and there is no syslog to reinstall.

I'd say that the situation is partly confused by the fact that one exists as a separate package and one comes from some unknown source. That is the first issue to users. Only second would be whether or not there was a benefit to moving to something else. Far, far more important is a really useful GUI for handling logs.

People want choices in programs thru packages that work similarly, and are actually available. I know I would try rsyslog if it were available today.

So that's one user's experience on the subject. I say, "Make it available".

It seems like switching is not a clear win. The vote thread on the arch-dev-public list was +2, with 3 people voting, and the caveat that we then keep maintaining both.

That seems like a fair amount of work and migration pain for marginal gain, at least at this juncture. If someone would like to step up and commit to maintaining two syslog demons in core/extra, please write in here before April bug day. Otherwise, I'm recommending we close this as "won't implement" on April 2010 bug day.

Another question I have is if this is as good for the average desktop user as syslog. That's kind of the bottom line, and it has not been made clear if the focus is on server application or general use. Its always hard to tell the intent of some comments because we can't read minds, but my comments are just regarding general desktop useage. And again, is there a gui for it?

Thanks.

@KitchM - There wasn't really a distinction made between server/desktop benefits to rsyslog over syslog-ng. I can't say I really know what the *average* arch-using desktop user wants, but as for me, I use arch on the server and on the desktop, and I would prefer to use rsyslog over syslong-ng for the easier syntax alone. Most of the advanced features are aimed more at server-installations, things like logging to SQL databases, remote logging with encryption, etc. but that doesn't mean they wouldn't also be useful on the desktop. As for a GUI, I don't know what exactly you want the GUI to do. I don't know of a GUI for configuring any logging daemon. As for viewing logs, there are several, such as gnome-system-log, and none of them that I have used rely on a certain backend because they just display and search text files (thus they can be used with rsyslog or syslog-ng).
Another reason that I haven't previously mentioned for having arch use rsyslog instead of syslog-ng is because I would really like to be able to use phpLogCon (http://www.phplogcon.org/) for analyzing my arch logs and syslog-ng isn't supported.

Thanks, hoban. I went right over to the page at phpLogCon, and it was pretty cool. Anyone who needs a good log tracking display will find that darn handy. It looked very descriptive and explanatory, so I understand your interest.

As for a GUI, it would be very nice to be able to configure the logging function and its parameters, as well as let it be the viewer for the files. The likely extension of the idea would be to include the ability to have popup notifications. In other words, a complete application.

I did check for gnome-system-log but couldn't find it in the repositories. I'll keep looking. I don't remember if I tried it or ksystemlog, but I will compare them again the next chance I get. (I believe I removed the KDE version because of the huge overhead of files that come along with it.)

Anyway, thanks again.

@KitchM -
$ pkgfile gnome-system-log
extra/gnome-utils

Thanks, hoban. I had a feeling I was looking for the wrong thing. It just occurred to me that the package search at Arch does not include all programs contained within other packages. You saved the day.

By the way, in case anyone is interested, here is a list of some of the advanced features that rsyslog provides which I care about:
Drop-in syslogd replacement
ipv6 support
tcp logging (and port specifications)
Definition of message output formats
Precise timestamps
log to SQL databases
better filtering
server failover (when using tcp)
built-in TLS encryption

Some of these features are provided by syslog-ng, others are not.

I'm having a little trouble with the drop-in idea. I installed it and my logs are not right. I'm going to keep working.

@KitchM - rsyslog is a drop-in replacement for *syslogd*. Arch runs syslog-ng which has a completely different syntax. So to be more clear, rsyslogd is a drop-in replacement for anyone who runs the legacy logging daemon, but unfortunately, arch doesn't fit into that category so if the config provided by the package doesn't fit your needs, there will be a bit more work ahead of you, let me know if you have a particular issue getting something configured and I'll be glad to look into it with you. (Though this thread may not be the best place for that conversation)

decision?

It seems clear that the real issue here is that Arch needs to remove syslog-ng as the default logger. The reason is that it is non-standard, if I understand it correctly; at the very least because it causes a change to standard syntax.

There really isn't any necessity for further reasons. However, it appears that the clear advantages to rsyslog are enough to over-ride the use of syslog-ng. I have not heard anything here that points out any benefits to syslog-ng over rsyslog, so that part is pretty settled.

Promoting a better program is justification enough. No one steps out to do much with another program, if the Arch distro leadership doesn't endorse it. (Actually, that's another reason to change.)

My personal experience is that trying to put in rsyslog after Arch changed to syslog-ng caused big problems in logging functionality. Evidently, that also leaves only the change back to syslog or a change up to rsyslog as the only reasonable option at this point.

What standard does it break? Links are appreciated.
We already break LSB anyways by not supporting RPMs. Standards aren't always that important. ;)
I'm not against the change. I just like to see the right facts.

louipc,
Evidently from what you can read here, the original standard by which these things were measured is the syntax of syslog. Also, since it is the basis (read "standard") for logging in 'nix, then a replacement should follow that trend or be more intuitive when it offers "improvements". From what I gather, syslog-ng does not do that.

Second, improving something implies a benefit. I have not experienced that with syslog-ng, nor evidently have most others. Let's support real improvements the end user can appreciate.

IMHO, my suggestion for you would be to compare these side-by-side with the oldest method first. What I gather from reading here is that there is real concern for the results of that comparison, and my personal experience has shown clearly that there is a need for vast improvement in Linux logging.

Well, obviously I'm biased for syslog-ng as I'm developing it. But if you think it is really important to have the old syslog.conf style format back, I can write a small plugin that does just that. Read the old file on-the-fly.

Is it something that you think would be a useful addition?

@Balázs: i think that would crush quite some contra-arguments... so.. GO! :)

Here you are:

http://bazsi.blogs.balabit.com/2010/09/syslog-ng-now-supports-the-syslog-conf-file-format/

bazsi, sorry for being slow to respond. I apologize.

As I understand it, the original issue in this thread was basically that people thought that a more progressive approach to logging would be appropriate. I not only agree with that, but consider it obvious that such should always be the case. (Either we move forward or we slide behind.)

But more than that is the issue of ease of use. If it is assumed, as implied in the discussion, that syslog is simple and easy to use, then anything which wishes to supplant that function should be at least as easy. But actually it should be even easier than the predecessor.

With that said, there is little question that forcing a change to the function and/or format of logging by going to syslog-ng instead of syslog or rsyslog is counter to the idea of being beneficial or to the KISS concept. The only way it could be viewed as otherwise would be if there were huge benefits to be gained by forcing users to learn a new way. I don't think so, because after all it is just logging.

Writing a compatibility function into syslog-ng to make it work with the old, well-known syntax of syslog is probably a good idea. But that still leaves out the stated benefits of rsyslog. The bottom line is that something would still be missing and the user has gained no additional functionality in most situations. IMHO, there is no need for plugins if the necessary functions are built into the program in the first place. Would there be the desired timestamp choices, as one example, without another plugin? Wouldn't it just be better to use another that already has all of this?

Anyway, that is how I understand the situation based on what has been written so far.

Thanks.

Is this sitting idle as a way of proving some kind of point?

rsyslog is the default logger for:
* Fedora
* openSUSE
* Debian GNU/Linux
* Ubuntu (and derivitives)
* Red Hat Enterprise Linux (also CentOS)
* Mandriva

original System V syslogd is the default logger for:
* FreeBSD (both rsyslogd and syslog-ng available via ports)
* OpenBSD (both rsyslogd and syslog-ng available via ports)
* PCLinuxOS (both rsyslogd and syslog-ng available in repo)
* Slackware

no default logging daemon:
* Gentoo (system V syslogd most common but both rsyslogd and syslog-ng ebuilds exist)

syslog-ng is the default logger for:
* Arch Linux

Keep in mind once again that several of the distros now using rsyslog by default used syslog-ng prior to rsyslog and were motivated enough to switch to rsyslog.

Please don't let this ticket remain stalled forever, more and more people coming to Arch from other distros are used to configuring logging using syslogd/rsyslog syntax, *not* syslog-ng syntax, are used to rsyslog features/frontends, have scripts and recipes for rsyslog, etc. and this will continue to be the reality going forward.

The above list takes care of the top 15 or so distros per distrowatch.com and I'm certain that the cycle would continue if I kept searching.

Once again I ask, please, make rsyslog the default logging daemon for Arch!
Thanks,
Tim

@hoban: You ask to make rsyslog the default, and yet you give no reason why.

1) syslog-ng works well, you raised no particular problem with it as a logging software as far as I can see.
2) If anyone wants rsyslogd, they can install rsyslogd. If you don't like the syslog-ng configuration syntax, install rsyslogd instead.

So, if you don't like this bug report sitting here, should we close it as "Won't implement"? There has been an internal discussion about the topic with no consensus, as nobody cares enough to make the switch. Maybe we should finish that discussion and come to a decision.

I don't really care, I am happy with syslog-ng, and I especially don't care enough to waste any time on changing some default for no real reason. Last time I checked, Arch users didn't care about defaults, and were experienced enough to adjust their system to do whatever they liked.

> more and more people coming to Arch from other distros are used to configuring logging using syslogd/rsyslog syntax,
> *not* syslog-ng syntax, are used to rsyslog features/frontends, have scripts and recipes for rsyslog, etc

So? pacman -R syslog-ng; pacman -S rsyslogd. That is no argument for a switch. Attracting or scaring away users isn't either.

> The above list takes care of the top 15 or so distros per distrowatch.com

One simple question: Who cares?

I agree with brain0. The main argument for rsyslogd is that others are using it, so we should too.
That reasoning is not what I have in mind when I think of archlinux.

As an alternative idea, does anything else in the base install require syslog? why not simply remove any default syslog, then there is no default at all. perhaps a package called something like "nosyslog", which when installed uses an init script to notify admins at startup that no syslog daemon is installed, and that one should be installed (or alternately, nosyslog could be removed to supress the warning message)

@brain0 @tomd123: It seems that neither of you bothered looking at previous comments before blasting me. I and others have given several reasons why rsyslog should replace syslog-ng as the default logging daemon. For your convenience, allow me to present once again some of those arguments:

a) Rsyslog is a drop in replacement for former syslog so:
*Anyone experienced with System V or BSD UNIX is familiar with syslogd syntax.
*Same goes for people with enterprise Linux experience (RHEL/SLES)
*syslog-ng syntax is ugly and harder to learn (so say hundreds of students I've taught logging to and I had to teach both rsyslog and syslog-ng).

b) The core Linux development group (Fedora/Red Hat engineers) don't like syslog-ng because of:
# Code Complexity
# Performance issues
# Incompatible format
# Dual licensing model where adding features available in the other version might cause friction with upstream.

c) Most Linux certifications (if not all that I've seen) require knowledge of syslogd syntax and _not_ syslog-ng syntax for what it's worth.

d) Compatibility
*Programs like phplogcon that support only syslogd-compatible logging daemons
*In multi-OS/Distro environments, to use advanced features such as TLS/tcp logging/precise timestamps/server failover, it is *much* easier (and sometimes the only possibility) when using the same logging daemon. Since nearly every other distro uses rsyslog, arch becomes the annoying one to configure because of adding one more multi-level step (yes, pacman -R syslog-ng; pacman -S rsyslogd as you mention starts the process, but then you still have to stop syslog-ng, start rsyslog, and edit /etc/rc.conf to switch default daemon).

d) rsyslog provides all the features of syslog-ng and more (what's the argument for using syslog-ng over rsyslog?)

f) Momentum
*This is specifically the piece that everyone keeps misunderstanding the importance of and as a result it becomes the focus of your rebuttal. To quote you both:

brain0 says (quoting me to begin with):
">> more and more people coming to Arch from other distros are used to configuring logging using syslogd/rsyslog syntax,
>> *not* syslog-ng syntax, are used to rsyslog features/frontends, have scripts and recipes for rsyslog, etc

>So? pacman -R syslog-ng; pacman -S rsyslogd. That is no argument for a switch. Attracting or scaring away users isn't either.

>> The above list takes care of the top 15 or so distros per distrowatch.com

>One simple question: Who cares?"

tomd123 continues:
>"I agree with brain0. The main argument for rsyslogd is that others are using it, so we should too.
>That reasoning is not what I have in mind when I think of archlinux."

Never once did I say that we should do it just because they are. I said:
"Keep in mind once again that several of the distros now using rsyslog by default used syslog-ng prior to rsyslog and were motivated enough to switch to rsyslog."

That right there is the reason I bring up the other distros. Since we are the only mainline distro that hasn't moved on, that ought to get us thinking about what has motivated so many other groups to make their decision to move. How many times does the battle have to play out with the same victor every time before we see the writing on the wall? rsyslog is superior. That's the conclusion that is being made time and time again. Why are we too stubborn to accept that fact? It doesn't hurt anyone to make the new default rsyslog. It will log as usual and those that don't care about logging won't notice a difference. Those that have existing installations won't be affected. Those that want syslog-ng can "pacman -R rsyslog; pacman -S syslog-ng" if they want (to use your own argument).

One other thing, to quote brain0 once more:
"I don't really care, I am happy with syslog-ng, and I especially don't care enough to waste any time on changing some default for no real reason. Last time I checked, Arch users didn't care about defaults, and were experienced enough to adjust their system to do whatever they liked."
That is a very "single-system" type of point of view. Yes, switching your single installation to a different logging system is easy. Do the install of rsyslog. Turn syslog-ng off, turn rsyslog on, change daemons line in /etc/rc.conf. Doing this on many machines gets annoying fast. I'm currently working for a company that has around 20,000 Linux servers and many more added each day. For new installations, kickstart/autoyast/whatever scripted installation can take care of those sorts of logging daemon changes at install time but let's assume we used arch in our environment at some point in the future. Making that kind of change on thousands of machines is somewhat more daunting even with scripts.

To sum it up:
There are several reasons mentioned above why rsyslog is superior to syslog-ng.
The current reality is that most other unix-type operating/distros use rsyslog, not syslog-ng.
It looks very certain that rsyslog will continue to be the logging daemon of choice.
Why are we keeping ourselves in the past? I seriously don't understand why there is such opposition to switching the default, especially when there is already a well-tested rsyslog package available.

@aarcane: I like that you're thinking outside the box on this one but having no logging daemon by default would mean nothing gets logged if/until the user installs and enables a logging daemon.

@hoban: Thanks for the summary. You bored me with a) and c). You got my attention with b) and you might have convinced me with d).

So, game-plan:
1) find a maintainer among the developers for rsyslogd (it won't be me, I am not in a position to add anything to my plate).
2) move rsyslog to core and add it to the base group.
3) move syslog-ng (and glib2 with it) to extra, remove it from the base group.

Point 1) will be the most difficult, and making sure that we have a syslog layout similar/identical to our current one. I will ask around among the developers who wants to take care of this.

@brain0: Sounds great. I've never maintained an arch package but I'd be open to it if you aren't able to find a developer who wants it. I like the idea of being more involved but there would be some learning curve obviously.

Hi,

I'm jumping in to clarify. I'm not an Arch user, so my opinion may not matter much, but I'm related to syslog-ng though (just to make my bias clear). Reading the same repeating arguments again and again is tiresome, especially if they may not be true anymore, and even if they had some merit, the explanation given here is not detailed enough and do not contain the whole picture.

> The core Linux development group (Fedora/Red Hat engineers) don't like syslog-ng because of:
> # Code Complexity

I guess this is a lot more about style. rsyslog is said to be a fork of sysklogd, and thus probably easier to follow in the simple cases. syslog-ng processing model has always been more complex but therefore the more complex functionality is easier to add and configure.

There are numerous complaints on the rsyslog config file format, which is neither simple or easy to understand once the more involved functionality is used.

but i guess it's just as easy for me to say bad things about rsyslog, just see for yourselves.

some users also came back to syslog-ng purely because of stability problems, and because of the quality of the documentation.

> # Performance issues

again, it depends which version, and what kind of performance issues. syslog-ng doesn't scale to multiple CPUs, but I'm quite certain it is faster in single threaded configurations, and probably has a smaller footprint.

and multi core support is already available in the development branch.

> # Incompatible format

syslog-ng now understands the syslog.conf format, and people who do need only the original stuff could as well stick with syslogd, and as mentioned earlier in this thread arch already has syslog-ng.

> # Dual licensing model where adding features available in the other version might cause friction with upstream.

this has been dealt with quite nicely. syslog-ng doesn't need copyright assignment anymore, the license is contribution friendly, and the authors have given a statement, that they would integrate contributions even if it competes with the commercial offering.

lwn article on the license change:

http://lwn.net/Articles/402298/

(read the comments too.)

http://bazsi.blogs.balabit.com/2010/08/syslog_ng_and_open_core_why_we_think_it_is_different/

> Keep in mind once again that several of the distros now using rsyslog by default used
> syslog-ng prior to rsyslog and were motivated enough to switch to rsyslog.

This is not true. Only suse was installing syslog-ng by default.


> d) Compatibility
> *Programs like phplogcon that support only syslogd-compatible logging daemons

I fail to see how phplogcon would differentiate SQL data based on what application inserted in the first place.

> *In multi-OS/Distro environments, to use advanced features such as TLS/tcp

syslog-ng has defined these log transports. rsyslog is compatible with it.

> logging/precise timestamps/server failover, it is *much* easier (and sometimes the
> only possibility) when using the same logging daemon.

this is a stretch. syslog has standards. precise timestamps are defined in RFC5424. failover is just the selection of an alternate destination when the first one fails.


> d) rsyslog provides all the features of syslog

well, online log correllation is one example.
a flexible name-value based message model, also supporting tagging is another
collecting process accounting files is another.
a new mongodb destination has just been merged.
...

syslog-ng is more than just syslog, and is certainly alive and kicking. and being completely independent of the legacy of sysklogd can be more flexible too.

I'll finish now, it's up to the guys doing the work to decide what happens.

The arguments here can go on for a VERY long time, both of these are healthy and active system loggers. I am only a lowly TU but I would be more than happy to make the move to rsyslog. Keeping in mind that my motivation is purely based upon the fact that "I have had better experiences with rsyslog" and that I use rsyslog over syslog-ng in my corporate deployments.
In a nutshell, I will be happy to take rsyslogd, but I am a lowly TU.

This is what I was afraid of: Now there are arguments for and against switching. In this case, I would still prefer to keep syslog-ng.

Follow this thread for the current developer discussion: https://mailman.archlinux.org/pipermail/arch-dev-public/2011-January/019155.html

@thatch45: Good to hear from you! (This is Tim Gelter) Glad to see you're still using arch!
@brain0: I'm done for my part...I don't want to cause bad blood in the community and even though I have a strong bias, I'll defer to what is decided in the mailing list and keep my peace.
@bazsi: Thanks for your perspective. Thanks also for explaining your bias. I think we'd be well off to hear a rebuttal by an rsyslog developer but I doubt that is going to happen. I can't comment for any of the developer-related gripes I read about as I haven't, nor will I ever be involved with that process. All I know is that from a system administration standpoint (which I do know quite a bit about) rsyslog is an easy choice over syslog-ng by myself, all current and prior coworkers, and students I taught in the classroom.

Whatever will be, will be. :)
Thanks

@thatch45: Good to hear from you! (This is Tim Gelter) Glad to see you're still using arch!
@brain0: I'm done for my part...I don't want to cause bad blood in the community and even though I have a strong bias, I'll defer to what is decided in the mailing list and keep my peace.
@bazsi: Thanks for your perspective. Thanks also for explaining your bias. I think we'd be well off to hear a rebuttal by an rsyslog developer but I doubt that is going to happen. I can't comment for any of the developer-related gripes I read about as I haven't, nor will I ever be involved with that process. All I know is that from a system administration standpoint (which I do know quite a bit about) rsyslog is an easy choice over syslog-ng by myself, all current and prior coworkers, and students I taught in the classroom.

Whatever will be, will be. :)
Thanks

Oops. Duplicate post.

I've discovered a minor (but still meaningful) strike against rsyslog, from the point of view of user support. It's one of the few cases where syslog-ng is clearly better than rsyslog - normally I consider them basically of equal quality.

Both syslog-ng and rsyslog provide a means of indicating the software version the config file was written to be used with. That way, if the software is upgraded, but the config is allowed to get stale, the correct behavior will still occur.

rsyslog requires one to indicate the correct version as a command-line flag to the daemon. This means it can easily get out of sync with the config itself, like so: the command-line flags are stored in /etc/conf.d/rsyslog (courtesy of a patch I recently submitted - previously rsyslogd was always started with no options). If a user modifies /etc/rsyslog.conf, but doesn't touch /etc/conf.d/rsyslog (probably a common case), then when the syntax version changes, pacman will upgrade /etc/conf.d/rsyslog in place, but will care a .pacnew for /etc/rsyslog.conf. Careless users who don't merge the .pacnew will then try to run an old config file with a new version flag. We may argue that we don't care about careless users, but this will create noise on the forum / Flyspray / mailing lists. Next thing you know, users will ask for weird hacks to be implemented to prevent this.

syslog-ng, on the other hand, stores the intended version in the config file itself. No chance of it getting out of sync there.

Again, this is minor, but it's worth considering. In the end, we will switch if some dev wants to own an rsyslog package, stay on syslog-ng if some dev wants to maintain that, or (unfortunately) stay on syslog-ng with the current minimal level of support if no one cares enough to do either. (For those who don't read arch-dev-public, that seems to be the real "problem" here if there is one - no dev currently actively maintains any syslog package.)

Last comment sounds more like a packaging problem than a software one.
Why not mark both config files as backup so they both get extracted as pacnew and so stay in sync ?

@shining: Both are marked as backup. It's just that since one file is much more likely to be changed by a user than the other, one will get a .pacnew and the other will just be overwritten. A second issue I just though of, is that it's entirely possible for the (Arch) maintainer to forget to change the conf.d file when rsyslog.conf changes (or not to be aware of the need for a change), since the conf.d file is our local creation, but the stock rsyslog.conf comes from upstream. I agree, it's really a packaging problem, but still something for any potential maintainer to be aware of.

I would like to contribute some points from the rsyslog camp. I am not an arch Linux user, but the main developer of rsyslog. I will try to clarify some positions from my PoV. I obviously have a bias, but I do not pretend to know what's the right choice for the arch linux community. With that said:

@baszi: Let me start with that I am a bit astonished that syslog-ng defines syslog/TLS. I thought the RFC5425 does this. And as far as I remember the discussions on the IETF mailing list, you were not very active at the time the standard was defined. Even more interesting, rsyslog was the first software to implement the draft version, shortly followed by an enhanced FreeBSD syslogd and very much later by syslog-ng. I think we should stay fair with who did what first and who followed ;)

On the feature set: online log correlation is not yet in rsyslog, but we are working to integrate a standard-based approach (CEE). And, yes, I know Balabit has finally joined that effort as well ;) The current master branch has some functionality. syslog-ng, as far as I can see, is still way ahead in this area. The same goes for name-value pairs. A patch for mongodb for rsyslog has been proposed and will soon be added.

IMHO it boils down that both syslog-ng and rsyslog have active community and development. Feature-wise, I think there is a large core set and each project has some features the other is missing.

On code quality/complexity: I have intentionally never looked at syslog-ng code for more than half an hour. But I guess you also never looked at rsyslog, at least if I read your comments on the legacy. IMHO both projects have rather complex code today, because both do complex things.

On the license change, I am glad to see that I managed to make you adopt the rsyslog model -- this was one of my motivations for creating rsyslog. You can read about the motivation here:

http://blog.gerhards.net/2007/08/why-does-world-need-another-syslogd.html

@all Some general points:
A main difference between rsyslog and syslog-ng is that syslog-ng is backed by a large commercial organisation and a commercial fork of syslog-ng. This is not necessarily bad. But it is a difference to how rsyslog is driven: I get some funding for rsyslog development from Adiscon, but Adiscon is much, much smaller. So rsyslog is more of a community project. I have to admit that we at Adiscon still hope to find some commercial funding, e.g. via support contracts or custom development. But we do have far less of the traditional commercial machinery behind that (maybe it would be better we had, at least for me ;)).

Being a mostly community project has some good and some bad consequences. The good one is that rsyslog is pretty independent from Adiscon and me by know: there is lot of knowledge in other folks. One of the really bad things is documentation: rsyslog documentation is sparse and not too well. I simply have not managed yet to find time for a decent writeup and nobody else cares (but Red Hat's writer team, to the best of my knowledge, has begun to do a decent writeup).

The config file format is another story: the initial goal was to keep it fully compatible with "regular" syslog.conf statements (because of all the complaints on syslog-ng syntax from first-timers). But we really have outgrown this model. Still, writing a new config system is a lot of effort. The current system seriously sucks for complex cases. syslog-ng's format is much better in those. My hope is that I can come up with a really good new format soon. We had a big discussion inside the community last year, but the thoughts were so diverse that I pushed back the idea of rewriting the system for a bit. V6 contains part of the minimal consensus and my personal feeling is that the minimal consensus is too minimal to be a solution to this issue.

On the version command line switch: this does NOT modify the way the config file is treated! It just changes defaults. The idea is that when a user upgrades, he will typically not like to change everything. So when the command line switch is not adjusted, everything works "just as before". Even new features work, but according to old defaults. We require explicit opt-in to activate new features. This concept can not work if the compatibility is stored in the config file itself (e.g. because some legacy command line switches needs to be supported, what we do not know at the time we read the config file).

On stability: rsyslog is also more a community project in that we release very early, including experimental versions. We are more a community project in that we rely on end user bug reports. That means the most recent versions are probably less stable than syslog-ng (especially as massive multi-threading with state-of-the-science methods brings some extra problems). To mitigate this, we offer various versions (v3,v4,v5,v6) each of which has stable, beta and development builds. v3 is what you want in the datacenter (if the feature set is sufficient), whereas v6, even stable, you will only want if you like to experiment or have a very urgent need. BTW: Some of the world's largest data centers run rsyslog at stunning message rates without any problems as part of their day-to-day operations.

Rsyslog is also often used as a testbed. For example, I used it when I wrote RFC5424, it was a testbed for syslog/tls -which provided very valuable feedback to the IETF syslog wg-, it is used as a testbed for the new Mitre CEE effort (part of the technology also made it into sagan, which helps drive the normalizer) and one of my next projects will be to try out new ways to implement highly parallel processing using lock-free algorithms (an academic project).

To sum up: both rsyslog and syslog-ng are quite good implementations, each with their own strength. I think the main difference is the process in which they are created (more commercial-focus vs. community/technology-focus).

Rainer

> @baszi: Let me start with that I am a bit astonished that syslog-ng defines syslog/TLS.

using syslog-ng + stunnel to implement encryption was mentioned in 2004 on the mailing list. It has been implemented in syslog-ng 2.1 in 2006.

> I thought the RFC5425 does this. And as far as I remember the discussions on the IETF mailing list, you
> were not very active at the time the standard was defined. Even more interesting, rsyslog was the first
> software to implement the draft version, shortly followed by an enhanced FreeBSD syslogd
> and very much later by syslog-ng. I think we should stay fair with who did what first and who followed ;)

Yes, you're probably right about RFC5425 support, it was released introduced in syslog-ng 3.0, published in 2008. (the development happened earlier that year, but 3.0.0 was published in November).

OK, if you consider *that* as syslog/TLS, than syslog-ng was probably the first syslogd that we know to be used with TLS. I even have a copy of the tutorial available from stunnel.org on Feb, 4th 2004[1]. I had just never thought that syslog-ng had "invented" this.

rsyslog with RFC5424 was released on May, 6th 2008. I am surprised to hear that syslog-ng had this support in 2008 as well, because when we implemented syslog-transport-tls, Martin Schütte (who did the FreeBSD implementation) and me looked for other "players", but nobody showed up... But on the other hand, both Martin and I did not dig deeper after summer 2008 (Martin finished his work around that time).

But that's all history...

[1] http://www.winsyslog.com/common/en/articles/stunnel-downloads/stunnel-syslog-ng.pdf

Just a thought: if you consider any protocol spoken by a syslogd, SDSC syslogd was probably the first one to implement TLS-based syslog: they supported RFC3195, which offers TLS via a BEEP tuning profile. My implementation of BEEP (done in 2003) never supported that profile, but I think SDSC's did.

@rgerhards:

Well, one thing that is sure that history is all about perceptions and different
people have different set of information which their perception is based upon. :)

I've now reread the sentence in my post, which reads as:

>> *In multi-OS/Distro environments, to use advanced features such as TLS/tcp
>
> syslog-ng has defined these log transports. rsyslog is compatible with it.

tcp() transport was introduced with syslog-ng 1.0, back in 1998.
TLS support as I understood the original poster was RFC3164 on TCP wrapped in SSL.
Which came up quite early in syslog-ng's life and got implemented by a lot of
people using syslog-ng + stunnel for years (I've found related posts back in 2000).
I'd say it was quite a routine to do and one motivations of using syslog-ng
over syslogd was its use of TCP, and thus the ability to integrate with stunnel.

I hope this explains what I meant in my original post.

I also wanted to react on other points made by Rainer, but I've felt that this forum is not necessarily the best one to use, so I've posted it on my blog, here:

http://bazsi.blogs.balabit.com/2011/02/syslog-ngs-development-drivers/

I'm closing this report as "Won't implement":

- No reasons to replace syslog-ng by rsyslog seem to be standing out. Both of them are good logging system and are active upstream.
- No developper seem to have the time and interest in investigating (and perhaps implementing) this and maintaining rsyslog in core. This bug report dates from 2008 and the recent thread on the mailing list didn't gather much interest.
- I have recently adopted syslog-ng and have pushed a syslog-ng 3.2.2-1 package in core which resolved all the old pending issues that were on the bug tracker. So the package will get more care that it had lately.
- Users who want to use rsyslog can easily install it from the community repo.