Welcome to the SmartList FAQ.
You are strongly advised to read the official SmartList Manual in advance. Many questions in this FAQ are only pointing to the appropriate chapter in the Manual.
This document is a collection of helpful hints and tips for getting the most out of SmartList. The sources for this information are numerous.
If you have a comment or suggestion regarding this FAQ, please forward it to sl-faq@hartzler.net. Note: Please do not direct requests for assistance with SmartList to this address, or they will be ignored. There are public lists dedicated to that purpose (see below), and you are much more likely to get satisfaction there.
Disclaimer: All information in this FAQ is provided AS IS and without any warranty of any kind. No person involved in this document assumes any liability for damages which may occur as a result of using this information. USE THIS INFORMATION ONLY AT YOUR OWN RISK.
It is suggested that you create a test list, and try changes on that test list before you place them into production.
This document was originally written by jason@mindwell.com.
It was then picked up and modified/enhanced by Werner Reisberger; <werner@pure.ch>.
This FAQ is currently maintained by Peter Hartzler, and is officially located at http://www.hartzler.net/smartlist.
You may also find it mirrored at http://www.pure.ch/doc/smartlist/faq.html
SmartList is an email list management system based on procmail.
The current "best" version of SmartList is 1.13.
The best documentation is the SmartList manual contained within the SmartList sources. Every list maintainer should first read this manual to understand how SmartList works and to save a lot of time if problems occurs later.
The SmartList manual is available at ftp://ftp.pure.ch/pub/smartlist/SmartList-Manual as well as in the .doc directory of a normal SmartList install.
Next, the distribution comes with considerable amounts of example code, which is to be found in the .examples directory of your installation.
There are actually several FAQs that people have put together, and you might very well find some useful bits in the others. There appears to be a considerable overlap in these documents, but what the heck. :) Check out:
You can also search in the list archive(s) (see question "Is there an archive of the mailing list?")
Finally, SmartList is heavily based on procmail, and in fact many questions about how to do specialized things in SmartList end up being mainly procmail questions. You might want to check out:
SmartList is based on a set of procmail scripts to process emails suitable for the purpose of a mailing list. If you want to know how the emails will be processed take a look into the SmartList manual (section "Schematic overview of what goes on behind the scenes")
SmartList needs also Procmail.
The latest version of Procmail can be found at: http://www.procmail.org or ftp://ftp.informatik.rwth-aachen.de/pub/packages/procmail/procmail.tar.gz
The latest version of Smartlist can be found at: ftp://ftp.informatik.rwth-aachen.de/pub/packages/procmail/SmartList.tar.gz
Yes. for more information, see http://MailMan.RWTH-Aachen.DE/mailman/listinfo/smartlist
Another mailing list, for developers is
smartlist-dev-request@cuci.nl
Yes, at
http://www.cnr.berkeley.edu/~casterln/smartlist-arc/maillist.html
Another view of that archive is available at
http://www.cnr.berkeley.edu/~casterln/smartlist-arc/threads.html
You can also access the archive via email. To learn how to, send an email to smartlist-request@informatik.rwth-aachen.de with the subject "archive help".
They let you enhance or alter the function of SmartList by calling procmail scripts at different states of email processing.
If SmartList sends out messages to the maintainer or the list, it adds message headers beginning with "X-Diagnostics:". Every maintainer should read these header lines to recognize problems with the list.
SmartList usually creates by default in every list a log file called "log". If you want to trace a problem with a list you can set the variable VERBOSE in rc.custom to "on". Subsequently, every processed message will create a lot of debugging information in the log file.
Don't forget to turn it off later.
This question seems very trivial but experience showed that many people have problems understanding how to (de)activate a variable in the rc.custom file.
To uncomment a line in the file, remove all hashmarks (#) from the front of the line (if there is more than one, be sure and remove them all), and change the value if necessary. For example, the moderate section in the standard distribution looks like this:
#moderated_flag ##moderated_flag= yes # uncomment this to make the list # moderated (you must create aFor example, to turn your list into a moderated list, remove the two hash marks from the line that says "moderated_flag = yes", so the section now looks like this:
#moderated_flag moderated_flag= yes # uncomment this to make the list # moderated (you must create aIn other words, anything after a hashmark on a line is considered a comment, and anything before any hashmarks (if any) is uncommented. So, you can uncomment a feature setting, and still have a comment on that line, out towards the end.
NOTE Be sure the text editor you use saves text in "UNIX" text format, with only a newline character at the end of each line, and not a carriage-return newline pair. Many windows based text editors insert that carriage-return, and this can break your configuration files.
If you end up with carriage returns in your text files, you can remove them with perl, like this: (this particular example assumes the file you want to clean is rc.local.s20, change this file to be the one that needs cleaning):
perl -p -i -e 's/\r//' rc.local.s20
To make the list closed subscription (i.e. the maintainer has to approve subscriptions), uncomment the auto_subscribe line in rc.custom, and set it to read:
auto_subscribe = noTo make the list closed for unsubscriptions, do the same thing with the auto_unsubscribe line.
- Set the variable auto_unsubscribe in rc.custom to an empty value.
- Remove the line '(Only addresses below this line can be automatically removed)' from the dist file.
Everything in the dist file preceding the line containing: '(Only addresses below this line can be automatically removed)' is write protected from changes by multigram (i.e. these addresses can never be automatically/accidentally unsubscribed).
By sending a message to the listname-request address with the message 'subscribe' or 'unsubscribe' in the body or the subject of the mail, if the list is open, they will be added to it, if it is a closed list the request will be forwarded to the list maintainer.
The password is stored in the rc.custom file.
By uncommenting the appropriate "auto_help" line in the rc.custom file the list will respond to every undecipherable request message as if it requested help. Messages that will still get through to the maintainer are those:
- that seem to come from a daemon.
- which look like a reply.
By setting the cc_requests or cc_unsubrequests variables in the rc.custom file to yes (making sure to uncomment it).
Instead of rejecting submissions by people not on the accept (dist) list, you can enable "force_subscribe". This will cause people submitting mails to the list to be autosubscribed to the list if they were not in the dist file.
Uncomment the line "RC_LOCAL_SUBMIT_20 = rc.local.s20" from your rc.custom file, and add the following to your rc.local.s20 file:
# # Adding a disclaimer in front of every mail: #
:0 fhw | cat - header.txt
# # Appending a footer to every outgoing mail: #
:0 fbw | cat - footer.txt
You will also need to create a file called header.txt or one called footer.txt, which contains the text to add to the message (this file can be empty though, so you can create the files, but only fill them in when you need to add something)
A small refinement here: if you add something like this:
# # Appending a disclaimer to every outgoing mail: #:0 fbBw * ! to unsubscribe | cat - footer.txt
then (assuming that the phrase "to unsubscribe" appears in your footer.txt file), the footer won't be repeated if some bozo list member quotes an entire message without trimming the footer. (Answer submitted by Doug Landauer)
An even cleaner solution, developed by Paul Thomas and Era Eriksson is to pull in content from your actual footer.txt to check and see if that text already exists in the message. This has the advantage of tending to automatically track changes you might make to your footer.txt file...
TEXT=`sed -n '2 p' <footer.txt` :0 fbBw * $ ! $\TEXT | cat - footer.txtWhat this does is load the variable TEXT with a line of text from your footer.txt file (the second line, in this example), and use that data for it's test. Note that you need two $'s -- one to say "here's the variable I want to interpolate" ($TEXT) and one in front of the condition to say "interpolate variables in this condition". Also note that $\TEXT is a special form of $TEXT where regex specials will be escaped in the interpolated value. So if the second line of the footer says "*** cut here ***" then that is not a valid regular expression, but "\*\*\* cut here \*\*\*" is.
You can create a file named "subscribe.files" in the list directory. It can contain any number of archive-server commands. The results (i.e. the files requested) will be sent to the new subscriber.
Yes. See Chapter 5 of the SmartList manual for the exact format and limitations.
However, according to Paul Thomas:
"Perhaps it should be noted that comments in the dist file confuse multigram and those addresses have to be removed by hand should they desired to be unsubsribed, start bouncing, etc."
See Chapter 4 of the SmartList manual.
Additionally, there are web interfaces to allow searching archives. A combination of Mhonarc, Wilma and Glimpse is reported to work well, and more information is available on the Mhonarc site.
- Mhonarc: http://www.mhonarc.org/
- Wilma: http://www.hpc.uh.edu/majordomo/#wilma
- Glimpse: http://www.webglimpse.org/
delink is a shell script in the .bin directory of your SmartList installation which lets you remove the hard link from a file. This is important for list files like rc.init which are by default hard linked across all lists.
Usage:
delink filenameA set of files which are hard linked together have the quality that a change made to any of those files will be reflected in all of them. In other words, all files in a hard linked set are always identical. If you want to make one file in a hard linked set different from the others, perhaps to customize one list differently from the others, then you must first remove that file from the hard linked set, using delink or the equivalent.
You can use the showlink tool to get some idea of what a file is hard linked to.
By default, the accept list (those who can submit to the list even if foreign_submit is off) is linked to the dist list. You may want to add another list of valid submitters, such as to allow both list and digest recipients to post to a list.
Additionally, you may want to add yet another accept list, which could include the list owner, or other valid senders:
- Unlink rc.submit.
- Find the line that ends with:
-x$listaddr -x$listreq accept accept2- Change it to:
-x$listaddr -x$listreq accept accept2 accept3(Tip o' the hat to Paul Thomas)
Here is another hint, submitted by Scott Pallack:
I use 3accept instaad of accept3. My list has both normal (dist linked to accept) and digest (dist linked to accept2) subscribers and I can put additional addresses, such as the listowner, alternative addresses for list members, etc., into the 3accept and still be abole to do things like:
grep -i user@domain.net acc* cat acc*|grep @|sort|uniq|wc cat acc*|grep @|sort|uniq -detc.
See chapter 2 of the SmartList manual.
It should be noted that it is possible to put multiple commands in one message. Combined with the example in .examples/rc.local.r00 this makes it easy to work with X-Commands by allowing you to place multiple commands within the body of your messages.
And no, you can't just put X-Commands in the Subject line of the header.
See chapter 2 of the SmartList manual.
See chapter 2 of the SmartList manual.
Modify the x_command script with the following patch:
51,66c51,60 < $echo $2 | $multigram -c -l32000 dist >/dev/null < if test $? -ne 0; then < $echo "From $2 " >$tmprequest < $echo "From: request ($listreq)" >>$tmprequest < $echo "Reply-To: $2" >>$tmprequest < $echo "To: $listreq" >>$tmprequest < $echo "Subject: subscribe $2" >>$tmprequest < $echo "$2" >$tmpfrom < $test -z "$subscribe_log" || < $echo "x_command: subscribe" >>$subscribe_log < $subscribe <$tmprequest || < $echo "X-Diagnostic: Unscreened, not subscribed" < else < $echo "" < $echo "$2 is already subscribed!" < fi --- > $echo "From $2 " >$tmprequest > $echo "From: request ($listreq)" >>$tmprequest > $echo "Reply-To: $2" >>$tmprequest > $echo "To: $listreq" >>$tmprequest > $echo "Subject: subscribe $2" >>$tmprequest > $echo "$2" >$tmpfrom > $test -z "$subscribe_log" || > $echo "x_command: subscribe" >>$subscribe_log > $subscribe <$tmprequest || > $echo "X-Diagnostic: Unscreened, not subscribed"
There is a perl cgi script which let's you send all X-Commands via the web. It's called x_command and is available at
ftp://www.pure.ch/pub/smartlist/
You can also use Matt Wright's "Formmail" script to provide a web based sign-up form. You can get the script at http://www.cgi-resources.com
See http://www.mplsrc.com/mailinglist.shtml for the form.
Thanks to Irwin Lazar for the Formmail suggestion.
There are several possible reasons why SmartList might fail to update the dist list. The two most common reasons seem to be:
- An incorrect installation (possibly from a faulty package), or
- Incorrect file permissions.
Assuming your installation is OK, check the file permissions of your dist file. The 's' (setuid/setgid) permissions of the flist binary determine what rights it has when it runs. The dist file must be writable by a process with those permissions.
See chapter 3b of the SmartList manual.
In addition you can use the confirm addon to forward all subscription requests at first to the maintainer who has to confirm the request before the new address will be added to the list.
See chapter 3c of the SmartList manual.
Disable the variable auto_unsubscribe in rc.custom or use the confirm addon.
No. email is an insecure tool and messages are easy to fake. SmartList cannot make it more secure. Each message could be faked on a list. To make email more secure use PGP or other encryption packages for email usage.
See chapter 4b of the SmartList manual.
Yes, You need an addon to SmartList to confirm subscriptions and/or unsubscriptions. The addon is called confirm and is available at:
ftp://www.pure.ch/pub/smartlist/
Also check out:
ftp://gen.free.de/smartlist/confirm-1.2.tar.gz
Confirmation of subscriptions is very useful to prevent the initial subscription of invalid addresses or the unwittingly subscription of addresses by a malicious person.
You need to apply the patch below to rc.submit (delink first) and copy the subcreen script within the .examples directory to yourlist_dir/postscreen.
E. g. if you want to allow all postings from anyuser@friendly.dom and reject all other addresses which are not in the accept file, the lines
case "$1" in *@friendly.dom exit 0 ;; # This one is ok esacexit 1 # reject everything else
will be sufficient.
Patch to rc.submit:
102a103,107 > * 2^0 ? if test -f postscreen; then \ > screenadress=`formail -xFrom: | sed -e 's/.*<\(.*\)>/\1/' \ > -e 's/\(.*\) (.*)/\1/'`; \ > postscreen $screenadress;\ > fi
If the string you're filtering agains is "foobar", then put something like this in rc.local.s00
:0 * ^Subject:[ ]*.*foobar.* | formail -rtk -I "Subject: Posting of followup rejected (was: $MATCH)"
"By default, posts that are too long are forwarded to the list owner, who has to write the person manually to explain what's happened. It would be more convenient if SmartList would send them an automated notice..."
Try this in rc.local.00:
:0 wfbh * !< $size_limit |formail -rk -I "X-Loop: $listreq" -I "From: $listreq" \ -I "Subject: Sorry, message size for $list is max.: $size_limit bytes!" |\ $SENDMAIL -tIt should bounce the whole message, mentioning the max. message size in the subject.Tip 'o the hat to Werner Reisberger for this solution.
See chapter 3a of the SmartList manual.
See chapter 3a of the SmartList manual.
See chapter 3a of the SmartList manual.
See chapter 3a of the SmartList manual.
Yes. If you use the sendmail replacement of postfix you don't need to change anything. Just make sure that SmartList can find the sendmail wrapper.
For large lists it may be advantageous to replace choplist with the direct sendmail wrapper call because postfix doesn't need a helper program for the load balancing.
Yes. If you use the sendmail replacement of qmail you don't need to change anything within SmartList.
For large lists Tracy R Reed <treed@freeside.ultraviolet.org> suggested the following:
To: djb-qmail@koobera.math.uic.edu, SmartList@Informatik.RWTH-Aachen.DE
Subject: smartlist and qmail
Date: Mon, 29 Sep 1997 14:10:42 -0700 (PDT)
I have noticed that using choplist to feed qmail's sendmail clone (because choplist expects sendmail) to distribute mail to my large mailing list was incurring massive amounts of overhead and really slowing things down. If I could just get smartlist to call qmail-inject directly, it would greatly speed things up. To this end, I modified my rc.init file:
alt_sendmail="smart-qmail dist $listreq"And put this perl script in ~list/.bin/smart-qmail
#!/usr/bin/perl $from = $ARGV[1]; $addyfile = $ARGV[0]; while() { $message .= $_; } open(BLAH,"<$addyfile") || die "Doh! No address file!\n"; $x = 0; while (
) { if(/Only addresses below this line can be automatically removed/) { next; } chop; $addy .= " ". $_; $x++; # 30 is kinda arbitrary, I just don't want to run into any ugly command # line length limits.
if ($x == 30) { open(INJECT,"|/var/qmail/bin/qmail-inject -f$from $addy"); print INJECT $message; close INJECT; $x = 0; $addy = ""; } } open(INJECT,"|/var/qmail/bin/qmail-inject -f$from $addy"); print INJECT $message; close INJECT;
Another note -- Paul Thomas points out that
"Qmail bounces email differently than sendmail. It does a 'local' bounce and a 'remote' bounce. SmartList will process to the best of it's ability a 'remote' bounce, but 'local' bounces are written to a qmail "Mailbox' file and overlooked by SmartList. In short, qmail will work with SmartList but to make everything work right one is looking at a major hack of SmartList."
If you log an error message something like
554 "|exec /path/to/flist listname"... Service unavailablethe reason is most probably that sendmail is using the smrsh "Sendmail Restricted SHell" environment for shell calls.The way a message initially gets to the SmartList system is through a sendmail alias which pipes the email to the program. The alias probably looks something like:
listname: "|exec /path/to/flist listname"However, some configurations of sendmail attempt to increase security by restricting allowable programs to those in a certain directory. In this case, you need to create a symlink to the flist binary in the smrsh directory.Try these steps:
- Find the correct location of your smrsh directory. You may have to examine your smrsh source, or use the strings command to examine your smrsh binary.
- Create a symlink to your flist binary in the smrsh directory:
ln -s /path/to/flist /location/of/smrsh/flist- Change the alias for the list to simply be
listname: "|flist listname"You don't need the path to flist now, since sendmail is only going to look in one place (your smrsh directory). You don't need the exec either.Don't forget to run newaliases after you modify your aliases file to effect your changes.
Yes apply the following patch (from Alan Stebbens) to your rc.request file (don't forget to remove the hard link before). After applying the patch you can use several maintainer addresses in rc.custom separated by a comma (don't include blanks between the addresses).
--- installed/rc.request Thu Oct 31 00:02:23 1996 +++ rc.request Fri Oct 18 09:11:58 1996 @@ -291,20 +264,20 @@ ## is there more than one maintainer? -maintainer_re=$maintainer -maintainers=$maintainer +maintainer_re="$maintainer" +maintainers="$maintainer" Maintainer=Maintainer
:0 * maintainer ?? , -{ maintainer_re=`echo "$maintainer" | sed -e 's/,/|/g' -e 's/\./\\\./g'` - maintainers=`echo "$maintainer" | sed -e 's/,/ /g'` +{ maintainer_re="`echo $maintainer | sed -e 's/,/|/g' -e 's/\./\\\./g'`" + maintainers="`echo $maintainer | sed -e 's/,/ /g'`" Maintainer=Maintainers }
:0 wf:dist.lock -* $^$X_COMMAND:.*$maintainer_re[ ]+$X_COMMAND_PASSWORD
If you have access to your aliases file, another approach is to set the maintainer in rc.request to a "role address" defined in the aliases file to expand to all maintainers, for example "mylist-owner". Maintainers would then use the "mylist-owner" address to issue list commands, and any email generated for the maintainer will be delivered to all maintainers defined in the aliases file.
There is an addon to SmartList which enables you to do this via email or a web interface. You should only use this package called rconfig if you know how SmartList works. Otherwise you could completely destroy a list. rconfig is available at
There is an addon to SmartList which enables subscribers to suspend their subscription for a given number of days. It's called suspend and is available at ftp://www.pure.ch/pub/smartlist/
SmartList Utilities and Patches - send email to alan.stebbens@software.com with the subject
"send smartlist library".
See chapter 3f in the SmartList manual.
There is also this package:
The messages that smartlist sends out are in the ~slist/listname directory, and can be edited using any available file editor. The files that it sends and their uses are:
- help.txt:
Sent as a response to a help command, or if anything smartlist doesnt understand is sent to the listname-request address.- subscribe.txt:
Sent to new subscribers.- unsubscribe.txt:
Sent to someone who just unsubscribed.- archive.txt:
A description of how to use the archive server, and the commands it understands.
In your rc.custom file, uncomment the line that reads "RC_LOCAL_SUBMIT_10 = rc.local.s10", and make a file called "rc.local.s10" in your list directory, in that file put this:
SUBJ=`formail -zx Subject:` # extract the subject :0fw * ! Subject:.*\[listname\] * ! Subject:.Re: * | formail -I "Subject: [listname] $SUBJ"Be sure to replace "[listname]" with what you would like to be at the beginning of the subject line. (Posted to SmartList@Informatik.RWTH-Aachen.DE by Leigh Wolenczak ).Another approach, courtesy of Tim Pierce:
:0f * !digest_flag ?? y * $!^Subject:.*\\[listname\\].* * ^Subject:\/.* | formail -I "Subject: [listname]$MATCH"
Uncomment the line "RC_LOCAL_SUBMIT_20 = rc.local.s20" from your rc.custom file, and add the following to your rc.local.s20 file:
# # Adding some custom headers (although it will work, this is not # needed for Reply-To related headers, see the reply_to variable in # rc.custom): #:0 fhw | formail -i "X-Subliminal-Message: SmartList is great" \ -I "X-Mailer: procmail, which is great too :-)"
Uncomment the line "RC_LOCAL_SUBMIT_20 = rc.local.s20" from your rc.custom file, and add the following to your rc.local.s20 file if you want :
# # To get rid of some headers: #
:0 fhw | formail -I X-Mailing-List: -I X-Mailer:
# # To get rid of all X- fields: #
:0 fhw | formail -I X-
This script was contributed to the SmartList mailing list by mark david mcCreary, you can run this from your list directory (where the dist file is located), and it will remove the duplicates and sort the entries that appear below the "(Only addresses below this line can be automatically removed)" line.
#! /bin/sh : # # rc.dedup.list # # mdm 11/5/97 borrow code from David W. Tamkin, with additional # suggestions from Martin Konold and Hal Wine # # # This script will sort and remove duplicates from a Smartlist # distribution file. Smartlist dist files contain a comment line # that looks like this # # (Only addresses below this line can be automatically removed) # # Addresses above and including this line will not be sorted. #lockfile=lockfile # /usr/bin/lockfile cat=cat # /bin/cat rm=rm # /bin/rm sed=sed # /bin/sed
# create lock file to discourage smartlist running a job
$lockfile -10 -l3600 -r11 -s30 rc.lock
$sed -e "1,/(Only addresses below this line can be automatically removed)/{ ;w slabove.$$ ;d; } " dist | sort -fu -o slbelow.$$
$cat sl?????.$$ > dist $rm -f sl?????.$$
$rm -rf rc.lock
Several good recipes for this have been posted to the smartlist mailing list.
- Mike Nolan contributed one, which rejects messages that are posted in html: http://www.mindwell.com/smartlist/contrib/reject_html.txt
- Rob Lingelbach contributed one which rejects messages in html, messages with a Return-Receipt request, uuencoded messages, winmail.dat files, non text encoded files, attached applications, inline attachments, and messages which have too much quoted text: http://www.mindwell.com/smartlist/contrib/reject_garbage.txt
The "Reply-To:" header element is designed to allow you to do this, and the reply_to variable within your rc.custom file gives you the ability to control this header element in list processed email.
There are several approaches. Arguably the simplest is, in the rc.custom file, uncomment the line:
reply_to = $listaddrand set it to something like this:
reply_to = "Reply-To: $listaddr"Paul Thomas notes that:
"The 'Reply-To: ' rule in rc.custom is overwritten if a post to the list already contains a 'Reply-To: ' with a different address. Putting somthing like::0 fhw | formail -I "Reply-To: test@somedomain.com"in rc.local.s20 might be better."
If you run "announcement" style lists, where you want to discourage list discussion by having replies go to the original author, then in the rc.custom file, uncomment the line:
reply_to = $listaddrand set it to something like this:
reply_to = "Reply-To: `formail -xFrom:`"Note that the formail -xFrom: command is enclosed in backticks (`), which cause procmail to run the formail command in a subshell and return that command's results.
The idea is to keep the footer text in a file called footer.template. In that file you embed $COUNT wherever you want the list population inserted. Then you cause your subscription and unsubscription processing to count the list members, and interpolate footer.template to produce a footer.txt file.
Perhaps your footer.template file might look like this:
--- To unsubscribe, blah blah blah. This list has $COUNT subscribers.For this to work, you need to uncomment
RC_LOCAL_REQUEST_20 = rc.local.r20from either your rc.init or your rc.custom. You also need to have cc_unsubrequests and cc_requests set to yes. Then, add this recipe to the end of normal sub/unsub processing; rc.local.r20:# Count list members, and update footer.txt using footer.template. :0 * ^X-Diagnostic: (Processed|Added to the subscriber list) { COUNT=`grep -cv '^(' dist` :0cWi | sed 's/\$COUNT/'"$COUNT"/g footer.template >footer.txt }
Yes. There is a recipe you can find in .examples/rc.local.r00 which has the effect of causing SmartList to look in the beginning of the message body as well as within the message header for commands. This trick is helpful if you have a MUA which makes it difficult or impossible to send custom headers.