Question posted in 
I send daily a newsletter to about 20K recipients. After moving to other server I need about six times more time to this using the same PHP scripts – I tried many different settings but had no luck with it (please see this post). The email server seems to be properly configured (with regard to spam, relay and viruses), the server runs with Plesk 11.5.30.
Doing many experiments, changing PHP scripts and Postfix settings I was able to achieve the best performance with the following:
- Sending emails from PHP using SwiftMailer.
- Using SMTP (local account) as transport.
- Dividing recipients list in four parts and starting separate PHP-CLI process for each one.
When I start only one process, PHP sends about 5 emails / second (which are about the same when I send emails using mail() function). Postfix has no problems with sending them “on the fly“ so there is almoust no queue.
When I start four processes, PHP sends about 10 emails / second, however postfix is no more able to send them „on the fly“ (it can send only about 5 emails / second) so the queue becomes bigger and bigger. When the PHP processes are over, I have about 4500 emails in my queue. And here comes the interesting part: when there are no more active PHP processes sending emails, Postfix „accelerates“ to sending about 20 emails / second. Would Postfix be able to send the emails at this rate while PHP is delivering new emails? there would be no queue and it would solve my problem.
I would appreciate any idea what could I do to improve the performance of my script and server sending the newsletter.
Here comes the Postfix configuration:
root@hostname:~# postconf -n
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases, hash:/var/spool/postfix/plesk/aliases
append_dot_mydomain = no
biff = no
config_directory = /etc/postfix
disable_vrfy_command = yes
inet_interfaces = all
inet_protocols = ipv4
mailbox_size_limit = 0
mailman_destination_recipient_limit = 1
message_size_limit = 51200000
mydestination = (none), localhost, localhost.localdomain, localhost
mydomain = domain.com
myhostname = myhostname.com
mynetworks = 127.0.0.0/8 [::1]/128 my.ip.add.ress
plesk_virtual_destination_recipient_limit = 1
readme_directory = no
recipient_delimiter = +
relayhost =
sender_dependent_default_transport_maps = hash:/var/spool/postfix/plesk/sdd_transport_maps
smtp_connect_timeout = 10s
smtp_helo_timeout = 100s
smtp_send_xforward_command = yes
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_use_tls = no
smtpd_authorized_xforward_hosts = 127.0.0.0/8 [::1]/128
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_rbl_client zen.spamhaus.org
smtpd_proxy_timeout = 3600s
smtpd_recipient_restrictions = permit_mynetworks, check_client_access pcre:/var/spool/postfix/plesk/no_relay.re, permit_sasl_authenticated, reject_unauth_destination
smtpd_sasl_auth_enable = yes
smtpd_sender_restrictions = check_sender_access hash:/var/spool/postfix/plesk/blacklists, permit_sasl_authenticated, check_client_access pcre:/var/spool/postfix/plesk/non_auth.re
smtpd_timeout = 3600s
smtpd_tls_cert_file = /etc/postfix/postfix_default.pem
smtpd_tls_key_file = $smtpd_tls_cert_file
smtpd_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_use_tls = yes
transport_maps = , hash:/var/spool/postfix/plesk/transport
virtual_alias_maps = $virtual_maps, hash:/var/spool/postfix/plesk/virtual
virtual_gid_maps = static:31
virtual_mailbox_base = /var/qmail/mailnames
virtual_mailbox_domains = $virtual_mailbox_maps, hash:/var/spool/postfix/plesk/virtual_domains
virtual_mailbox_limit = 0
virtual_mailbox_maps = , hash:/var/spool/postfix/plesk/vmailbox
virtual_transport = plesk_virtual
virtual_uid_maps = static:110
3
Answers
Thank you for all your answers and your effort. Fortunatelly I was able to find the main cause of the problem - it was saving the queue on the hard disc. We changed it to RAM and are now able to send about 30 emails per second.
Hey I cannot find the exact problem with your settings but I have read the comments which might help to find the exact time lag. Please refer http://ubuntuforums.org/showthread.php?t=2007309 and also you can recheck your code by http://liangsun.org/posts/configure-mail-service-with-php-mail-function-and-postfix/
And for settings of the swiftmailer refer https://bbs.archlinux.org/viewtopic.php?id=82062
Since you write about your PHP process are sending about 5 or 20 mails per seconds, I’ll suppose you are already measuring in detail your PHP code, so if this delay is in the PHP code you should be already discovered.
Reading the comments I also suppose your systems are quite big, so you have enough ram and disk.
In my experience a very common problem that could cause such kinds of delay are the DNS resolutions, well, I mean a wrong configuration of DNS, that leads in a delay all around.
But this hypothesis is all to be proved.
On the other hand, I even remember that postfix have a throttling configuration in order to avoid server abuses. http://www.postfix.org/TUNING_README.html
At last, if you don’t find anything in the postfix configuration, what I can suggest you is to try to debug the postfix process during its execution, with the utility strace.
It is a quite useful utility, I used it often when I have to debug non common problems.
In short: strace lists all system calls done by the process. There are a lot of options available in strace, but I usually use only:
During the debug you’ll see all the system calls executed by postfix main process and all its children. If you add the -c option then you’ll see the time spent for every system call.