Outbound Rate Limiter Exception Working?

[dciwebworks]

We set the outbound rate at the default levels Magic Spam provided.

We then made an exception for a specific user that emails a newsletter to a large group weekly on the WHM Per-User Rate Limiter Whitelist, a list of local addresses that are excluded from outbound rate limit restrictions

The user didn't get an exemption. Here's a bounced email:

Connecting to mail.irishsnugrunners.com (69.##.###.79)

220-vps.dciwebworks.com ESMTP Exim 4.80.1 #2 Wed, 04 Dec 2013 23:52:35 -0500

220-We do not authorize the use of this system to transport unsolicited,

220 and/or bulk e-mail.

>EHLO irishsnugrunners.com

250-vps.dciwebworks.com Hello c-71-###-###-95.hsd1.co.comcast.net [71.###.###.95]

250-SIZE 52428800

250-8BITMIME

250-PIPELINING

250-AUTH PLAIN LOGIN

250-STARTTLS

250 HELP

>AUTH LOGIN

334 VXNlcm5hbWU6

>###############################

334 UGFzc3dvcmQ6

>###############################

235 Authentication succeeded

>MAIL FROM:<myuser@irishsnugrunners.com>

250 OK

>RCPT TO:<member@mac.com>

554-You have sent too much mail. Try again later.. Protection provided by

554 MagicSpam 2.0-2.5 http://www.magicspam.com

When we raised the overall rate limit on the WHM to the level that is sent in an hour, the mail went through. Should we be creating this exception at the domain level in cPanel rather than WHM? Why doesn't the exception, in fact, provide an exception?

[magicspam]

Hello!

Note: We made some edits to your post to put the logs into a "QUOTE" block and to remove the base64 encoded email address and password used with SMTP AUTH.


We would like to verify that the IP address in your settings is correct. You don't need to post it here, but please examine the contents of your /etc/magicspam/control/rl_whitelist.list file and ensure that the IP address is correct.

Also, if the IP address was rate limited before you whitelisted them, then this would still cause them to have their messages blocked. You can configure what your block timeout is in your MagicSpam dashboard.

If you think there is still an issue, we might suggest taking a screenshot of your rate limiter settings page and sending it to us in an email at support@magicspam.com.

Thanks!
-- MagicSpam Support Team --

[dciwebworks]

Thank you for your kind reply.

Hmmm. Perhaps the problem is that there IS NO /etc/magicspam/control/rl_whitelist.list file located in that or any other directory under magicspam. I have rl.db, rl.block, rl.count and rl.flag. The rate exception entry IS contained in /etc/magicspam/ra_whitelist.list.

Can I generate or build rl_whitelist.list? Can you forward an example file? I assume the IP you're looking for here is the main IP used for the Exim SMTP server.

I've sent a screen shot to your email address.

[dciwebworks]

The response to my email and screen shot noted that the limiter threshold shown was very high. Yes, it was. We wanted that sender listed in our exception list to leap over the constraints of the prior settings one way or another. We'll bring it back down when we can get the Outbound Rate Limiter Exception working.

In the meantime, you offered no thoughts on the missing /etc/magicspam/control/rl_whitelist.list file, which has me on the edge of my seat awaiting your thoughts. All the best.

[magicspam]

Thank you for your detailed information regarding this issue.

To give some background on how the rate limiter system works in MagicSpam PRO, there are 2 unique separate rate limiters:

One is IP based - intended to limit inbound spam floods

The second is Authentication based - intended to limit compromised local account outbound spam

We have seen that the settings you applied for the authentication rate limiter appeared to work around the issue - but our concern at this time is that perhaps the IP based rate limiter was also triggered - and that the work around was applied after the IP rate limiter session had timed out already.

Now.. there is talk about a missing file /etc/magicspam/control/rl_whitelist.lst file. This specific file stores the entries for the IP rate limiter exemptions only. Have you made entries in the IP rate limiter exemption interface that you were expecting to apply? (please note that the 'per user / authentication' rate limiter is in a separate file /etc/magicspam/control/ra_whitelist.lst - we do apologize if there was any confusion over this)

If you could please confirm if there are corresponding entries in the IP rate limiter exemption list that would be appreciated. In the meantime we will confer with our development team to see what the current progress is on separating the rate limiters further (eg: IP based rate limiter should not apply to authenticated connections)

Thanks!

[dciwebworks]

Well, you've challenged me here to look at your dialogs in Magic Spam in a new way.

I've been referring to the Limiters as OUTBOUND and INBOUND. You're referring to them as IP and AUTHENTICATED USER. Clearly, the Inbound Rate Limiter only has exceptions based on IP address, and just as clearly the Outbound Rate Limiter only has exceptions based on AUTHENTICATED USERS.

Our Outbound Rate Limiter was set to defaults at the time, about 50 emails allowed per 5 minutes or shutoff for 6 hours.

But we entered our AUTHENTICATED USER in the list (and he appears in /etc/magicspam/ra_whitelist.list.) in advance of running the distribution newsletter, and the result was

>MAIL FROM:<mysender@mymagicspamprotecteddomain.com> << this sender appears in the ra_whitelist.list
250 OK
>RCPT TO:<newsletterrecipient@email.com>
554-You have sent too much mail. Try again later.. Protection provided by
554 MagicSpam 2.0-2.5

You ask "you could please confirm if there are corresponding entries in the IP rate limiter exemption list that would be appreciated". I can confirm, there are NONE. Because we are expecting that the inbound limiter doesn't affect our outbound mail.

But I can see what you're saying...the email came INBOUND to the server from the client so that it could be sent OUTBOUND, and therefore we may need to add the active IP address of the sending client's ISP connection before doing these mailings. Is that, in fact, what you're saying?

[magicspam]

That is exactly correct! The IP and Per user are both applied. The reason there is even a different set of settings for these two is to allow for finer granularity (eg: block a single sender that might have a virus when operating behind a NAT'd IP without killing service for everyone else behind the NAT).

As mentioned, our development team has been apprised of this to look into alternate methods for splitting these up further but for the interim looking at an addition IP exemption should do the trick for this case.

We hope this helps!

[dciwebworks]

Well, unfortunately, there seems to be something wrong with the Outbound/User rate limiters when used in conjunction with our bulk mail utility, Gammadyne Mailer.

The Per-IP (Inbound) Rate Limiter seems to have nothing to do with the Per-User Rate limiter and here is why…

1. Our initial newsletter send operation started failing at 149 messages in just over 1 minute. We changed the “Max # of messages” in the Per-User Rate Limiter to 750 messages, and the send operation continued successfully.

2. The Per-IP Rate Limiter has been set to the default (150/5/360) and has been unchanged…

3. Yesterday, I added the IP address for the computer performing the newsletter send operation to the Per-IP Rate Limiter as an exception, began the send operation as usual, and after 149 messages the send operation began to fail. Again, I increased the Per-User Rate Limiter, max messages permitted setting, and again the send operation continues successfully.

Please advise additional troubleshooting steps. It would seem that the Per-IP rate limiter is not working, as it has yet to block the “inbound” emails. It would also seem that the exception to the Per-User Rate Limiter is not working, as it has yet to allow the outbound messages from the sending email account.

This has been a learning experience, but I’d say at this point, we understand how the rate limiter system is meant to work…it just isn’t working.

[dciwebworks]

And now I come back to you with some rather disturbing news.

With a rate limiter set universally at 150 messages per hour, one account has managed to send 22,308 messages and never get stopped, interrupted, or place on hold.

With no response even to my last post, I'm wondering if any of this rate limiter stuff ever works except to block legitimate email.

[magicspam]

Please check whether the sender was in the rate-limiter whitelist or not.

If the sender was not in the rate-limiter whitelist, if you could, please provide us the following additional information:

- the sender's email address
- screenshot of your rate limiter settings, including whitelists
- any log that you have regarding the mass outbound emailing

Due to the sensitive nature of the data, we suggest emailing the above information to support@magicspam.com.

And we will be able to assist you further from there.