Story Author: Indiana University Information Technology Services
Publish Date: January 13, 2005
Source Publication: Not Available

UITS and many other organizations face tough challenges in regards to the messaging environment we use today. Indiana University receives roughly one and a half million messages a day, and roughly seventy percent of those are spam.

Some of you might be aware of the recent problems Comcast and InsightBB users have experienced when trying to send email to IU users. Analysis of the inbound mail traffic reveals that SMTP connections from hosts on Comcast and InsightBB networks have very quickly and repeatedly exceeded their bandwidth thresholds. Further analysis shows that a large percentage of the mail from these providers is being sent from domains that do not exist. As part of IU's anti-virus/spam strategy, we do not deliver mail from domains that do not exist.

One of the ways IU protects itself from these misbehaving mail servers is by limiting each host to sending several hundred messages per minute.
This figure is very dynamic and changes, as needed, to provide IU users with the best mail experience possible.

The rate-limit measures IU uses are reasonable and dynamically adjusted, and allow well-behaved mail servers to connect and deliver mail. Hosts that exceed these thresholds are sending hundreds (if not thousands) of messages per minute. Prior to implementing the thresholds, there were delays in receiving mail across the board. These delays were caused by only a handful of machines at any particular time, but they affected mail from all of the well-behaving hosts.

It is true that legitimate email is being adversely affected by this behavior, but the thresholds are needed to balance the resources in our mail environment, thereby allowing us to receive mail from other email environments. In a nutshell, we are trying to minimize delays for the good citizens, but the bad citizens still get a piece of the pie. We will not, however, let them consume the entire pie.

It is important to note that the SMTP standard specifies that it is the sending SMTP host's responsibility to process deferred or failed connections, and then communicate back to the user (sending client) when an email is not delivered. This is customarily done via an NDR (Non-Deliverable Report).

We often find IU users asking us why they did not receive an email message that they were expecting. If that message was sent from a host outside our environment, there are multiple points of failure that are possible as the message traverses the Internet. In many cases, the failure occurs before the email even reaches our network.

In other instances, a message might reach our network, then be refused due to the connection thresholds described above, or deferred because our resources are at capacity. Regardless, the sending server is still responsible for reporting a delivery failure or delay to the sender.

UITS Messaging works hard to ensure that the university email environment adheres to RFC standards, and delivers appropriate error codes and NDRs. UITS is very aware of the importance of email to the academic mission, and dedicates as many resources as possible to provide our users with the most reliable mail experience possible. In fact, the Registrar has a policy regarding the use of email for official university communication with students. Please see the Knowledge Base document, "At IU, what is the policy about official communications from the university to students?" (http://kb.indiana.edu/data/aozw.html).

Additionally, a perception has been that one or another IU email system is "at fault", and this is not the case. The IU email system architecture separates internal university mail flow from inbound/outbound Internet mail. The protection layer between IU and the Internet is the same regardless of the email system (IMAP servers, Exchange or even departmental servers) you use. It is the huge volume of spam currently on the Internet that is degrading timely email service for most, if not all, email services around the world.

This message was written for the LSP community. We are working on communication for the general user population which will be published in the Knowledge Base and on the IT Notices page.

We appreciate your patience with this matter. If you have any questions or concerns, LSPs may contact us at [PROTECTED]