Azure SMTP Restrictions & Resolution with SMTP Relay Services

This post explains how Microsoft has strengthened Azure platform security against unauthenticated SMTP traffic to maintain Azure IP stack reputation and how 3rd party SMTP API can be used to overcome these restrictions.

Background info

Recently I have set up a Microsoft Exchange hybrid training lab in an Azure Computing environment for one of my clients. The client has registered Azure for a 30 days trial subscription. I have setup exchange server, setup MX and SPF records as appropriate checked entries on MXtoolbox and initiated email send/receive tests to and from the Internet.

I was able to receive emails from the external world but when I tried to send email to external users, they kept bouncing with an error that the recipient system is not accepting emails right now. My external emails were nothing more than my Gmail and Hotmail accounts. Yet those accounts were able to receive emails from the Internet except for mine.

I then ran a message trace and found that mail was not going out of my exchange server. I started the queue viewer and indeed, mail was stuck in the queue with the error message. To further test, from exchange server I tried to telnet on SMTP port 25 to google MX and my O365 exchange online – the telnet test failed.

This started to be a real pain for me to establish a fully functional hybrid lab. So I started searching the Internet and came across an article which says that Microsoft has banned outbound SMTP communication towards the Internet from Azure compute resources in trial subscriptions and hence I must use a 3rd party smart host to deliver emails from Azure compute resources to the external world.

Article: https://blogs.msdn.microsoft.com/mast/2017/11/15/enhanced-azure-security-for-sending-emails-november-2017-update/

Azure SMTP restrictions

Why has Azure stopped outbound emails (standard SMTP protocol) from Azure compute subscriptions?

When we say outbound emails, it applies to unauthenticated email delivery on TCP 25 to the external world through direct DNS MX lookups. The fact is that Microsoft offers Azure trial subscriptions worth 200$ for free and people all over the world take benefit of trial subscriptions to test/host their custom applications/servers.

If any email transactions happen from these apps/servers without proper sender identity verification/authentication (SPF / DKIM) records, it leads to spoiling Azure data center public IP addresses blacklisting, Abuse or negative IP reputation. Since 15th November 2017, Microsoft has restricted outbound SMTP (TCP 25) communication to external world to specific subscriptions only. The purpose is to reduce/minimize negative IP reputation and eventually blacklist Microsoft Azure datacentre IP addresses.

Well, not all Azure subscriptions are blocked, SMTP blocking applies to new subscriptions registered post 15th Nov 2017. Enterprise customers are not affected by this blockage. Customers having a Pay-As-You-Go subscription can send a support request with a valid reason to Azure, and Microsoft conducts a few checks against the request and if the request is approved, they will then unblock SMTP restrictions.

Any new Azure trial subscriptions including MSDN, Azure Pass, Azure in Open, Education, BizSpark, registered post 15th Nov 2017 would by default be blocked for outbound SMTP delivery on port 25. There is no plan for allowing these subscriptions for direct outbound SMTP delivery to the external world. Any support request pertaining to this will not be entertained. These subscribers if needed must use 3rd party email gateways (Smart host) for outbound SMTP delivery. Further SMTP communication must have happened on either secure SMTP port (TLS 587) or other non-standard port such as TCP 2525. TCP 25 would be blocked permanently

Microsoft recommends Azure customers use 3rd party authenticated SMTP relay services with TLS support (TCP port 587 or 443) to send email from Azure VMs or Apps to the internet. These services are specialized in IP / domain reputation and reduce the possibility of blacklisting and email rejection by ISPs and organizations.

These 3rd party SMTP service providers also provide other benefits along with IP and domain reputation such as bulk messaging, transactional email services, marketing campaigns across messaging platforms. Use of these e-mail delivery services is not restricted in the entire Azure infrastructure, regardless of subscription type.

Resolution – Authenticated SMTP Relay Services

An SMTP Relay Service (often known as a Smart Host) is defined as an intermediary SMTP relay server between a sender’s email server and the recipient’s email server. The purpose is to either overcome a limitation by the sending email server or provide bulk email services.

Common reasons why companies consider using an SMTP Relay service:

ISP blocked outbound SMTP 25 port or have set daily / hourly limit on SMTP transactions
A sending SMTP server IP is getting blacklisted frequently
A need to send bulk emails/mass mailing for advertising/marketing while at the same time, maintain IP / domain reputation (Avoid Blacklisting)

In our case, we need a smart host because of network port restrictions (1st point as listed above) imposed by Microsoft with the Azure network.

There are a number of 3rd party SMTP relay services available in Market such as SparkPost, MailGun, turboSMTP, ElasticEmail, ManDrill, SendGrid and so on, just to name a few. One of their service offerings is an SMTP API which allows you to relay emails to the external world on your behalf securely.

All you need to do is to register a free account with them and subscribe for free (if offered) or a paid email plan and you need to authenticate/verify your custom SMTP domain with their email gateway which allows you to deliver emails from their gateway. These gateways listen on various SMTP protocols like 25, 587, 465 including few non-standard ports like 2525. ID and password would be required to connect to the SMTP services. The Password remains very complex as most of the time “API KEY” (encrypted text) is used as a password to avoid password guesses.

What we are looking here, how we can integrate Azure Subscription with such SMTP relay services. The available SMTP relay service should provide the qualities listed below:

Authenticated Connection
Greater spam control
TLS Support
Clean IP and domain reputation
Sender Domain authentication ability
- DKIM authentication
- SPF authentication
Message tracking
Maximum uptime
Bulk messaging facility
Wide range of SMTP protocol support (25, 587, 465, 443, 2525 etc)

Microsoft has provided us the option to integrate an Azure Subscription with SendGrid for free. Once integrated, you can configure the SendGrid SMTP API (Smart host functionality) within an Azure hosted app/server to deliver emails to the external world. Sendgrid offers you 1st 25000 email deliveries free of charge per month. If your requirement is more than 25K, you need to purchase sendgrid plans.

The process is very seamless and secure, works on TCP 587 OR 465 OR 2525, normally you can use TCP 587 if the client supports or you can use 2525. SPF and DKIM checks are managed by Smart Host provider in this case.

Below I have added step by step Azure subscription integration with Sendgrid. Steps are simple:

Azure – Sendgrid Integration – Create a Sendgrid Account as an Azure resource From your Azure Subscription
SMTP API (Smart Host) Configuration – Point applications/services / Exchange server to the Sendgrid host as a smart host relay by using an ID/password created by the Azure integration
Test Email Flow – Test outbound email flow, Sendgrid will take care of SPF and DKIM authentication in this case.
Domain Authentication – This will enable DKIM authentication for your domain. Domain authentication is optional and would be required if we wanted to build our SMTP domain reputation, however, If we have a DMARC record already in place, we must complete domain authentication with Sendgrid. Domain authentication with Sendgrid can be achieved by adding a few DNS records in our DNS zone. This step allows us to pass an existing/new DMARC record for our domain.
Modify SPF Record – Add Sendgrid SPF host to our domain SPF record, this step is recommended for organizations who already have DMARC record created for there SMTP domain. Normally SMTP Relay service providers enforce this requirement as a mandatory prerequisite before they activate service for our domain.
Validate Domain Authentication – Test email flow through Sendgrid and validate its authenticity

Azure – Sendgrid Integration

Microsoft has included SendGrid Email delivery with Azure Marketplace and we can add it as connector resource in the Azure portal. Login to Azure Portal with Global/tenant administrator.

Under the Azure management portal, under “All resources”, click Add and type Sendgrid Email Delivery under new and click search.

Click Create. This will land on Sendgrid registration page within the Azure portal itself.

Fill up name (username for SendGrid), Password, select Azure active subscription and define Azure Resource group to be used to create sendgrid connector. Under the pricing tier, select “Free tier” which provides you 25000 emails per month.

Further, In the same form, fill out the contact info and accept the legal agreement, wait for validation to be successful. Once validation is successful, click Create.

The connector is successfully created in the Azure resource group we selected during deployment. It generated long random characters as user ID (azure_xxx…@azure.com) which is hard to guess. It also provides an SMTP server DNS name to be used as a Smart Host. We can see this information on the connector properties page in the Azure Portal. Note that the Sendgrid account username is different from the username created for Azure purposes.

In this case, “MaheshP” is the Sendgrid account username (located in the extreme top left in above screenshot) and the username created for Azure is located at the extreme right side of above screenshot.

Both usernames share a common password which we set during connector creation in the Azure portal. Former is used to login to the Sendgrid management portal and the latter is used to authenticate against Sendgrid SMTP API services within Azure VMs and apps.

Now click on Manage. It will automatically take us to Sendgrid website landing page to initiate the email verification process. You will receive an email from SendGrid asking you to verify your account and login to your account. Please complete the account verification.

SMTP API (Smart Host) Configuration

We can use the user ID created by Azure – Sendgrid integration (azure_xxx…@azure.com) along with the password in Azure-hosted applications/servers to relay emails to the Sendgrid Gateway and it will accept those emails as authenticated and process further.

The process would be simple. Within applications/servers, configure the email relay section, add smtp.sendgrid.net as a Smart Host server along with the required port (587 TLS OR 2525 OR 465 SSL) as SMTP relay server. Provide the Azure generated ID and password as the authenticator.

I had already deployed Exchange server in Azure for my lab purpose, in order to send external emails, I needed to configure the above user ID with my Internet send connector on Exchange server.

Locate the send connector properties configured for the Internet and on the Network tab, add smtp.sendgrid.net as a smart host (SMTP relay server) and click Change to enter credentials.

Enter the User ID generated by Azure – Sendgrid connector along with the password, select Basic Authentication over TLS and click OK

Next step would be configuring the send connector to operate on a custom port as by default Exchange servers send connectors to operate on the default SMTP port (TCP 25) since TCP 25 is blocked by Azure, hence we will change it to 2525, a non-standard port.

First, we need to ensure that we can reach to smtp.sendgrid.net on TCP 2525

The Telnet session connected successfully, so now we need to configure the send connectors to use TCP 2525 for outbound email delivery.

Run “Get-SendConnector <Internet> | Set-SendConnector -Port 2525” against all internet facing send connectors.

We can see that the Sendgrid Host is added as a Smart Host entry under the available message queue with a ready status.

Test Email Flow

Now we are ready to test external email flow. Login to webmail with an on-premise Exchange server account and send a test email.

In the above example, the sender is user3@exchangelabs.in which is my on-premise native mailbox. user1@exchangelabs.in is my account migrated to O365 and Mahesh PM is nothing but my Gmail account.