MTA-STS: Why and How I Did It

When I tell people I run my own mail server, they picture this. (Photo by Ivan N on Unsplash)

There is always something needed when it comes to a home network, but when it comes to public-facing servers there are additional steps one has to go through. Now this is going to sound like a combination of boredom and insanity, but the mail server I run for NMRC which is running Sendmail (yes that is a link to an FTP server) needed to implement RFC-8461 aka MTA-STS. Okay, it didn’t need it but I wanted to do it, and I’ve decided to document what I did since existing documentation didn’t seem to answer all the questions I had. Granted this is not complete documentation itself, but it is the steps I went through.

Even though the benefit of this is to the mail server, since I had already completed a few steps literally years earlier, I did not have to actually touch my mail server, except to monitor logs. So if you’re still awake, read on my nerds.

So What Is MTA-STS?

MTA-STS (Mail Transfer Agent Strict Transport Security) is a standard for improving SMTP (Simple Mail Transfer Protocol). In essence, it helps close a security hole when email servers talk to each other. Now granted this hole is quite small, the attack vector is not easily exploited if you are in fact vulnerable, but it is there.

Modern mail servers are set up to exchange email via an encrypted channel to prevent eavesdropping and being able to read the plaintext email while in transit over the Internet. My mail server is listening for inbound connections from other mail servers, and it is configured so that one of the first things it says during this server-to-server connection is that it wants to continue the conversation while encrypted. It does this by flashing up a keyword of STARTTLS which tell the other server to start using TLS encryption - basically the same type of encryption that your browser uses for web traffic. Obviously all of this goes on under the covers, so to speak.

What MTA-STS does is allow for a form of “advertising” that says yes, if you’re going to send email to my domain, here is the mail server for my domain, and when you do see the STARTTLS thing you will need to check the TLS certificate that I have which proves that I am who I say I am. This prevents an attacker from launching a “man-in-the-middle” (MITM) attack and basically prevent the STARTTLS part from happening so they can force unencrypted traffic they can sniff. More or less.

Don’t get me wrong - this doesn’t solve the entire problem by any means. Email is typically stored in plaintext on the sender’s end as well as the receiver’s end. And it doesn’t force your mail server to only talk using STARTTLS, it forces other mail servers that are talking your mail server to honor the MTA-STS policy and use it. Also, if you’ve ever done any attacking on the open Internet, pulling off a MITM attack is not exactly easy. Truthfully, stealing creds and breaking into the email account or accounts is way easier, which is something to keep in mind.

So why do it then? Defense in depth. This is not the only configuration option or choice on my mail server, but it is one of many. I’ve talked about my mail server settings before, this is just something extra I added. MTA-STS adds a bit of security help for inbound email like DMARC allows a bit of security help for outbound email. I already had DMARC, DKIM, and SPF configured, so for me MTA-STS made sense.

Before You Start

I guess the first thing is you should have your own mail server, or at least you manage one (perhaps at work). Second, you’re going to need STARTTLS configured. If you’re literally running your own mail server (using Sendmail like I am, or something similar like Postfix), this can be a bit complex. To configure things you’re going to need a public certificate. Yes, like a web server that uses HTTPS much have a valid and publicly provable certificate, your mail server is going to need that. Granted if you are setting things up within your cloud provider (for example, your mail presence is hosted by Google and it a part of the Gmail setup) then things will be a lot easier and they have decent step-by-step instructions for all of this, from STARTTLS to MTA-STS and many other settings.

I am not going to go through these details for any Sendmail or Postfix setup, as there are plenty of resources online for that, and we’re discussing MTA-STS. Suffice to say I had previously configured STARTTLS with a public certificate (via Let’s Encrypt), and had also set up DMARC, DKIM, and SPF so this wasn’t exactly unfamiliar territory for me. MTA-STS is going to require the creation of a few DNS records for your domain, and the publishing of a web page on a specific subdomain based off of your domain, so a bit of familiarity with DNS will also be of help.

What I Did

Talon is my mail server, and the associated MX record points to that. I did a blog post about Talon that is now rather outdated, but it does mention that SMTPS and all the other fun stuff is set up and configured. So basically I did nothing to Talon at all as STARTTLS was already there for incoming email from other mail servers. This has to be set up first, you have to use a public certificate (via Let’s Encrypt which is great), and like I said there is decent documentation online for these steps elsewhere. Just know you can’t skip this step.

MTA-STS needs to be on its own domain. Therefore I created a new subdomain website in nginx to answer for mta-sts.nmrc.org. This has very little content - there is an index.html page on the root with a link to the main NMRC website in case someone gets a little curious and accesses it directly, and a subdirectory called .well-known which contains a single file called mta-sts.txt. That’s the entire subdomain website. It’s not like this was hard to do or anything, as I already had a complete website, but it was required for things to work.

This probably isn’t immediately obvious, but talon.nmrc.org is the server where the mail service is, and rigor-mortis.nmrc.org is the server where web services are. Rigor-mortis handles the main NMRC website and the Mastodon instance, as well as a couple of other odd domains, so setting this up for mta-sts.nmrc.org was literally a bit of cutting and pasting and editing a few files. So you don’t need to run this mta-sts web presence on the same server as the mail server, it does work this way.

Anyway, the contents of mta-sts.txt is as follows:

version: STSv1
mode: testing
mx: talon.nmrc.org
max_age: 604800

That’s it, that’s the “policy”. The version must be STSv1, the mx must be the host listed in your domain’s DNS mx record (yes you can list multiple as in mx: thishost.example.com and mx:thathost.example.com each on its own line), and the max_age is the lifetime of the policy and must be between 86400 and 31557600 seconds. The mode can be testing, enforce, or none. I recommend setting it to testing first just to make sure everything is working (I did it for a week no problems) before setting it to enforce. If you’re going to set it to none, well, what’s the point of doing any of this? I guess to temporarily turn the policy off or something.

The certificate for the MTA-STS web server must also be a public certificate. This is another Let’s Encrypt thing, unless like me you are doing this on the same server as the main domain. Since I had set up the main web server’s certificate with Let’s Encrypt for one cert to handle nmrc.org and www.nmrc.org, I simply added the new subdomain:

certbot certonly --cert-name nmrc.org -d nmrc.org,www.nmrc.org,mta-sts.nmrc.org

Magically certbot stepped me through the process rather painlessly.

Now we needed to create some DNS records. First up I did the new website. I created a DNS “A” record which when done looked like this:

mta-sts 1800 IN A 162.196.226.76

The IP address is the same address as www since they are on the same host, and it all works out.

There were two more DNS records, these were TXT records. The first one is for mta-sts, and the second one (which is optional but highly recommended to help spot errors) is for SMTP TLS reporting:

_mta-sts 1800 IN TXT "v=STSv1; id=20240627092600;"

_smtp_tls 1800 IN TXT "v=TLSRPTv1;rua=mailto:aggregate@nmrc.org"

In both of them, the v is for version and must be the same as I have listed. The id value in _mta-sts just needs to be unique, to simplify it I used a format involving the date and time, although any letter-number combination is fine. A change to the id value signifies to a potential sender of email that some is new, if the value is the same as their cache they can still use the same policy. The rua value in is the email address to send reports to. As I was already sending DMARC reports to a specific address, I simply used the same address for MTA-STS. After I gave it a few minutes for DNS to propagate, I made sure I could access the new host and it’s new file without some certification error or browser complaint.

There are not a lot of ways to test unless you have mail accounts on other systems. But the best way I found was to visit MXToolbox and use the SuperTool. Type in your domain, click the down arrow button and find “MTA-STS Lookup”. This can give you a quick method of ensuring you’ve set things up correctly. The SuperTool may not work if your mail server is sitting behind a service like Cloudflare, so you might just have to wait for those email reports (which typically come in daily). And of course check your mail logs. Granted I know for me easily 99% of the email my mail server receives is already encrypted, which is why I waited a week before I set things to enforce, so give it time.

Results

I guess the question is, how did things turn out? Well, it’s been about a week and I have yet to get a single report of any failure or error of any sort. I’ve edited the policy from testing to enforce. Frankly, I’ve benefited much more from the addition of DMARC than MTA-STS as far as the logs go. Was all of this in vain? Who cares! I’m a nerd, I live for this kind of goofy stuff, and I had this configured and working during a lunch break.