TL;DR: Unless you have a complex SPF record that exceeds the SPF DNS query limits, you should not flatten SPF records. Flattening SPF records can cause email delivery issues and can harm the trustworthiness of your domain.
Recently we got a support call from a customer who was experiencing a strange email delivery issue.
Using DMARC monitoring they noticed that about one in 20 emails they would send would fail SPF validation at the receiver. Other email from the same email provider sent to the same receiver passed without any problems. The SPF record checker also reported that everything was fine.
So, what caused the email to fail SPF validation occasionally?
A quick investigation from our side revealed the culprit: they had 'optimised' their SPF DNS records by flattening all the SPF include terms.
If you search online you will find numerous tools that supposedly optimise your SPF records by flattening all include, a and mx terms into a single, large SPF record consisting only of ip4 and ip6 terms.
So, an SPF record like this:
v=spf1 include:spf.mailhardener.com ~all
Might be 'optimised' to this:
v=spf1 ip4:188.241.151.48 ip6:2a01:7c8:aaca:567::1 ~all
The former record requires the receiver to make multiple DNS queries to validate a sender, the latter requires only one. The motivation behind flattening is that it reduces the number of DNS queries the receiver has to make during SPF validation.
Some claim that flattening the records reduces the chance of your email being flagged as spam, because it is easier for the receiver to validate SPF. This is not how SPF works. There is no penalty in SPF validation based on the number of required DNS fetches. Unless the SPF record exceeds the limit of 10 DNS queries required for evaluation, the complexity of your SPF record will not impact your domain's rating.
An SPF record is essentially a list of senders that are allowed to send email for a domain. A receiver will attempt to match the IP address of the sender to the SPF record found in the domain of the sender email address. The sender is trusted if the IP address matches one of the mechanisms in the SPF record.
For a more in-depth article about SPF, see our SPF knowledge base.
A term is a matching rule, it can be an IP address or range, but also a dynamic entry that points to another DNS record. For instance: the SPF mx mechanism points to the DNS MX record(s) of the domain.
If the sender IP matches any of the MX records, the sender passes SPF validation.
Most SPF mechanisms can even point to a completely different domain, for example mydomain.com may have an SPF record with term mx:otherdomain.com which points to the MX records of otherdomain.com.
There is also the include mechanism, which is used to include another SPF record.
If the sender IP address matches the included SPF record, it will also match for the domain that includes it.
Regardless of where the mechanisms point to, a simple SPF record can quickly require many DNS queries. Say that mydomain.com has the following SPF record:
v=spf1 mx ~all
For a receiver that receives an email from mydomain.com, it must perform the following DNS queries for SPF validation:
TXT record itself of mydomain.comMX type DNS record at mydomain.comMX type record: query the A or AAAA DNS records for the address in the MX record.That's a lot of DNS queries to make, for a typical domain with 2 MX records this already requires up to 4 queries.
The address argument in the include, a and mx mechanisms allow for decoupling of administrative responsibility.
Email service providers give you an address to include into your SPF record.
The email service provider will manage this SPF record for you.
If the email service provider makes a change to their infrastructure they can adjust their own SPF record, without having to instruct all their customers to change their SPF record.
Even if you manage the network yourself, it will be safer to use the a mechanism than the ip mechanism, because it removes the possibility of forgetting to update your SPF record if you ever change the IP address of your SMTP host.
The problem with flattening may be obvious now: if one of your email service providers makes a change to their infrastructure, it won't be reflected in your SPF record.
Even if it is an SPF term pointing to your own domain: the a mechanism is safer to use than an ip4 or ip6 mechanism, albeit at the expense of an extra DNS query on the receiver side.
This is exactly what happened to our customer: one of their email service providers added a new IP address to their pool. Once every 20 messages or so their load balancer would use this IP address, which would fail SPF validation at the receiver side.
The current SPF standard RFC7208 section 4.6.4 (DNS Lookup Limits) states that evaluating an SPF record should not require more than 10 DNS lookups.
SPF implementations MUST limit the total number of those terms to 10
during SPF evaluation, to avoid unreasonable load on the DNS. If
this limit is exceeded, the implementation MUST return "permerror".
The other terms -- the "all", "ip4", and "ip6" mechanisms, and the
"exp" modifier -- do not cause DNS queries at the time of SPF
evaluation (the "exp" modifier only causes a lookup at a later time),
and their use is not subject to this limit.Note that this limit applies to all mechanisms and modifiers inside the SPF record, the lookup of the initial SPF record itself does not count. So the SPF validation can cost up to 11 queries.
If the number of queries exceeds 10, the receiver is allowed to stop the SPF validation with a permerror error.
If DMARC monitoring is used, the permerror error will be reported by the receiver.
For most domains the limit is not an issue, you'll need a fairly complex SPF record to surpass the limit.
If you want to validate the number of DNS queries that it would cost to evaluate the SPF record for your domain, you can use our SPF record validator.
Yes, you can to some degree, but not by flattening the record.
An SPF record is evaluated from left to right (hence the reason why the all mechanism always comes last). Once the IP address of the sender matches one of the mechanisms in the SPF record, there is no reason to evaluate the rest of the mechanisms.
Here are some practical ways to make it easier for receivers to evaluate your SPF terms:
mx mechanism, it increases the number of required DNS lookups. If you use an externally hosted email solution that has an include term, you probably don't even need the mx mechanism. Remember: the DNS MX records are for email delivery, not necessarily for sending.redirect modifier.ptr mechanism.a entry from your SPF record.To improve email deliverability and reduce spam we must supply email receivers with as much usable information as we can. With a well configured SPF setup, in combination with DMARC and preferably DKIM you supply a receiver with a strong statement on how to treat email from your domain.
Flattening your SPF record does not achieve that goal, it merely reduces the network traffic of the receiver with a few hundred bytes at best, while sacrificing reliability. You risk misconfigurations that would either cause deliverability issues, or force the receiver to treat the SPF validation result less trustworthy.
DNS queries are cheap to perform, so they are rarely an issue for those who operate email services. Unless your SPF configuration requires more than 10 lookups to evaluate, you should not try to optimise by flattening.
If you want to help receivers, give them as much information as you can on how to reliably distinguish legitimate email from spam. This means:
reject), this indicates to the receiver that it should trust senders that pass SPF and/or DKIM in a strict manner.~all in your SPF record, see why Mailhardener recommends SPF softfail over fail.Obviously, you must be sure that your SPF record is complete (contains all senders that are allowed to send email for your domain) before you commit to SPF ~all and DMARC reject.
To do this, you should use DMARC monitoring to validate that all your senders pass SPF and DKIM.