The Sender Policy Framework (SPF) is a method aimed at reducing spam and fraud for email. SPF allows domain owners to publish a policy about which senders are allowed to send email for that domain. Receivers use SPF as one of the methods for spam detection.
The SPF policy is published as a DNS record by the owner of the domain.
SPF is a basic security mechanism for email. It's easy to set up, but it does have some flaws and shortcomings. Additional mechanisms such as DKIM and DMARC complement SPF to allow for better security.
By default, any computer connected to the internet can send email to any email inbox with any sender name.
This means that, without additional countermeasures, anyone could send an email as president@whitehouse.gov.
The Sender Policy Framework (SPF) is a standard that is part of the email ecosystem that aims at preventing this form of email identity fraud. SPF is also used as one of the factors in detecting spam messages.
An SPF policy is a list of senders (computers) that are allowed to send email on behalf of a domain. The policy is published as a DNS record under the domain it applies to.
When an email message is received by an email server, the receiver uses SPF to determine if the computer that sent the message was allowed to do so. If the sender does not pass SPF validation, the message is likely to be rejected, or flagged as spam or fraud.
The SPF standard is defined in RFC7208, which was finalized in 2014.
To understand how SPF works and why it is needed, you must first have a basic understanding of how email works. If you are already familiar with SMTP, you can skip this section.
Email is transmitted over the internet using the Simple Mail Transfer Protocol (SMTP). SMTP is designed around the principles of paper mail. Like with paper mail, SMTP features the concept of an envelope. The envelope contains the address of the intended receiver and the address of the sender to which the mail can be returned if the mail is undeliverable. Inside the envelope is the letter itself, just like on a formal letter it contains the address of the sender and the receiver. Once the email is delivered to the intended receiver (your mailbox) the envelope is discarded.
Sticking with the paper letter analogy: the postman (called the Mail Transfer Agent (MTA) in SMTP) uses the address on the envelope, and the receiver (your email program) uses the address on the actual letter.
The problem is that (just like with paper mail) anyone can write any sender address on the envelope or the letter. Before SPF, there used to be no way for the MTA (the postman) to determine if the letter actually came from the sender address.
| Paper mail | Electronic mail (email) |
|---|---|
| Postal service | Mail Transfer Agent (MTA) - the SMTP server |
| Mailbox | Mail User Agent (MUA) - the email client |
| Envelope | SMTP protocol |
| Letter | Email message |
With SPF the receiving MTA checks if the computer that sent the email was allowed to use the domain name in the address on the SMTP envelope.
If it turns out the sender was not authorized to use the address, the MTA may mark that email as spam or refuse to deliver it to the receiver. The exact action taken by the receiver upon SPF failure can vary. Most receivers use the SPF result as one of the factors in detecting spam. The receiver may still reject an email that passes SPF for other reasons. Vice-versa the receiver may also still accept email that fails SPF.
A domain owner can publish the SPF policy through the Domain Name System (DNS) by adding a DNS record, which we'll call the SPF record.
The SPF record is a DNS record of type TXT that is placed in the (sub) domain that the SPF policy is to be applied to.
The SPF record contains the list of senders that are allowed to send email for the domain the SPF record is placed under.
It also contains a policy on how to process email from a sender that does not match the list.
The SPF syntax is as follows:
[modifier]=[value] {prefix}[mechanism]{:value}An SPF record consists of 1 or more terms, which are separated by whitespace. A term can be a modifier or a mechanism. A mechanism has an optional prefix and value. Modifiers always have a value.
An SPF record must start with the v (version) modifier with value spf1.
So, an SPF record always starts with the text v=spf1.
This allows MTAs to distinguish the SPF TXT record from other TXT records.
A DNS query on a (sub)domain should result in a maximum of 1 TXT record starting with v=spf1. If more than 1 record is found it is not guaranteed which record will be used by the MTA.
Modifiers are used to instruct the receiver on how to use this SPF record.
Modifiers always have a value, the modifier and value are separated by an equal sign (=).
| Modifier | Value | Effect |
|---|---|---|
v |
version | The SPF version that this record should be used for. This modifier is mandatory and should always be at the beginning of the record. Currently, the only supported value is spf1 |
redirect |
domain name | Use the SPF record found at the given domain name for this domain. Setting a redirect value will cause the MTA to ignore any other mechanisms in this record. |
exp |
domain name | Use the txt type DNS record found at the given domain as a human-friendly error text if the email fails SPF validation. Rarely used. |
Read more about the SPF redirect modifier
The mechanism prefix is optional and instructs the receiver what to do with the sender if it matches the mechanism.
If no prefix is set, the + (pass) prefix is assumed.
| Prefix | Name | Effect |
|---|---|---|
+ |
pass | Email from matching host will pass SPF evaluation |
? |
neutral | SPF result on match will not be used, receiver will only rely on other mechanisms (like DKIM) instead. |
~ |
soft fail | Email originating from a host that matches the term will not pass SPF evaluation, but will still be tested for other mechanisms (like DKIM). |
- |
fail | Email originating from a host that matches the term will be rejected, no further mechanism (like DKIM) may be used. |
Note that because the + prefix is implicit, most operators only use the prefix in combination with the all mechanism, but you are free to use prefixes on each term.
For example:
v=spf1 +ip4:1.1.1.1 -ip4:2.2.2.2 ~all(note the +, - and ~ prefixes to each term respectively)
The example policy above will:
1.1.1.1 (the + prefix could be omitted here)2.2.2.2 (DKIM may not be checked)The mechanism that is used to match the sender IP address. Most mechanisms have optional values.
The mechanism and value are separated by a colon (:), if no value is used the colon should be omitted.
| Mechanism | Value | Effect |
|---|---|---|
mx |
domain name (optional) | Match any sender that the DNS mx record resolves to for the domain. If no domain is specified, the current domain is used |
a |
domain name (optional) | Match any sender that the DNS A or AAAA record resolves to for the domain. If no domain is specified, the current domain is used |
ip4 |
IPv4 address | Match any sender that matches this IP address, the value may be one IP address, or an IP range in CIDR notation |
ip6 |
IPv6 address | Match any sender that matches this IPv6 address, the value may be one IPv6 address, or an IP range in CIDR notation |
ptr |
domain name (optional) | Match any sender whose reverse-DNS lookup results in this domain name. If no domain is specified, the current domain is used. Notice: the ptr mechanism is considered dangerous and should no longer be used |
include |
domain name | Match any sender found in the SPF record found in the specified domain name |
exists |
macro | Expand the given macro and perform a DNS query on the resulting value. If the query yields a result the sender matches the mechanism |
all |
Match any sender |
The SPF record is evaluated from left to right, the first mechanism that matches is used.
It is therefore common to end the SPF record with a ~all mechanism to reject any sender that does not match any of the preceding mechanisms.
If no prefix is set, the default prefix + is used, so:
v=spf1 mx a ~allIs the same as:
v=spf1 +mx +a ~allAllow email from this domain's mail exchange (MX), soft-fail all other senders:
v=spf1 mx ~allAllow email from this domain's mail exchange or any service with an IP address in range 10.0.0.1 - 10.0.0.255 and soft-fail all other senders:
v=spf1 mx ip4:10.0.0.1/24 ~allAllow email from this a sender whose IP address matches an address found in the domain DNS A record, or any service that matches the SPF record found at _spf.google.com. Soft-fail all others.
v=spf1 a include:_spf.google.com ~allDeny any email for this domain (no email is expected to be sent by this domain):
v=spf1 -allYou can use our SPF record validator to inspect the SPF settings for your domain.
Performing DNS queries costs the validator (the receiver) resources such as bandwidth, time, CPU and memory. So to avoid 'unreasonable load' on the validator, RFC7208 section 4.6.4 states that evaluation of an SPF policy may not exceed 10 additional lookups. The DNS query for the SPF policy record itself does not count towards this limit.
According to the RFC, a validator (the receiving email system) must not proceed after 10 lookups, and reject the SPF validation with a permerror error.
Additionally, the RFC states that a DNS query of a hostname found in an MX record must not yield more than 10 A or AAAA records.
If a DNS PTR query (reverse-DNS lookup) yields more than 10 results, only the first 10 results are to be used.
Note: the SPF ptr mechanism is considered insecure, it should no longer be used.
We have a separate article where the SPF lookup limit is explained.
SPF was one of the early attempts at reducing spam and fraud. Since its introduction it became apparent that it has some flaws and shortcomings. You should still use SPF, but you shouldn't rely on it solely. We recommend using SPF in combination with DKIM for a better protection against fraud.
We explained that an email has a sender address on both the envelope and the message itself. The address on the envelope and the letter itself can be different, even from different domains. The MTA (postman) uses the address on the envelope, but your email program discards the envelope and displays the sender address that is on the letter.
SPF is checked by the MTA, not your email program. Thus, SPF is only applied to the address on the envelope. A fraudulent sender can simply use a sender address of a domain that they control on the envelope and use a different domain as the sender address on the letter. Because they control the domain on the envelope sender address, they can publish an SPF record allowing themselves to send email for that domain.
Some email services (for example: Gmail) will display a notification in their web interface if the domain in the sender address differs between the envelope and the letter. It will add via <server> behind the sender address.
Receiving MTAs may also apply stricter spam checking on emails where the address on the envelope and the letter differ.
SPF can break certain forms of email relaying (forwarding). We must be clear here that this only affects automated forwarding, not manually forwarded emails by users.
On an email client, if you forward an email it will be wrapped in a new envelope. This is because your email client has already discarded the original envelope. The receiver will see the address of the forwarder as the sender, it is clear that the message inside is forwarded by that person. SPF will not affect this.
However, some mail services offer automated forwarding (sometimes known as 'relaying'), where email is forwarded while retaining the original envelope, and thus the sender address. This may be used when a domain name is no longer in use and all email must be forwarded to a new domain name. This scenario is common for businesses that rebrand to a new name. The original sender address is retained so that the receiver sees the name of the real sender, and can use the reply button to answer the email. The issue here is that this server is now sending email for a different domain (the one in the sender address on the envelope). This server is most likely not allowed to send email for this domain by the SPF policy.
That is why it is recommended to complement SPF with DKIM, as DKIM signatures are stored in the email content (the letter) and not the envelope. So a DKIM signature will remain intact when the email is forwarded.
A sending MTA may conceal its real IP address with a different IP address. This is known as spoofing. By spoofing its IP address a sender may trick the receiver into believing that the email came from a legitimate sender.
The difference between the fail (-) and softfail (~) prefixes are often poorly understood.
In a typical email scenario, the fail/softfail prefix is used in combination with the all mechanism.
Historically, the ~ soft fail prefix was used when domains were not willing to give a strong statement about a sender.
The - fail prefix was then used as a stronger level of rejection.
However, as we now know, SPF will break when emails are relayed (forwarded), which may cause SPF failures as the sender IP effectively changes. This problem has been solved with the introduction of DKIM, which is not affected by relays
But, as we have learned in the 'prefix' section of this article, the fail (-) prefix instructs a receiver to reject a sender completely if it matches the -all term.
With a hard fail, the receiver will immediately drop the email based on SPF, this it will not perform DKIM authentication.
Hence, with the introduction of DKIM (and later DMARC), it is no longer recommended to use -all for typical email scenarios.
It is recommended to use the ~all term instead.
The DMARC specification also warns about this rfc7489, section 10.1
Some receiver architectures might implement SPF in advance of any DMARC operations. This means that a "-" prefix on a sender's SPF mechanism, such as "-all", could cause that rejection to go into effect early in handling, causing message rejection before any DMARC processing takes place. Operators choosing to use "-all" should be aware of this.
So, as a rule of thumb:
For typical email use, domain administrators should use
~all(softfail).The
-all(fail) prefix is not 'more secure' than~all(softfail). The fail prefix has unintended consequences which are not desirable for typical email use.
There is still one use-case for -all, which is to completely prevent a domain from being able to send email.
For more information on that subject, see hardening unused domains.
Most organizations do not send email from on-premise services, but rely on third-party senders instead.
Most third party senders (such as Microsoft 365, Gmail, Mailchimp, etc.) will offer an SPF record to include in the SPF record of the domains they are sending email for.
One consideration here is that including this third-party sender could allow any other user of that service to send email using your domain name.
In practice all large email providers won't allow a user logged in as domain_a.com to send email for domain_b.com, but that is something you'll need to trust the service for.
In other words: by adding a shared email service to your SPF, you must trust this service to never let any of their other customers send email using your domain name.
Hence, why you may sometimes see domains that use ?include (neutral prefix) for third party services, which causes the receiver to take a neutral stance on the SPF result, and use DKIM instead.
Before an administrator switches to neutral SPF, it must ensure the third party sender has DKIM support, and 100% DKIM coverage should be confirmed by using DMARC reports.
Despite the shortcomings described above, it is still recommended to use SPF for legacy email systems. However, domains should not rely on SPF alone, but use additional mechanisms such as DKIM and DMARC to properly harden outbound email for the domain.
For domains that do send email, it is recommended to create the 'all' term with a softfail prefix (~all) and use a DMARC monitoring service such as Mailhardener to make an inventory of all legitimate senders and verify that they pass SPF authorization at the receiving MTAs.
The use of -all (hard fail) is not recommended for domains used to send email, for those the recommended value is ~all.
This is because -all may cause rejection of the email on SMTP level, preventing verification of other authentication methods (such as DKIM) of the email.
The -all term should only be used for domains that do not send email.
See hardening unused domains for more details.
SPF records have a 10 query lookup limit, which unfortunately is way too restrictive. Domain administrator should verify that the SPF record does not exceed the lookup limit.
rfc4406 'Sender ID' was an experimental protocol developed by Microsoft, around the same time as the rfc4408 SPF protocol described in this article. Sender ID never gained any substantial traction, and was soon abandoned. It is now considered obsolete, and should not be relied on in production environments.
The version identifier spf2.0 may lead some to believe that Sender ID is a successor of SPF (which uses the v=spf1 version identifier), but this is not true.
SPF is a basic method of authorizing senders to send email on behalf of a domain.
SPF unfortunately has many shortcomings, both in practicality and security.
Though flexible, SPF has also proven to be confusing for domain administrators to maintain, which led to many misconfigured domains. Due to common misconfigurations, receivers often cannot rely on SPF alone to determine if a sender is authenticated to send on behalf of a domain.
We generally advise domain administrators not to rely on SPF alone, and focus on DKIM coverage instead. However, it is still recommended to implement SPF as a fallback method for legacy email services that are not capable of DKIM signature validation.
On last thing: If you have questions, comments or thoughts on this article, don't hesitate to shoot us an email.
You can also follow and reach us on X @Mailhardener.