Why Your Emails Get Blocked: Security Protocols and Deliverability Explained

The era of easy email automation is ending.

Microsoft is aggressively pushing "New Outlook," and with it, the death of COM automation for email. For Access developers who have relied on Outlook's object model for years, this change represents more than just a technical hurdle—it's a fundamental shift in how we must approach email integration.

When we send emails via Outlook, we don't need to worry about the intricacies of email deliverability. Outlook handles the authentication, security protocols, and delivery mechanics behind the scenes.

That luxury is disappearing.

As we transition to alternatives like Mailgun, SendGrid, or direct SMTP integration, we're suddenly responsible for understanding and implementing email security protocols that ensure our messages reach their recipients. Without proper configuration, our carefully crafted emails will disappear into the digital void.

Let's explore why your legitimate emails might be getting blocked and how to ensure they make it through.

The Trust Problem: Medieval Messaging in a Modern World

Email was built on trust.

The original email protocols assumed everyone would play by the rules, much like King Arthur assuming everyone would recognize his sovereignty. But as email became ubiquitous, this trust-based system became vulnerable to abuse.

Today's email landscape resembles the French castle from Monty Python, where suspicious guards taunt you and fart in your general direction.

This transformation happened because the original email system had no built-in way to verify who actually sent a message. Anyone could claim to be from your bank, your boss, or your friend.

Modern email security protocols exist to distinguish legitimate messages from imposters.

SPF: The Authorized Courier List

Imagine you're a guard at the castle gate.

A messenger arrives claiming to be from an important domain. How do you know they're legitimate? You check if they're on the official list of authorized couriers.

SPF (Sender Policy Framework) works exactly this way for email.

When a domain owner publishes an SPF record, they're saying, "These are the only servers authorized to send email on behalf of my domain." Receiving servers check if incoming mail came from an authorized server on that list.

If a message claims to be from Bank of America but comes from an unlisted server, it fails authentication—much like someone falsely claiming royal authority. This simple check prevents the most basic form of email spoofing, where attackers send messages pretending to be from trusted domains.

For your emails to pass this checkpoint, your sending server must be included in your domain's SPF record. This is why services like Mailgun require you to add their servers to your SPF record during setup.

Without SPF, anyone could claim to send email from your domain.

DKIM: The Royal Wax Seal

SPF alone isn't enough to secure email.

In medieval times, important messages carried a wax seal impressed with the royal signet ring. This seal verified the sender and ensured the message hadn't been tampered with.

DKIM (DomainKeys Identified Mail) is the digital equivalent of this royal seal.

When a mail server sends a DKIM-signed message, it adds a cryptographic signature using a private key that only the legitimate sender possesses. The receiving server verifies this signature using the sender's public key, published in DNS records.

This confirms two critical things:

• The message truly came from the claimed domain
• The contents haven't been altered during transit

For your emails to pass this authentication step, you need to complete a two-step process:

1. Configure your sending system to implement DKIM signing. Email service providers like Mailgun handle this part for you by generating the private keys and signing outgoing messages.
2. Add the provider's public keys to your domain's DNS records. If you're automating emails on behalf of a client or building an application for them, your client will need to perform this DNS configuration step on their domain.

This two-part setup creates the complete authentication system that verifies your messages as legitimate.

DMARC: The Guard Captain's Instructions

Even with courier lists and royal seals, castle guards need clear instructions.

DMARC ties SPF and DKIM together with policies that tell receiving servers what to do when messages fail authentication. Domain owners can specify whether receivers should:

• Deliver the message anyway (monitor mode; i.e., do nothing)
• Place suspicious messages in the spam folder (quarantine mode)
• Reject failed messages outright (reject mode)

Additionally, DMARC provides reporting mechanisms that work even in monitor mode, helping you identify legitimate messages that are failing authentication and potential abuse of your domain.

For predictable email delivery, implementing a clear DMARC policy is essential as you transition away from Outlook automation.

How DMARC Reports Actually Work

While monitoring mode is valuable, it only works if you actually receive and review the reports. Here's how that process works:

DMARC reports are delivered via email to addresses you specify in your DMARC record. This happens through two reporting mechanisms:

1. Aggregate Reports (rua): These are XML files sent daily containing statistical data about messages claiming to be from your domain. You configure this by adding an rua=mailto:reports@yourdomain.com tag to your DMARC record.
2. Forensic Reports (ruf): These provide details about specific messages that failed DMARC, including content samples. Configure with ruf=mailto:forensics@yourdomain.com.

For example, a DMARC record with reporting might look like:

v=DMARC1; p=none; rua=mailto:dmarc-reports@example.com; ruf=mailto:forensic@example.com

However, these XML reports aren't human-readable without processing. You have three main options:

1. Use a DMARC analysis service: Services like dmarcian, Valimail, or PowerDMARC provide dashboards and analytics.
2. Self-hosted tools: Open-source options like parsedmarc can process reports and load them into databases for analysis.
3. Manual processing: For low volumes, tools like the DMARC Analyzer XML viewer can help interpret individual reports.

The reports reveal which messages are passing or failing authentication, which sending sources are legitimate, and what specific authentication mechanisms are failing. This information is crucial for identifying configuration issues before moving to stricter policies.

Without properly configured reporting and a system to analyze the data, "monitor mode" provides little value beyond maintaining the status quo.

It's important to note that generating and sending DMARC reports is voluntary for receiving mail servers.

While most major email providers participate in this ecosystem, not all email systems will send reports. Your visibility will typically cover the majority of consumer email services, but there may be gaps with some organizational or regional email systems.

When Guards Become Overzealous: False Positives

Sometimes legitimate messengers get turned away at the gate.

While (most) email systems won't demand shrubberies to deliver your emails, they sometimes block legitimate messages for reasons that seem just as random and frustrating.

These false positives occur for several reasons:

• Email forwarding can break SPF authentication
• Email modifications can invalidate DKIM signatures
• Aggressive spam filters might block messages despite passing authentication
• Receiving servers may implement additional checks beyond standard protocols

When troubleshooting delivery issues, verify your configurations, check email headers, test deliverability to different providers, and review DMARC reports for clues.

It's worth noting that policies can vary wildly among email providers.

Gmail, Outlook, Yahoo, and corporate email systems each implement their own interpretation of these standards, sometimes with additional proprietary filtering mechanisms. What passes through one provider's gates might be rejected by another's, making comprehensive testing across multiple providers essential for critical communications.

Here Be Dragons: The Perilous Territories of Misconfiguration

In medieval maps, dangerous areas were marked with "Here be dragons." In email security, certain configurations present similar dangers.

SPF misconfiguration represents the highest risk. When you modify an SPF record, changes can take days to propagate. During this time, errors could block all your emails.

The most dangerous mistake is using the "all" mechanism incorrectly. A record ending with "-all" (hard fail) tells servers to reject any email not explicitly authorized. If you don't include your legitimate sending servers before the "-all" mechanism, you've instructed the world to reject all your email—a mistake that would make even Brave Sir Robin run away.

DKIM offers safer implementation. You can test keys before making them mandatory and quickly revert if something goes wrong.

The safest implementation sequence is:

1. Start with monitoring-only configurations (DMARC p=none)
2. Implement and test DKIM signing
3. Carefully configure SPF
4. Gradually increase DMARC enforcement after confirming everything works

When configuration goes wrong, having a disaster recovery plan is crucial. Keep previous DNS records documented so you can quickly revert changes, and consider using short TTL (Time To Live) values during initial setup to speed up propagation of fixes.

For Access developers transitioning away from Outlook automation to services like Mailgun, the lesson is clear: approach email security configuration with the same caution you'd use when deploying critical database changes to production.

Maintaining Your Kingdom's Reputation

In the world of email, your sending reputation significantly impacts deliverability.

Even with perfect authentication configuration, emails can be blocked if they come from a domain with a poor reputation.

Email reputation is built on sending history, complaint rates, bounce rates, content quality, and technical compliance.

For Access developers implementing email functionality through services like Mailgun:

• Start with low volumes and gradually increase
• Remove bouncing addresses promptly
• Implement feedback loops for complaints
• Ensure content is relevant to recipients
• Maintain consistent sending patterns

Reputation problems can be invisible to senders—you might see perfect authentication results while your messages land in spam folders.

Conclusion

Email authentication represents a balancing act, but not in the way you might think.

Historically, permissive configurations might have seemed safer for ensuring delivery, but this paradigm has shifted dramatically. Today's email ecosystem increasingly penalizes insufficient authentication. Major providers like Gmail, Yahoo, and Microsoft now scrutinize or outright reject messages from domains with weak or missing authentication.

Properly implemented SPF, DKIM, and DMARC aren't just security best practices—they're becoming prerequisites for reliable delivery.

Start with monitor mode (p=none) to understand your email ecosystem before gradually tightening security. Regularly review your DMARC reports to identify legitimate senders that might be affected by stricter policies. As you gain confidence in your configuration, moving to quarantine or reject policies not only improves security but can actually enhance deliverability by establishing your domain's reputation.

The effort invested in proper email authentication configuration pays dividends in both security and deliverability—making it well worth the initial learning curve.

Acknowledgements

• Article title generated with the help of Claude-3.7-Sonnet
• Article excerpt generated with the help of Claude-3.7-Sonnet
• Initial draft generated with the help of Claude-3.7-Sonnet
• Cover image generated by Imagen3