What exactly is the Confidence Factor℠?

Question

What is the method/logic behind //email/testTo: ("TestReceiver") testing? What is actually done to arrive at the Confidence Factor℠?. Asking because we are using CheckTLS.com as part of our corporate security policy for vendor and customer email.

Answer

A CheckTLS Confidence Factor reports our expert opinion on the security of the Internet facing portion of an email system. Many of the terms used in this FAQ are explained elsewhere, so a complete understanding of Confidence Factor may require familiarity with email security technologies and CheckTLS, but even without this background knowledge the explanation of Confidence Factor here should make sense.

CheckTLS has four separate Confidence Factors:

The Thru Confidence Factor reports on an email sent from CheckTLS to an email system and then returned from that email system to CheckTLS. It is simply a combination of the Sender and Receiver Confidence Factors (below).

The Sender Confidence Factor reports on an email sent from an email system to CheckTLS. It scores the Sender’s use of TLS, SPF, DKIM, DMARC, BIMI, DNSSEC, and FCrDNS. These email security technologies are described elsewhere, and how CheckTLS takes into account the use and quality of each technology and combines it into the Sender Confidence Factor is proprietary to CheckTLS.

The Receiver Confidence Factor reports on what would happen with an email sent across the Internet to the Receiver’s email system. It scores the Receiver’s use of TLS, MTA-STS, DANE, DNSSEC, and FCrDNS. These security technologies are described elsewhere, and how CheckTLS takes into account the use and quality of each technology and combines it into the Receiver Confidence Factor is proprietary to CheckTLS.

The Domain Confidence Factor reports on the face that a Receiving email system presents to the Internet. It is a combination of the Receiving Confidence Factors for all of the email servers (MX’s) that the domain publishes. It combines these individual Confidence Factors using a proprietary algorithm that takes into account their quantity and preference.

More on the Domain Confidence Factor

If you run //email/testTo: ("TestReceiver") with Output Format set to Detail, you will see the raw data that we use to compute the Confidence Factor.

From a high level, the Confidence Factor is a measure of the security of each MX weighted by the likelyhood of the MX being used.

For example, with an MX, a strong SSL/TLS version counts more than a strong cipher. And for the weighting, an MX with weaker security that is a lower MX priority doesn't hurt as much as a one near the top.

While the actual formula is proprietary, it works like this:
First we score each MX by looking at:

  • the version of SSL/TLS used
  • the cipher strength
  • certificate validity and name match
  • MTASTS (if tested)
  • DANE (if tested)
  • FCrDNS (if tested)
  • DNSSEC (if tested)

    • Then the MX scores are combined into the single Confidence Factor using a weighted average based on the MX's Preference (from DNS) and if it connected (tempered by the IgnoreNoConnect option).