Certificate problems when sending mail via smpt (tream_socket_enable_crypto(): SSL operation failed with code 1)

I know there were already a lot of these issues on this board, but none solved mine, unfortunately. Be assured that I searched quite a lot (for hours) but could not solve it. I hope I have not overlooked a good answer.

This post is somewhat long – I am sorry – but I did want to include some debugging-information since I guess it might be helpful in finding a solution.

Thank you very much for taking the time to read and (hopefully) respond.

Summary: I tried to send mails (from a php script) via smtp. The site uses an external mailserver (MX mail.domain.com → mailgate.otherdomain.com → A to external IP). When trying to send the mail, I get an error: SSL routines:tls_process_server_certificate:certificate verify failed. Trying to connect via SSH results in Verification error: self signed certificate which is true for the webserver, but not for the external mailserver. CheckTLS states: Cert Hostname DOES NOT VERIFY (mailgate.wwwshort.com != localhost.localdomain). I am in charge of the webserver (managed server) but not of the mailserver (externally provided).

The Setup

Website runs on wwwdomain.com – they also have a shorter version, let’s call it wwwshort.com

Mail server is provided by another company and is found on mailgate.wwwshort.com (this is the MX for all (wwwdomain, wwwshort, other domains) the domains. In the DNS for wwwshort.com there is an A record pointing to the IP). There is also an A record mail.* pointing to mailgate.wwwshort.com.

I try to send emails from a craftcms3 installation via smtp. Mailserver is mail.wwwdomain.com

All domains have self-signed certificates. The server behind mailgate.shortdomain.com has a proper SSL certificate which also mentions all the possible domain names (e.g. mail.wwwdomain.com, mail.wwwshort.com, mailgate.wwwshort.com etc).

My client uses this mailserver daily (I guess it is an Microsoft Exchange Server) and has no problems whatsoever.

The Problem

When trying to send mails, I get the following error in my log:

Error sending email: stream_socket_enable_crypto(): SSL operation
failed with code 1. OpenSSL Error messages: error:1416F086:SSL
routines:tls_process_server_certificate:certificate verify faile

I was able (some time ago) to perform the workaround suggested here but I don’t want to do that anymore since it is not really secure.

Troubleshooting

I set up a script using PHPMailer to get a more complete log. This was what I got:

2021-01-23 18:30:30 Connection: opening to mail.wwwdomain.com:587, timeout=300, options=array()
2021-01-23 18:30:30 Connection: opened
2021-01-23 18:30:30 SMTP INBOUND: "220 mailgate.wwwshort.com ESMTP - DSS"
2021-01-23 18:30:30 SERVER -> CLIENT: 220 mailgate.wwwshort.com ESMTP - DSS
2021-01-23 18:30:30 CLIENT -> SERVER: EHLO www.wwwdomain.com
2021-01-23 18:30:30 SMTP INBOUND: "250-mailgate.wwwshort.com"
2021-01-23 18:30:30 SMTP INBOUND: "250-PIPELINING"
2021-01-23 18:30:30 SMTP INBOUND: "250-SIZE 105080012"
2021-01-23 18:30:30 SMTP INBOUND: "250-ETRN"
2021-01-23 18:30:30 SMTP INBOUND: "250-STARTTLS"
2021-01-23 18:30:30 SMTP INBOUND: "250-ENHANCEDSTATUSCODES"
2021-01-23 18:30:30 SMTP INBOUND: "250-8BITMIME"
2021-01-23 18:30:30 SMTP INBOUND: "250 DSN"
2021-01-23 18:30:30 SERVER -> CLIENT: 250-mailgate.wwwshort.com250-PIPELINING250-SIZE 105080012250-ETRN250-STARTTLS250-ENHANCEDSTATUSCODES250-8BITMIME250 DSN
2021-01-23 18:30:30 CLIENT -> SERVER: STARTTLS
2021-01-23 18:30:30 SMTP INBOUND: "220 2.0.0 Ready to start TLS"
2021-01-23 18:30:30 SERVER -> CLIENT: 220 2.0.0 Ready to start TLS
2021-01-23 18:30:30 Connection failed. Error #2: stream_socket_enable_crypto(): SSL operation failed with code 1. OpenSSL Error messages:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed [/usr/www/users/client/test-mail/PHPMailer/SMTP.php line 467]
SMTP Error: Could not connect to SMTP host.
2021-01-23 18:30:30 CLIENT -> SERVER: QUIT
2021-01-23 18:30:30 
2021-01-23 18:30:30 
2021-01-23 18:30:30 
2021-01-23 18:30:30 
2021-01-23 18:30:30 
2021-01-23 18:30:30 Connection: closed
SMTP Error: Could not connect to SMTP host.
Message could not be sent. Mailer Error: SMTP Error: Could not connect to SMTP host.

Now I searched the web and did a lot of investigations. I will list the results here.

Checking CA certificates

I followed the advice from PHPMailer and checked the CA certs by

echo QUIT | openssl s_client -crlf -starttls smtp -CAfile /etc/ssl/cacert.pem -connect smtp.gmail.com:587

This worked perfectly fine.

Now I tried the same with mail.wwwdomain.com:

echo QUIT | openssl s_client -crlf -starttls smtp -CAfile /etc/ssl/cacert.pem -connect mail.wwwdomain.com:587

Here the problem started again:

CONNECTED(00000003)
depth=0 O = NA, CN = localhost.localdomain
verify error:num=18:self signed certificate
verify return:1
depth=0 O = NA, CN = localhost.localdomain
verify return:1
---
Certificate chain
 0 s:O = NA, CN = localhost.localdomain
   i:O = NA, CN = localhost.localdomain
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFMDCCAxigJANm1uWBG8V5qMA0GIb3DQEBDAUAMC0xCzAJBgNV
BAoMAk5BMR4wHAYDDBVQub2WjAtMQswCQYDVQAJOQMBwGA1UEAww
bG9jYWxob3N0LmxvY2FsZG9tYWluMIICIjANBgkqhk0BAg8AMIIC (shortened)
CgKCAgEAyXZvXtT7f
-----END CERTIFICATE-----
subject=O = NA, CN = localhost.localdomain

issuer=O = NA, CN = localhost.localdomain

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2481 bytes and written 483 bytes
Verification error: self signed certificate
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 0225B94178B4DE0B4499DAFB0C0D3AD4BE5519CCFBA4458E1333FF56B56D700D
    Session-ID-ctx: 
    Master-Key: 103159001B6597C40E8C35A31B5DC240AE52D081BEE153A0B904A71C618D235AE5DD21192A784FBD35084130A1A36688
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - 71 f2 0b 8f 85 4b e2 9a-cb bc 21 1f 5a c6 a7 b4   q....K....!.Z...
    (shortened)

    Start Time: 1611427542
    Timeout   : 7200 (sec)
    Verify return code: 18 (self signed certificate)
    Extended master secret: yes
---
250 DSN
DONE

The output was equal for mailgate.wwwshort.com and the IP-Adress of the Mailserver.

Two things caught my attention:

SSL handshake has read 2481 bytes and written 483 bytes
Verification error: self signed certificate

and the mentioning of localhost.local

0 s:O = NA, CN = localhost.localdomain
  i:O = NA, CN = localhost.localdomain

SSL Report

I used the SSL Report from SSL Labs to test the domains.

• wwwlong.com was correctly recognized as "self-signed" and lists only the main domain and www.* It got an B rating.
• wwwshort.com also
• mail.wwwdomain.com was correctly recognized as not-self-signed and listed as "alternative names" all the other possible domains pointing to this server. It got an A+ rating.

Check-TLS

I used CheckTLS to run some tests on wwwdomain.com. It identified the MX server correctly as mailgate.wwwshort.com (with correct IP) and showed the following output.

[000.000]       Trying TLS on mailgate.wwwshort.com[217.86.144.15:25] (10)
[000.099]       Server answered
[000.595]   <‑‑ 220 mailgate.wwwshort.com ESMTP - DSS
[000.595]       We are allowed to connect
[000.595]   ‑‑> EHLO www12-do.checktls.com
[000.691]   <‑‑ 250-mailgate.wwwshort.com
                250-PIPELINING
                250-SIZE 105080012
                250-ETRN
                250-STARTTLS
                250-AUTH PLAIN LOGIN
                250-ENHANCEDSTATUSCODES
                250-8BITMIME
                250 DSN
[000.691]       We can use this server
[000.691]       TLS is an option on this server
[000.692]   ‑‑> STARTTLS
[000.788]   <‑‑ 220 2.0.0 Ready to start TLS
[000.788]       STARTTLS command works on this server
[001.010]       Connection converted to SSL
SSLVersion in use: TLSv1_2
Cipher in use: ECDHE-RSA-AES256-GCM-SHA384
Perfect Forward Secrecy: yes
Certificate #1 of 1 (sent by MX):
Cert VALIDATION ERROR(S): self signed certificate
So email is encrypted but the recipient domain is not verified
Cert Hostname DOES NOT VERIFY (mailgate.wwwshort.com != localhost.localdomain)
So email is encrypted but the host is not verified
Not Valid Before: Jan  9 20:31:46 2021 GMT
Not Valid After: Oct  6 20:31:46 2023 GMT
subject= /O=NA/CN=localhost.localdomain
issuer= /O=NA/CN=localhost.localdomain
[001.013]   ~~> EHLO www12-do.checktls.com
[001.111]   <~~ 250-mailgate.wwwshort.com
250-PIPELINING
250-SIZE 105080012
250-ETRN
250-AUTH PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
[001.111]       TLS successfully started on this server
[001.111]   ~~> MAIL FROM:<[email protected]>
[001.212]   <~~ 250 2.1.0 Ok
[001.212]       Sender is OK
[001.212]   ~~> QUIT
[001.311]   <~~ 221 2.0.0 Bye

Highlighted was this part:

Certificate #1 of 1 (sent by MX):
Cert VALIDATION ERROR(S): self signed certificate
So email is encrypted but the recipient domain is not verified
Cert Hostname DOES NOT VERIFY (mailgate.wwwshort.com != localhost.localdomain)
So email is encrypted but the host is not verified

I wondered whether the self-signed certificate of the webserver caused the problem (since it was a wildcard *.wwwdomain.com initially) so I replaced all the certificates with self-signed ones that only include the main domain. However, the problem persists (although I am not sure whether I just have to wait longer).

My "not-understanding"

To my (not)understanding there are reports on two problems: 1) the certificate is self signed (which is true for the webserver,but not for the mailserver) and 2) the domain names don’t match up.

I am absolutely not an expert on all this. Can anyone tell me whether the problem is on my side (webserver, craft, etc.) or on "their side" (mailserver).