Categories: Office 365

Emails between O365 and On-premises do not work

Emails between O365 and On-premises do not work

When sending an e-mail from O365 migrated users to On-premise users the On-premise users  don’t get e-mails.

Failure Message

From: Microsoft Outlook <MicrosoftExchange329e71ec88ae4615bbc36ab6ce41109e@domain.onmicrosoft.com>
Date: 4 Nisan 2014 22:35:30 GMT+3
To: <test@domain.com.tr>
Subject: Undeliverable: deneme

Delivery has failed to these recipients or groups:

User (User@domain.com.tr)
The server has tried to deliver this message, without success, and has stopped trying. 

Please try sending this message again. If the problem continues, contact your helpdesk.
 

User2 ( Company ) (User2@domain.com.tr)
The server has tried to deliver this message, without success, and has stopped trying. 

Please try sending this message again. If the problem continues, contact your helpdesk.
 

Diagnostic information for administrators:

Generating server: DB4PR03MB532.eurprd03.prod.outlook.com
Receiving server: emea01-internal.map.protection.outlook.com (10.47.216.25)
 

User (User@domain.com.tr)
4/4/2014 7:35:30 PM – Remote Server at emea01-internal.map.protection.outlook.com (10.47.216.25) returned ‘550 4.4.7 QUEUE.Expired; message expired’


4/4/2014 7:27:34 PM – Remote Server at emea01-internal.map.protection.outlook.com (10.47.216.25) returned ‘450 4.7.0 Proxy session setup failed on Frontend with ‘451 4.4.0 Primary target IP address responded with: “451 5.7.3 STARTTLS is required to send mail.” Attempted failover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts. The last endpoint attempted was 210.179.31.5:25

User2 ( Company ) (User2@domain.com.tr)
4/4/2014 7:35:30 PM – Remote Server at emea01-internal.map.protection.outlook.com (10.47.216.25) returned ‘550 4.4.7 QUEUE.Expired; message expired’

4/4/2014 7:27:34 PM – Remote Server at emea01-internal.map.protection.outlook.com (10.47.216.25) returned ‘450 4.7.0 Proxy session setup failed on Frontend with ‘451 4.4.0 Primary target IP address responded with: “451 5.7.3 STARTTLS is required to send mail.” Attempted failover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts. The last endpoint attempted was 210.179.31.5:25”

Original message headers:

Received: from DB4PR03MB610.eurprd03.prod.outlook.com (10.141.234.156) by DB4PR03MB532.eurprd03.prod.outlook.com (10.141.235.143) with Microsoft SMTP Server (TLS) id 15.0.908.10; Wed, 2 Apr 2014 19:31:29 +0000 Received: 

from DB4PR03MB610.eurprd03.prod.outlook.com (10.141.233.156) by DB4PR03MB610.eurprd03.prod.outlook.com 

(10.141.234.156) with Microsoft SMTP Server (TLS) id 15.0.898.11; Wed, 2 Apr 2014 12:49:18 +0000 Received: from DB4PR03MB610.eurprd03.prod.outlook.com ([10.141.233.156]) by DB4PR03MB620.eurprd03.prod.outlook.com 

([10.141.233.156]) with mapi id 15.00.0913.002; Wed, 2 Apr 2014 12:49:17 +0000 Content-Type: multipart/mixed; boundary=”_000_2c4cf07ee43e4faab98dc52f068a566fDB4PR03MB620eurprd03pro_” 

 From: test <test@domain.com.tr> To: “User ( Company )” <user@domain.com.tr>, “User2 ( Company )” <User2@domain.com.tr> Subject: deneme Thread-Topic: deneme Thread-Index: Ac9Oce26frtuRTMySYWFyAvAom/lyQ== Date: Wed, 2 Apr 2014 12:49:16 +0000 Message-ID: <2c4cf07ee43e4faab98dc52f068a566f@DB4PR03MB620.eurprd03.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: <2c4cf07ee43e4faab98dc52f068a566f@DB4PR03MB620.eurprd03.prod.outlook.com> x-originating-ip: [78.186.201.28] X-Forefront-Antispam-Report: SFV:SKI;SFS:;DIR:INB;SFP:;SCL:-1;SRVR:DB4PR03MB610;H:DB4PR03MB620.eurprd03.prod.outlook.com;FPR:;LANG:tr;;SKIP:2; MIME-Version: 1.0 X-MS-Exchange-CrossPremises-AuthAs: Internal X-MS-Exchange-CrossPremises-AuthMechanism: 03 X-MS-Exchange-CrossPremises-AuthSource: DB4PR03MB620.eurprd03.prod.outlook.com X-MS-Exchange-CrossPremises-SCL: -1 X-MS-Exchange-CrossPremises-messagesource: StoreDriver X-MS-Exchange-CrossPremises-BCC: X-MS-Exchange-CrossPremises-originalclientipaddress: 78.186.201.28 X-MS-Exchange-CrossPremises-avstamp-service: 1.0 X-MS-Exchange-CrossPremises-antispam-scancontext: DIR:Originating;SFV:SKI;SKIP:0; X-MS-Exchange-CrossPremises-processed-by-journaling: Journal Agent X-MS-Exchange-CrossPremises-ContentConversionOptions: True;00160000;True;; X-OrganizationHeadersPreserved: DB4PR03MB610.eurprd03.prod.outlook.com Return-Path: test@domain.com.tr X-OriginatorOrg: domain.com

Symptoms

When you try to telnet the Office 365 hub transport from Exchange on-premises server it won’t recognize the telnet commands on the SMTP server.

Resolution:

451 4.4.0 Primary target IP address responded with: “451 5.7.3 Must issue a STARTTLS commnd first” Office 365 Hybrid


If you have an Office 365 hybrid configuration you may experience issues sending emails between on premise and cloud users (in either direction).

The Exchange 2013 (or 2010) on premises queue viewer may show:

‘451 4.4.0 Primary target IP address responded with: “451 5.7.3 STARTTLS is required to send mail.” Attempted failover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts. The last endpoint attempted was xxx.xxx.xxx.xxx’

The Office 365 Message Trace Console shows the delivery status of ‘None’
 


Office 365 Message Trace

The errors suggest the TLS connection cannot be made but a TLS certificate IS present and during the Hybrid Connection Wizard the required connectors are automatically created so should not require an additional configuration.

When an email is sent between on premise & cloud (Office 365) users of your SSO domain it is sent across one of the automatically created send connectors. These connectors are secured using TLS.

So, assuming you have ruled out all the normal stuff its now time to get baffled. We know the on premise server can send and receive external email. We also know that the Office 365 service can send and receive email. It is just the email between the two services that does not work.

I was banging my head against a wall for ages until I used Telnet to connect from my on premise Exchange server to Microsoft cloud gateway.

What I got is shown below:


This is not correct. As you can see the server has not recognised the “ehlo” statement and the banner does not “look right”…

A bit of digging around the firewall I noticed that packets were being dropped when TLS was attempted.

The firewall is a Cisco PIX 515. I disabled ESMTP inspection but that made no difference so I discounted this as the cause.

After a lot more digging around and raging I remembered that the PIX was behind another Cisco firewall – this time an ASA 5510. So I accessed this device and sure enough this edge firewall was also inspecting and dropping TLS over SMTP.

Once both firewall were configured not to inspect ESMTP the default configuration that was set by the Hybrid Configuration Wizard started working straight away.

The commands to disable ESMTP inspection are:

pix(config)#policy-map global_policy
pix(config-pmap)#class inspection_default
pix(config-pmap-c)#no inspect esmtp
pix(config-pmap-c)#exit
pix(config-pmap)#exit
Now telnet the cloud server and you should see a correct banner:

moh10ly

Recent Posts

Reset passwords for Active Directory Users

Reset and manage your Active Directory users' Passwords Active Directory is one of the most…

2 years ago

Finding Exchange Database hidden mailboxes. ​

Finding Exchange Database hidden mailboxes. Story:Maybe you have been in this situation before, trying to…

2 years ago

Setting up ADConnect and PTA (Password auth through) servers agents behind proxy

If you're using a Proxy server in your firewall or in your network and have…

2 years ago

Get Report of Active Directory Locked Accounts and Machine they logged in from

Story:I got some clients  that have reported some of their users being locked out and…

2 years ago

Checking and Providing Full and SendAs delegate access on O365 Exchange Online

Delegate Permissions This is a code that I have wrote recently to check if an…

2 years ago

Retrieving attachments from Exchange mailbox using python

Story: I got a request from a client who constantly gets CVs and have to…

3 years ago

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298

Warning: Undefined array key "adsense_ad_type" in /datadrive/wwwroot/www.moh10ly.com/wp-content/plugins/quick-adsense-reloaded/includes/amp-condition-display.php on line 298