Professional Documents
Culture Documents
Agenda
Introduction to Exchange Server 2007 Troubleshooting Troubleshooting Client Access Server (CAS) Troubleshooting Hub Transport Server (HT) Troubleshooting Mailbox Server (MBX)
Microsoft Confidential
Introduction
Exchange Services Overview Active Directory Provider
Microsoft Confidential
Provides Active Directory topology information to several Exchange Server components This service does not have any dependencies EDGE not dependent
(MSExchangeMonitoring)
Provides a remote procedure call (RPC) server, used to invoke diagnostic cmdlets.
Microsoft Confidential
Microsoft Confidential
Microsoft Confidential
Notifies Hub Transport server located in the Mailbox server's Active Directory to pickup from a sender's outbox Dependent upon AD Topology Service
Microsoft Confidential
cont
10
Microsoft Confidential
cont
11
Microsoft Confidential
12
Microsoft Confidential
What is the AD Provider? Components Who uses the AD Provider? Exchange Active Directory Topology Service AD Topology vs Exchange Topology
13
Microsoft Confidential
The majority of components and services in Exchange 2007 are built on managed code:
Replay Service Exchange Transport Service Mailbox Assistants Search Unified Messaging
Unmanaged components
Information Store Service System Attendant DAV
14
Microsoft Confidential
15
Microsoft Confidential
16
Microsoft Confidential
18
Microsoft Confidential
19
Microsoft Confidential
Exchange Topology
AD sites, site links and costs Subnets VDirs Location of Exchange servers Examples of use: Mail Routing Mapping of Client Access, Hub Transport and Unified Messaging server to the appropriate Mailbox server PF referrals
20
Microsoft Confidential
Agenda
Introduction Overview Locating CAS Configuration and Topology Data Troubleshooting:
Autodiscover Availability Service Offline Address Book Client Access Security Outlook Web Access Exchange ActiveSync
22
Microsoft Confidential
23
Microsoft Confidential
25
Microsoft Confidential
Agenda
Introduction Overview Locating CAS Configuration and Topology Data Troubleshooting:
Autodiscover Availability Service Offline Address Book Client Access Security Outlook Web Access Exchange ActiveSync
26
Microsoft Confidential
IMAP4/POP3
RPC (MAPI)
SMTP
Site A
Encrypted RPC (MAPI) Encrypted RPC (MAPI) Mailbox server roles SMTP over TLS
Site B
27
Microsoft Confidential
Agenda
Introduction Overview Locating CAS Configuration and Topology Data Troubleshooting:
Autodiscover Availability Service Offline Address Book Client Access Security Outlook Web Access Exchange ActiveSync
29
Microsoft Confidential
File system \ClientAccess\PopImap Active Directory configuration container and file system, including the Web.config file \ClientAccess\exchweb\ews IIS metabase Active Directory configuration container Active Directory configuration container File system, including the Web.config file in the \ ClientAccess\Sync folder IIS Metabase Active Directory configuration container and file system \ClientAccess\ IIS metabase
Microsoft Confidential
Agenda
Introduction Overview Locating CAS Configuration and Topology Data Troubleshooting:
Autodiscover Availability Service Offline Address Book Client Access Security Outlook Web Access Exchange ActiveSync
31
Microsoft Confidential
Autodiscover
What Autodiscover does:
Automatically configure Outlook profiles without knowing where the mailbox is located Provide Web Service URLs to Outlook 2007 Clients Use both RPC and HTTPS connection
32
Microsoft Confidential
SCP Objects are accessed by domain joined Outlook 2007 clients to locate AutoDiscover service Non domain joined clients rely on DNS to locate AutoDiscover service
33
Microsoft Confidential
Autodiscover
Configuring Outlook 2007 profiles and Web services URLs
1
Outlook uses e-mail address to locate an Exchange Client Access servers at a pre-defined location (autodiscover.domain.com)
HTTP Request
XML Config
Outlook Anywhere settings Server locations Web service URLs Authentication information OAB download location
AD Lookup 0
34
Microsoft Confidential
AD
Configuration Information
If domain joined, Outlook automatically fills out the users email address and password
Locating Autodiscover
To locate AutoDiscover:
Internal domain joined clients use SCP Non domain joined or external clients use DNS
For Outlook Anywhere and remote clients an host record for Autodiscover server should be created on an external DNS Without AutoDiscover access client can access mailbox but certain functions like F/B, OOF, OAB and UM will not be accessible If AutoDiscover is located via DNS, Outlook will try predetermined order of URLs to connect to AutoDiscover Server. For example:
https://domain.com/autodiscover/autodiscover.xml https://autodiscover.domain.com/autodiscover/autocover.xml
36
Microsoft Confidential
38
Microsoft Confidential
Troubleshoot Autodiscover
Client side
Test E-mail Autoconfiguration Outlook Logging
Server side
Test-OutlookWebServices Event Logs Exchange Management Shell
40
Microsoft Confidential
Results tab: Web service URLs Log tab: URLs used and error codes Popular error codes
80072EE7 ERROR_INTERNET_NAME_NOT_RESOLVED 80072EFD ERROR_INTERNET_CANNOT_CONNECT 80072F17 ERROR_INTERNET_SEC_CERT_ERRORS
Outlook logging
OLKDISC.log in temp directory OLKAS directory
41 Microsoft Confidential
42
Microsoft Confidential
Event logging
Three MSExchange AutoDiscover event categories
\Core \Provider \Web Set-EventLogLevel "MSExchange AutoDiscover\Core" Level:Expert
43
Microsoft Confidential
Agenda
Introduction Overview Locating CAS Configuration and Topology Data Troubleshooting:
Autodiscover Availability Service Offline Address Book Client Access Security Outlook Web Access Exchange ActiveSync
44
Microsoft Confidential
Availability Service
Calendaring functionality for free/busy, meeting suggestions and Out-of-Office (OOF) depends on Availability Web Service Availability Service is used only by Exchange 2007 Mailboxes For Exchange 2007 Mailboxes, Calendar data will be read from users mailbox directly OL 2007/Exchange 2007 users access Exchange 2003 mailbox free/busy data by using Availability Service to look up free/busy Public Folders on Exchange 2003 Servers
45
Microsoft Confidential
46
Microsoft Confidential
47
Microsoft Confidential
Test-OutlookWebServices Cmdlet
Test-OutlookWebServices -id:user1@contoso.com -TargetAddress: user2@contoso.com
48
Microsoft Confidential
49
Microsoft Confidential
Test-OutlookWebServices
Test-OutlookWebService is a diagnostic task to verify AutoDiscover , Availability Service, RPC/HTTP and OAB distribution configuration for connectivity only
Test-OutlookWebServices -Identity <Alias, Domain\User or SMTP address> -ClientAccessServer <FQDN or NetBIOS name> -TargetAddress <Alias, Domain\User or SMTP address>
Returns information about SSL Certificate problems Determines the validity of the returned service URLs The request is made for one day of free busy data and the data is not returned in the task output.
50 Microsoft Confidential
52
Microsoft Confidential
Example:
20070305-110303994-fb.log
54
Microsoft Confidential
MessageText Contains information about the failure. ExceptionCode Contains the exception that caused the failure. ResponseCode Contains the web response code for the failure.
56
Microsoft Confidential
Permissions Error : Engage the target mailbox owner to confirm calendar permissions
<MessageText>Caller does not have access to free busy data. </MessageText> <ResponseCode>ErrorNoFreeBusyAccess</ResponseCode>
57
Microsoft Confidential
Legacy Free/Busy Failures : Public folder store on Exchange 2003 is not mounted or inaccessible
<MessageText>The remote server returned an error: (503) Server Unavailable.. <ResponseCode>ErrorPublicFolderRequestProcessingFailed</ResponseCode >
58
Microsoft Confidential
Agenda
Introduction Overview Locating CAS Configuration and Topology Data Troubleshooting:
Autodiscover Availability Service Offline Address Book Client Access Security Outlook Web Access Exchange ActiveSync
61
Microsoft Confidential
Relies on:
OABGen Exchange File Distribution OAB Virtual Directory Autodiscover
BITS Client does not support self-signed certificates, so by default OAB Distribution Points use HTTP SSL can be enabled with a fully trusted certificate in IIS
62
Microsoft Confidential
64
Microsoft Confidential
Outlook 2007
Make sure Autodiscover works and the URLs are correct Check the OAB distribution on nearest CAS server Check IE proxy settings (KB939765)
Agenda
Introduction Overview Locating CAS Configuration and Topology Data Troubleshooting:
Autodiscover Availability Service Offline Address Book Client Access Security Outlook Web Access Exchange ActiveSync
68
Microsoft Confidential
SSL handshake
SYNC (TCP_Port = 443)
Outlook clicks on URL to access Secure Webmail Server (https://mail.msft.com)
The browser establishes a TCP connection on the HTTPS TCP Port 443
SERVER_DONE
To continue with the authentication process, client should verify the servers certificate
1 2 3 4
Is today's date within the validity period? Is the issuing Certificate Authority (CA) a trusted CA? Does the issuing CA's public key validate the issuer's digital signature? Does the domain name in the server's certificate match the domain name of the server itself? 5 The server is authenticated.
Microsoft Confidential
69
71
Microsoft Confidential
Certificate Request
72
Microsoft Confidential
Outlook Anywhere
74
Microsoft Confidential
This allows OL2007 to get complete the Autodiscover phase of Outlook Anywhere profile creation Domain-joined clients do not display Invalid CA certificate warnings
Note: Self signed certs would generate warnings for end user and we recommend our customers to buy the required certificates before deploying CAS for the end-users.
75 Microsoft Confidential
To fix this:
If the domain name parameter includes Netbios or Server FQDN in the certificate request, the certificate should be enabled for SMTP service. Run Enable-ExchangeCertificate command to enable for SMTP Alternatively, do not use Netbios or Server FQDN in the certificate request, use only Public FQDN
76
Microsoft Confidential
Using new certificate without considering Autodiscover ISA server is not publishing Autodiscover URL
78
Microsoft Confidential
EXTRA Tracing
Enable the following components/tags: Common\Certificate Validation Networking Layer\Certificate Transport\Certificate Tool to troubleshoot transport security problems Used to troubleshoot authentication errors while accessing web service or any other web page Unofficial utility to dump the msExchServerInternalTLSCert value in AD into a readable format
80
Microsoft Confidential
82
Microsoft Confidential
85
Microsoft Confidential
86
Microsoft Confidential
Agenda
Introduction Overview Locating CAS Configuration and Topology Data Troubleshooting:
Autodiscover Availability Service Offline Address Book Client Access Security Outlook Web Access Exchange ActiveSync
87
Microsoft Confidential
/owa
IIS SSL
/exchange /exchweb /public
88
Microsoft Confidential
Authentication Methods
Authentication Method Basic authentication Digest authentication Security Level Low (unless Secure Sockets Layer [SSL] is enabled) Medium How Passwords Are Sent Base 64-encoded clear text Client Requirements All browsers support Basic authentication Microsoft Internet Explorer 5 or later versions Internet Explorer 2.0 or later versions for Integrated Windows authentication; Microsoft Windows 2000 Server or later versions with Internet Explorer 5 or later versions for Kerberos
Hashed Hashed when Integrated Windows authentication is used Kerberos ticket when Kerberos is used Integrated Windows authentication includes the Kerberos and NTLM authentication methods Uses cookies to help secure a user's name and password
High
Forms-based authentication
High
Internet Explorer
90
Microsoft Confidential
Intercept
Owaauth .dll
Redirect to owa/auth/logon.asp
Anonymous GET /owa/auth/logon.asp Return FBA logon page POST including username + password Redirect to /owa + set Auth Cookie
Auth Cookie
91
Microsoft Confidential
Note: Looking for a specific command? Use Get-Help with correct wildcards Example: get-help *OWA*
93 Microsoft Confidential
Check Mapped application for Legacy Virtual Directories (/Exchange, /Public and /EXCHWEB)
CAS Only: exprox.dll CAS + MBX: davex.dll
95
Microsoft Confidential
Registry Keys
Disable LDAP Encryption (Troubleshooting ONLY)
Key: HKLM\SYSTEM\CurrentControlSet\Services\MSExchangeADAccess DWORD: Disable LDAP Encryption Value: 1 = LDAP Encryption disabled
96
Microsoft Confidential
IMPORTANT: Fiddler is a Client Side debugging Tool and should never be installed onto a production Exchange server. Serious problems have been reported with Active Sync when Fiddler is installed on an Exchange 200x server!
98 Microsoft Confidential
EAS
Yes
No
OWA/EAS not available
No
Yes
Best CAS has ExternalURL Set?
Note: CAS to CAS proxying is not supported between virtual directories that use Basic authentication, the virtual directories must use Integrated Windows authentication.
Yes
Note: Redirection is only supported for OWA
99
No
Comments/ Consequences
OWA EAS Web Services used by 3rd party LOB applications Availability Service used by Outlook 2007 Outlook Anywhere WebDAV and OWA 2000/2003
No No
No
No
Yes
Must have a CAS server in each Exchange AD site to use OWA/EAS/Web Services
No
Unnecessary: Autodiscover
Unnecessary: Autodiscover
Yes
Yes, RPC
Not applicable Proxying to legacy E2003 server IMAP/POP clients must access a CAS in the mailbox AD Site directly
Yes, HTTP
No
Not applicable
IMAP4/POP3
No
No
No
101
Microsoft Confidential
Between an Exchange 2007 Client Access Server and an Exchange Server 2003 Back-end server when:
OWA Clients connect to /Exchange virtual directory EAS Clients connect to /Microsoft-Server-ActiveSync virtual directory
103
Microsoft Confidential
104
Microsoft Confidential
106
Microsoft Confidential
108
Microsoft Confidential
110
Microsoft Confidential
Value #>0 #>0 1000000 > # > 0 1000000 > # > 0 Valid Path #>0 #>0
Default Value 1000 200 5000 (in KB) 5000 (in KB) %SYSTEMROOT%\Temp 1000 (in MB) 20 (seconds)
111
Microsoft Confidential
113
Microsoft Confidential
114
Microsoft Confidential
Agenda
Introduction Overview Locating CAS Configuration and Topology Data Troubleshooting:
Autodiscover Availability Service Offline Address Book Client Access Security Outlook Web Access Exchange ActiveSync
115
Microsoft Confidential
PIN reset
117
Microsoft Confidential
118
Microsoft Confidential
Additional resources
Support WebCast: Introduction to AutoDiscover in Microsoft Exchange Server 2007 http://support.microsoft.com/kb/935438 White Paper: Exchange 2007 Autodiscover Service http://technet.microsoft.com/en-us/library /59adba4e-44e1-4aa2-b09d-06988cbeab2d.aspx Autodiscover and Exchange 2007 http://technet.microsoft.com/en-us/library /7c44814d-bb46-4fb8-9b6b-a082be35afdc.aspx Managing the Autodiscover Service http://technet.microsoft.com/en-us/library/aa995956.aspx Exchange 2007 Autodiscover and certificates http://msexchangeteam.com/archive/2007/04/30/438249.aspx More on Exchange 2007 and certificates - with real world scenario http://msexchangeteam.com/archive/2007/07/02/445698.aspx Exchange 2007 Offline Address Book Web Distribution http://msexchangeteam.com/archive/2006/11/15/431502.aspx
120 Microsoft Confidential
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
Mail Submission
Process of picking up a message from a users mailbox and getting it into the Submission queue on a local Hub Transport server
123
124
125
Dumpster
Submission Queue
StoreDriver
Convert to MAPI
RPC
RPC
JET
Outlook Client
STORE
STORE
STORE
126
Dumpster
Convert to MIME
RPC
RPC
Server
Mailbox Server DN, Senders MB MailSubmission GUID, Messages RPC Client EntryID, etc
Outlook Client
STORE
STORE
STORE
127
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
129
130
131
132
133
134
135
136
137
138
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
140
SMTP
Used for communication when messages are relayed between SMTP servers
141
142
144
145
147
148
150
152
153
154
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
156
157
158
161
162
163
164
165
166
167
168
169
170
171
For Hub servers default Receive connector will need to be modified to allow anonymous connections
172
LAB 5 Mailflow
173
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
175
Description
ESE Database file that stores all the queued messages Temp database file used to verify the queue database schema on startup Transaction logs that record all changes to the queue database Temporary transaction log created in advance Tracks the log entries that have been committed to the database Transaction reserve log files. Used when the hard disk drive that contains the transaction log runs out of space to stop the queue database cleanly
176
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
178
179
180
181
182
183
184
185
186
187
188
189
190
PICKUP/REPLAY DIRECTORY
QUEUING
SUBMISSION
AGENTS
RESUBMIT
STOREDRIVER
UNREACHABLE
MAPI
OUTBOUND
191
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
If a queue is or messages are in a retry state look at the Last Error value to help isolate the reason for failure Use the Next Hop Domain value to help determine where the message is being delivered
193
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
SMTP X-AnonymousTLS
Hub to Hub Hub to Edge/Vice Versa
Direct Trust Authentication
198
Microsoft Confidential
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
From the remaining list, pick the best certificate (in order of preference)
Trusted CA Issued Certificate preferred over Self-Signed Newest installed certificate over oldest
An older 3rd Party CA issued certificate would be used over a newer self-signed certificate
203
Microsoft Confidential
Newest valid certificate wins changed in SP1 Does not support Wildcards such as *.fourthcoffee.com changed in SP1
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
Troubleshooting - STARTTLS
Tools Used
Exchange Management Shell TELNET.EXE (is STARTTLS advertised?) TCP.EXE (What certificate is being served?) CertUtil to verify certificates Application Logs
Troubleshooting - STARTTLS
What is the FQDN of the Receive Connector Get-ExchangeCertificate DomainName <FQDN> Get-ExchangeCertificate DomainName <FQDN> | FL * Is there more than one certificate?
Are any certificates issued from a trusted CA? Are the certificates valid? What is the newest certificate? Can we access the CRL Distribution Point? Proxy?
Troubleshooting
[PS] C:\>Get-ExchangeCertificate 91BFBDD870D8928018E22B736922411645218B85 | fl *
CertificateDomains
CertificateRequest IisServices
: {clt-e2k7.fourthcoffee.com, clt-e2k7}
: : {IIS://clt-e2k7/W3SVC/1}
: : : :
Troubleshooting
[PS] C:\Get-ExchangeCertificate 6CC3257C2236DFC88BA40CD9A374C9E53CC18E2B | fl *
CertificateDomains
CertificateRequest IisServices
: {clt-e2k7, clt-e2k7.fourthcoffee.com}
: : {}
: : : :
Troubleshooting
TCP.EXE
Troubleshooting
Certutil Sample
[PS] C:\>certutil -verify certnew.cer Issuer: CN=LON-E2K7 DC=fourthcoffee DC=com Subject: CN=clt-e2k7 Cert Serial Number: 610cbc3c000000000010
dwFlags = CA_VERIFY_FLAGS_DUMP_CHAIN (0x40000000) ChainFlags = CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT (0x40000000) HCCE_LOCAL_MACHINE CERT_CHAIN_POLICY_BASE -------- CERT_CHAIN_CONTEXT -------ChainContext.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
Troubleshooting
Certutil Sample
CRL 15: Issuer: CN=LON-E2K7, DC=fourthcoffee, DC=com 93 a6 c0 1b ad cf 8f 9a 91 3b 6e b5 7e bc 93 ed 53 89 89 5c Delta CRL 16: Issuer: CN=LON-E2K7, DC=fourthcoffee, DC=com 35 9e 39 96 9d e8 08 ce 3c 16 a5 99 d5 aa 28 89 d1 54 db 3e Application[0] = 1.3.6.1.5.5.7.3.1 Server Authentication
cont
::
CertContext[0][1]: dwInfoStatus=10c dwErrorStatus=0 Issuer: CN=LON-E2K7, DC=fourthcoffee, DC=com Subject: CN=LON-E2K7, DC=fourthcoffee, DC=com Serial: 115643e01d0eab874e228cc4545d7e6c d6 80 20 2f 11 ad f2 39 53 b5 92 df c1 5a 26 28 c4 5c e5 90 Element.dwInfoStatus = CERT_TRUST_HAS_NAME_MATCH_ISSUER (0x4) Element.dwInfoStatus = CERT_TRUST_IS_SELF_SIGNED (0x8) Element.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100) Exclude leaf cert: ef af 86 b5 a7 9e 25 51 44 18 98 b6 69 f9 df 62 c5 65 31 66 Full chain: a8 51 bc 17 d8 b4 94 5a 6a 3d b9 01 89 bc c6 63 37 8e 0b ea Issuer: CN=LON-E2K7, DC=fourthcoffee, DC=com Subject: CN=clt-e2k7 Serial: 610cbc3c000000000010 Template: WebServer 06 99 41 18 54 db 2d 8b 2c ae 0a 5d d7 b5 27 54 42 d8 20 0b
The revocation function was unable to check revocation because the revocation se rver was offline. 0x80092013 (-2146885613) Revocation check skipped -- server offline ERROR: Verifying leaf certificate revocation status returned The revocation func tion was unable to check revocation because the revocation server was offline. 0 x80092013 (-2146885613) CertUtil: The revocation function was unable to check revocation because the rev ocation server was offline.
CertUtil: -verify command completed successfully.
212
Microsoft Confidential
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
Used to establish Direct Trust authentication after XAnonymousTLS negotiation between a Hub and Edge server. Used to establish secure LDAP connections from Hub for Edge Synchronization Used to encrypt and decrypt EdgeSynchronization credentials which are stored in the directory. Direct Trust Certificates = Default Certificate
Error:
Internal Transport
Error when Certificate Store cannot be found
Occurs when the Direct Trust certificate has been forcibly removed from the system (e.g MMC)
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
Tools Used:
Exchange Management Shell TELNET Protocol Logs CertLib.ps1
: : : : : : :
clt-e2k7\5 SmtpRelayWithinAdSiteToEdge edgesync - northamerica to internet 4039e148-2af7-4889-9de8-6cf6f1b2716e Retry 1 451 4.4.0 Primary target IP address responded with: 454 4.7.0 Temporary authentication failure. Attempted failover to alternate host, but that did not succeed. no alternate hosts, or delivery failed to all alternate hosts . 9/19/2007 8:45:33 AM 9/19/2007 8:50:33 AM True Unchanged
Is ExchangeServer authentication enabled on the receive connector? Make sure both Hub and Edge show X-ANONYMOUSTLS available
[PS] C:\>gettlscertfromad e2k7-edge1 Running on an Edge Server - pulling cert details from Adam System.DirectoryServices.DirectoryEntry Running on an Edge Server - pulling cert details from Adam (&(objectclass=msExchExchangeServer)cn=e2k7-edge1) Getting Prop Thumbprint Subject ---------------3B1C9C4472B9ED9E9981262F48F164E4EDB02F0D CN=E2K7-EDGE1
Agenda
Troubleshooting Mail flow
Local Delivery & Mail Submission: Architecture Local Delivery & Mail Submission: Troubleshooting Remote Delivery: Architecture Troubleshooting Mail Flow
SP1 Changes
Certificate Logging Get-ExchangeCertificate Domain FOO.COM Opportunistic TLS fallback Direct Trust Certificate
Changing Terminology -Internal Transport Certificate Updated anytime SMTP provided as service Warnings upon update Dont have to re-subscribe w/update on Hub No more 1037/2019 events Attempts to fallback to alternate certificates if cant be loaded
POP/IMAP
Selection now prefers PKI Supports Wildcards
SP1 Changes
Logging Sample
EBDDE4C98F71840199B4256B9368F265A92DDB0C: Rejected. Unable to access the associated private key for the certificate.
Searching for a certificate that has one of the following FQDNs :
A7C7D7B88B767BAA6003A1F0DBE0DF0937210C68: Rejected. Has a key size less than 1024 bits, dropping from consideration.
Considering certificate CA979162AC2854BBE389153D965358379F8CD43E CA979162AC2854BBE389153D965358379F8CD43E: Selected. PKI issued certificate.
Agenda
Databases
Public Folders Recipient Management Distribution Lists & Address Lists Offline Address Book Exchange Search ExMON
ESE System
Transaction Log File
7 15 5 25 4 15 8 4 1 10
Memory
4 8 12 7 17 10 15 8 25 3 4 1
1 5 9
2 6 10
3 7 11
Memory
1001 980
7 15 5 25 4 15 8 4 1 10
1 5 9
2 6 10
3
1001
4 8 12
7 11
Checkpoint File
Exx.chk
Exxnnnnn.log
Storage Groups
Set of all databases that share common log files Separate instance of Jet Up to 50 Storage groups per Server (Enterprise) 5 Databases per Storage Group but 50 maximum databases (Enterprise) Recommendation is to have one database for one storage group
Utilities in 2007
No change compared to 2003!
Eseutil command and options are the same but now can be also used on Hub and Edge transport databases Isinteg command and options are the same
236
Microsoft Confidential
237
Microsoft Confidential
Performance counters
Many counters available to monitor Continuous Replication
238
Microsoft Confidential
Uses half of the Online Defrag Maintenance Window time Will notify corruption via eventlog (-1018, -1022 etc)
121:1
If Read:Freed ratio is greater than 100:1 then the OLD window can be reduced If Read:Freed ratio is less than 50:1 then the OLD window should be increased
Why reduce?
Increase backup window Reduce snapshot/block level differential sizes (DPM v2) Validate that Online Checksum/Page Zeroing can be introduced with current OLM window
Agenda
Databases
Public Folders
Recipient Management Distribution Lists & Address Lists Offline Address Book Exchange Search ExMON
CCR cannot have a public store if there is more than one PF store
Troubleshooting Referrals
Connection status in Outlook Get-PublicFolderDatabase UseCustomReferralServerList CustomReferralServerList Link cost (AD and RGC)
Agenda
Databases Public Folders
Recipient Management
Distribution Lists & Address Lists Offline Address Book Exchange Search ExMON
Recipient Management
Simplified Recipient Provisioning for the Exchange Administrator Support for Split Permissions within a single forest Ability to delegate Recipient management to a lower level administrator Ability to create Active Directory object and mail- or mailbox-enable it Instant-on recipients no need to wait or kick the RUS to stamp objects Rich filtering support includes domain- and forest-wide scoping Allows administrators to see only the objects relevant to them New recipient types plus clear distinction of all recipient types Conference Room and Equipment Mailbox (Resource Mailbox) Policy support for select mailbox settings Ability to apply the same settings to all recipients associated with a policy Unified Messaging, Messaging Records Management, and ActiveSync Recipient Policies still exist but are now called E-Mail Address Policies
Page 253
Page 255
Working with Recipients and ADUC Active Directory Users and Computers (ADUC) is no longer extended to manage Exchange recipients
It is not supported to mailbox-enable user accounts using ADUC when the mailboxes will be housed on Exchange 2007 servers. If there is an Exchange Server 2003 RUS server operational, the ADUC mailbox operation will succeed, so the mailbox will be able to send and receive messages Mailbox is considered legacy and certain features or actions, or properties will be blocked Set-Mailbox -ApplyMandatoryProperties
Page 256
Scoping
Recipient Configuration Center supports domain- and forest-wide scoping Ability to specify which DC Console should connect to Scope is configurable, even down to OU $AdminSessionADSettings session variable (in shell) Domain Scope is default behavior Determined by domain of which the Server is a member: Only recipients (e.g., redmond\evand) in selected domain can be found Referenced recipients (e.g., Membership, Delegate, Owner, etc.) are exempt Reduces issues related to replication Forest Scope can display and find all recipients within the forest Provides a complete view of the GAL
Page 259
Enable/Disable Adds or removes Exchange attributes from existing Active Directory objects Enable adds attributes to an existing Active Directory object mail-enabled or mailbox-enabled Disable removes attributes returning Active Directory object to non-Exchange state StoreMailbox in MDB will fall under mailbox retention and will eventually be purged
New/Remove* Creates or deletes Active Directory objects plus adds and removes Exchange attributes in one step New creates Active Directory object and mail-enables or mailbox-enables the object Default Remove removes Active Directory object. StoreMailbox in MDB will fall under mailbox retention and will eventually be purged -Permanent: removes Active Directory object and StoreMailbox in MDB will be purged immediately (shell only) * Must have Account Operator privileges
Page 262
Managing Mailboxes
Well-known functionality are still there New mailbox Move mailbox Delete mailbox Change Mailbox properties
Page 265
Moving Mailboxes
You can use the Exchange Management Console or the Exchange Management Shell to move mailboxes You can move mailboxes across mailbox databases, across servers, across domains, across Active Directory sites and across forests You can also move mailboxes among different versions of Microsoft Exchange Server (2000/2003/2007 only) Move mailbox is more resilient (Pre-Validation) Exchange Management Shell Command: move-mailbox More options available Note: You cannot use the Exchange Management Console to move mailboxes across forests. You must use the Exchange Management Shell instead.
Page 269
Email Address Enforcement IgnoreRuleLimitErrors cmdlet option Damaged or corrupted messages BadItemLimit cmdlet option Skip errors validation from EMC Move-Mailbox Wizard MfcMapi Isinteg
Page 274
Import-Mailbox
Imports from PST Must run 32-bits console
Restore-Mailbox
278
Microsoft Confidential
Agenda
Databases Public Folders Recipient Management
Exchange 2007 Distribution List can only use Universal group scope
Common Issues
Unable to send to the Distribution Group from users external to the Organization
When Require that all senders are auhenticated flag is set on DL Properties To solve the issue run: Set-DistributionList RequireSenderAuthenticationEnabled $true
Issues
Unable to Edit the Address List Properties (Address List must be upgraded)
If ALs created by using Exchange 2003 Upgrade them to Exchange 2007 to use OPATH filters
Set-GlobalAddressList "Default Global Address List" -RecipientFilter {(Alias ne $null -and (ObjectClass -eq 'user' -or ObjectClass -eq 'contact' -or ObjectClass -eq 'msExchSystemMailbox' -or ObjectClass -eq 'msExchDynamicDistributionList' -or ObjectClass -eq 'group' -or ObjectClass -eq 'publicFolder'))}
Agenda
Databases Public Folders Recipient Management Distribution Lists & Address Lists
An Offline copy of the Global Address List Used by Outlook clients in Offline mode or Cache mode Several versions appeared over time Version 2 appeared in Exchange 5.5
For clients Outlook 98 and later
292
Microsoft Confidential
On all Client Access servers, an OAB virtual directory is created to serve the OAB The Exchange File Distribution Service that runs on the CAS servers is responsible to getting the OAB content from the OABGen server The virtual directory points to the directory %programfiles%\microsoft\exchange server\ClientAccess\OAB In that directory, the different OABs are stored per <guid>
The .lzx files contains the OAB data in V4 format The oab.xml contains metadata for Outlook 2007
Outlook 2007 is configured to retrieve the OAB via the OAB URL that is obtained through AutoDiscover. Otherwise it will download OAB from public folders like all other legacy clients
3/24/2009 | Page 294
Started to become an issue with Outlook cache mode deployments No limit for Public Folder connections OAB throttling to control network bandwidth usage Outlook Random Full OAB Request Timer
Key: HKCU\Software\Microsoft\Exchange\Exchange Provider DWORD: Max Full OAB Download Wait Value: Integer >=1
295
Microsoft Confidential
OAB V4 Improvements
297
Microsoft Confidential
User B
Legend
CAS Server
User A
The Internet
User C
Page 299
OAB generated files are kept within the System Attendant mailbox Deleted files from the mailbox role OAB share will be copied back Deleted files from the CAS web virtual directory will be copied back from the Mailbox OAB share
302
Microsoft Confidential
Outlook clients will connect through RPC to the public folder server holding a replica of the OAB To reduce bandwidth usage you should: Make sure to use OAB V4 Replicate the OAB on a public folder in every Active Directory site holding a Mailbox role Or create an OAB per site and assign the mailbox stores to the local OAB Dont forget OAB Threshold registry setting
303
Microsoft Confidential
Set Diagnostic level Set-EventLogLevel -Identity MSExchangeSA\OAL Generator -Level Expert Read event logs Using the event viewer Using Powershell Get-EventLog Application | Where {$_.Category -eq "OAL Generator"}
309
Microsoft Confidential
Tool to simulate
Client connection to download OAB files from public folder store
Does not yet test web distribution (should be available soon)
If run from the server install CDO 1.2.1 to test MAPI access. Downloadable from:
http://www.microsoft.com/downloads/details.aspx?FamilyID=e17e7f31079a-43a9-bff2-0a110307611e&DisplayLang=en
312
Microsoft Confidential
OAB Generation Errors Exchange server configured to generate the OAB By default it is first Exchange Server in org which is Exchange 2003 in mixed modes In mixed mode:
Move OAB from Exchange 2003 server to Exchange 2007 server Local replicas of OAB on Exchange 2007 server should be successfully replicated All mailbox stores on Exchange 2007 server under Client Settings tab should have Default Offline Address Book associated
314
Microsoft Confidential
CCR cluster Only one node is generating OAB When the node becomes passive OAB is not updated Logs error event 9395 How to Fix:
HKLM\System\CurrentControlSet\Services\MSExchangeSA\Parameters\Se rver-Name\EnableOabGenOnThisNode ="ThisNodeName"
316
Microsoft Confidential
318
Microsoft Confidential
320
Microsoft Confidential
321
Microsoft Confidential
324
Microsoft Confidential
327
Microsoft Confidential
Agenda
Databases Public Folders Recipient Management Distribution Lists & Address Lists Offline Address Book
Exchange Search
ExMON
Exchange Search
Understanding Exchange Search Difference between Exchange Search and store search Unexpected results scenarios Troubleshooting Exchange Search
Microsoft Exchange Server 2007 Search is a feature that allows you to quickly search text in messages through the use of pre-built indexes Indexes occupy approximately 5 percent of the total mailbox database size Kept separately in same location as database files
Performance Enhancements
Used by OWA and Outlook online mode Outlook cached mode uses new client-side search Windows Desktop Search Instant Search goes through attachments in Outlook Can be extended to use any filter in Windows
Page 331
Performance Improvements
Outlook in online mode Exchange Server 2007 Search Indexer and advanced find in Outlook 2007 Faster indexing than Exchange Server 2003 and Exchange Server 2000 New messages indexed in under a minute Small storage tax (~5%) for indexes Indexes/searches message bodies and attachments Uses any filter installed in Windows Can install new filters later Outlook in Cached Exchange Mode On Windows XP, Outlook uses Windows Desktop Search On Windows Vista, Outlook uses Vistas built-in search engine
Page 332
Mailbox
Index
restrictions
* Attachments types that are supported by the installed filters
Step 1
Is the IndexEnabled parameter set to true Step 2 Get-MailboxDatabase |ft Name,IndexEnabled Has the Exchange database been crawled? Step 3 MSExchange Search Indices performance object=0 Run the Test-ExchangeSearch
Step 4
Check Event Viewer Step 5 Source: MSExchangeSearch Indexer restart the Microsoft Search
Step 6
Test-ExchangeSearch
The Test-ExchangeSearch cmdlet creates a message and attachment that only the Microsoft Exchange search can find. Unless a mailbox is specified in the Identity parameter, the message is stored in the System Attendant mailbox. The command waits for the message to be indexed and then searches for the content. The command reports success if the message content is found. The command reports failure if the content is not found after the interval set in the IndexingTimeout parameter has elapsed. To run the Test-ExchangeSearch cmdlet, the account you use must be delegated the following: 1. Exchange Recipient Administrator role -and2. Exchange Server Administrator role and local Administrators group for the target server
Programmatically: use the ResetSearchIndex.ps1 Manually stop the service and deleting the file GetDatabaseForSearchIndex.ps1 When the index directory files are provided, this script returns the associated mailbox database names. GetSearchIndexForDatabase.ps1 This script returns index directories for the specified mailbox database names.
339
Microsoft Confidential
Agenda
Databases Public Folders Recipient Management Distribution Lists & Address Lists Offline Address Book Exchange Search
ExMON
What is Exmon?
Originally developed by Microsoft to understand user load on servers Shows per user activity in details Allows to track down high users
Introducing ExMon
Administrators can view the following using ExMon: IP addresses used by clients Microsoft Office Outlook versions and mode (Cached Exchange Mode versus classic online) Outlook client-side monitoring data CPU usage Server-side processor latency Total latency (network and processing) Network bytes In Exchange 2007 works for all mailbox access
342
Microsoft Confidential
Some Outlook users are complaining about latencies regarding mailbox access RPC Average Latency is high Want to know what Outlook versions are really in use Want to find high RPC activity users Are they in cache mode? Want to know who is working among connected users Determine usage pattern on healthy systems
343
Microsoft Confidential
Viewing ExMon Data Must be viewed on same OS or higher as collected on Windows 2003 Server required to view data from Windows 2003 Server Large files can take a long time to open, use CPU Saving your work Command Line can save any 'By ' without displaying UI File->Save will save all 'By ' views in one .csv By Event (for a given user) can be saved only in UI The Save Icon on the toolbar instructs ExMon to save ETL files captured during Live Capture
Know your environment Establish baseline to compare Detect RPC Average Latency peak using the performance wizard
CPU Time Server Latency Client Latency Foreground Client Latency Network Bytes
Basic Principles
Focus on the most expensive Users or Operations (unless you are troubleshooting a particular user) Statistics are best for expensive operations or LOTS of inexpensive one Look for problem to repeat in ExMon, then tackle Longer captures are better than short ones Expect some expensive operations to happen Full sync of an OST Occasional searches and sorts Trick is to find the ones that happen frequently or really hurt When looking at an individual user Look for patterns of repetition Compare to 'normal behavior'
Terms of Use
2008 Microsoft Corporation. All rights reserved. Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. For more information see Microsoft Copyright Permissions at http://www.microsoft.com/permission Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. The Microsoft company name and Microsoft products mentioned herein may be either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners. THIS DOCUMENT IS FOR INFORMATIONAL AND TRAINING PURPOSES ONLY AND IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, WHETHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT.