Windows server network interface priority, DNS and Exchange “554 5.4.4”

My friend´s Exchange 2013 implementation was experiencing problem with e-mail delivery. E-mails were returned from inside as well as outside of organization with NDR pointing to DNS failure.

I was investigating this issue, but not all e-mails were lost. Only some of those. I knew, that there was a change recently, that new network interface for VPN connection has been added to Exchange server.

Symptoms:

DNS for next hop is not resolvable returning DNS error

“554 5.4.4 SMTPSEND.DNS.NonExistentDomain; nonexistent domain”.

Investigation:

I started to investigate what could cause the problem. It was DNS error, so I started to use NSLOOKUP tool and tried to resolve DNS name for next hop domain. The answer was different, then I thought it would be. I have received Time Out answer from VPN DNS server, even there is another DNS server set in production environment. I realized from my previous Exchange 2010 projects, that priority of network interfaces is added from Highest to lowest based on sequence interfaces are added to system. So for example if you configure Production network interface before Replication, Replication interface will get higher priority and if you use DNS servers in your replication network (for whatever reason), wrong DNS server will answer to your request and this might lead to unexpected failures.

Now the naughty stuff:

If you use IPv6, you are still OK, since IPv6 DNS servers will answer even though Replication interface has higher priority then production one.

If you disable IPv6 on your interfaces, you have troubles. Once interface priority is changed to incorrect order, you will not be albe to at least resolve DNS queries correctly. Incorrect priority might also lead to packet loss.

Solution:

Very simple:

  • Right click network icon in notification area (right low corner) of your server and click Open Network And Sharing Center
  • Click on Change Adapter Settings
  • Press ALT key and select Advanced -> Advanced Settings
  •  In the Advanced Settings window select interface with incorrectly set priority and press arrows to move interface to correct place

Results before priority change

priority_before

priority_before_cmd

Results after priority change

priority_after

priority_after_cmd

Exchange 2010 – All access rights from all mailbox folders

Let’s try this request:22-03-2013 19-42-05
Get all access rights for all mailbox folders.

We should use cmdlet Get-MailboxFolderPermission which requires declaration like alias:\Inbox\folder. So we need only another cmdlet Get-MailboxFolderStatistics and convert slash characters in its folderpath property.

PS] C:\>$folders = Get-MailboxFolderStatistics filip | %{$_.folderpath} | % {$_.replace("/","\")}

[PS] C:\>$folders[7..11]
\folder 0
\folder 0\folder ? test
\folder 0\folder 1
\folder 0\folder 1\ffolder2
\folder 0\folder 2

22-03-2013 20-51-29We should have all folder paths from the mailbox (filip = alias) in $folders variable and be able to use it for Get-MailboxFolderPermission. We can expect few errors for “system” folders such as Top of Information Store, Recoverable Items, Deletions, Versions, Purges but also for folders which contains the slash character in name because of converting.

[PS] C:\>$folders = $folders | %{ Get-MailboxFolderPermission "filip:$_" | select foldername,user,accessrights}`
 | ? { $_.AccessRights -notlike "None" -and $_.User -notlike "*filip*" }

The operation couldn't be performed because 'filip:\Top of Information Store' couldn't be found.
 + CategoryInfo : NotSpecified: (:) [Get-MailboxFolderPermission], ManagementObjectNotFoundException
 + FullyQualifiedErrorId : 28783069,Microsoft.Exchange.Management.StoreTasks.GetMailboxFolderPermission

The operation couldn't be performed because 'filip:\folder 0\folder ? test' couldn't be found.
 + CategoryInfo : NotSpecified: (:) [Get-MailboxFolderPermission], ManagementObjectNotFoundException
 + FullyQualifiedErrorId : 18857875,Microsoft.Exchange.Management.StoreTasks.GetMailboxFolderPermission

The operation couldn't be performed because 'filip:\Recoverable Items' couldn't be found.
 + CategoryInfo : NotSpecified: (:) [Get-MailboxFolderPermission], ManagementObjectNotFoundException
 + FullyQualifiedErrorId : ABE546E,Microsoft.Exchange.Management.StoreTasks.GetMailboxFolderPermission

The operation couldn't be performed because 'filip:\Deletions' couldn't be found.
 + CategoryInfo : NotSpecified: (:) [Get-MailboxFolderPermission], ManagementObjectNotFoundException
 + FullyQualifiedErrorId : 8321BED7,Microsoft.Exchange.Management.StoreTasks.GetMailboxFolderPermission

The operation couldn't be performed because 'filip:\Purges' couldn't be found.
 + CategoryInfo : NotSpecified: (:) [Get-MailboxFolderPermission], ManagementObjectNotFoundException
 + FullyQualifiedErrorId : 65847762,Microsoft.Exchange.Management.StoreTasks.GetMailboxFolderPermission

The operation couldn't be performed because 'filip:\Versions' couldn't be found.
 + CategoryInfo : NotSpecified: (:) [Get-MailboxFolderPermission], ManagementObjectNotFoundException
 + FullyQualifiedErrorId : 8A11CF7B,Microsoft.Exchange.Management.StoreTasks.GetMailboxFolderPermission

Valid objects are in the folders variable:

22-03-2013 20-49-14

Shortly:

  1. $folders = Get-MailboxFolderStatistics alias | % {$_.folderpath} | % {$_.replace(“/”,”\”)}
  2. $folderPermissions = $folders | %{ Get-MailboxFolderPermission “alias:$_” | select foldername,user,accessrights}
  3. $folderPermissions

Exchange – Single public IP address and easy SMTP High availability

This article is about use high availability of very simple kind, when you have single public IP address without possibility to forward traffic to more than one hosts. In my case I have single public IP address in my LAB, I use Steve Goodman´s Exchange 2010 HAProxy (http://www.stevieg.org/e2010haproxy/), which is not compiled for SMTP traffic. I have 2 node Exchange 2013 DAG with CAS/MAILBOX roles on each node.
This configuration simply means, that I cannot use Win NLB, because DAG cannot operate on the same machine as Win NLB does.

Previously

I used single node to route SMTP traffic to and in case of node failure SMTP traffic was held on gateway till the node came up.

Current setup

Well. If I think about DAG itself, it is high available cluster solution for Exchange 201x. For me there are 2 aspects good for SMTP high availability:

  • DAG has its own IP address
  • IP address is assigned to node running Active Manager and quorum

Yes, these 2 things are essential. If I route SMTP traffic to DAG IP, I will have it allways online and available, because if Active Manager is not online and accessible on single DAG node, DAG is in serious problems and most probably some or all databases will not work.

What to do to make it work?

  • On each DAG node create new internet receive connector bound to Frontend Transport Service, which is stateless SMTP proxy (running on Client Access role), routing traffic to Transport Service on Mailbox server role. Bind the connector to DAG IP address.

New-ReceiveConnector -Name “From Internet” -Bindings “192.168.1.55:32” -PermissionGroups AnonymousUsers -TransportRole FrontEndTransport -Usage Internet

Identity                                Bindings                                Enabled ——–                                ——–                                ——-

FRONTEND1\FromInternet                  {192.168.1.55:32}                       True

  • Set receive connectors to access traffic from smart hosts if needed.
  • Set up routing of SMTP traffic to virtual DAG IP address
  • Check firewall to be sure SMTP traffic is allowed to traverse the network
  • The node to which traffic will flow is the one owning DAG IP Address (Active manager). There is info how to determine Active manager in my article: http://ficility.net/2012/09/02/exchange-2010-dag-active-manager-determinemove/

Downsides:

  • SMTP traffic is not load balanced for external traffic

PowerShell – Functions and examples how to work with IP addresses (IPv4)

The IP addreass can be declared by data type System.Net.IPAddress. These examples are equivalent:

[System.Net.IPAddress]$Address = "192.168.23.12"
[IPAddress]$Address = "192.168.23.12"

$Address object contains:

Address : 202877120
AddressFamily : InterNetwork
ScopeId :
IsIPv6Multicast : False
IsIPv6LinkLocal : False
IsIPv6SiteLocal : False
IsIPv6Teredo : False
IPAddressToString : 192.168.23.12

TypeName: System.Net.IPAddress
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object comparand)
GetAddressBytes Method byte[] GetAddressBytes()
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
Address Property long Address {get;set;}
AddressFamily Property System.Net.Sockets.AddressFamily AddressFamily {get;}
IsIPv6LinkLocal Property bool IsIPv6LinkLocal {get;}
IsIPv6Multicast Property bool IsIPv6Multicast {get;}
IsIPv6SiteLocal Property bool IsIPv6SiteLocal {get;}
IsIPv6Teredo Property bool IsIPv6Teredo {get;}
ScopeId Property long ScopeId {get;set;}
IPAddressToString ScriptProperty System.Object IPAddressToString {get=$this.Tostring();}

As can be seen the IPAddress Class has some limitation for advanced tasks, but the following article uses its functionality really nice related to breaking up decimals in low-/high-bytes and creating IP address ranges (http://powershell.com/cs/blogs/tobias/archive/2011/02/20/creating-ip-ranges-and-other-type-magic.aspx)

New-IPRange

function New-IPRange ($start, $end) {
 # created by Dr. Tobias Weltner, MVP PowerShell
 $ip1 = ([System.Net.IPAddress]$start).GetAddressBytes()
 [Array]::Reverse($ip1)
 $ip1 = ([System.Net.IPAddress]($ip1 -join '.')).Address
 $ip2 = ([System.Net.IPAddress]$end).GetAddressBytes()
 [Array]::Reverse($ip2)
 $ip2 = ([System.Net.IPAddress]($ip2 -join '.')).Address
 for ($x=$ip1; $x -le $ip2; $x++) {
 $ip = ([System.Net.IPAddress]$x).GetAddressBytes()
 [Array]::Reverse($ip)
 $ip -join '.'
 }
}

New-IPRange

Do not use  “0.0.0.0”  and “255.255.255.255” because you will wait on 4294967296 addresses :-).

Of course we can specify IP range through  Microsoft.Exchange.Data.IPRange:

LowerBound : 157.144.33.22
UpperBound : 157.144.33.22
Netmask :
CIDRLength :
RangeFormat : SingleAddress
Size : ::1

LowerBound : 157.144.202.0
UpperBound : 157.144.202.127
Netmask : 255.255.255.128
CIDRLength : 25
RangeFormat : CIDR
Size : ::80

TypeName: Microsoft.Exchange.Data.IPRange

Name MemberType Definition
---- ---------- ----------
CompareTo Method int CompareTo(Microsoft.Exchange.Data.IPRange x)
Contains Method bool Contains(ipaddress ipAddress), bool Contains(Microsoft.Exchange.Data.IPvxAddress ipAddress)
Equals Method bool Equals(System.Object obj), bool Equals(Microsoft.Exchange.Data.IPRange other)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
CIDRLength Property System.Int16 CIDRLength {get;}
LowerBound Property Microsoft.Exchange.Data.IPvxAddress LowerBound {get;}
Netmask Property System.Net.IPAddress Netmask {get;}
RangeFormat Property Microsoft.Exchange.Data.IPRange+Format RangeFormat {get;}
Size Property Microsoft.Exchange.Data.IPvxAddress Size {get;}
UpperBound Property Microsoft.Exchange.Data.IPvxAddress UpperBound {get;}

But I would like to  focus on converting numbers to binary and back (http://www.powershellmagazine.com/2012/10/16/converting-numbers-to-binary-and-back/) because it is basic principle for network addressing.

New-IPv4toBin

function New-IPv4toBin ($ipv4){
 $BinNum = $ipv4 -split '\.' | ForEach-Object {[System.Convert]::ToString($_,2).PadLeft(8,'0')}
 return $binNum -join ""
}

New-IPv4toBin

New-CidrToBin

function New-CidrToBin ($cidr){
 if($cidr -le 32){
 [Int[]]$array = (1..32)
 for($i=0;$i -lt $array.length;$i++){
 if($array[$i] -gt $cidr){$array[$i]="0"}else{$array[$i]="1"}
 }
 $cidr =$array -join ""
 }
 return $cidr
}

New-CidrToBinGet-Broadcast (require New-IPv4toBin)

function Get-Broadcast ($addressAndCidr){
 $addressAndCidr = $addressAndCidr.Split("/")
 $addressInBin = (New-IPv4toBin $addressAndCidr[0]).ToCharArray()
 for($i=0;$i -lt $addressInBin.length;$i++){
 if($i -ge $addressAndCidr[1]){
 $addressInBin[$i] = "1"
 } 
 }
 [string[]]$addressInInt32 = @()
 for ($i = 0;$i -lt $addressInBin.length;$i++) {
 $partAddressInBin += $addressInBin[$i] 
 if(($i+1)%8 -eq 0){
 $partAddressInBin = $partAddressInBin -join ""
 $addressInInt32 += [Convert]::ToInt32($partAddressInBin -join "",2)
 $partAddressInBin = ""
 }
 }
 $addressInInt32 = $addressInInt32 -join "."
 return $addressInInt32
}

Get-BroadcastTest-IPinIPRange (require New-IPv4toBin)

function Test-IPinIPRange ($Address,$Lower,$Mask) {
 [Char[]]$a = (New-IPv4toBin $Lower).ToCharArray()
 if($mask -like "*.*"){
 [Char[]]$b = (New-IPv4toBin $Mask).ToCharArray()
 }else{
 [Int[]]$array = (1..32)
 for($i=0;$i -lt $array.length;$i++){
 if($array[$i] -gt $mask){$array[$i]="0"}else{$array[$i]="1"}
 }
 [string]$mask = $array -join ""
 [Char[]]$b = $mask.ToCharArray()
 }
 [Char[]]$c = (New-IPv4toBin $Address).ToCharArray()
 $res = $true
 for($i=0;$i -le $a.length;$i++){
 if($a[$i] -ne $c[$i] -and $b[$i] -ne "0"){
 $res = $false
 } 
 }
 return $res
}

The first IP address is checked regarding IP range (IP,mask or IP,cidr)

Test-IPinIPRange

New-IPv4fromBin

function New-IPv4fromBin($addressInBin){
 [string[]]$addressInInt32 = @()
 $addressInBin = $addressInBin.ToCharArray()
 for ($i = 0;$i -lt $addressInBin.length;$i++) {
 $partAddressInBin += $addressInBin[$i] 
 if(($i+1)%8 -eq 0){
 $partAddressInBin = $partAddressInBin -join ""
 $addressInInt32 += [Convert]::ToInt32($partAddressInBin -join "",2)
 $partAddressInBin = ""
 }
 }
 $addressInInt32 = $addressInInt32 -join "."
 return $addressInInt32
}

New-IPv4fromBinAdvanced examples:

IPAddressExamplesDownload: IPAddressFunctions.ps1 (click for content)

How to load all functions above into Powershell:

FilePathLoadPS1

Exchange – Offline Address Book – OAB download methods, Cached vs Online

Theory

By default OAB is a point in time snapshot of global address list and it is used as cached source of information about Exchange recipients properties. OAB is stored on Exchange servers (see my previous article http://ficility.net/2013/03/04/oab-differences-between-exchange-2010-and-exchange-2013-in-brief/) and downloaded to client once Outlook is configured in Cached mode. I would like to test modes of using address book,
while Outlook is in cached mode. There are several methods to download OAB. These methods depends on registry settings of Outlook (full article here:http://support.microsoft.com/kb/823580)
shortly:

If the following registry key is present (XX.0 means office version – 15.0 for Office 2013), Outlook behaves upon the DWORD value inside:

HKEY_CURRENT_USER\Software\Microsoft\Office\XX.0\Outlook 
HKEY_CURRENT_USER\Software\Policies\Microsoft\Office\XX.0\Outlook

DownloadOAB DWORD supported values:
  0 = The Offline Address Book does not download automatically.
  1 = The Offline Address Book uses the Download Full Items download mode. This is the default setting.
  2 = Download the Offline Address Book in any download mode, but download a differential update in the Header only download mode.
  3 = Always download the Offline Address Book and a differential update in one of the following download modes:
Download Headers and then Full Items
Download Full Items
Download Headers

The goal of this article is to test differences in OAB behavior between modes 0 and 1. I want to see how it looks when:

1) Mode 1 – The Offline Address Book uses the Download Full Items download mode.
a)Download OAB and check
b)Change GAL and test downloading OAB instantly
c)Update OAB and check

2) Mode 0 – The Offline Address Book does not download automatically.
a) Test behavior once mode 0 is configured while old OAB files are still on the client
b) Test behavior once mode 0 is configured while old OAB files are removed from client

Test scenarios

1) Mode 1 – The Offline Address Book uses the Download Full Items download mode.

By default OAB is downloaded from server hosting Active mailbox database with Organizational mailbox. (or in 2010 from CAS server distribution point or in 2007 and 2010 from Public Folders)

a) Download OAB and check

I have opened OAB from my mailbox and result is in the Picture

1-Before change

b) Change GAL and test downloading OAB instantly

Creation of a mailbox doesn´t updtate OAB itsetf. To create mailbox use command

New-Mailbox OAB_Test_o1 -UserPrincipalName OABTestO1@salonovi.cz
WARNING: A script or application on the FRONTEND1.SALONOVI.CZ remote computer is sending a prompt request. When
prompted, enter sensitive information such as credentials or password only if you trust the remote computer and the
application or script requesting it.

cmdlet New-Mailbox at command pipeline position 1
Supply values for the following parameters:
Password: ********

Name                      Alias                ServerName       ProhibitSendQuota
----                      -----                ----------       -----------------
OAB_Test_o1               OABTestO1            backend1         Unlimited

c) Update OAB and check

To update OAB use command

Get-OfflineAddressBook | Update-OfflineAddressBook

Now I have tested if OAB change is reflected in client computer (should not be)

1-After_change_Before_Download

And now I have downloaded new OAB to client and tested again

1-After change and download

2) Mode 0 – The Offline Address Book does not download automatically

One registry key setting and Outlook client will work online from Addres book point of view, BUT! This setting also requires to clean up OAB files from client computer to behave correctly. I will test both possibilities and try to find differneces in behavior.

First I will set up registry key and restart Outlook

In my lab I dont use any special GPO so yhe setting is done via registry key HKEY_CURRENT_USER\Software\Microsoft\Office\XX.0\Outlook

2-Registry change - Added

OAB files are left on the client

a) Test behavior once mode 0 is configured while old OAB files are still on the client

New mailbox should be immediately visible for client without need to download OAB since information should be available online.

b) Change GAL and test if it appears immediately to client

Change in OAB is not visible immediately, because we have OAB files on the client computer and Outlook use those!

New-Mailbox OAB_Test_o2 -UserPrincipalName OABTestO2@salonovi.cz

2-RAfter change - no download!

c) Update OAB automatically by restarting Outlook

If Outlook is restarted, OAB version is checked / downloaded from Exchange server.

2-RAfter change - after download!

AND NOW THE NAUGHTY stuff! OAB is not updated anymore even you download files successfully from Exchange. In this stage Outlook is stuck somewhere between mode 0 and 1 and updates are not received by client.

d) Update OAB and download change to client manually even mode 0 is used

In this scenario I would like to prove, that setting mode 0 is not the only thing to consider to have Outlook work correctly.

  • To update OAB use command
Get-OfflineAddressBook | Update-OfflineAddressBook
  • Download OAB to client and check if changes are displayed to client.

2 - OAB manual download result

 Changes are only reflected if I try to manually download full or incremental copy of address book. OAB is not downloaded during the client startup!

OAB files are removed from client

a) Test behavior once mode 0 is configured while old OAB files are removed from the client

I added new mailbox again, updated OAB, but for now I have removed all OAB-Related files from client. Changes should appear to client immediately.

To remove OAB files

  • locate:
    c:\Users\<USERNAME>\AppData\Local\Microsoft\Outlook\Offline Address Books\
  • Delete folders
    Example: (c:\Users\lelicek\AppData\Local\Microsoft\Outlook\Offline Address Books\6a285982-48d6-43ee-979b-f84dd5b7d989\)
  • Start Outlook
  • After Outlook startup, folders will be re-creaded, but will be empty. It proves we have mode 0 set to Outlook client.
  • open Offline Address Book
  • Newly created mailbox should be here

2 - OAB folders removed change present

 d) Do change in GAL and test if it appears immediately to client

  • Create new mailbox
New-Mailbox OAB_Test_o5 -UserPrincipalName OABTesto5@salonovi.cz
  • Open Offline Address Book and new mailbox should be again there

2 - OAB folders removed change present again

e) Download OAB manually and test if the changes made to GAL after OAB download will be imediatelly visible to client

2 - OAB manual download

Conclusion

Mode 1

  • Conclusion is , that mode 1 works fine and as it should and user will get updated OAB after regularOAB update schedule or after manual run of Update-OfflineAddressBook command.
  • Note that in Exchange 2010 you must restart File distribution service, to distribute updated OAB to WEB distribution points. In Exchange 2013 it is not needed anymore.

Mode 0

  • Use mode 0 only in case, that you do a lot of changes in GAL and you need clients to see changes immediatelly while taking advantages of Outlook Cached connection.
  • Once mode 0 is used, administrator has to make sure, that OAB files will be removed from client computer (for example by logon script / GPO), otherwise user must use manual OAB update via Send/Receive -> Send/receive groups / Download Address Book.
  • If user tries to manual download OAB while mode 0 is used, Online functionality will STOP working from that time until OAB files are deleted again!

Backup Exec – Removing BKF Files By Script

Sometimes we need to remove bkf files from disk manually due to the media set “Keep Data Infinitely – Do Not Allow Overwrite” in Backup Exec. Zbynek has created a script for that purpose and here is my extended version.

# Variables for reporting
$date = get-date -format "dd-MM-yyyy-hhss"
$rep = "D:\RemovedBKFfiles\$date-Log.txt"

# Incremental variables (retention period and source folder)
$retentionIncremental = -7 #(it means seven days back)
$folderIncremental = "I:\BEData"

# WeeklyFull variables (retention period and source folder)
$retentionWeeklyFull = -27
$folderWeeklyFull = "J:\BEData"

# function RemoveBkfFiles
function RemoveBkfFiles ($folder,$retention,$report) {
 $files = @()
 $files = Get-Childitem -literalpath $folder | where-object {$_.Extension -like ".bkf"}
 Foreach ($file in $files) 
 {
 if (($file -ne $null) -and ($file.lastwritetime -gt (get-date).AddDays($retention))){
 $fileName=$file.fullname
 $fileModify=$file.lastwritetime
 $text="$fileName modified $fileModify - Stays on drive."
 $text | out-file -filepath $report -append
 }
 if (($file -ne $null) -and ($file.lastwritetime -lt (get-date).AddDays($retention))){
 $fileName=$file.fullname
 $fileModify=$file.lastwritetime
 $text="$fileName modified $fileModify - Will be deleted - retention is over." 
 $text | out-file -filepath $report -append
 get-item $file.fullname | Remove-Item -force # file is deleted here!!
 }
 }
}

# Calling the function
RemoveBkfFiles $folderIncremental $retentionIncremental $rep
RemoveBkfFiles $folderWeeklyFull $retentionWeeklyFull $rep

The function RemoveBkfFiles expect to get parameters: source folder (location for bkf files), retention period (how old the bkf file should be remove), destination file for txt report. So its calling could be also:

RemoveBkfFiles "I:\BEData" "-7" "D:\RemovedBKFfiles\Test-Log.txt"

Report looks like the following:

Removed-BKF_filesFeel free to update the script according to your needs and let us know if you have any question.

Exchange 2010 – Background Cleanup Failed (Event ID: 10001)

Let me make a short look at the following error:

Log Name: Application
Source: MSExchangeIS Mailbox Store
Date: 3/9/2013 7:21:21 PM
Event ID: 10001
Task Category: Background Cleanup
Level: Warning
Keywords: Classic
User: N/A
Computer: ExCas32.lifa.intra
Description:
Mailbox cleanup could not completely remove the mailbox for user 9e7c5f65-9cdd-44d9-af51-d2b16b786157 (Root Fid 6fa-4FD75D76E9) following mailbox removal. Encountered error 0xfffffae8. Should this message continue to persist for the same mailbox, it may be indicative of a problem that requires further investigation.

This event usually indicates failed cleanup process which cannot clean corrupted fields in the mailbox database. It is mainly related to mailbox moving (either for the source database with SoftDeleted mailbox, the destination database or both of them). It can occur also for the moving which was completed successfully.

How to stop this event?
In many cases this problem has been resolved by dismounting and mounting the database. If there are many databases having the problem restart of IS service or reboot the computer could help also.

How to find the affected database?
It could be achieved through ExchangeGuid from the event 018e1b17-524c-4ee6-958c-2d8721d3714d.

Get-Recipient -Identity "9e7c5f65-9cdd-44d9-af51-d2b16b786157"

If you do not have a success, you can try to use LDAP Query:

Get-ADObject -LDAPFilter "(&(objectclass=*)(msExchMailboxGuid=\65\5f\7c\9e\dd\9c\d9\44\af\51\d2\b1\6b\78\61\57))"

How to specify ExchangeGuid for LDAP Query:

  1. Original GUID from the Event Log: 9e7c5f65-9cdd-44d9-af51-d2b16b786157
  2. Divide it with backslashes: \40\34\19\4C\21\C1\4D\C1\95\1D\65\13\20\57\7A\N0
  3. Invert the first 3 columns (9e7c5f65-9cdd-44d9): \4C\19\34\40\C1\21\C1\4D\95\1D\65\13\20\57\7A\N0

Source: How to find the Object that belongs to a GUID

If you do not have the success again, you can try to re-mount all active/mounted mailbox databases on reported server from the event:

$dbs = Get-MailboxDatabase | where {$_.server -like "ExCas32.lifa.intra"} 
$dbs | Dismount-Database
$dbs | Mount-Database

OAB Differences between Exchange 2010 and Exchange 2013 in brief

I wanted to summarize OAB differences (and what have not been changed) between Exchange 2010 and Exchange 2013 for my next article about OAB update problems, so here it is:

Server side OAB defaults:

In Exchange 2010

  • OAB is generated daily on specified mailbox server at 5AM. To gather current config use
Get-OfflineAddressBook | select identity,server,schedule
  • OAB is generated by MS Exchange System Attendant service and then distributed to CAS servers virtual directories by File Distribution Service
  • PF distribution is also enabled in Exchange 2010 and Exchange 2007 Clients are getting OAB URL from autodiscover service and upon URL clients download OAB from one of the distribution points (CAS servers)

In Exchange 2013

  • New OAB must be generated for Exchange 2013 in coexistence scenario (either Exchange 2007 or 2010)
  • OAB is generated daily in special mailbox “Organization Mailbox” set with persistent capabilities “OrganizationCapabilityOABGen
Get-OfflineAddressBook | select identity,server,schedule (Server attribute is empty in Exchange 2013)
  • The information about Organization mailboxes can be gathered by the following command
get-mailbox -arbitration | select identity,persistedcapabilities | fl

Identity              : domain.com/Users/SystemMailbox{1f05a927-1445-4b2f-9d3c-f5a07705c8cc} PersistedCapabilities : {}
Identity              : domain.com/Users/SystemMailbox{e0dc1c29-89c3-4034-b678-e6c29d823ed9} PersistedCapabilities : {OrganizationCapabilityUMDataStorage}
Identity              : domain.com/Users/FederatedEmail.4c1f4d8b-8179-4148-93bf-00a95fa1e042 PersistedCapabilities : {}
Identity              : domain.com/Users/SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c} PersistedCapabilities : {51, OrganizationCapabilityUMGrammarReady, OrganizationCapabilityMailRouting, 
                        OrganizationCapabilityClientExtensions, OrganizationCapabilityGMGen
,                         OrganizationCapabilityOABGen, OrganizationCapabilityUMGrammar}
Identity              : domain.com/Users/Migration.8f3e7716-2011-43e4-96b1-aba62d229136 PersistedCapabilities : {OrganizationCapabilityManagement}
  • OAB is generated by OABGeneratorAssistant assistant running under Microsoft Exchange Mailbox Assistant service and stored to Organizational mailbox first and then copied to %ExchangeInstallPath%\ClientAccess\OAB\ on the mailbox server, where database hosting Organization mailbox is active
  • OAB generation process is under workload policy management. It is stopped / lowered its priority / increased its priority based on load on the server hosting active mailbox database with Organization mailbox
  • Current configuration of the workload policies can be gathered by the command below
Get-WorkloadPolicy *OAB*
Get-WorkloadPolicy OABGeneratorAssistant | fl

RunspaceId               : a0640926-b38f-42ca-b0ec-793f101c8c30 
WorkloadClassification   : InternalMaintenance 
WorkloadType             : OABGeneratorAssistant 
Name                     : OABGeneratorAssistant 
WorkloadManagementPolicy : DefaultWorkloadManagementPolicy_15.0.505.0
  • Thresholds for the particular level of the workload management can be gathered by
Get-ResourcePolicy | select Identity,InternalMaintenance*

Result is in the Picture:

workload management

Client defaults:

For both Exchange versions are settings for clients similar. Upon result of Autodiscover service client contacts the server hosting the copy of OAB and downloads it.

On Exchange 2010

  • Client contacts load balanced address, which is then redirected to some CAS server

On Exchange 2013

  • Client contacts load balanced address, which is then redirected to mailbox server hosting active database with Organization mailbox
  • If there is recent failover and database with Organization mailbox become active on another mailbox server, OAB files are not present in %ExchangeInstallPath%\ClientAccess\OAB\ and must be extracted from Org. mailbox prior client can download it.
  • If DB stays active on another node during next scheduled generation time, node with active database generates OAB again into organization mailbox

Offline accessible properties:

Default properties included in OAB are same for Exchange 2010 SP3 and Exchange 2013. Properties are up to 1 day old, byt can be accessed even client is offline.

Properties gathered online by default:

It means that these properties are always up to date, but cannot be accessed when client is offline.

  • Custom properties in Active Directory that an administrator has added (for example, the Employee ID of each employee)
  • Organization hierarchy information
  • Group membership information