Email Migration ADMT

1.Introduction and Migration Tool Overview #

Migration Guide Part 1: Introduction and Migration Tool Overview

Carve-out Migration Guide Part 1: Introduction and Migration Tool Overview

The final stage of any carve-out project, where a corporate segment of a large parent company (“Parent”) is being separated to become its own independent organization (“NewCo”), involves the complete migration of all IT resources and services. Basic services such as Active Directory authentication and Exchange email, which were historically handled by the Parent IT department, must be gracefully transitioned to the NewCo’s datacenter without causing any major disruptions. This process inevitably involves accessing and making changes to the Parent’s server infrastructure and this oftentimes poses problems. The nature of carve-out projects is such that once the deal is finalized, the once-helpful Parent IT staff may not prioritize the carve-out project needs above other projects and may be resistant to provide access to their environment. They essentially become a foreign organization.

Whereas many cross-forest migration guides exist online, many require Domain Admin or other administrative rights in the source domain and very little is written about what to do when your source domain admins simply won’t provide the permissions you require. This series of posts aims to address those unique problems faced during a carve-out migration.

This series is derived from West Monroe’s experience with several large-scale carve-out projects involving organizations of hundreds or thousands of employees being separated from their corporate parent. Once the new target domain is created and its supporting infrastructure built-out, the five stage migration process is started. Active Directory and Exchange are always the two major IT services that need to be transitioned. Their respective objects to be migrated are as follows:

Active Directory

  • Users
  • Security Groups
  • User Workstations
  • Group Policies
  • User Permissions

Exchange

  • User mailboxes
  • Distribution Groups
  • Public Folders
  • Outlook Profiles
  • Mobile Profiles

Although there are many different migration tools, most recently been using the following combination of products:

BinaryTree_Logo_Pantone282_600w

BinaryTree CMT Coexistence for Exchange Directory Synchronizer (DirSync)

  • Used to migrate AD Groups and Users
  • Migrates all Exchange-related attributes
  • Does not migrate SID history
  • Does not migrate non-primary SMTP addresses for Distribution Groups

BinaryTree E2E (E2E)

  • Used to migrate all mailboxes
  • Enables the automatic creation of objects required for mail coexistence
  • Does not migrate public folders (without Domain Admin rights)
  • Does not allow for Outlook Profile Updates

MSlogo

Microsoft Active Directory Migration Tool (ADMT)

  • Used to migrate Workstations
  • Used to migrate SID history
  • Does not migrate any Exchange-related attributes

Microsoft Exchange Inter-Org Replication Tool (IORepl)

  • Used to migrate public folders
  • Used to bi-directionally sync public folders, including free/busy

This combination of tools, used in the correct sequence and with the proper preparations, allows for the migration of all Active Directory and Exchange services while minimizing any disruptions to the business. The high-level migration process by which West Monroe uses these tools is as follows:

Pre-migration steps:

  1. Establish and Test Mailflow Coexistence
  2. Install, Configure, and Test all Migration tools

Big-bang migration steps:

  1. Migrate all Groups with DirSync
  2. Migrate all Users with DirSync
  3. Migrate SID history for all Groups with ADMT
  4. Migrate SID history for all Users with ADMT
  5. Migrate and sync public folders with IORepl

Site-by-site, phased migration steps:

  1. Migrate Workstations with ADMT
  2. Migrate Mailboxes with E2E

Post-migration steps:

  1. Disable Mailflow Coexistence
  2. Change public MX records
  3. Establish public autodiscover records
  4. Remove Trust

In part 2 of this guide, I will be discussing the steps required to establish and maintain Exchange mail-flow coexistence between the Parent and NewCo organizations.

2.Establishing Exchange Mail-Flow Coexistence #

Part 2: Establishing Exchange Mail-Flow Coexistence

In the first blog post of this series, I provided an introduction on how companies undergoing carve-outs can effectively migrate of all their IT resources and services to a new entity. In this blog, I discuss  the steps required to establish and maintain Exchange mail-flow coexistence between the Parent and NewCo organizations.

In lieu of a big-bang-style weekend migration where all mailboxes are migrated at once, cross-forest Exchange coexistence is required. Because MX records cannot be changed until all of the migrations are complete, the coexistence design must involve all mail flowing through the Source Exchange environment. Once the mail comes in, it must go directly to the inboxes of those users who have not yet been migrated or get routed to the Target domain for those users who have been migrated. This could be achieved by changing the receive connector for TargetX.com to Internal Relay on both the Source and Target domains, however this could also cause a mail loop when an email is sent to a user who does not exist in either domain.

The solution, therefore, is to use Mail Enabled User objects (MEUs) on the Source domain to essentially forward mail for migrated users. When a user is migrated, the Mailbox object is replaced with an MEU having an external email address of TargetX.net, which is a previously unused email domain. This external email address is essentially a forwarding address, so rather than delivering the email to a mailbox, Exchange sends it to the external email address. This domain is also not made public via MX records, however, so a send connector for TargetX.net also needs to be created on the Source hub transport servers in order to route all mail sent to TargetX.net to the Target Exchange servers. The overall coexistence design is illustrated below:

Kyle blog post

The use of MEUs on the Source domain will therefore enable cross-forest mailflow for migrated users. The only additional consideration to make is migrated users replying to old emails or using their auto-complete (nickname) cache in Outlook. Both of these actions use the Distinguished Name (DN) of the recipient instead of the standard SMTP address. In order to accommodate this functionality, MEUs for all users will also need to be created on the Target domain, and they must contain X500 records which match the DN from the Source environment and also have an external email address of SourceY.com. The easiest way to “stamp” these MEUs with the proper X500 record is using powershell, but the syntax will depend on the DN format in the Source domain. Essentially these MEUs are used to translate the DN back into a SMTP email address so that it can be properly routed using either a send connector to the Source domain or via MX records. Either way will work, however the use of a send connector will improve mailflow efficiency.With MEUs created on both sides, mailflow coexistence should be functional. Here is a mailflow matrix which can be used to test all possible cross-forest mailflow scenarios.

Picture1

I will be discussing the permissions and access requirements of the selected migration tools and how to best approach these requests with the Parent organization.

3.Migration Access Requirements #

Part 3: Migration Access Requirements

In Part 1 of this 4 part series, I provided an introduction on how companies undergoing carve-outs can effectively migrate of all their IT resources and services to a new entity. In part 2, I discussed the steps required to establish and maintain Exchange mail-flow coexistence between the Parent and NewCo organizations. Here, we take a deeper dive into migration access requirements.

When requesting access to systems within the Parent company’s IT organization during a carve-out, it’s always best to consolidate the access requests into a single, summarized document. It can appear a bit overwhelming, but going back to the Source administrators with a string of requests and follow-ups such as “Oh and could we also open port 80?” can hurt your credibility and make the admins question what the final limit to your access requirements is going to be. Our experience is that it’s best to “rip the band-aid” and have all of the difficult conversations up-front. Identifying any access requirements that will not be accommodated is also important because it will require a change in Migration tool selection or strategy.

The request requiring the most amount of effort from the Source admins is to isolate all users, groups, and computer objects being migrated in their own Organizational Unit (OU) and mailbox database. From there, a service account must be created (svc_sourcemigrator) with full control and exchange admin rights over that subset of users. This is required for some tools to function, but is also very useful when you need to make changes to objects in the source environment (such as adding SMTP addresses to existing users, creating/modifying MEUs, creating test users, etc.).

Assuming that this first request is granted, here are the remaining access requirements for each migration tool:

Provision Migrator accounts for Binary Tree (E2E) and ADMT, Provision Required Access Rights

  1. Add ADMT Migrator account (Target\svc_targetmigrator) to the Source Local Domain Builtin\Administrators group. (This is not absolutely required and will likely get denied as it’s essentially equivalent to Domain Admin; however it will make things easier if the Source admins will allow it.)
  2. Delegate ADMT Migrator account (Target\svc_targetmigrator) “Read all user information” permission on the OUs containing all computers and users to be migrated.

Create Firewall Exceptions for Binary Tree (E2E) and ADMT

  • Open ports for the ADMT console (From Target network to Source DCs):
    • 389 (TCP): LDAP
    • 1024-5000 (TCP): RPC
  • Open ports for the E2E console: (From Target network to DCs and Exchange):
    • 808 (TCP): Mailbox Replication Service
    • 53 (TCP): DNS
    • 135 (TCP): RPC End Point
    • 389 (TCP): LDAP
    • 3268: LDAP
    • >1024 (TCP): MAPI
    • 88 (TCP): Kerberos
    • 445 (TCP): Microsoft-DS Service
    • 443 (TCP): HTTPS
    • 80 (TCP): HTTP

Domain prerequisites

  • Forest trust configured to use domain-wide authentication.
  • Create a domain local security group in Users container called SOURCE$$$.
    • This specific group (with this specific name) is required for ADMT to audit SID history operations.
    • Members should not be added to this group, otherwise SID history migration will fail.
  • Disable SID filtering to allow for SID migration.
    • Netdom trust SOURCE /domain:Target.net /quarantine:No /userD:SOURCE\<DomainAdministratorAcct> /passwordD:<DomainAdminPwd>
  • Configure Audit Account Management.
    • Open the Default Domain Controllers group policy.
    • Browse to Computer Configuration\Policies\Windows Settings\Security Settings\Local Policies\Audit Policy.
    • Select Audit Account Management and enable for both Success and Failure.
    • Select Audit Directory Service Access and enable for Success.
    • Click OK and close the Group Policy.
    • Run gpupdate /force on all domain controllers in SOURCE.

 Exchange Prerequisites

  • Run an exchange powershell command to give the E2E migrator account access to the Exchange 2007 metadata
    • Get-Exchange Server | where {$_.IsClientAccessServer -eq $TRUE} | ForEach-Object {Add-ADPermission -Identity $_.distinguishedname -User (Get-User -Identity svc_migrator  | select-object).identity -extendedRight ms-Exch-EPI-Impersonation}

End-user workstation Prerequisites

  • Push the following patch to all user workstations to be migrated:
  • Enable connectivity to all end user workstations (either disable Windows Firewall via GPO or push the file and print sharing firewall exception).

With all of the prerequistes in place, pre-migration staging can begin. Please stay tuned for Part 4 of this series where I’ll document the pre-migration staging process.

4.Pre-Migration Account Staging #

Part 4: Pre-Migration Account Staging

In Part 1 of this 5 part series, I provided an introduction on how companies undergoing carve-outs can effectively migrate all of their IT resources and services to a new entity. In Part 2, I discussed the steps required to establish and maintain Exchange mail-flow coexistence between the Parent and NewCo organizations. In Part 3, I covered all of the access requirements to perform a cross-forest migration. Here, I’ll cover the first stage of migration, which I refer to as “pre-migration account staging.”

Rather than waiting for phased migrations to migrate all Active Directory (AD) groups and users, these objects should be migrated (copied, not moved) to the target domain before the first site migration. There are several reasons to take this approach:

  • In general, you should try and minimize the number of steps required for each site migration.
  • Mail-Enabled Users (MEUs) for each user must be created in the Target domain in order to facilitate mailflow coexistence.
  • Assuming that Domain Admin access to the Source Domain is not granted, migrating Security Identifier (SID) history will require scheduling a screen sharing session with a Source Domain Admin, so it’s much more efficient to migrate it all at once.
  • Migrating Users and Groups has no effect on the AD objects in the Source environment, so the operation is entirely non-impactful.

However, this approach also has some special considerations:

  1. When having AD accounts in both the Source and Target domains and having a trust in place, users could potentially log on to their machine with their Target account before their migration date, causing a new local profile to be created and major confusion to occur (“all the files that were on my desktop are gone!!”). This is prevented by not migrating user passwords and not telling the users what their new password is until they’ve been migrated. This is also a good approach from a user experience standpoint, helping to drive the point that this is a new company with a new network and a new password.
  2. Once all users are migrated with SID History, any new hires created on the Source side will need to be migrated to the Target side before their migration date. This is a hassle because it requires a screen sharing session with a Source Domain Admin. Alternatively, you could also shift the onboarding process over to the Target domain directly after the pre-staging migration and ensure that new users are instructed to log on to the Target domain. Keep in mind that in addition to creating an AD object and Mailbox, a MEU will also need to be created for each new user on the Source domain.
  3. Once Groups have all been migrated, any changes to Group memberships (including Distribution Groups) on the Source domain will not be reflected in the Target domain. This can be resolved easily by re-migrating all of the groups with the Active Directory Migration Tool (ADMT) with the merge option. This does not re-migrate SID history and therefore does not require a Source Domain Admin to be involved.

With these considerations taken into account, the pre-migration account staging is executed as follows:

STEP 1: Migrate Groups and Users with DirSync
Although Groups and Users will also be migrated with ADMT, it’s important to use DirSync (please see Part 1 of my series for an explanation of DirSync and other migration tools used) in order to bring over Exchange Attributes for Users and Mail-Enabled Universal Groups. DirSync will automatically create an MEU for each migrated user. Migrate the users and groups to a temporary staging Organizational Unit (OU), I called mine “Group Staging.” This will help you find any groups whose SID history was not migrated successfully.

Special Notes on Using DirSync
Although the tool is relatively straightforward to configure and use, there are some special considerations to keep in mind. Once the settings are saved and the migration is ready to begin, you can start the tool with the “CMT Coexistence for Exchange” Service. However, this will cause the migration to occur over and over again on a set time frame. This is a feature to supposedly allow for constant syncing between domains, but there are a couple of bugs which make this feature impossible to use. The first is that any SMTP addresses added to MEUs in the Target domain will be erased every time that DirSync runs. This is a major issue because you want to add Target.net to all users in order to enable mailflow coexistence. The second issue is that any Users with a “Manager” attribute will fail to migrate due to a big in the software. The workaround is to do the following:

On the BinaryTree Server, open a command prompt and navigate to the CMT Coexistence folder.
Run Binarytree.coexistance.dirsync.exchange.exe /repush
This will pull all of the Users from the Source domain and place them in the DirSync SQL Database
On the SQL Database, open the DirSync database and run the following command
update BT_Person set Manager = null
This will clear all manager attributes for all users.
Then, back on the BinaryTree server, run Binarytree.coexistance.dirsync.exchange.exe /repull
This will push all of the Users from SQL to the Target domain

STEP 2: Migrate Groups and Users with ADMT
ADMT is used with the merge option to migrate SID History. Please see the following guide for step-by-step instructions: http://portal.sivarajan.com/2011/05/user-account-migration-and-merging-part.html

If you’d like to keep the same password for every user as set by DirSync, make sure that “Do not update passwords for existing users” is checked. The option to move merged groups to the target OU should also be selected, with the final OU for groups and users targeted. This will allow you to check the staging OU for any Groups or Users whose SID history was not successfully migrated.

Special Notes on Using ADMT
When SID history is selected for migrating, ADMT performs two pre-checks on the Source domain to ensure that the SID history can be migrated. It checks for and (if necessary) creates a group called SOURCE$$$ (where SOURCE is the name of the Source domain) and checks for the two auditing settings mentioned in part three. In order to perform these checks, the account running ADMT must be a user of the Builtin/Administrators group in the Source domain. Source domain admins will likely be unwilling to provide you with this level of access. Therefore, ADMT must be run with the account of a Source Domain admin. In my case, this was achieved by scheduling a screen-sharing session with an admin in order for him to input his password when prompted and supervise the use of his credentials. In order for that to work, the Source admin also had to be added to the local administrators group on the ADMT server, the Builtin/Administrators group in the Target domain (only temporarily, for migrations) and also a member of the dbowner group on the ADMT SQL Database. With all of those permissions in place, the Source domain admin should be able to successfully Migrate SID history for all Users and Groups. The last issue to watch out for is that having the Source Admin input his password via screenshare caused some issues for me. The shift key did not work properly, so the Admin had to temporarily change his password to something all lowercase.

STEP 3: Sync Public Folders and Free/Busy information Using InterOrg Replication (IORepl)
At the time of this project, the public folder syncing feature of BinaryTree E2E required Source Domain Admin credentials in order to function properly. This was not allowed by the Source Domain admins, so IORepl had to be used as a substitute. This was not ideal, as it required significant efforts on the part of the Source Domain Admins and the process of troubleshooting the system was left almost entirely to them. It did eventually work, however. IORepl was configured according to this guide:
http://www.msexchange.org/articles-tutorials/exchange-server-2010/migration-deployment/deep-dive-into-rich-coexistence-between-exchange-forests-part12.html

STEP 4: Prepare Mail Enabled Users (MEUs) for Migration
All MEUs must have a @Target.net SMTP address in order for the email migration to work successfully. This can be added pretty easily with Powershell:

Get-MailUser -Resultsize Unlimited| foreach {
$NewSMTP = $_.PrimarySMTPAddress.Local + “@target.net”
Set-MailUser -EmailAddresses @{add=$NewSMTP}
}

With those four steps successfully completed, phased site migrations can begin which will be covered in part 5 of this series.

5.Phased Migrations #

Phased Migrations

In Part 1 of this 5 part series, I provided an introduction on how companies undergoing carve-outs can effectively migrate all of their IT resources and services to a new entity. In Part 2, I discussed the steps required to establish and maintain Exchange mail-flow coexistence between the Parent and NewCo organizations. In Part 3, I covered all of the access requirements to perform a cross-forest migration. In Part 4, I detailed the “pre-migration staging” steps required before migrations can begin. Here, in the final part of the series, I’ll document the steps involved in each phased migration.

With all of the preparation work complete, it’s finally time for the fun part: actual migrations. When dealing with more than 50 users, it usually makes the most sense to break the users into chunks and migrate each chunk in separate phases. This is largely due to support requirements; no matter how much preparation or testing is done beforehand, some users will encounter issues post-migration. It’s typical to have users who have trouble logging in, or some third party applications that don’t handle the user profile migration well. IT support staff should be expecting a larger-than-normal call volume on the morning after each migration and ideally they will be on-site to provide desk-side support.

With these considerations taken into account, the phased migrations are executed as follows:

STEP 1: Prepare to Migrate Workstations
The first step is to identify the list of machine names to be migrated and ensure that it matches a physical inventory taken by the on-site resource. It is very helpful for the on-site resource to create a map of where all of the machines are in order to quickly find any computers that are having issues. In order to minimize any failed migrations, a pre-flight check should be executed to check for any issues known to cause problems with ADMT migration. I’ve used a pre-flight script that checks the following:

  • Communication via ping
    • Any computers not responding need to be tracked down by an on-site resource to determine the cause. They are typically asleep or powered off.
  • Free Disk Space
    • ADMT requires 1 GB of free space in order to perform the migration.
    • Additionally, if the workstations are being patched post-migration (recommended), then there must be enough free space to accommodate all Windows Update packages.
  • Necessary Windows XP hotfixes
    • If any of the following hotfixes are missing on Windows XP, the workstation migration will fail.
      • KB944043 – Netlogon Update for Environments with RODCs (required in order for ADMT to migration XP systems, even if RODCs are not deployed) http://support.microsoft.com/kb/944043
      • KB944505 – DFS Client Update http://support.microsoft.com/KB/944505
      • KB968730 – Crypto Engine Update to Support SHA256 Certificate Enrollment http://support.microsoft.com/kb/968730
      • KB2535512 – Security Update for DFS Client http://support.microsoft.com/kb/2535512

Once all machines are responding, patched, and showing enough free disk space, all computers should be restarted via script in order to clear any locked profiles. Once all machines are back online, the migration can begin.

STEP 2: Migrate Workstations
Workstation migrations should be performed all at once. Please refer to this guide for a step-by-step migration guide: http://blog.thesysadmins.co.uk/admt-series-11-computer-migration-wizard.html.

The “Post-Check” status is the ultimate sign of success or failure. “Failed” or being stuck at “Not Started” are both indicators that the migration failed. The potential reasons for failure, in order of likelihood, are the following:

  • The required hotfixes are not installed (for XP)
  • Target\Svc_targetmigrator is not a local administrator
  • The wireless adapter or secondary NIC has not been disabled
  • Windows Firewall has not been disabled
  • The remote registry, server, workstation, and/or netlogon services are not running

If all of those potential issues are addressed and the workstation still will not migrate, check the logs on the ADMT server (C:\Windows\ADMT\Logs) to try and determine the root cause.

It’s important to note that migrating AD Computer objects from the Source domain does not disable or remove the Source objects (although that is an option) so it’s possible for users to log on to the source domain from their machine post-migration. This will cause a new local user profile to be created (because the original profile was migrated) and cause some confusion for the user. In order to eliminate this issue, consider setting ADMT to disable Source accounts as they’re migrated. If that’s not an option, communicate to users that they must log on to the Target domain and run a script post-migration to set the default domain accordingly. For an example of such a script see this article: http://support.microsoft.com/kb/555050.

STEP 3: Migrate Email
Email migrations for all users at each site should be performed all at once. Please refer to the BinaryTree E2E documentation for migration instructions. Be sure and set the “Bad Item Limit” to 50 to avoid any corrupt emails causing the migration to fail. The details of any failed migrations can be seen in the Application Event Viewer. The potential reasons for a failed mail migration are, in order of likelihood:

  • The mail-enabled user on the Target Exchange servers does not contain @target.net as an SMTP address.
  • The “Bad Item Limit” is not set high enough.
  • The mailbox being moved is larger than the target database will allow.

If all of these potential issues are addressed and the mail still will not migrate, dig through the event viewer on the E2E server to try and determine the root cause.

As E2E migrates mailboxes, it automatically creates MEUs on the Source domain to enable coexistence mailflow (as discussed in Part 2).

STEP 4: Reconfigure Outlook and Activesync Devices
With all of the workstations cut over to the Target domain, Outlook clients will no longer be able to communicate with the Source Exchange servers. This is a non-issue for Outlook 2007 and above, as those user profiles will try and autodiscover their new mail server and succeed in finding the Target Exchange servers. For Outlook 2003, however, the user profile has to be updated manually.

BinaryTree offers an Outlook Profile Update Service (OPUS), but it is not designed for cross-forest migrations and therefore will not work. You can deploy a .PRF file to update local Outlook profiles, but this will cause a new profile to be created and therefore cause every user to re-download their .OST file (assuming they are all using Cached Mode). This could potentially put too much load on network site links.

The best solution uses what I have termed “the poor man’s autodiscover” and involves editing the hosts file of each workstation running Outlook 2003. I’m normally no fan of adding hosts file entries, but this solution is very effective. A startup script is deployed via GPO which checks for Outlook 2003 and, in cases where it is present, adds an entry to the hosts file pointing the namespace for the Source Exchange server to the IP address of a Target mailbox server. For example:

mail.source.net 10.10.75.12

This causes Outlook to connect directly to one of the Target mailbox servers and then Exchange automatically reconfigures Outlook to connect to the Target CAS Array. If the hosts file entry were to point Outlook directly to the Target CAS Array, Outlook would not reconfigure itself as it would appear that nothing has changed. This would work, but is not ideal because Outlook would always rely on that hosts file entry to function going forward. By pointing Outlook to a mailbox server, Outlook is forced to reconfigure itself (similar to autodiscover) and the hosts file entry is no longer needed and can therefore be removed.

Activesync devices will also need to be re-pointed to the new exchange environment. This is simply performed by changing the Exchange server from mail.source.com to mail.target.com and adjusting the user’s credentials accordingly.

With workstations and mailboxes migrated and email clients reconfigured, the migration is finally complete! Congratulations! Repeat this process for each “chunk” of users and all that remains is to end coexistence and cut off all ties to the source organization. The steps for this final process are as follows:

Email

  1. Confirm that there are no outstanding mailboxes which need to be migrated
  2. Migrate and merge all mail-enabled groups one final time to update Distribution List memberships
  3. Make Target Hub Transport Servers authoritative for Target.com
  4. Point MX records for Target.com to the Target Exchange servers
  5. Deploy a public autodiscover record for Target.com
  6. Remove send connector to Source Exchange servers
  7. Have the Source domain admins remove their send connector to Target Exchange servers

Active Directory

  1. Ensure that all fileshares have been migrated
  2. Removed DNS Conditional Forwarders for the Source domain
  3. Have Source domain admins remove the DNS Conditional Forwarders for the Target domain
  4. Verify that all workstations were migrated via network traffic logs and migrate any outstanding workstations
  5. Remove trust to Source domain

With the Active Directory and Exchange coexistence features removed, the Target organization is completely free-standing and independent of the Source org. Carve-out complete!

Help Guide Powered by Documentor
Suggest Edit