When you install Windows 10 using SETUP.EXE, or any time you install a new feature update either using SETUP.EXE from media or installing via the Windows Update agent, the installation process will attempt to grab a set of additional “stuff” to make the installation process go as smoothly as possible.

The simplest way to see all of this work:  Take a virtual machine running Windows (any applicable version or SKU).  Make sure the machine isn’t joined to a domain (you don’t want WSUS polices to get in the way).  Then make the task harder by installing some language packs manually using the Settings app.  Navigate to Time & language –> Region & language and then add one or more language packs.  (It’s best doing this before patching the OS, since installing a language pack will require reinstalling the latest cumulative update.)

It’s easiest to kick off an upgrade manually using Windows 10 1703 media.  Insert the ISO, or use an extracted copy, and run “SETUP.EXE /AUTO UPGRADE”.  Once the upgrade is complete, take a look at the C:\WINDOWS\PANTHER\SETUPACT.LOG file to see what happened.  Search for “DU Client Application Information” to find all the dynamic update checks.  Here’s an example, shortened some to remove some of the noise:

2017-09-29 21:05:31, Info                  SP     DU Client Application Information
…
2017-09-29 21:05:31, Info                  SP     Executing download operation: Download Language Pack Dynamic Updates
…
2017-09-29 21:05:31, Info                  SP     Searching for dynamic updates
2017-09-29 21:05:31, Info                  SP       Update types:
2017-09-29 21:05:31, Info                  SP         Component updates
2017-09-29 21:05:31, Info                  SP       Category IDs:
2017-09-29 21:05:31, Info                  SP         6111a83d-7a6b-4a2c-a7c2-f222eebcabf4
…
2017-09-29 21:05:31, Info                  SP     1 updates were found

You should see a few of them:

• Setup updates (category e4b04398-adbd-4b69-93b9-477322331cd3)
• Critical dynamic updates
  • Component updates (category e4b04398-adbd-4b69-93b9-477322331cd3)
  • Driver updates (category e4b04398-adbd-4b69-93b9-477322331cd3)
• OS updates
  • Component updates (category abc45868-0c9c-4bc0-a36d-03d54113baf4)
• Driver updates
  • Driver updates (categories 405706ed-f1d7-47ea-91e1-eb8860039715, 34f268b4-7e2d-40e1-8966-8bb6ea3dad27, 0ba562e6-a6ba-490d-bdce-93a770ba8d21, 06da2f0c-7937-4e28-b46c-a37317eade73)
• Language pack updates
  • Component updates (category 6111a83d-7a6b-4a2c-a7c2-f222eebcabf4)

That should raise at least one question:  What are those category IDs?  Well, it turns out that they aren’t really categories at all, at least not in WSUS terminology.  Instead, they map to “products.”  If we get a list of the products from WSUS using the “Get-WsusProduct” PowerShell cmdlet (available on Windows Server 2016), you can see these items for Windows 10:

Title                                                                               ID
-----                                                                               --
Windows 10 and later drivers                                                        05eebf61-148b-43cf-80da-1c99ab0b8699
Windows 10 and later upgrade & servicing drivers                                    34f268b4-7e2d-40e1-8966-8bb6ea3dad27
Windows 10 Anniversary Update and Later Servicing Drivers                           bab879a4-c1af-4b52-9617-0f9ae1286fb6
Windows 10 Anniversary Update and Later Upgrade & Servicing Drivers                 0ba562e6-a6ba-490d-bdce-93a770ba8d21
Windows 10 Creators Update and Later Servicing Drivers                              cfe7182c-14a0-4d7e-9f5e-505d5c3a66f6
Windows 10 Creators Update and Later Servicing Drivers                              f5b5092c-d05e-4eb1-8a6a-919770378ff6
Windows 10 Creators Update and Later Upgrade & Servicing Drivers                    06da2f0c-7937-4e28-b46c-a37317eade73
Windows 10 Dynamic Update                                                           e4b04398-adbd-4b69-93b9-477322331cd3
Windows 10 Feature On Demand                                                        e104dd76-2895-41c4-9eb5-c483a61e9427
Windows 10 GDR-DU LP                                                                6111a83d-7a6b-4a2c-a7c2-f222eebcabf4
Windows 10 GDR-DU                                                                   abc45868-0c9c-4bc0-a36d-03d54113baf4
Windows 10 Language Interface Packs                                                 7d247b99-caa2-45e4-9c8f-6d60d0aae35c
Windows 10 Language Packs                                                           fc7c9913-7a1e-4b30-b602-3c62fffd9b1a
Windows 10 LTSB                                                                     d2085b71-5f1f-43a9-880d-ed159016d5c6
Windows 10 S and Later Servicing Drivers                                            c1006636-eab4-4b0b-b1b0-d50282c0377e
Windows 10                                                                          a3c2375d-0c8a-42f9-bce0-28333e198407

I highlighted all of the matching items from the SETUPACT.LOG list – they line up nicely (well, except for one of the driver ones, but we’ll leave that mystery for another time).  Some of you asked what these categories were for when they showed up in WSUS – now you know.  Originally, if you selected these products in WSUS, you never saw any updates show up for them.  But that’s no longer the case – some of these will have something in them.

But before we enable those, it’s useful to do one more test:  Do another upgrade, but this time, point the machine at a WSUS server.  (You can do this via local GPO if you want to keep your AD domain out of the way.)  Afterwards, again look at the “DU Client Application Information” sections.  Here’s one example (again cutting out some of the noise):

2017-09-29 23:30:10, Info                  SP     DU Client Application Information
…
2017-09-29 23:30:10, Info                  SP     Executing download operation: Download Language Pack Dynamic Updates
…
2017-09-29 23:30:10, Info                  SP     Searching for dynamic updates
2017-09-29 23:30:10, Info                  SP       Update types:
2017-09-29 23:30:10, Info                  SP         Component updates
2017-09-29 23:30:10, Info                  SP       Category IDs:
2017-09-29 23:30:10, Info                  SP         6111a83d-7a6b-4a2c-a7c2-f222eebcabf4
…
2017-09-29 23:30:10, Info                  SP     No updates to download. Leaving...

And that illustrates the problem:  When your devices point to WSUS, they can’t get the needed packages for dynamic update to work, so it doesn’t do anything – no patches, no drivers, no language packs, no updated compatibility information, etc.

OK, so now let’s go back to WSUS and enable the categories highlighted above.  Once you do that and synchronize, you’ll see a bunch of new updates are found.  If you look at the “All updates” view sorted by the arrival date descending, you can see all of them in the list.  But I prefer to create a better view.  Right click on “Updates” and choose “New update view".  Choose “Updates are for a specific product”, and then select the products highlighted above:

Select the new view, and change the criteria at the top to “Any Except Decline” and “Any,” then refresh to see the full list.  These are all the updates that dynamic update could seek out.  If you add the “Classification” column to the list, you can see that there are critical updates, security updates, and updates, and there are lots of language packs and related language packages in the list.  Also notice that there are no drivers, as those aren’t published to WSUS.  (That’s probably a good thing, as there would be lots of them.)

Now let’s take the next step:  Select all of those items and approve them for “All computers” and for “install.”  Make sure your server has sufficient disk space, that’s a lot of language packs.)  That will take a while, first to approve the updates, and then to download all the needed stuff.  A good time for a lunch/dinner/weekend break, a stroll through the park, etc.

Once everything is downloaded, we’re ready to try again, upgrading from Windows 10 1607 (with an extra language installed) to Windows 10 1703.  Again, I’ll use media and kick off the upgrade using “SETUP.EXE /AUTO UPGRADE”.  After it finishes, the SETUPACT.LOG again provides a view into what happened, in this case for language packs:

2017-09-30 11:17:07, Info                  SP     DU Client Application Information
…
2017-09-30 11:17:07, Info                  SP     Executing download operation: Download Language Pack Dynamic Updates
…
2017-09-30 11:17:07, Info                  SP     Searching for dynamic updates
2017-09-30 11:17:07, Info                  SP       Update types:
2017-09-30 11:17:07, Info                  SP         Component updates
2017-09-30 11:17:07, Info                  SP       Category IDs:
2017-09-30 11:17:07, Info                  SP         6111a83d-7a6b-4a2c-a7c2-f222eebcabf4
…
2017-09-30 11:17:08, Info                  SP     1 updates were found

Yeah!  And we can confirm that with PowerShell by running “Get-WindowsCapability -online” to see what’s installed:

Name  : Language.Basic~~~en-US~0.0.1.0
State : Installed

Name  : Language.Basic~~~de-DE~0.0.1.0
State : Installed

Name  : Language.Handwriting~~~en-US~0.0.1.0
State : Installed

Name  : Language.Handwriting~~~de-DE~0.0.1.0
State : Installed

Name  : Language.OCR~~~en-US~0.0.1.0
State : Installed

Name  : Language.OCR~~~de-DE~0.0.1.0
State : Installed

Name  : Language.Speech~~~en-US~0.0.1.0
State : Installed

Name  : Language.Speech~~~de-DE~0.0.1.0
State : Installed

Name  : Language.TextToSpeech~~~en-US~0.0.1.0
State : Installed

Name  : Language.TextToSpeech~~~de-DE~0.0.1.0
State : Installed

Name  : Language.UI.Client~~~en-US~
State : Installed

Name  : Language.UI.Client~~~de-DE~
State : Installed

That looks good, two languages (English and German) with all the needed language pieces in place.  (That’s a separate conversation that I think I’ve blogged about before – you need several components to have full support for a new language.)

Another difference that can be seen:  Even though I’m using the original Windows 10 1703 ISO (build 15063.0), the upgraded machine has a later patch installed:

That’s not the absolute latest build (15063.540 installed, latest patch would be 15063.632 at the time of writing), but it is the expected result:  The dynamic update process doesn’t necessarily grab the latest update, it grabs the latest cumulative update tagged as a dynamic update.  So it should be no big surprise that when you check for updates after upgrading, it finds updates to install:

But that does illustrate one other important point:  All of those dynamic updates that I approved are only used by the SETUP installation process (again, whether initiated from media, from WSUS, etc.).  You’re not going to suddenly see all of your machines trying to install lots of language packs after you approved them in WSUS.

Thanks to Sudhagar Thirumoolan for getting the needed packages published to WSUS to make this work, and to former WSUS PM Steve Henry for repeatedly saying this was possible.

Some of you might be thinking “what about ConfigMgr.”  The challenge there is that you would need to go through the same WSUS steps to make this work with ConfigMgr (task sequences or Windows 10 servicing), and the ConfigMgr team doesn’t support that sort of manipulation.

In the future, we’ll be making some changes to the way these components are published, as part of the Unified Update Program work.  Once that’s done, then this will be much easier:  Everything needed will be part of the same package bundle.  As mentioned at Ignite this week, that bundle will be large (since it contains language packs, cumulative updates, features on demand, and more), but clients will be smart enough to only download what they need.  Stay tuned for more on that.

Some of you may have seen me do a demo at previous events showing how you could change from Windows 10 Pro to Windows 10 Enterprise, just by changing the product key – the change is immediate, no reboot is required.  That capability was added in Windows 10 1607.

Now in Windows 10 1709, we’ll add one more OS into the fold.  Now you can also change from Windows 10 Enterprise to Windows 10 Education, again just by changing the product key.  Here’s a short video showing how to do this:

In case you are wondering, the product keys shown in that video are publicly documented at https://technet.microsoft.com/en-us/library/jj612867(v=ws.11).aspx – they don’t actually activate the OS, they just cause it to try to find a KMS server that can activate it.

We announced a similar capability recently, Windows 10 Subscription Activation, which leverages the same technology behind the scenes, but today that only supports stepping up from Windows 10 Pro to Windows 10 Enterprise.  Check out the Ignite session that Kamal and I did on this topic, available at https://myignite.microsoft.com/sessions/53410.  The ability to support stepping up from Windows 10 Pro to Windows 10 Enterprise automatically through Subscription Activation is something we’re looking at for the future.

p.s. One of the questions asked at Ignite, and more recently on Twitter, was how to move from Windows 10 Enterprise LTSB to Windows 10 Education.  While you can’t upgrade directly from Windows 10 Enterprise LTSB to Windows 10 Education, you can instead upgrade from Windows 10 Enterprise LTSB to Windows 10 Enterprise 1709, then use the mechanism I described above to change the product key to Windows 10 Education – one additional step in the process.

Since we’ve started talking about Windows as a service (almost three years ago now), we’ve published a lot of good information.  And while most people know about the main set of documentation (http://aka.ms/waas) and the Microsoft Mechanics video (http://aka.ms/waasvideo), there are some other sources of information that are also useful to help connect the dots between the theory and the current state.  Here’s an overview of some key resources:

• The Windows lifecycle fact sheet.  This page includes an “Updates and Service Packs” section that provides the expected end of servicing dates for each Windows 10 feature update, so you’re never surprised by one of the dates.  For example, here you can see that Windows 10 1511 end of servicing date is October 10, 2017 (this week), the Windows 10 1607 end of servicing date is tentatively March 2018 (effectively no earlier than that, although it could shift a little later), etc.  And of course the "End of support” table contains a good reminder of the Windows 7 end of support date, January 14, 2020.
• The Windows 10 release info page, http://aka.ms/win10releaseinfo.  This page shows the status of each release, the date of release, and all of the cumulative updates that have been released.  This is a good reference site.  (It doesn’t include the end of servicing dates yet, but we expect this page to be reworked sometime next year to include this information.)
• The Windows 10 release notes page, http://aka.ms/win10releasenotes.  This page provides details on significant fixes included in each cumulative update.
• The Configuration Manager team publishes details to tell you which releases of Windows 10 and the Windows 10 ADK are supported by each Configuration Manager current branch release on their Support for Windows 10 for System Center Configuration Manager page.
• We publish an overview of new features in each Windows 10 feature update at https://docs.microsoft.com/en-us/windows/whats-new/.
• We publish (and recently updated) the Windows Roadmap page to talk about features recently delivered, ones in preview, ones in development, etc.
• For long-term servicing releases, the end of support dates can be found via the Support Lifecycle search page.

In one of the sessions that we did at Ignite around Windows 10 Subscription Activation, we included this slide that talked about the progression that has been made through the years in simplifying the process of changing the installed SKU of Windows, primarily between Windows Pro and Windows Enterprise:

Judging from the reaction to my other blog that talked about the one new path that we added with Windows 10 1709 to easily change between Windows 10 Enterprise and Education SKUs, it appears that there are lots of people that aren’t familiar the options that are available, or even how those options changed over the years.

To remedy that, let’s give you a history lesson, similar to what’s in the Ignite session recording:

• With Windows 7, if you wanted to change from Windows 7 Professional to Windows 10 Enterprise, you had to redeploy the operating system – a full wipe-and-load process.
• With Windows 8.1, we added support for a Windows 8.1 Pro to Windows 8.1 Enterprise in-place upgrade (considered a “repair upgrade” because the OS version was the same before and after).  This was a lot easier than wipe-and-load, but it was still time-consuming.
• With Windows 10 1507, we added the ability to install a new product key via a provisioning package or using MDM to change the SKU.  This would require a reboot, which would install the new OS components.  This took several minutes to complete, but it was a lot quicker than in-place upgrade.
• With Windows 10 1607, we made a big leap: Now you can just change the product key and the SKU instantly changes from Windows 10 Pro to Windows 10 Enterprise.  In addition to provisioning packages and MDM, you can just inject a key using SLMGR.VBS (which just injects the key into WMI), so it’s now trivial to do this via a command line too.
• With Windows 10 1703, we made this “step-up” from Windows 10 Pro to Windows 10 Enterprise automatic for those that subscribed to Windows 10 Enterprise E3 or E5 via the CSP program.
• With Windows 10 1709, we added support for Windows 10 Subscription Activation, very similar to the CSP support but for large enterprises, enabling the use of Azure AD for assigning licenses to users; when those users sign in to an AD or Azure AD-joined machine, it automatically steps up from Windows 10 Pro to Windows 10 Enterprise.

Enough history lesson then.  Let’s switch to “how-to” and focus on the most common scenarios today.

Scenario #1:  Using KMS for activation, just purchased Windows 10 Enterprise E3 or E5 subscriptions (or for some reason have had an E3 or E5 subscription for a while but haven’t yet deployed Windows 10 Enterprise), and you are using Windows 10 1607 or above.

All you need to do to change all of your Windows 10 Pro machines to Windows 10 Enterprise is to run this command on each machine:

cscript.exe c:\windows\system32\slmgr.vbs /ipk NPPR9-FWDCX-D2C8J-H872K-2YT43

In case you are wondering, that key came from https://technet.microsoft.com/en-us/library/jj612867(v=ws.11).aspx.  It causes the OS to change to Windows 10 Enterprise and then seek out the KMS server to reactivate.  (And yes, if you really want to, you can inject the Windows 10 Pro key from that page to step back down from Enterprise to Pro.)

Scenario #2:  Using Azure AD-joined devices or Active Directory-joined devices, with Azure AD synchronization set up (“hybrid join,” as discussed in the video above), and you are using Windows 10 1709 or above.

All you need to do is to follow the steps in https://docs.microsoft.com/en-us/windows/deployment/windows-10-enterprise-subscription-activation to acquire a $0 SKU to get a new Windows 10 Enterprise E3 or E5 license in Azure AD, then assign that license to all of your Azure AD users (which can be AD-synced accounts).  Then the device will automatically change from Windows 10 Pro to Windows 10 Enterprise when that user signs in.

Summary

If you have a Windows 10 Enterprise E3 or E5 subscription, but are still running Windows 10 Pro, it’s really simple (and quick) to move to Windows 10 Enterprise using one of the scenarios above.

If you’re running Windows 7, it can be a lot more work.  While a wipe-and-load approach works fine, it would be easier to upgrade from Windows 7 Pro directly to Windows 10 Enterprise (yes, that’s a supported path) which takes care of the move in one hop.  (That works for Windows 8.1 Pro as well.)

With the release of Windows 10, the Windows Update log changed from a text file to a binary file.  A new PoweShell cmdlet, Get-WindowsUpdateLog, was added to format the binary file, generating the equivalent text file that many of you were used to.  But there was one challenge:  The formatting process required access to Windows symbols.  As long as the corresponding symbols were available on the Microsoft symbol server (typically used by developers to debug Windows) and you were able to reach the symbol server (on the internet, potentially via a proxy), the formatting would work fine. 

But sometimes the formatting didn’t work fine, typically because the symbols weren’t available.  That was usually a temporary situation, waiting for the symbols to be uploaded to the server and indexed, but it seems it would always happen when people had a need to format the log.

With Windows 10 1709 (including in the current Insider Preview builds), the formatting process has changed.  You still need to use the Get-WindowsUpdateLog cmdlet to format it, but it no longer needs access to the symbols to do that formatting.  Notice this screenshot:

Even though the computer has no network connection at all (see the icon at the bottom right), the Get-WindowsUpdateLog worked successfully.  And more importantly, we end up with a usable log file:

Happy formatting

As discussed in the documentation at https://docs.microsoft.com/en-us/windows/deployment/windows-10-auto-pilot, you can harvest the hardware details from existing Windows 10 devices, then upload those to AutoPilot via the Microsoft Store for Business, the Partner Center, or Intune (coming soon).  I published a PowerShell script called Get-WindowsAutoPilotInfo to the PowerShell Gallery that helps with gathering that info and formatting it into a CSV-style file that can be uploaded.

In order to gather the hardware details, the device must be running Windows 10 1703 or later and the script needs to be run with admin rights (elevated when running locally).  It can’t run on earlier versions of Windows, nor will it run in Windows PE.

It is possible to gather hardware details remotely, accessing WMI over the network, but the remote machine still needs to be running Windows 10 1703 or later.  With the most recent script update published last week, you can optionally provide credentials for connecting to the remote machine (not supported for the local machine); just add the “-Credential” parameter (along with the -ComputerName parameter) and specify either a PSCredential object or a user ID (which would result in prompting for the password).

If you are trying to gather the hardware details remotely, you will need to make sure that the Windows Firewall enables access to WMI.  To do that, open the “Windows Defender Firewall” control panel and click on “Allow an app or feature through Windows Defender Firewall.”  Find “Windows Management Instrumentation (WMI)” in the list and enable for both Private and Public, then click OK:

Or you can do the same thing via Group Policy – nothing special about what the script is doing, it just needs remote WMI access to the classes that contain the details.  The script should also accept a piped-in list of computer names, so you could grab them from AD or ConfigMgr and do a whole batch at once.

Some have also asked if it’s possible to have ConfigMgr inventory these details.  Due to the length of the hardware hash (4KB), that doesn’t work.  You could use ConfigMgr to run the PowerShell script on each machine, then collect the resulting CSV files.

If you’ve watched the Microsoft Mechanics video or Ignite sessions (presented by Sidd or me) on Windows AutoPilot, you’ve seen what is supposed to happen:

• Manually choose language, region, keyboard
• Connect to a network (if not wired)
• Azure AD authentication using custom branding (org name, icons, etc.)
• The device is joined to Azure AD
• Automatic MDM enrollment (as configured in Azure AD)
• Automatic logon as the specified Azure AD user
• MDM configuration is applied (with progress display if configured)

But what if that isn’t what actually happens?  The following sections provide some troubleshooting suggestions.

Manually choose language, region, keyboard

If you don’t see these standard out-of-box experience (OOBE) screens, there is either something wrong with your OS image, an unexpected unattend.xml being used, or OOBE was partially completed before the computer restarted.  Either start from scratch with a good image, or complete OOBE the rest of the way creating a local account, then reset the OS from the Settings app to start over.

A tip:  If you are using a Hyper-V virtual machine, configure the VM to create standard (not production checkpoints).  Create a checkpoint before the VM is powered on for the first time.  Then start the VM and let it get to the first OOBE screen to choose a language before creating a second checkpoint.  These checkpoints give you an easy way to repeatedly test the process.  (The first one gives the full experience; the second just saves little time.)

Another tip (as covered in my Ignite session):  You can capture the hardware hash from the machine after it gets to the language screen in OOBE.  Press Shift-F10 to open a command prompt, then run PowerShell.exe.  From there, you can set the execution policy to a reasonable level (RemoteSigned or Bypass) and then install and run the Get-WindowsAutoPilotInfo.ps1 script to create a CSV file with the hardware details.  Copy that CSV file to a network location or USB key, then import it into the Microsoft Store for Business.  The machine is still early enough in the process that it hasn’t yet checked to see if it is known to AutoPilot.  Just exit out of PowerShell and the command prompt after you’ve uploaded the machine.  (See https://blogs.technet.microsoft.com/mniehaus/2017/12/12/gathering-windows-autopilot-hardware-details-from-existing-machines/ for more on Get-WindowsAutoPilotInfo.)

Connect to a network (if not wired)

If the machine doesn’t have a wired connection and it has a wireless adapter, you should be prompted to connect to an available wireless network.  If you don’t have any wired or wireless adapters, nothing is going to help – OOBE is going to short-circuit and end up creating a local account.  Assuming you have an wireless adapter, you should be able to connect to a wireless network, even if it requires a captive portal – if it does, you should see a browser window open.

So what can go wrong here?  Proxy servers can cause challenges if they aren’t automatically discoverable (e.g. WPAD) or if they require user authentication (there’s no user yet).  Wireless networks that require certificates likely won’t work as there will not yet be any certificates.  Firewalls might make Windows think that there isn’t a path to the internet.

We’re talking about basic network troubleshooting here, so you may want to use Shift-F10 to open a command prompt for further troubleshooting (IPCONFIG, NSLOOKUP, PING, etc.).

Azure AD authentication using custom branding (org name, icons, etc.)

After the network connection is in place, Windows will check with the AutoPilot deployment service (in the cloud) to see if the device is known.  If it is, then the customized branding can be displayed with the org name and logo.  The customizations you should see are highlighted in red:

If the device isn’t configured to AutoPilot, then you get standard choices (personal vs. work/school if running Windows 10 Pro, a generic Azure AD sign-in screen with Windows 10 Enterprise, OneDrive, Cortana, privacy, etc.).  What if you don’t get what you expect?  There are a few possible causes:

• The device really isn’t defined to AutoPilot.  A simple way to verify is to gather the hardware details again, then either search for the same machine in the existing list or just try to upload it again to see if you get an error about it being a duplicate.  (Note that significant changes to the hardware, e.g. swapping out the motherboard, can change the hardware hash.  Also, make sure that your device has a serial number, as the script expects that.  Some devices, e.g. Intel NUCs, don’t have a serial number by default.  See http://amicasto.com/2017/11/22/customizing-intel-nuc-bios-with-intel-integrator-toolkit/ for details.)
• The Azure AD tenant doesn’t have an Azure AD Premium license (or an equivalent suite license such as EM+S E3).  That’s required for customized branding.  (Interestingly, the license doesn’t have to be assigned to the user – at least not for this piece.  That’s required in the next step though.)
• You haven’t configured the company branding settings in Azure AD.  See the Azure AD documentation for information about that.

Assuming you type in a valid username (UPN, e.g. user@contoso.com) and password, you should continue to the next step.

Azure AD Join

Now that the credentials have been validated, the next step is to join the device into Azure AD.  This is usually the easy piece, but there are some gotchas that can get in the way:

• Have you enabled the user to join devices to Azure AD?  That is an obvious requirement – in order for AutoPilot to be used by end users, all of them need to be able to join devices to Azure AD.  The steps to enable that are covered at https://docs.microsoft.com/en-us/azure/active-directory/device-management-azure-portal#configure-device-settings.  If the user doesn’t have rights, you’ll get error 801C0003 (“Something went wrong”):
   
• Has the user reached the maximum number of devices that they can join?  There can be a limit, but if you configure it to “unlimited” then you don’t need to worry about this one.  If you do set a limit and the user reaches the limit, you’ll get the same 801C0003 (“Something went wrong”) error as above.

Automatic MDM enrollment

Once the Azure AD join completes, the MDM enrollment will happen.  This automatic MDM enrollment is an Azure Active Directory Premium feature.  If that doesn’t happen as expected, here are some things to check:

• Was automatic MDM enrollment enabled?  This needs to be configured in Azure Active Directory via the Azure Portal.  See https://docs.microsoft.com/en-us/intune/windows-enroll#enable-windows-10-automatic-enrollment for details.  If this isn’t configured, you won’t see any errors, but the machine will not be managed by Intune.
• Is the user in the group that was configured for MDM automatic enrollment?  If they aren’t, you won’t see any errors, but the machine will not be managed by Intune.
• Does the user have an Azure AD Premium license (or a suite that includes that license)?  (Azure AD Premium P1 or P2 both work fine.)  If not, they’ll see an 80180018 error (“Something went wrong”), indicating that the device could not be enrolled in MDM:
   
• Does the user have an Intune license (or a suite that includes that license)?  If not, they’ll see the same 80180018 error (“Something went wrong”) as above.
• Has the user reached the limit of devices that they can enroll in Intune?  This limit is configurable and can range from 1 to 15; see https://docs.microsoft.com/en-us/intune/enrollment-restrictions-set for more information.  (Interestingly, this is an issue that I’m not presently able to duplicate.  I just keep adding more and more devices…)

Automatic logon as the specified Azure AD user

This is the easy part – I’ve never seen this fail, since the credentials have already been verified and the device is already joined using the same credentials.  So there’s no reason why the third usage of those credentials shouldn’t work.  After the logon, you’ll see the pulsing status screen while apps are being installed, an MDM progress display (at least once that is enabled by Intune) showing what policies still need to be applied, and finally the desktop.

At that point, the whole AutoPilot-driven provisioning process is complete.  To see that the device is indeed being managed by Intune, launch the Settings app, navigate to Accounts –> Access work or school, click the “Connected to <tenant>” entry (which indicates the device is joined to Azure AD) and then click the “Info” button to see the MDM enrollment details:

This will show you information about the policies applied, server URLs, etc.  You can also generate an HTML diagnostics report from here, or manually initiate a sync with Intune.

One useful link I found while going through this process:  https://docs.microsoft.com/en-us/windows/client-management/mdm/azure-active-directory-integration-with-mdm has a table that is useful to look up the error codes that you might encounter (e.g. 801C0003).

In my last post, I talked about issues you might encounter with Windows AutoPilot.  But what if you still can’t figure it out?  Typically the support and development teams would want more information to see what’s going on.  While I’m sure most IT pros would like to see log files or event logs that show the step-by-step flow, you’ll have to dig a little deeper in order to get something that can be used for a “post-mortem” analysis.  What does that mean?

First, Windows AutoPilot uses Event Tracing for Windows (ETW) to capture events during the AutoPilot process.  Those events need to be enabled and captured so that they can be inspected and interpreted.  How do you do that?  Here are the basic steps:

• On the first OOBE screen where you select a language, press Shift-F10 to open a command prompt.
• Insert a USB key containing the AutoPilot.wprp file (attached to this blog, extract it from the zip file) that defines the events to collect, or map a drive to a network share that contains the file.
• Start the trace using the Windows Performance Recorder (WPR) tool that is built into Windows.  (I talked about that a few years ago in a different context, https://blogs.technet.microsoft.com/mniehaus/2012/09/13/using-the-windows-performance-toolkit/.)  The command line to use:wpr.exe -start <folder location>\AutoPilot.wprp
• Exit from the command prompt.
• Finish your AutoPilot process, continuing at least as far as you need to go to reproduce the issue.
• Stop the recording using WPR.EXE, specifying where to put the resulting trace (press Shift-F10 to open another command prompt if you are still in OOBE):
  wpr.exe -stop C:\AutoPilot.etl

That part is easy enough.  The next challenge is reading and interpreting that file.  There are some options for doing that:

• Windows Performance Analyzer, provided as part of the ADK.
• Microsoft Network Analyzer (formerly called Network Monitor).
• TRACEFMT.EXE, a command-line utility that can convert the binary ETL file into XML or other formats.

My personal preference is to use Windows Performance Analyzer (WPA).  To get started with this process, let’s look at an ETL file captured from a 100% successful AutoPilot deployment, which I captured using the process described above.  After opening the file in WPA, you will see this view:

Exciting, right?  Well, not until you dig in more.  First, click the triangle before the “System Activity” label to get to this view:

Then double-click on the “Generic Events” block to show the details:

Now we can at least see some data.  Let’s focus in on two specific providers:

• Microsoft.Windows.Shell.CloudExperienceHost.Common.  This is where you’ll find the AutoPilot-generated events.  Some of these are prefixed with “ZTD” (related to the old codename for AutoPilot), while others aren’t prefixed.
• Microsoft.Windows.Shell.CloudDomainJoin.Client.  Events that are part of the initial “is this an AutoPilot deployment,” the AAD join process, and MDM enrollment are from this provider.

So let’s expand those for two “normal” cases.  First, for a device that is configured for AutoPilot:

Compare that to a device that was not enrolled in AutoPilot, but was still joined to Azure AD and automatically enrolled in Intune:

Not surprisingly, there are fewer events for the non-AutoPilot scenario.  But let’s dig into those events a little more, first the “GetCloudAssignedAadServerData” event.  Click on the triangle to the left of the “GetCloudAssignedAadServerData” task name, then select the “win:Stop” opcode (which shows the result).  Scroll to the right to see the data for an AutoPilot-defined machine:

Notice the “wasConfigured” value is “True,” indicating that this device is indeed registered with AutoPilot.  Compare that with a device that isn’t:

And not surprisingly, that shows “False” for the device that isn’t registered.  OK, so back to the registered device.  To see what Azure AD tenant it is registered with, we can check the “LogTenantId” and “LogTenantDomain” tasks.  Expand the “win:Info” opcodes for each and again scroll to the right:

There, you can see the Azure AD tenant ID (a GUID, which you can also find in the Azure Portal in the Azure AD tenant’s properties) and the tenant name (in this case, “contosomn.onmicrosoft.com”).  (Do you see the “IsDevicePersonalized” task name?  That’s for a coming AutoPilot feature that enables a device to be assigned to a user to further customize the “welcome” experience.  That’s not enabled yet, so it always returns “0” today.)

Let’s look at one more item, the “GetCloudAssignedForceStandardUser” task name (with the same routine: expand, select win:Stop, scroll right):

This shows that it’s configured (wasConfigured is “True”) to not force the Azure AD user to be a standard user (forceStandardUser is “False”).

Now lets look at a different trace, this one showing a failure with Azure AD join, error 801C0003 (“Something went wrong”) covered in the previous troubleshooting blog.  We’ll take a different approach this time, just searching for a specific “Event Name” value.  Click on the magnifying glass at the top of the “Generic apps” (right-hand) pane to get a search box, and search for “CDJUIError”.  That will take you directly to the row that has the error, showing you the 801C0003 error code (which you can then look up, see the link at the bottom of the previous blog):

That same approach would work for any Azure AD join issue.  But what about MDM enrollment issues?  Let’s look at another error, the 80180018 error (“Something went wrong”) from the previous post indicating that the device could not be enrolled in MDM.  Guess what?  It shows up in the exact same “CDJUIError” field:

That leaves one key piece of information:  What are the actual profile settings (e.g. skip privacy settings, skip EULA, etc.) that were configured for the device?  Those are reported in a bitmap:

SkipCortanaOptIn = 1,
OobeUserNotLocalAdmin = 2,
SkipExpressSettings = 4,
SkipOemRegistration = 8,
SkipEula = 0x10

The first four were present in Windows 10 1703, while the last one was added in Windows 10 1709.  In my case, I know that I enabled all options except the “OobeUserNotLocalAdmin” one (I wanted the user to have admin rights), so the value should be 1+4+8+16 = 29.  If we look at the trace, we see that is shown in all the “ZTP_GetConfigActivity” tasks (again, expand the entries, expand the win:Stop opcodes, and then one more level with process “Unknown”):

Sure enough, each of those (which are checking individual settings given the complete bitmap value) shows the value 29 in the “Field 2” column.  If you configured different options, the value would be different in that column.

I won’t claim that I’m particularly good with ETW or Windows Performance Analyzer, but hopefully this will give you some hints on what to look for in the traces.  If you pick up any additional tidbits looking at these traces, let me know.  Also, in case you were to open a support case for AutoPilot, don’t be surprised if they ask you to recreate the problem with a trace running, using the steps described at the top of this blog, sending the resulting ETL file so that they can walk through it and look at what happened.

AutoPilot.wprp.zip

In yesterday’s AutoPilot troubleshooting blog, https://blogs.technet.microsoft.com/mniehaus/2017/12/13/troubleshooting-windows-autopilot-level-300400/, I showed a variety of captured traces for AutoPilot (and one non-AutoPilot) deployment scenario, which required actually reproducing some different scenarios, including some failure cases.  As some of those aren’t particularly simple to reproduce outside of a lab environment (where you can purposely reconfigure Azure AD or Intune), a few people requested copies of those traces – analyzing the failures without having to cause the failures, as an educational exercise.  So, here they are; download the zip file at the bottom of this blog.

Here’s a quick overview of the included traces, all of which were captured from (unpatched) Windows 10 Enterprise 1709 virtual machines:

• 1629915.NoAutoPilot.etl.  This trace was captured from a VM that is not known to AutoPilot (hash was never captured or uploaded).  The device went through the standard OOBE process, manually.
• 1629915.Success.etl.  This is a normal, successful AutoPilot deployment (hash was captured and uploaded, with an AutoPilot profile of settings applied).  The device went through the optimized OOBE process.
• 1629915.801C0003.etl.  This trace, from the same VM as the success case, shows what happens when the user does not have the ability to join Azure AD.
• 1629915.801C0003-5limit.etl. This trace, also from the same VM as the success case, shows what happens when the user can join no additional machines to Azure AD (device limit was set to 5 in Azure AD, this would have been device #6).  (From what I can tell, this trace is identical to the previous one, which just indicates that there is no way to tell the difference between the two failure scenarios.)
• 1629915.80180018.etl.  This trace, also from the same VM as the success case, shows what happens if you take away the Azure AD Premium or Intune license from the user who was setting up the device.  (They did again have rights to join Azure AD.  I only captured one trace, even though I did run through both scenarios separately – once I saw that “no AAD Premium license” and “no Intune license” returned the same error, I didn’t bother capturing a second trace.)

Let me know if you find any other interesting tidbits in these traces.

AutoPilotTraces

When you use Windows AutoPilot to deploy new devices, you want the process to be friendly and familiar for the users going through the process.  One of the ways that is done is by customizing the logon experience to include logos and company-specific text.  This leverages an Azure Active Directory Premium feature called company branding, describe at https://docs.microsoft.com/en-us/azure/active-directory/customize-branding.  The basic steps:

• Sign into the Azure Portal as a tenant admin.
• Navigate to Azure Active Directory –> Company branding.
  
• Click Edit to configure the needed settings.
  
• Fill in all the customizations.
  
  

You will need some bitmaps to do this:

• A square logo, 240 pixels by 240 pixels, PNG or JPG, 10KB or smaller.
• A banner logo, 280 pixels by 60 pixels, PNG or JPG, 10KB or smaller.
• A background image, 1920 pixels by 1080 pixels, PNG or JPG, 300KB or smaller.

If you need to resize existing bitmaps, or reduce the size of existing bitmaps by reducing the color pallet, you may need to use something like Paint.NET.

So then you just need to figure out where those values are used, so let’s look at the resulting Windows AutoPilot screen captures:

So let’s look at the mapping from each of the highlighted and numbered items in the screenshots above:

• (1) and (5) correspond to the “square logo image” (240x240px).
• (2) and (3) don’t come from the company branding.  Instead, you can set that value in the “Name” field of the Azure AD tenant properties:
  
  (Watch out if you include special characters in the text, e.g. accented characters, as they may not display properly.  That’s being investigated.)
• (4) and (6) come from the “sign-in page text” field.  (You might notice a bug in the above screenshot:  The text, which can be up to 256 characters, wraps on the username screen, but not on the password field.  I’ll check on getting that fixed.)
• Notice that the “user name hint” property specified in AAD is ignored.  (I’ll check on that too.)

Some of you might have noticed a different password screen too.  If you are using ADFS, you’ll be presented with a web view to specify the password (since it’s the ADFS servers verifying the password for Azure AD).  ADFS has its own customization capabilities, described at https://docs.microsoft.com/en-us/windows-server/identity/ad-fs/operations/ad-fs-user-sign-in-customization, which includes this useful sample:

Also note that you can have Intune present a “terms and conditions” page (presented in a web view as part of the MDM enrollment process).  See https://docs.microsoft.com/en-us/intune/terms-and-conditions-create for details on how to set that up.

I’ve recorded a variety of videos (especially for Windows AutoPilot, but prior to that for other Windows deployment scenarios) that involve capturing the initial phases of a Hyper-V VM’s boot process.  Normally, these default to a resolution like 1024x768, although they can be changed to a better resolution (e.g. 1366x768 or 1920x1080) manually after Windows has started up.  The end result is a rather “square” image:

But what if you want the entire video to be fixed at a particular resolution?  If you are running Hyper-V on Windows 10 or Windows Server 2016, there is an easy way to do this using the Set-VMVideo PowerShell cmdlet.  First, let’s try switching to 1366x768:

PS C:\> Set-VMVideo -VMName Demo -ResolutionType Single -HorizontalResolution 1366 -VerticalResolution 768

The result (effective the next time the VM starts up):

The "-ResolutionType Single” is required in this case to force the VM to a specific resolution by telling Windows that is the only supported video resolution.  (And yes, OOBE does add black bars to left and right.)

You can also change to 1080p (1920x1080) the same way:

PS C:\> Set-VMVideo -VMName Demo -ResolutionType Single -HorizontalResolution 1920 -VerticalResolution 1080

with this result (yes, more and bigger black bars from OOBE):

You may still see the VM start up with a smaller resolution and then switching to a higher resolution a second or two later.  There’s no easy way around that, but if you set the VM to be full screen (e.g. using a 1080p monitor) and then capture a video from that monitor, you don’t notice the switching.

If you want to change the VM back to its original, default behavior, you can:

PS C:\> Set-VMVideo -vmname Demo -ResolutionType Default

I’ve been surprised by the number of organizations that have never tried to join a Windows 10 device into Azure Active Directory.  So let’s spend some time talking about that.

You’ve been able to join a Windows device to Active Directory domains for as long as there have been Active Directory domains.  With Windows 10, there is now the ability to join Azure Active Directory.  Azure Active Directory is a multitenant directory, so you aren’t joining a domain, you’re joining a tenant.  And most of your organizations already have such a tenant, since it’s used by Office 365, Intune, Azure services, and many other things.  More reading if you want to know more:

• What is Azure Active Directory?  Summary:  It’s a multitenant, cloud-based directory and identity management service.
• Set up Azure AD joined devices.  Summary:  You can join a device to Azure AD via Settings, but you can also do it via OOBE or via a provisioning package.
• Azure Active Directory Features.  Summary:  Azure AD has lots of capabilities, well beyond just joining devices and authenticating sign-ins.  There are free and paid versions; free is good enough for Azure AD join.
• Integrate your on-premises directories with Azure Active Directory.  Summary: You can sync all your AD accounts with Azure AD, with three options for validating passwords:  Leave them in AD only and use Windows Servers with ADFS to validate them; leave them in AD only and leverage Azure Active Directory Pass-Through Authentication, or synchronize password hashes between AD and Azure AD.  (If you are just starting, Pass-Through is really simple, ADFS provides more advanced capabilities, and password hash synchronization often gives security teams high blood pressure.)

And a couple of things that Azure AD is not:

• It’s not Active Directory running in the cloud.  (We do provide a service like that, called Azure Active Directory Domain Services, designed for scenarios where you want to join Azure VMs into an AD domain.  It’s not designed for on-prem PCs or servers.  And yes, that’s a confusing name.)
• It’s not for device management.  (To manage devices joined to Azure AD, you would use Intune or another MDM service, potentially combined with System Center Configuration Manager via co-management.)

To confuse matters a little more, there is one more scenario to discuss called Hybrid Azure Active Directory Join.  Why is this confusing?  Well, the device is joined to Active Directory (just like most of you do today), so it has access to all the AD features (e.g. GPOs).  If you look at the Settings app, you won’t see anything about Azure AD.  For all intents and purposes, it behaves just like an AD-joined device (because it is), with a few extra benefits.  For the purposes of this blog post, we’ll ignore this scenario because we want to focus on devices that are joined directly to Azure AD, and users who sign on using Azure AD credentials, with no need for any domain controllers or any other on-prem resources.

Setup

Let’s shift focus then to setting all of this up.  I performed the following tasks:

• Using my MSDN Azure subscription, I created a new Azure AD tenant called “autopilotrocks.onmicrosoft.com” and added a cloud-only admin account to it, admin@autopilotrocks.onmicrosoft.com (used later).
• Using my GoDaddy account, I created a new “autopilotrocks.com” DNS domain, then added it as a custom domain to the Azure AD tenant.  (You don’t have to do this, but it’s much nicer to sign on with @autopilotrocks.com instead of @autopilotrocks.onmicrosoft.com.)
• I created a new Windows Server 2016 domain controller for “autopilotrocks.com” (the AD domain/forest).  (I made this a DHCP server on its own private Hyper-V network, and connected it to an RRAS server for routing to the internet.  Internet connectivity is needed.)  I created a new domain account, Michael, so that I have an account to use later.
• I download, installed and configured AAD Connect to sync my AD domain to AAD, setting up Pass-Through Authentication to make things simple.  (Setting up ADFS is a fair amount of work.)  The click-by-click:
  
  I won’t use express settings because I want to see the options available:
  
  Nothing I want to customize here:
  
  A few pre-reqs get installed:
  
  Then other configuration is done:
  
  Now I can choose Pass-Through Authentication:
  
  And specify my cloud admin account:
  
  Then I can click to add my AD domain:
  
  And specify the domain admin details (because I’m too lazy to create a separate account):
  
  I then clicked “Next” through the remaining screens, and then confirmed on the final screen by clicking Install:
  
  And finally, done:
  
  

Total elapsed time: about two hours.  Total cost: $12.17 to register the new autopilotrocks.com domain name – there are no Azure costs at all.

One final piece worth pointing out:  By default, all accounts have rights to join devices to Azure AD.  That can be changed if you want via the Azure portal (but if you are planning to use Windows AutoPilot, I wouldn’t change the default):

Adding a device

So now on to the next piece:  Joining a device into Azure AD.  As I mentioned before, there are three ways of doing this.  Since you’ve already shown some patience by reading this far, we might as well walk through all three.

1. Join from OOBE

After clicking through the first few screens in OOBE, you are asked to specify Azure AD credentials:

Note that in this case the machine didn’t ask if this was a work or home machine, because I’m using the Enterprise SKU.  (It also won’t ask if it finds that this device is registered with AutoPilot.  But this device isn’t registered with AutoPilot – at least not yet.)

I can type in my Azure AD account that I created earlier in AD and synced with AAD Connect into Azure AD:

and then I can type in my password, which is automatically validated against my AD domain controller using Azure AD Pass-Through Authentication (woo-hoo, works great).  After a few more OOBE screens, the device is joined and the user is automatically signed on.  Once at the desktop, you can see in Settings that the device is joined to Azure AD:

(Notice how there is no “Info” button?  That’s a sign that this device isn’t MDM-managed.)  It’s also interesting to look at this from a command prompt using “dsregcmd.exe /status”:

2. Join from Settings

If you have an existing install with a device that’s using a local account in a workgroup, or an MSA, you can join the device to Azure AD through the Settings app.  Navigate to Accounts –> Access Work or School and then click the Connect button:

While you might be tempted to type in the user ID (e-mail address), that would just “add a work account” – that’s not the same as Azure AD join.  To do the Azure AD join, click the link lower on the wizard pane that says “Join this device to Azure Active Directory”:

Then type in the user ID (e-mail address):

and password:

A confirmation is needed:

And then we’re done:

We can see the end result is exactly the same as doing this via OOBE:

3. Join via a Provisioning Package

I’m going to cheat for this one and recycle some older screenshots from Windows Configuration Designer.  To create a provisioning package to join Azure AD, you need to choose Azure AD (obviously):

Then you can click “Get Bulk Token” which will ask for Azure AD credentials:

The token (which can be used for up to 30 days to join an unlimited number of devices to Azure AD) is returned from Azure AD and stored in the provisioning package.  Install that provisioning package on the device in one of the supported ways:

• Inject it offline using DISM, before the OS boots for the first time.
• On the first OOBE screen, press the Windows key five times (really) and browse to a USB key that contains the provisioning package.
• Manually install the provisioning package via Settings:
  
• Install the provisioning package using PowerShell (using the Install-ProvisioningPackage cmdlet).
• Install the provisioning package by double-clicking on it and acknowledging the prompts.

Once installed, you can see the same views via Settings or from the DSREGCMD /Status command that we looked at previously.

Limited-Time Offer

Want to try this out yourself?  Send me an e-mail at mniehaus@microsoft.com and I’ll send you a user ID and password that you can use to join a device into my Azure AD tenant.  (There’s no MDM in place, and I reserve the right to accidentally or purposely remove the device – no a big deal since you probably won’t find a device joined to my tenant to be very useful.)

Summary

Thanks for reading this far.  You’re probably thinking “how can there possibly be a part 2” after all of this, but there certainly will be (and needs to be).  It might take me a little while to finish the next one, but stay tuned.

When I originally accepted a job with Microsoft, just over 14 years ago, I chose the company because of the sheer number of different roles that were available – lots of opportunities for new challenges.  That has taken me through roles that included consulting, development, program management, product marketing, and product management.

Starting this week, I have changed roles again, moving from Windows commercial product marketing (which covered the product marketing and product management roles above) to the Windows & Devices Group (WDG), the group that creates Windows, as a principal program manager on the modern deployment team that works on Windows AutoPilot, MDM management, subscription activation, and related technologies.

While that will likely change the typical makeup of my day quite a bit, there will certainly be areas that won’t change:  You’ll still see me blogging (something that wasn’t in my previous job description either), participating in events (talking about modern deployment and management – someone else can talk about Windows as a service), working with customers, MVPs and partners, working with various product groups to address enterprise concerns, etc.  And I’ve been contemplating some new things, e.g. podcasts, videos, and that sort of thing.  So I’m definitely not going to disappear.

The last five years in Windows commercial marketing were a lot of fun, filled with plenty of successes and a fair number of failures too.  Now I’m looking forward to the path ahead.

As always, feel free to reach out to me via e-mail (mniehaus@microsoft.com), Twitter (@mniehaus), or LinkedIn (http://www.linkedin.com/in/mniehaus/).  And keep deploying   Now, back to work…

p.s. If you send me a LinkedIn request, introduce yourself, describe how you know me, explain what you do, etc. -- one downside of having a "director of marketing" title (my old job) is that you get lots of requests from people who want to sell you things.  So if I don't recognize you and just see a generic connection request, I'm likely to ignore the request.

In the part 1 blog, I talked about the mechanics of joining Windows 10 devices to Azure AD.  Now let’s shift focus and talk about the impact of doing it.  I already mentioned that Azure AD doesn’t provide management capabilities (those should be provided via Intune, System Center Configuration Manager, or other management tools).  But what about authentication with existing on-prem resources like web servers, file servers, etc.?  That’s the focus for this post.

Baseline:  AD-joined

Let’s start with the basics, things that you expect to “just work” because of Active Directory: users in AD, signed into devices joined to AD, accessing resources in AD.  Behind the scenes, Windows 10 talks to an AD domain controller to get a Kerberos TGT (ticket-granting ticket), which can then be used to access AD-joined resources.  No big surprise, these work just fine:

• Access a UNC path on an AD member server.
• Access a AD member web server configured for Windows integrated security
• Running Active Directory Users & Computers to administer all AD objects
• Running DHCP snap-in to administer an AD-joined DHCP server

So, let’s then look at another scenario:  Using the same AD-joined servers, let’s try using an Azure AD-joined device, where the user is signing into the device using an Azure AD account that has been synchronized with the AD domain.  What do you expect to happen in that case?  Before answering that question, let’s dig in a little deeper on what happens when you sign into an Azure AD-joined device.  Instead of talking to an AD domain controller, it talks to the Azure AD service, which provides a PRT (Primary Refresh Token), which can then be used to access Azure AD-protected resources. 

Next Step:  Azure AD-joined

But what about AD-joined resources being access from an Azure AD-joined device?  The PRT doesn’t do any good in that case, because AD doesn’t know anything about it.  So to make this work, Windows 10 performs an extra step: When it sees that it is on a network with an available AD domain controller, it reaches out to that server to get a Kerberos TGT.  So now it has everything it needs to access both: a PRT for Azure AD, and a TGT for AD. 

But how does this work? Through the magic of Azure AD Connect, Windows knows the AD domain name (synchronized from the on-prem AD into Azure AD) and can use that to look for an AD domain controller.  When it finds an AD domain controller, it can they get the TGT  needed to access AD-joined resources.  Let’s look at the results:

• Access a UNC path on an AD member server
• Works perfectly.
• Access a AD member web server configured for Windows integrated
  security
• Works perfectly.
• Running Active Directory Users & Computers to administer all AD
  objects
• You have to manually specify the domain that you want to connect to, as the MMC snap-in can’t use LDAP to figure that out itself.  Just use "Change domain” and then it will work perfectly.
• Running DHCP snap-in to administer an AD-joined DHCP server
• You may need to specify the DHCP server name or address (may work automatically if you used another AD-linked MMC snap-in first), but once you do it will work perfectly.

No surprises, you can successfully authenticate, although you might need to help some tools locate the AD domain you are trying to administer or the resource you are trying to access (if it’s located using AD).

Another Twist:  Windows Hello for Business

By default, devices joined to Azure AD will be enabled for Windows Hello for Business, which enables devices to sign on using a PIN or biometrics.  (You can turn it off via MDM policies, but you’re not doing your users any favors by doing so – avoiding passwords is a great thing.)  This works well when accessing only Azure AD resources, but does have some implications when accessing AD-joined resources.

For me, this was definitely a learning experience, pouring over the Windows Hello for Business documentation, including a detailed deployment guide.  I could probably write several blogs about that, but I’ll just provide the cliff notes here:

• You want to do a “hybrid” deployment of Windows Hello for Business.
• There are two choices for setting this up:
• Key trust, which requires at least one Windows Server 2016 DC, Certificate Services set up in the domain, Kerberos certificates on each DC, the trusted root cert deployed to each device using Intune, and an accessible certificate revocation list (CRL).
• Certificate trust, which needs the Windows Server 2016 schema, but works with older DCs.  But for this to work, you need to issue certificates to everyone, which requires NDES, more Intune configuration, deploying the trusted root cert, and having an accessible CRL.  (You’ve probably already done this work if you are using smart cards, so it might not be so bad).

Once you have either a key trust or certificate trust set up and properly configured (took me several days to get the CRL right), then all the above tests work just as they did without Hello for Business:

• Access a UNC path on an AD member server
• Works perfectly.
• Access a AD member web server configured for Windows integrated
  security
• Works perfectly.
• Running Active Directory Users & Computers to administer all AD
  objects
• You have to manually specify the domain that you want to connect to, as the
  MMC snap-in can’t use LDAP to figure that out itself.  Just use "Change
  domain” and then it will work perfectly.
• Running DHCP snap-in to administer an AD-joined DHCP server
• You may need to specify the DHCP server name or address (may work
  automatically if you used another AD-linked MMC snap-in first), but once you do
  it will work perfectly.

More Options Available

Also note that you might choose to leverage something like the Azure AD Application Proxy to access internal web apps.  With that, you can authenticate using Azure AD (including support for multi-factor authorization) and let Azure handle the internal authentication and redirection for you (no client-side VPN required).  And as announce last week, you can now configure these apps using PowerShell; see https://cloudblogs.microsoft.com/enterprisemobility/2018/02/15/faster-application-deployment-in-azure-ad-application-proxy/ for more details.

Summary

For those afraid of joining devices to Azure AD because of the number of Active Directory-secured apps, services, shares, etc. that are being used in your environment, I think you’ll be presently surprised at how well it can work.  Obviously your results may vary (people always find creative ways to configure things, so there’s definitely a chance that some apps might not work, at least not without some reconfiguration), so you still want to do some testing on your own.  But it will likely work well enough for you to try it out yourself – take a few devices, maybe even your own primary device, and join it to Azure AD to see what happens.  (See part 1 for more details on that.)

Back in December, a new build of MDT was released.  Continuing the pattern established with the release before it (8443), it’s identified only by its build number, hence it’s called MDT 8450.  As Aaron detailed in the announcement blog at https://blogs.technet.microsoft.com/msdeployment/2017/12/21/mdt-8450-now-available/, this release is primarily for compatibility with the latest ADK, Windows 10, and ConfigMgr releases, and includes fixes for a variety of bugs (with the full list included).

Here’s a quick rundown of every change made to the MDT scripts and templates for those that are curious:

• Templates:
• SCCM_Client.xml.  Changed the UEFI recovery partition size from 300MB to 499MB to make sure it’s big enough (same as what MDT was already using for Lite Touch).
• SCCM_Server.xml.  Changed the UEFI recovery partition size from 300MB to 499MB to make sure it’s big enough (same as what MDT was already using for Lite Touch).
• Scripts:
• DeployWiz_ProductKeyVista.vbs.  Fixed logic that caused an “invalid deployment type” error.
• LiteTouch.wsf.  Fixed a variety of logic related to LTI Windows 10 upgrade task sequences (which also fixed an issue related to autologon after a reboot prior to the upgrade).
• LTIApply.wsf.  Changed the BCDBOOT logic to always run it the same way for UEFI devices, regardless of the OS version, to address some boot-loop issues on bare metal UEFI deployments on some devices.
• LTICleanup.wsf.  Inconsequential changes (line alignment).
• ServerManager.xml.  Fixed component names that prevented the installation of Windows Media and IIS Management Console features on Windows Server 2016.
• SetupComplete.cmd.  Fixed logic related to the changes in LiteTouch.wsf for Windows 10 upgrade task sequences.
• SetupRollback.cmd.  Ditto.
• ZTIBde.wsf.  Changed the BitLocker pre-provisioning logic to not try to do anything with the TPM while in Windows PE, to avoid putting the TPM into a reduced functionality state.
• ZTIGather.wsf.  Added some new chassis types (30, 31, 32 for laptops; 35 and 36 for desktops; 28 for servers).
• ZTIMoveStateStore.wsf.  Fixed the logic that moved the state store so that it didn’t use a hard-coded \StateStore folder location.
• ZTIOSRole.wsf.  Fixed the logic so that it works for multiple calls to get the source location.
• ZTIUtility.vbs.  Fixed logic to ignore disabled “Install Operating System” steps (caused problems with some types of task sequences).

Additionally, all the standalone task sequencer binaries (used to run LTI task sequences) were updated to the latest code from ConfigMgr.

So, it’s a very minor update overall.  If you have existing task sequences created with MDT 8443, you shouldn’t need to recreate them, although if you have ConfigMgr task sequences you might want to edit the “Format and Partition” UEFI steps to specify 499MB instead of 300MB for the recovery partition size.

And as always, back up your deployment share before upgrading (especially if you’ve made any script edits), reintegrate your changes if needed, and make sure you update your boot images (including on WDS, USB boot media, boot ISOs, etc.) as mismatched versions will cause all sorts of problems.

I’ve stated this publicly a few times, but many people don’t realize the distinction so I’ll repeat it again here.  There are provisioned apps included in Windows 10 (that you can remove, see https://blogs.technet.microsoft.com/mniehaus/2015/11/11/removing-windows-10-in-box-apps-during-a-task-sequence/) and there are apps that are installed from the Microsoft Store when you first sign in (that you can prevent from installing on some SKUs, see https://blogs.technet.microsoft.com/mniehaus/2015/11/23/seeing-extra-apps-turn-them-off/).  For example, Weather is a provisioned app that you can remove; Network Speed Test is an app that comes from the store when the user signs in.

The documentation team has an article at https://docs.microsoft.com/en-us/windows/application-management/apps-in-windows-10 that has the current list of provisioned (in-box) and installed (from the store on first logon) for various Windows 10 releases.  But one thing that isn’t reflected in that documentation:  the list of apps installed from the store can be different depending on the type of account you sign in with, by the SKU that you are using, and even by the region of the world the device is in.

This is what the default start menu layout looks like in all Windows 10 1709 SKUs:

Each of those down-arrow icons will be replaced by an app from the store (some of which are actually installed from the store, some of which are just “shortcuts” that will trigger the installation if you launch them), after the user signs in for the first time.

So let’s do a little comparison.  First, let’s look at a local account being used on Windows 10 Pro 1709:

OK, very consumer-y, with lots of games.  Now, let’s look at a local account signing into Windows 10 Enterprise 1709:

Much better – no games, with most of the apps focused more on productivity.  And signing into Windows 10 Enterprise 1709 with a Microsoft Account, an Active Directory account, or an Azure Active Directory account yields the same result.

But on Windows 10 Pro 1709, you will see a different result when you sign in with an Active Directory account or an Azure Active Directory account.  They will look the same as Windows 10 Enterprise 1709:

So a quick summary: You’ll get games on Windows 10 Pro 1709 only when using local or MSA accounts; you’ll lever get games for AD or AAD users, or for anyone on Windows 10 Enterprise. 

Also note that Windows 10 Pro for Workstations will soon have the same behavior as Windows 10 Enterprise, as described in the Windows Insider blog:

Productivity focused out of box applications: In the Windows 10 Fall Creators Update, the out of box experience for Windows 10 Pro for Workstations draws from the Pro edition of Windows 10. One area where this is noticeable is the suite of applications installed out of the box visible as tiles in the start menu. In the next release for Windows, you will see for Windows 10 Pro for Workstations productivity and enterprise focused applications in place of consumer applications and games. This was one of the top feedback shared with us by our partners and users and we’re delivering this in our next update.

Also note that on Windows 10 Enterprise, you can turn off the installation of apps from the store altogether by configuring the “Turn off Microsoft consumer features” GPO or the equivalent MDM policy.

After some campaigning and some quick work from the ConfigMgr team for the ConfigMgr tech preview 1802 release, you can now use non-OSD task sequences, such as those that install apps or perform feature updates, even for internet-connected (and Azure AD-joined) devices.  See https://docs.microsoft.com/en-us/sccm/core/get-started/capabilities-in-technical-preview-1802#windows-10-in-place-upgrade-task-sequence-via-cloud-management-gateway for more details.

Here’s a snapshot of what that looks like, first showing an internet-connected client (connecting through the ConfigMgr cloud management gateway) running the TP1802 client (build 8627):

And then the view in Software Center:

where you can see the task sequence named “Non-OSD Task Sequence” (just to be clear, that’s what I named it).  And all I had to do to get that to show up (after setting up the cloud management gateway and installing TP1802) is to check the “Allow task sequences to run for client on the internet” box:

And if you look back at the Software Center picture above, you can also see another new feature in TP1802 too: It’s showing apps that target users, even on this Azure AD-joined device.  That feature is described at https://docs.microsoft.com/en-us/sccm/core/get-started/capabilities-in-technical-preview-1802#use-software-center-to-browse-and-install-user-available-applications-on-azure-ad-joined-devices.

So in a short period of time, ConfigMgr has gained some important new capabilities for Azure AD-joined devices, which will be very useful for Windows AutoPilot deployments.

Since most of us have been using Active Directory for ages, you probably also take for granted the process for printing to your on-premises printers:  They are published to Active Directory, you can search for them, and you can easily connect to them.  But what happens when you join your devices to Azure AD instead?  As discussed in https://blogs.technet.microsoft.com/mniehaus/2018/02/21/afraid-of-windows-10-with-azure-ad-join-try-it-out-part-2/, you can certainly connect to a Windows Server (AD-joined) hosted print queue if you know its UNC path (authentication works via a Kerberos ticket).  But how do you discover that printer in the first place?  The low-tech approach is to label the printer with its UNC path, so you just walk up to it, read the UNC, and type it in.

Fortunately, we’ve released a higher-tech approach that solves two different problems:

• Discovering Windows Server-hosted printers from Azure AD-joined devices
• Printing to those printers from anywhere in the world

This solution is called Hybrid Cloud Printing because it connects those Azure AD-joined devices to your existing Active Directory-joined Windows Server printing infrastructure.  That was announced back in February via the Enterprise Mobility blog.  As you can probably guess from the diagram included in that blog, there are a few components in the solution:

First, some requirements:

• Windows 10 1703 (Creators Update) or higher.
• Windows Server 2016 on your print servers.
• An Azure AD tenant.
• Azure AD Connect, to synchronize your Active Directory with Azure AD.
• An MDM service, e.g. Intune, to configure the print settings on each device.

Then, you need to set it up.  To make the connection from internet-facing Azure AD-joined devices to those on-prem Windows Server 2016-hosted services, Azure Application Proxy is used.  This makes an outbound connection to Azure, which is used to then allow inbound traffic to the published services.  There are two ways this can be set up:

• Azure Active Directory pre-authentication, where Azure AD makes sure the user is authenticated before the traffic passes through the proxy.
• Passthrough authentication, letting Windows Server authenticate the users via Kerberos.

Typically the first option is recommended, to support things like conditional access and multi-factor authentication.  I recommend sticking with that that recommendation, following the “five” steps at https://docs.microsoft.com/en-us/windows-server/administration/hybrid-cloud-print/hybrid-cloud-print-deploy:

1. Install Azure AD Connect to sync between Azure AD and AD.  I covered that in https://blogs.technet.microsoft.com/mniehaus/2018/01/19/afraid-of-windows-10-with-azure-ad-join-try-it-out-part-1/.
2. Install the Cloud Print package on the print server.
3. Install the Azure Application Proxy.
4. Configure the MDM policies.
5. Publish desired shared printers.

Since each of those steps includes multiple sub-steps, it’s really more like a 20-25 step process.  And initially I walked through all of those steps – and it didn’t work at all.  Fortunately, one of the ConfigMgr/Intune MVPs, Sandy Yinghua, has a blog that documents how she did it.  I used those steps to verify my configuration and with that was able to get everything working.  I would note that I did things a little differently:

• I didn’t use a custom internet URL (mcs.smsboot.com in her example) for the Azure App Proxy-published websites.  Instead, I used the generated msappproxy.net URLs because then I didn’t need additional certs.
• I did get a public cert for my print server, so that I didn’t need to distribute a trusted root cert to the client machines when they talk to the print server via the Azure App Proxy.  Self-signed certs aren’t any fun (although Intune has no issues deploying them, so it wouldn’t be that bad).

But otherwise, my setup matches.  Here are the device configuration settings I configured in Intune:

• Printer discovery URL (my configured Azure App Proxy URL, pointing to the internal print server’s IIS):
  https://clouddiscoveryproxy-autopilotrocks.msappproxy.net/mcs/
• Print access authority URL (the logon URL for your Azure AD tenant, with the GUID from the tenant properties):
  https://login.microsoftonline.com/0b458c6e-97ce-4c7e-bc5e-1d29552989a5
• Azure native client app GUID (copied earlier from the native API app that was added):
  23772c3e-1a3b-4f28-86f9-ec614a53a776
• Print service resource URI (the registered app):
  http://MicrosoftEnterpriseCloudPrint/CloudPrint
• Maximum printers to query (Mobile only) (guess a number):
  500
• Printer discovery service resource URI (the other registered app):
  http://MopriaDiscoveryService/CloudPrint

What those look like in Intune:

And here are a few commands that I used as I was experimenting with the printer publishing process:

Publish a test printer

Publish-CloudPrinter -Printer EcpPrintTest -Manufacturer Microsoft -Model FilePrinterEcp -OrgLocation '{"attrs": [{"category":"country", "vs":"USA", "depth":0}, {"category":"organization", "vs":"AutoPilotRocks", "depth":1}, {"category":"site", "vs":"Redmond", "depth":2}, {"category":"building", "vs":"109", "depth":3}]}’ -Sddl "O:BAG:SYD:(A;;LCSWSDRCWDWO;;;S-1-5-21-501278528-1731656756-2472999879-1114)(A;OIIO;RPWPSDRCWDWO;;;S-1-5-21-501278528-1731656756-2472999879-1114)(A;OIIO;GA;;;CO)(A;OIIO;GA;;;AC)(A;;SWRC;;;WD)(A;CIIO;GX;;;WD)(A;;SWRC;;;AC)(A;CIIO;GX;;;AC)(A;;LCSWDTSDRCWDWO;;;BA)(A;OICIIO;GA;;;BA)" -DiscoveryEndpoint https://clouddiscoveryproxy-autopilotrocks.msappproxy.net/mcs -PrintServerEndpoint https://cloudprintproxy-autopilotrocks.msappproxy.net/ecp -AzureClientId 23772c3e-1a3b-4f28-86f9-ec614a53a776 -AzureTenantGuid 0b458c6e-97ce-4c7e-bc5e-1d29552989a5

Query published printers

Publish-CloudPrinter -Query -DiscoveryEndpoint https://clouddiscoveryproxy-autopilotrocks.msappproxy.net/mcs -AzureClientId 23772c3e-1a3b-4f28-86f9-ec614a53a776 -AzureTenantGuid 0b458c6e-97ce-4c7e-bc5e-1d29552989a5

Unpublish a printer

Publish-CloudPrinter -Unpublish -Printer EcpPrintTest -DiscoveryEndpoint https://clouddiscoveryproxy-autopilotrocks.msappproxy.net/mcs -PrintServerEndpoint https://cloudprintproxy-autopilotrocks.msappproxy.net/ecp -AzureClientId 23772c3e-1a3b-4f28-86f9-ec614a53a776 -AzureTenantGuid 0b458c6e-97ce-4c7e-bc5e-1d29552989a5

Trying it out

Let’s look at the client experience then.  The first sign of something different is on the “Printers & scanners” page in Settings.  There is a new “Search for cloud printers” link:

When you click that, you can then browse from a list of available locations, presented in a hierarchy that you can define:

And you can search for printers by keyword (or just leave the keyword blank to get all printers in that location):

And finally, select the printer and click “Add device” to add it:

Finally, I have a printer, with a slightly different icon to show that it’s a cloud printer:

And printing works, from any internet-connected location (thanks to the Azure Application Proxy).  The particular “printer” in this case is useful for testing, as it doesn’t waste any paper:  It just drops an XPS file into a folder on the server.

Check out the Ignite 2017 video at https://www.youtube.com/watch?v=Bvt1L--lqE4 for more information.  And with any luck, this will be integrated into Windows Server 2019 and easier to set up.

A while back, I published a sample script called Get-WindowsAutoPilotInfo, which is designed to help define existing computers with the Windows AutoPilot deployment service, See https://blogs.technet.microsoft.com/mniehaus/2017/12/12/gathering-windows-autopilot-hardware-details-from-existing-machines/ for more details about that.

Some people have asked to take this one more step, to automate the process of uploading the gathered hardware details.  Fortunately, this can be done using the Intune Graph API (exactly what the Intune portal uses behind the scenes).  To show you how to do that, I’ve published a sample PowerShell module called WindowsAutoPilotIntune on the PowerShell Gallery.  Here’s an overview of how to use it:

Install the needed modules

There are two modules needed, the WindowsAutoPilotIntune module and the AzureAD module that it uses for authenticating with Intune.  These can both be installed using the Install-Module PowerShell cmdlet:

Install-Module AzureAD
Install-Module WindowsAutoPilotIntune

Here’s what that looks like:

Then we can exercise it a little by getting a list of the devices that have already been uploaded.  First, load the module and connect to Intune by first specifying the user to use:

Import-Module WindowsAutoPilotIntune
Connect-AutoPilotIntune

After specifying the user principal, you’ll be prompted for a password (and if this is the first time you’ve used the Intune Graph APIs, you’ll also be prompted for permission).  Make sure you specify a user that has sufficient rights.  (Note that the authentication will time out after a period of time – the Azure AD sign-in process generates an access token that only has a limited life.)

Then you can get a list of devices:

Get-AutoPilotDevice

So let’s try something a little more involved: Adding some new devices.  I’ll start with a CSV file created with the previously-mentioned Get-WindowsAutoPilotInfo script.  It has five machines in it, one of which have already been imported (to show what an error looks like):

Import-AutoPilotCSV -csvFile C:\Demo.csv

As each device is added to Intune, each new object created in Intune is returned and displayed.  But that doesn’t mean that they are immediately added to the AutoPilot deployment service.  Instead, Intune takes care of this as a background process running asynchronously.  We can check on the progress to see if it is done yet, and once it is done we can see if the devices were added successfully.

The script keeps checking every couple of seconds, and shows you the number of devices that are still being worked on.  It looks like there is no progress being made, but then all complete at the same time (all part of the same batch).  Once all the devices are done, the final status is retrieved and displayed, and as we can see 4 devices were added successfully, while one had a 640 StorageError (which in this case means the device is already registered).

One final step is performed after the final status is retrieved: an object for each device is deleted, leaving just the four new devices, which can be seen with the original Get-AutoPilotDevice cmdlet.

Check out the Import-AutoPilotCSV function to see a few other functions that are used to complete this task:

• Get-AutoPilotImportedDevice, which retrieves the status objects for each device being added.
• Add-AutoPilotImportedDevice, which adds the device.  (Notice that you can specify an order ID for these devices, which could be useful for grouping devices using dynamic Azure AD device groups.)
• Remove-AutoPilotImportedDevice, which removes the status objects once the processing is complete.

Notice that there is no function to remove the added devices.  Just like in the Intune portal, there’s no way to do this yet – you can remove the device via the Microsoft Store for Business, but you’ll still see it in Intune.  That’s still being worked on.

As always, the PowerShell module is provided as an example, provided as-is with no warranty or support, and could easily have bugs. See the Intune Graph API documentation for more details on the REST calls being leveraged, and the PowerShell Intune Samples on GitHub for more on interacting with Intune via the Graph API. 

Let me know via comments on this blog, via Twitter (@mniehaus), or via e-mail (mniehaus@microsoft.com) if you are planning to use this, as we’re interested in understanding the scenarios you might be considering.

As we documented at https://support.microsoft.com/en-sg/help/2769827/sysprep-fails-after-you-remove-or-update-windows-store-apps-that-inclu, sysprep will fail when creating a custom Windows 10 image if there are any apps installed during the image creation process that aren’t also provisioned in the OS.  Depending on how you create your image, this can be problematic because during the process (typically an MDT or ConfigMgr task sequence), the Microsoft Store can download and install stuff from the internet.  And when that happens, you’ll see error messages logged by sysprep (which you can find in the C:\Windows\System32\Sysprep\Panther\Setupact.log file) like this:

SYSPRP Package Microsoft.MicrosoftOfficeHub_17.8830.7600.0_x64__8wekyb3d8bbwe was installed for a user, but not provisioned for all users. This package will not function properly in the sysprep image.

The longer your image creation process takes, the more likely such a failure is.  That also means this failure can be sporadic: sometimes you’ll sysprep before the Store installs updates, other times you won’t.

So how do you fix this?  My favorite way:  Don’t build your images on a VM that has internet access, since there’s no way the Microsoft Store can download any apps when there is no internet connectivity.  Or you can try to purposely disable internet access, using something like Brandon’s blog.

But if you want to have internet access or need internet access, there are other ways.  You can follow Johan’s blog, which leverages existing Group Policy settings (which are a little tricky to set on a workgroup machine when creating an image, but there are scripts to help with that).  Or you can add an extra step to the task sequence to clean up the offending apps prior to running sysprep.  Just to see how hard that would be, I decided to try it – turns out it’s not that bad.

But first, let’s set up a repro.  The easiest way to do that is to add a step to an MDT task sequence to suspend the task sequence.  Then you can launch the Store, force the installation of app updates, install other apps (I installed Facebook manually for good measure), etc.  (And if you think doing this manually has any value, e.g. trying to get new versions of the apps in your image, you would be wrong.)

So now I have a VM that is guaranteed to fail.  So let’s look at what I’ve added to the task sequence to fix this:

That step is placed right before the “Execute Sysprep” step (so if you are doing a sysprep-only task sequence execution, you would want to add it to the “Sysprep Only” group too).  The full command line:

powershell.exe -executionpolicy bypass -file %ScriptRoot%\RemoveUserApps.ps1

Resuming the task sequence shows the step running:

Then sysprep runs:

And if that succeeds, the machine will reboot to Windows PE and capture the image.

So yeah, success!

The script itself is attached to a zip file at the bottom of this blog.  While it gets the job done, it’s not perfect:

• It tries to remove framework packages (basically, dependencies) and fails.  No big deal, as sysprep doesn’t really care about these.  Ideally the script would filter these out, but it’s not worth the effort.
• It can’t count.  It tries to count up the number of apps that it removes, but doesn’t take into account failures (e.g. those same framework packages).  Error handling wasn’t worth the effort.

Feel free to try it out yourself.

RemoveUserApps.zip