Outlook 2016 constantly prompts for password on first start with Azure AD / Office 365

Hey folks,

After many years since my last, I’m doing yet another Office 365 migration. I’m fortunate that everyone is on Outlook 2016 and Windows 10… not too much legacy baggage there. This time around, I’m using Azure AD Sync + Pass-through authentication. The last migration I did, this functionality wasn’t even available.. I had to use Okta and then eventually OneLogin to accomplish my goals.

At first, everything seemed to be working fine, no password prompts using OWA with IE/Edge and Chrome. This was all good for me, since I tend to use OWS 100% when I’m on Office 365.

SADLY: Outlook itself was prompting for a password on first start / launch. The dreaded Outlook credential begging window in all its glory:

I went back through the manuals, looked at all of my Azure AD config… couldn’t figure it out. In my desperation, I ran across this blog: Jaap Wesselius: Single Sign-On and Azure AD Connect Pass-Through Authentication

The key step I missed (or didn’t know about?):

Set-OrganizationConfig -OAuth2ClientProfileEnabled:$true

After enabling that, I was immediately in business and no more password prompting from Outlook! Hopefully this helps someone, because this is yet another dark alley I see a lot of people struggling through.

Til next time…

Making Datadog report mount points vs. device names

This will seem like a really dumb post, but I scratched my head on this for a bit. I really wanted my Datadog storage dashboards to report mount point name vs. device names. I couldn’t really find much help within Datadog’s documentation either.

After going down a few paths, I bumped into this in the datadog.conf:

# Use mount points instead of volumes to track disk and fs metrics
use_mount: no

I changed that over to yes, and boom … problem solved!

EMC Avamar Windows Server 2008 R2 VSS backup fails with: System Writer is not present

Welcome back everyone,

Today’s random backup failure is brought to you by the number: infinity. Well, thats how many .NET temp files I seemed to have on a server that refused to complete its VSS backup.

On this particular SharePoint 2010 machine, when a VSS backup would run it bombed with this error:

2013-07-11 18:04:50 avvss Info : VSS: Creating vss version 6.0 or greater object
2013-07-11 18:04:50 avvss Info : Gathering writer metadata…
2013-07-11 18:04:51 avvss Error : Can not continue disaster recovery backup because the System Writer is not present, exiting.
2013-07-11 18:04:51 avvss Info : Final summary generated subwork 0, cancelled/aborted 0, snapview 0, exitcode 536870919
2013-07-11 18:04:56 avvss Info : uvss returning with exitcode 536870919

I tried all of the usual VSS writer DLL re-register & permissions fix tricks I knew (which technically aren’t recommended on Server 2008 R2!), but alas nothing would bring the System Writer back. Becoming almost apathetic about the issue, I then bumbled onto this TechNet Social post.

It gets interesting about halfway down with a post from “Rosaceae” & “Microbolt”. I’ve quoted their discussion below, should that link ever die.

I checked out my .NET Framework temp directories, and there were about 100k files in there going back to 2009. I cleaned them out, restarted the Cryptographic Service and wouldn’t you know it, the VSS System Writer came back and my backup was successful!

By the way, the Cryptographic Service is probably about the most unintuitive service name that could relate to a VSS component Microsoft could think of.

I’m going to keep my eye on this and see if I end up needing to relocate my .NET Framework temp files like Microbolt did, but I’m guessing not. It looked as if some developer was trying out some new/bad code and caused it.

So thanks to both of those people, I would’ve been stumped without it.

Till next time…

Hi,
I’ve got this problem about a month ago. I refer to MSP.
The problem was caused due to stack full. When we list system writer using “vssadmin list writers”, it will go through all the system files. To do that, the OS use a search algorithm with a stack which has a size limitation of 1000. When the stack was full, it failed to continue listing files and log an event in the application event log.
In my case, the following folder contains too many subdirectory and caused the problem.
C:\Windows\Microsoft.Net\Framework64\v2.0.50727\Temporary ASP.NET Files\*
1. Open C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\CONFIG\Web.config
2. Add tempDirectory attribute to compilation tag. For example:
< compilation tempDirectory=”c:\ASPTEMP”>
And also grant the folder with the same privilege with as “C:\Windows\Microsoft.Net\Framework64\v2.0.50727\Temporary ASP.NET Files”.
3. Restart the IIS Service.
4. Backup and delete all files under “C:\Windows\Microsoft.Net\Framework64\v2.0.50727\Temporary ASP.NET Files”.
5. restart the Cryptographic Service.
6. Try “vssadmin list writers” again.
Hopes this brings idea for you to solve it.

Thanks Rosaceae!
After going on a wild goose chase setting permissions and nothing working I got looking around in the .Net Folders per your advice. It looks like in my case I had the same issue with you except in the Framework instead of Framework64 (as most of my web apps are running x86).
I’ll share what I did incase it helps anyone (Ignore that last two of each step if you don’t have .Net 4.0 Installed):
Created 4 Folders:
C:\Asp.net Temp Files\2.0.50727\x86
C:\Asp.net Temp Files\2.0.50727\x64
C:\Asp.net Temp Files\4.0.30319\x86
C:\Asp.net Temp Files\4.0.30319\x64
Set Permissions on the folder (This is how I set them, may be different on your server. Check existing “Temporary ASP.NET Files” directory for permissions on your server
icacls “c:\Asp.net Temp Files” /grant “BUILTIN\Administrators:(OI)(CI)(F)”
icacls “c:\Asp.net Temp Files” /grant “NT AUTHORITY\SYSTEM:(OI)(CI)(M,WDAC,DC)”
icacls “c:\Asp.net Temp Files” /grant “CREATOR OWNER:(OI)(CI)(IO)(F)”
icacls “c:\Asp.net Temp Files” /grant “BUILTIN\IIS_IUSRS:(OI)(CI)(M,WDAC,DC)”
icacls “c:\Asp.net Temp Files” /grant “BUILTIN\Users:(OI)(CI)(RX)”
icacls “c:\Asp.net Temp Files” /grant “NT SERVICE\TrustedInstaller:(CI)(F)”
Add tempDirectory attribute to compilation tag. This will keep you from having the problem again in the future. Add the following attribute to these files:

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\CONFIG\Web.config

C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\CONFIG\Web.config

C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\CONFIG\Web.config

C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319\CONFIG\Web.config

Restart IIS so that it will use the new Temp Directory
iisreset
Deleted old Temp Files
rmdir /s /q “C:\Windows\Microsoft.Net\Framework64\v2.0.50727\Temporary ASP.NET Files\root”
rmdir /s /q “C:\Windows\Microsoft.Net\Framework\v2.0.50727\Temporary ASP.NET Files\root”
rmdir /s /q “C:\Windows\Microsoft.Net\Framework64\v4.0.30319\Temporary ASP.NET Files\root”
rmdir /s /q “C:\Windows\Microsoft.Net\Framework\v4.0.30319\Temporary ASP.NET Files\root”
Restart Cryptographic Service
net stop cryptsvc
net start cryptsvc
Now if all goes well you should be able to see the “System Writer” again!
vssadmin list writers

Configuring impersonation for use with Veeam Exchange Explorer.

So, … you just got your Veeam platform upgraded to 6.5, and now you’re ready to use Exchange Explorer to do some item level recovery hotness. You choose the “Restore to …” option and POW you get this error:

“The account does not have permission to impersonate the requested user.”

I’m not much of an Exchange engineer anymore, but I remembered something like this in Exchange 2007 when you wanted to do item level restore with EMC Avamar. To get past this, you must grant the account you’ve launched the Veeam managment console with the ability to impersonate the target user.

This is easily accomplished with the following Exchange Managment Shell command:

New-ManagementRoleAssignment -Name:impersonationAssignmentName -Role:ApplicationImpersonation -User:enter_your_account_here

If this is a role you’re not comfortable leaving enabled on this account, you can quickly remove it with this command:

Get-ManagementRoleAssignment | Where {$_.Role -eq “ApplicationImpersonation” -and $_.RoleAssigneeName -eq “enter_your_account_here“} | Remove-ManagementRoleAssignment

It’s also possible to this on a PER mailbox if you really want, but I’d just reccomend keeping this managment role enabled on your Veeam service account.

As a side note…. Exchange Explorer is a pretty amazing plug-in for Veeam 6.5. The features they continue to develop on this software is amazing, especially when you consider its price point.

Til next time…

VMware vSphere ADAM/LDS issues after an in-place OS upgrade.

I’m not a huge fan of doing in-place OS upgrades, but sometimes its just a necessary evil. Today I upgraded one of our vCenter servers from Server 2003 R2 64-bit to Server 2008 R2, which is a supported upgrade path from Microsoft.

The OS upgrade itself went smoothly, but about 10 minutes after the final reboot vSphere Service Status alerted on these two issues:

• LDAP replication health monitor – Failed to initialize LDAP instance manager
• LDAP backup task monitor – JoinTool initialization error

If you didn’t already know, vCenter relies upon Microsoft ADAM which was renamed Lightweight Directory Service. vCenter uses this as a repository for things like roles, license keys and many other metadata-ish stuff.

I searched around the VMware KB & Google and really didn’t find anything useful. Then I dug through the vCenter Webservices Logs (vws.log), and ran into this:

Action: Local ldap environment verification
Problem: LDAP tools not found in “C:WindowsADAM”

I zeroed in on: Problem: LDAP tools not found in “C:WindowsADAM and compared that directory between a known working vCenter on Server 2008 R2 and this problem child server.

Sure enough 3 files were missing:

• DSACLS.EXE
• LDIFDE.EXE
• REPADMIN.EXE

After synchronizing that directory with the working server, I restarted vCenter Web Services and the errors went away! My assumption would be that during the upgrade those files were nuked as the LDS role was re-applied.

For folks who might not know, here’s a VMware KB article for log file locations: KB1021804

Till next time…

EMC CLARiiON CX Disk Offset Configuration

I’ll be updating this with various OS method of setting the disk offset. This is mostly for me to consolidate my notes. It should be noted this is valid for MOST current EMC disk technologies, but you should always consult the documentation to make sure.

If anyone has an OS to add, or sees an error let me know.

Microsoft Windows Server 2003: 

1. Start -> Run -> cmd.exe
2. diskpart.exe
3. List disk and find the new LUN you’re wanting to offset by number
4. select disk #, where # = the LUN you wanted in step 3.
5. create partition primary align=X, where X = 32, 64 or 128 (in my case, 64).
6. format the disk in disk manager/assign a letter/use a mount point.

Microsoft Windows Server 2008:

Technically this is no longer required. This is because Server 2008 automatically sets the offset to 1MB on partition creation.

Linux / older (2.x) ESX / etc:

1. On service console, execute “fdisk /dev/sdX” (or “fdisk /dev/emcpowerX” for clariion systems), where X is the device on which you would like to create the new partition (a, b, c, etc).
2. Type “n” to create a new partition
3. Type “p” to create a primary partition
4. Type “1” to create partition #1
5. Select the defaults to use the full disk.
6. Type “t” to change partition type
7. Type “1” to select partition #1
8. Depending on your Linux environment and need: type “83” to set type to Linux partition, or type “82” to set type to Linux swap, or type “8e” to set type to Linux LVM, or type “fb” to set type to VMFS (vmware file system). For other partition types, type “L” to display the list of codes.
9. Type “x” to get into the expert mode
10. Type “b” to specify the starting block of partitions
11. Type “1” to select partition #1
12. Type “128” to make partition to align on 64KB boundary (block No. 128)
13. Type “w” to write new partition information to disk.
14. Exit fdisk and format the partition with your favorie filesystem.

Solaris:

To be added.

Making Exchange 2007 perform on ESX.

For a couple years we’ve maintained a full lab environment of our production hosted Microsoft Exchange 2007 CCR cluster. I have to be honest, and the Exchange administrator would agree, its never really performed that well. Finally the Exchange admin pretty much got ticked at the performance the other day and while he was out on vacation I thought I’d see what I could do. Its now performing about 100x the speed it ever did, and when he comes back Monday I hope he’s happy. More than anything we’ve just not had the time to really dig into the issue(s).

Needless to say over these couple years we’ve all learned quite a bit about how to eek more performance out of ESX, and in particular Exchange on ESX. I thought I’d share a bunch of the concepts and tidbits in one spot I used to arrive at better performance. The old tricks of throwing RAM and vCPU’s at the problem just didn’t cut it.

Its worth noting that people these days might not choose to use CCR on a virtualized environment (or never did), however I feel these concepts bleed over into stand-alone or maybe even FT/vLockStep implementations going forward.

First and foremost, I invite you to read this article on Exchange 2007’s memory managment strategy:

http://msexchangeteam.com/archive/2008/08/06/449484.aspx

Okay, now that you’ve read that lets continue. Suffice it to say Exchange literally grabs every piece of memory and page it can…. if you let it (which most people do for cache/performance reasons).

As most of you reading know, ESX has quite a few tricks up its sleeve in the memory management department itself and I invite you to read about those concepts in the vSphere/ESX manuals. Obviously the VM’s have access to physical RAM, shares RAM pages when possible, begs/borrows/steals from other VM’s (balloon) and when necessary swaps to disk as a last resort. After careful examination of the performance logs of the Exchange VM’s, it became very obvious ESX was swapping.

Now, we happen to have an entire lab cluster and nobody really cares about performance … well nobody except the persnickity Exchange admin anyway. After doing some research I came to the conclusion I didn’t want the Exchange servers to swap memory … period. I then set a reservation on the VM to the exact same size as the RAM I’d granted. In this case 3GB. This effectivly disables the vswap since the host has no choice but to ante up.  Doing  just this provided an incredible performance boost to the Exchange cluster, but I scratched a little deeper.

Within the VM itself I observed that it was paging quite a bit, and as the above referenced article shows… it always will. So to extract maximum paging performance I decided to create a couple LUN’s and mapped them raw (RDM) to the Exchange servers. I then did some research and came to the conclusion 4kb was the optimal block size for a raw paging volume and if anyone has differing opinions on that PLEASE post them. I then created page files equal to granted memory +20MB. After doing all of that and rebooting, I could tell we’re really cooking with gas now.

Going forward I would like to talk to the Exchange adminstrator about migrating his VMDK based message stores to RDM as well.

So, in closing … hopefully some of these ideas will help you come up with your own Exchange performance issue resolution. I’m sure there are more tricks I need to find, but right now I’m pretty happy with the results.

Till next time…

Why did sysprep get so hard for Server 2008, Vista and Windows 7?

I know I’m way behind the curve on this, but why did sysprep have to get so difficult on the newer server and workstation operating systems? For quick hit deployments, the 20 line .ini file worked just fine. XML, the wave of the future. I’m thinking about writing a quick webpage that cranks out the XML for you without having to crack open WIM’s and install an almost 3GB deployment toolkit.

That’s really all I had to say after a long lapse in posting 🙂

Till next time…

Evicting a broken cluster node via command line.

Like I’ve said before, I mostly have this blog for me. I often forget how to do stuff due to getting old.

Ran into a situation where a Microsoft Cluster node ate itself, and I wasn’t able to evict it with the GUI Cluster Administrator. This second proceedure at this link fixed it up: http://support.microsoft.com/kb/282227

If you cannot start the Cluster service, or if you have trouble removing the node, you can manually unconfigure the Cluster service:

1. Run the Cmd.exe program to open a command prompt.
2. At the command prompt, type cluster node nodename /forcecleanup, and then press ENTER.

Getting show desktop back into your quick launch.

I’m not sure why, but for whatever reason the Show Desktop shortcut disappeared from my quick launch menu. You’d think this would be really easy to get back, but either I’m stupid or it isn’t that easy. I’m sure there’s some more elegant way to do this, but I never claimed to be an expert. Whatever the case, not having this shortcut really bugs me … too lazy for keyboard shortcuts 😎

Anyway, here’s how I got mine back.

Start -> Run -> notepad

Paste in the following and save it as Desktop.scf:

[Shell]
Command=2
IconFile=explorer.exe,3
[Taskbar]
Command=ToggleDesktop

Copy / paste this file into: C:Documents and Settingsyour_user_hereApplication DataMicrosoftInternet ExplorerQuick Launch

Then right click on the task bar and disable/enable Quick Launch to force a re-read.

Till next time…